Tailoring coaching conversations with virtual health coaches
Due to the COVID-19 crisis the PhD defence of Tessa Beinema will take place (partly) online.
The PhD defence can be followed by a live stream.
Tessa Beinema is a PhD student in the research group Biomedical Signals and System (BSS). Her supervisor is prof.dr.ir. H.J. Hermens from the Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS).
While a healthy lifestyle is important for our well-being and can prevent illnesses, changing our behaviour to adopt a healthier lifestyle can be difficult. Digital health (eHealth) applications can support people in this process. However, it has been found that the use of these applications decreases rapidly when the novelty effect wears off. Two causes for this are the lack of human involvement and the lack of personalised content. Therefore, conversational agents are added to eHealth applications in the role of health coaches to provide a social incentive, and adjustment of their dialogue content to users is investigated to increase personal relevance (tailoring). Such tailoring has previously been found to be effective in eHealth applications, but has not extensively been investigated for interactive two-way communication over a longer period of time, such as in coaching conversations. The objective of this thesis research was therefore to address the following question: “How can we tailor users’ coaching conversations with conversational agents to improve engagement?”
To provide users with tailored coaching conversations, dialogue content is needed that can be adapted. In addition, it is also important that such dialogue content is in line with health coaching guidelines and that the advice the user receives is safe. Thus, it is beneficial if dialogues can also be written and tested by (non-technical) domain experts. Chapter 2, therefore, describes the development of a set of tools that support this process: the WOOL Dialogue Platform. These tools allow domain experts to author and test dialogues, while they also include dialogue execution libraries to support agent developers. The chapter describes the developed tools, dialogue language, and use cases from projects in which the WOOL Dialogue Platform is used.
Following the development of dialogue tools, Chapter 3 describes a study that investigates if the long-term strategies that are underlying to a coaching dialogue can be tailored. In the study, 108 participants interacted with a group of conversational agents, during which they answered a questionnaire on motivation to live healthy, were classified into motivation groups and were presented by the agents with two strategies: a positively and a negatively tailored strategy for their motivation group. Participants were asked to rate these strategies and to choose their favourite. In addition, they were asked to rate the agents' likeability at two moments during the experiment. Results showed that it is indeed possible to tailor coaching strategies to users' motivation to live healthy in the context of coaching dialogues, and they confirm that users can be both intrinsically motivated and externally regulated. Furthermore, the agents' likeability did not seem to affect users' ratings for the strategies, but rather the discussed dialogue content seemed to affect agents' likeability scores.
Where Chapter 3 investigated tailoring of the underlying strategies, the last two chapters in this thesis focus on dynamically tailoring coaching dialogues during interaction. To that end, Chapter 4 discriminates five levels in the process of tailoring conversations, namely the domain, topic, action, dialogue act and utterance level. The chapter then focuses on the definition of a blueprint topic model to support automatic tailoring on the topic level. The model was constructed on the basis of literature on behavioural psychology, persuasive technology and relational agents. It includes 30 hierarchically structured topics that cover 115 actions that a conversational agent could perform as a coach. The blueprint topic model was validated in a closed card-sort study with 11 experts. Results showed that most of the defined topics were confirmed to cover their core actions. Cards that were sorted incorrectly were often missing an immediacy indicator in their description or were based on actions for which it is more complex to imagine a corresponding dialogue.
Following the blueprint topic model, in Chapter 5 a proof-of-concept implementation of a component that performs tailored selection of conversational topics is evaluated. The chapter describes the setup and results of a micro-randomized trial that compares two conditions. In the first condition, the conversational agent suggested a specific topic to discuss (coach initiative), which is an application of the topic selection component. In the second condition, the user was shown a menu dialogue from which they could select what they wanted to discuss (user initiative). The condition was randomly selected each time a participant interacted with the conversational agent. In the study, 82 participants interacted with the application over 4-8 weeks, which resulted in 1,011 interactions. Results showed that the coach suggesting a specific (tailored) topic was perceived as a natural interaction and that acceptance of these suggestions by participants was high (71.1% overall, 75.8% for coaching topics). Participants did care about the type of topic that was suggested, as supported by the lower acceptance rate for background story suggestions in the second round. Furthermore, there were indications that a coach's initiative might need to be tailored to users' active involvement in their behaviour change process.
Finally, Chapter 6 discusses the main findings and conclusions of this thesis and does so following three themes, namely: 1) content and system design, 2) tailoring content and conversations, and 3) evaluation of tailored content. In short, it concludes that content should be carefully constructed and reported, that tailoring approaches should be combined, and that evaluation of tailored applications remains complicated due to the many options for tailoring and factors of influence in long-term daily life situations. The second part of the chapter then discusses considerations for future work on tailoring and conversational agents in eHealth and does so for five themes: safety, verification and validation; autonomy and trust; social interaction; input and output modalities; and open science.