HomeEducationDoctorate (PhD & EngD)For current candidatesPhD infoUpcoming public defencesFULLY DIGITAL (UNTIL FURTHER NOTICE) : PhD Defence Silke ter Stal | Look who's talking - Appearance of embodied conversational agents in Ehealth

FULLY DIGITAL (UNTIL FURTHER NOTICE) : PhD Defence Silke ter Stal | Look who's talking - Appearance of embodied conversational agents in Ehealth

Look who's talking - Appearance of embodied conversational agents in Ehealth

Due to the COVID-19 crisis measures the PhD defence of Silke ter Stal will take place online (until further notice).

The PhD defence can be followed by a live stream.

Silke ter Stal is a PhD student in the research group Biomedical Signals and Systems (BSS). Her supervisor is prof.dr.ir. H.J. Hermens from the Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS).

As we live longer and more people develop chronic diseases, the demand for care increases. The focus of our health care has to shift from curing acute complaints to (secondary) prevention of complaints via long-term care and coaching. A solution widely investigated to offer this long-term care and coaching is eHealth. But, for eHealth applications to be effective, use of the applications is crucial. In that sense, engagement with an eHealth application is necessary; those who are more engaged are significantly less likely to stop using it. To engage users in long-term care and coaching, Embodied Conversational Agents (ECAs) can be used. As humans, we immediately form ideas about such ECAs based on their appearance when we interact with them for the first time, similar to what we do when interacting with another human for the first time. Then, when we have a positive first impression of another human, we tend to interact more with that human. This might also apply to our interaction with ECAs. Therefore, ECAs could stimulate users to start or continue interacting with the underlying eHealth application (at first glance or after short interaction respectively) to eventually reach long-term interaction. By researching how an ECA’s appearance should be designed to trigger positive impressions in these different phases of interaction, we can stimulate users to continue to next phase and eventually reach long-term engagement.

The main aim of this thesis was:

“How to design the appearance of an ECA in eHealth to promote user engagement?”

 The sub objectives of this thesis were:

  1. How to design the appearance of an ECA in eHealth for positive impressions at first glance?
  2. How to design the appearance of an ECA in eHealth for positive impressions after short
    interaction?
  3. How to design the appearance of an ECA in eHealth for positive impressions after long-term interaction?

Chapter 2 investigated the state-of-the-art research on an ECA’s appearance in eHealth. The chapter presented a literature review identifying the researched design features for ECAs in eHealth, the outcome variables that were used to measure the effect of these design features and what the found effects for each variable were. Results showed that an ECA’s speech and/or textual output and its facial and gaze expressions were the most common design features. Little research was performed on the ECA’s looks. The measured effect of these design features was often on the perception of the ECA’s and user’s characteristics, relation with the ECA, system usage, intention to use, usability and behaviour change. Furthermore, results showed that emotion and relational behaviour seem to positively affect the perception of the ECA’s characteristics and that relational behaviour also seems to positively affect the relation with the ECA, usability and intention to use. However, results showed that these design features do not necessarily lead to behaviour change. Overall, chapter 2 showed that consensus on design features of ECAs in eHealth is far from established.

As chapter 2 showed that little research is performed on the ECA’s looks, chapter 3 explored users’ perceptions of ECAs with different looks. The chapter described effects of an ECA’s age (young or old), gender (male or female) and role (peer or expert) on perceptions of the ECAs at first glance (sub objective 1). In an online survey, participants were shown a set of eight static ECA images, positioned as coaches on healthy living. The chapter investigated which ECAs participants preferred at first glance, how they perceived the characteristics of these ECAs, and how likely they were to follow the advice of these ECAs. Results showed that people do have preferences for particular ECAs at first glance, and do rate ECA’s different depending on the ECA’s age, gender and role. For both a general and elderly population: 1) people seem to prefer images of young, female agents over old, male agents, 2) the a) age, b) gender and c) role of the agent image affect the perception of the agent’s characteristics and the likeliness of following the agent’s advice, and 3) people seem to prefer an agent image that is similar in a) age and b) gender.

Chapter 4 investigated whether similar findings could be found after short interaction with an ECA (sub objective 2). Three validated health assessment questionnaires were translated into dialogues between a user and an ECA. Participants interacted with the ECAs in a lab setting. In half of the dialogues, participants interacted with a young, female peer ECA and in the other half of the dialogues with an older, male peer ECA. Results showed that the older male ECA was perceived as more authoritative than the young female ECA, but that no other differences were found between the ECAs. Besides effects of an ECA’s looks, this thesis investigated effects of an ECA’s emotion after short interaction. Chapter 5 presented a study in which an ECA’s emotions expressed in text and an ECA’s emotions in facial expressions were compared on their effect on users’ perceptions of rapport. A happy or neutral facial expression was combined with a happy or neutral textual expression. The ECAs were presented to the participants in an online survey and were positioned as health coaches on healthy living. Participants interacted with one of the four coaches. Results showed no significant difference in overall rapport between the conditions, but that a happy textual expression for an ECA with a neutral facial expression resulted in higher ratings of the individual rapport items helpfulness and enjoyableness.

Chapter 6 and 7 described experimental studies on long-term interaction with an ECA in daily life. In these chapters, an ECA’s appearance was researched in the actual context of use (sub objective 3). Chapter 6 presented a study in which an ECA was developed within an eHealth self-management intervention for patients with both Chronic Obstructive Pulmonary Disease (COPD) and Chronic Heart Failure (CHF), which was offered for four months. Patients interacted with the application at home. Results showed that patients’ perceptions of the ECA’s characteristics did not change over time, but that after three weeks of use, the patients were less likely to follow the ECA’s advice. The ECA’s messages were perceived as non-personalised and the feedback as inappropriate, affecting the ECA’s perceived reliability. Chapter 7 described a study in which an ECA’s appearance was researched in a multi-agent eHealth application that provided holistic coaching on healthy ageing. Participants interacted with the application at home. In line with findings of chapter 3, the chapter explored whether, in such an application, older adults are more satisfied with ECAs that are similar to themselves in age and gender, both at first glance and after long-term use. Results did not confirm this hypothesis. In addition, results showed that preferences for particular ECAs change over time.

Lastly, chapter 8 presented a set of design strategies for an ECA’s appearance in eHealth to contribute to user engagement. The initial literature review on the state-of-the-art research on an ECA’s appearance for eHealth was updated in this chapter. Results showed that current studies on an ECA’s appearance for eHealth are of low, fair or moderate quality. For the majority of the design features researched no or preliminary evidence exists for effects on user engagement. Only for an ECA’s relational, empathic behaviour research shows moderate evidence and for an ECA’s emotion, agency, rendering style, clothing, body shape, gender, age and cultural tailoring research shows preliminary evidence. The findings of the state-of-the-art with at least preliminary evidence were synthesised into the following design strategies: 1) adapt the ECA’s demographics to the user’s preferences, 2) adapt the human-likeness of the ECA to its functionalities, 3) convey an ECA’s expertise using its appearance, 4) implement emotion in the ECA’s expressions, and 5) implement an ECA’s relational behaviour. These strategies should be validated, updated and extended in future research. Lastly, the chapter showed how these strategies can be used by an implementation example of an ECA as part of a mobile physical activity application.