PhD Defence Evania Fasya | Salient and subtle behaviors in social interaction | Investigating the social effects of virtual humans

Salient and subtle behaviors in social interaction | Investigating the social effects of virtual humans

Evania Fasya is a PhD student in the department Human Media Interaction. (Co)Promotors are prof.dr. D.K.J. Heylen from the faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS), University of Twente and prof.dr. M.E. Kret and dr. E.J. van den Bos from Leiden University.

Humans’ nonverbal behaviors in social interaction, such as facial expressions, eye gaze, and even subtle cues such as pupil dilation, may have social effects. For example, they may be associated with higher trust by the interaction partner or may influence the interaction partner’s emotional responses. Moreover, anxiety in social situations may influence the expression and evaluation of behaviors, as well as emotional responses.

Recent advances in technology enable people to interact with artificial agents such as virtual humans. In psychological research, virtual humans are beneficial for investigating social behaviors as they can balance experimental control and ecological validity. At the same time, a systematic investigation of virtual humans’ social effects can support their development, including applications aimed at reducing anxiety in social situations.

Throughout three empirical studies, this dissertation investigates how salient and subtle behaviors in social interaction with virtual humans affect social evaluation and emotional responses, and whether anxiety in social situations influences these social effects. The results show that, first, virtual humans’ positive behaviors and pupil dilation mimicry (in modulation with eye contact) enhance social evaluations of virtual humans (Chapter 4 and 3), suggesting their potential for improving future applications of virtual humans. Importantly, it generalizes the social effects of pupil mimicry to a richer situation involving storytelling with virtual humans. Second, humans’ own behavior contributes to the social evaluation; mimicking smiles and making more eye contact enhances humans’ evaluation of the virtual humans (Chapter 2 and 3), generalizing their effects from human-human interaction to human-virtual human interaction, and may provide an alternative way to assess humans’ evaluation of virtual humans. Third, anxiety in social interaction moderates the social effects of virtual humans, but only when the situation implies higher social risks (Chapter 4), highlighting the importance of customizing social situations and measuring anxiety levels, especially in applications that aim to benefit people with social or public speaking anxiety.

Collectively, this dissertation contributes to a more systematic investigation of the social effects of virtual agents, as well as the study of salient and subtle behaviors in social interaction in ways that balance ecological validity and experimental control.