PROFILE
Dirk Heylen is Professor Socially Intelligent Computing. His research interests cover both the machine analysis of human (conversational) behaviour and the generation of human-like (conversational) behaviour by virtual agents and robots. He is especially interested in the nonverbal and paraverbal aspects of dialogue and what these signals reveal about the mental state (cognitive, affective, social). These topics are explored both from a computational perspective and as basic research in the humanities, reflecting his training as a computational linguist. After his studies of Linguistics, Computer Science and Computational Linguistics at the University of Antwerp he moved to the Institute of Dutch Lexicology in Leyden, to develop tools for enriching natural language databases. After a couple of years he went on to the Utrecht University and got involved in the big European project Eurotra on Machine Translation. After coordinating a follow-up EU project, he started his PhD project on a logical approach to natural language analysis and parsing (Type Logical Grammar). At the University of Twente he started working on embodied dialogue systems (aka virtual agents or embodied conversational agents). This made his interests shift from pure linguistic analysis to body language, from text analysis to real-time human-machine interaction, and from the logical analysis to a much broader concern with emotion and social relations in interaction. He has been involved in several European projects such as Humaine, AMI and AMIDA, COST 2102, Semaine, SSPNet, and SERA. Dialogue management, human behaviour modeling, cognitive modeling, human-robot interaction, methodological issues in corpus studies and annotation, are some of the topics dealt with in these projects. Current Dutch national projects he is involved in include BrainGain in which neural correlates of a user's experience are explored for Human Computer Interaction using Brain Computing Interfaces and GATE dealing with cognitive models of virtual agents in training applications. He is the project leader of the COMMIT project Interaction for Universal Access.Topics of research: embodied conversational agents, social signal processing, ambient intelligence, affective computing spoken dialogue systems, human computer interaction, backchannels, brain computer interfaces.
Publications
Jump to: 2024 | 2023 | 2022 | 2021 | 2020
2024
Using Default Logic to Create Adaptable User Models for Behavior Support Agents (2024)In HHAI 2024: Hybrid Human AI Systems for the Social Good: Proceedings of the Third International Conference on Hybrid Human-Artificial Intelligence (pp. 350-359) (Frontiers in Artificial Intelligence and Applications; Vol. 386). IOS. Wolff, J., de Boer, V., Heylen, D. & Van Riemsdijk, M. B.https://doi.org/10.3233/FAIA240207Risk Assessment Using A Virtual Assistant In A Digital Genetic Counselling Application (2024)[Contribution to conference › Abstract] 13th Supporting Health by Technology Conference 2024. Beinema, T., Kolkmeier, J., Klaassen, R., van den Heuvel, L., Siemelink, M., van Lingen, M., van Essen, D., van Tintelen, P. & Heylen, D.Acquiring Semantic Knowledge for User Model Updates via Human-Agent Alignment Dialogues: Scenarios & Dialogues (2024)[Dataset Types › Dataset]. 4TU.Centre for Research Data. Chen, P.-Y., Tielman, M., van Riemsdijk, M. B., Jonker, C. M. & Heylen, D. K. J.https://doi.org/10.4121/b7a321df-640a-483d-8c32-a18fe21e7204Smile mimicry smoothens human-virtual human interactions (2024)[Dataset Types › Dataset]. DataverseNL. Fasya, E. L., van den Bos, E., Heylen, D., Kret, M. & Kret, M.https://doi.org/10.34894/hfxsmmSmile mimicry smoothens human-virtual human interactions (2024)International journal of human-computer studies, 183. Article 103182. Fasya, E. L., Bos, E. v. d., Heylen, D. K. J. & Kret, M. E.https://doi.org/10.1016/j.ijhcs.2023.103182Assessing the HI-ness of Virtual Heritage Applications with Knowledge Engineering (2024)In HHAI 2024: Hybrid Human AI Systems for the Social Good - Proceedings of the 3rd International Conference on Hybrid Human-Artificial Intelligence (pp. 173-187) (Frontiers in Artificial Intelligence and Applications; Vol. 386). IOS. Javdani Rikhtehgar, D., Tiddi, I., Wang, S., Schlobach, S. & Heylen, D.https://doi.org/10.3233/FAIA240193Ontology-Based Modeling of User-Environment Interaction in a Virtual Reality Exhibition (2024)In Metadata and Semantic Research: 17th Research Conference, MTSR 2023, Milan, Italy, October 25–27, 2023, Revised Selected Papers (pp. 241-252) (Communications in Computer and Information Science; Vol. 2048). Springer. Javdani Rikhtehgar, D., Wang, S., Schlobach, S. & Heylen, D.https://doi.org/10.1007/978-3-031-65990-4_22
2023
Drone-Driven Running: Exploring the Opportunities for Drones to Support Running Well-being through a Review of Running and Drone Interaction Technologies (2023)In HAI 2023: Proceedings of the 11th Conference on Human-Agent Interaction (pp. 209-220) (ACM International Conference Proceeding Series). Association for Computing Machinery. Balasubramaniam, A., Reidsma, D. & Heylen, D.https://doi.org/10.1145/3623809.3623831Toward Standard Guidelines to Design the Sense of Embodiment in Teleoperation Applications: A Review and Toolbox (2023)Human-computer interaction, 38(5-6), 322-351. Falcone, S., Englebienne, G., van Erp, J. B. F. & Heylen, D. K. J.https://doi.org/10.1080/07370024.2022.2039147A multidisciplinary investigation to unravel the complexity of the sense of embodiment in teleoperation (2023)[Thesis › PhD Thesis - Research UT, graduation UT]. University of Twente. Falcone, S.https://doi.org/10.3990/1.9789036558815Touch Technology in Affective Human-, Robot-, and Virtual-Human Interactions: A Survey (2023)Proceedings of the IEEE, 111(10), 1333-1354. Olugbade, T., He, L., Maiolino, P., Heylen, D. & Bianchi-Berthouze, N.https://doi.org/10.1109/JPROC.2023.3272780Exploring Runners' Preferences of Drone Based Feedback to Support their Well-Being (2023)In Proceedings of 2nd International Conference of the Greece ACM SIGCHI Chapter, CHIGreece 2023. Article 10 (ACM International Conference Proceeding Series). Association for Computing Machinery. Balasubramaniam, A., Reidsma, D., Obaid, M. & Heylen, D.https://doi.org/10.1145/3609987.3609997Machine learning for semi-automated scoping reviews (2023)Intelligent Systems with Applications, 19. Article 200249. Mozgai, S., Kaurloto, C., Winn, J., Leeds, A., Heylen, D., Hartholt, A. & Scherer, S.https://doi.org/10.1016/j.iswa.2023.200249Personalizing Cultural Heritage Access in a Virtual Reality Exhibition: A User Study on Viewing Behavior and Content Preferences (2023)In UMAP 2023 - Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization (pp. 379-387). Association for Computing Machinery. Javdani Rikhtehgar, D., Wang, S., Huitema, H., Alvares, J., Schlobach, S., Rieffe, C. & Heylen, D.https://doi.org/10.1145/3563359.3596666Developing an eMental health monitoring module for older mourners using fuzzy cognitive maps (2023)Digital Health, 9. Brandl, L., van Velsen, L., Brodbeck, J., Jacinto, S., Hofs, D. & Heylen, D.https://doi.org/10.1177/20552076231183549Acquiring Semantic Knowledge for User Model Updates via Human-Agent Alignment Dialogues: An Exploratory Focus Group Study (2023)In HHAI 2023: Augmenting Human Intellect: Proceedings of the Second International Conference on Hybrid Human-Artificial Intelligence (pp. 93-107) (Frontiers in Artificial Intelligence and Applications; Vol. 368). IOS. Chen, P.-Y., Tielman, M. L., Heylen, D. K. J., Jonker, C. M. & Riemsdijk, M. B. V.https://doi.org/10.3233/FAIA230077Design of the Interaction and Virtual Assistant in a Digital Genetic Counseling Application (2023)[Contribution to conference › Abstract] 12th Supporting Health by Technology Conference 2023. Beinema, T., Klaassen, R., Kolkmeier, J., van Lingen, M., van den Heuvel, L., Siemelink, M., van Essen, D., van Tintelen, P. & Heylen, D. K. J.Development of an alcohol refusal training in Immersive Virtual Reality for patients with Mild to Borderline Intellectual Disability and Alcohol Use Disorder: Co-creation with experts in addiction care (2023)JMIR Formative Research, 7. Article e42523. Langener, S., Kolkmeier, J., VanDerNagel, J., Klaassen, R., van Manen, J. G. & Heylen, D.https://doi.org/10.2196/42523Toward a Scoping Review of Social Intelligence in Virtual Humans (2023)In 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition, FG 2023. IEEE. Mozgai, S., Beland, S., Leeds, A., Winn, J. G., Kaurloto, C., Heylen, D. & Hartholt, A.https://doi.org/10.1109/FG57933.2023.10042532Correlating Eye Gaze with Object to Enrich Cultural Heritage Knowledge Graph (2023)[Contribution to conference › Poster] 22nd International Semantic Web Conference, ISWC 2023. Wang, S., Kulyk, D., Javdani Rikhtehgar, D., Heylen, D. K. J. & Rieffe, C.Correlating Eye Gaze with Object to Enrich Cultural Heritage Knowledge Graph (2023)CEUR workshop proceedings, 3632. Wang, S., Kulyk, D., Rikhtehgar, D. J., Heylen, D. & Rieffe, C. J.diVRsity: exploring diversity in VR (2023)[Contribution to conference › Abstract] ICT Open 2023. Javdani Rikhtehgar, D., Wang, S., Heylen, D. K. J. & Rieffe, C.Ontology-Based Modeling of User-Environment Interaction in a Virtual Reality Exhibition (2023)In Proceedings 17th International Conference on Metadata and Semantics Research, MTSR 2023. University of Milano-Bicocca. Javdani Rikhtehgar, D., Wang, S., Schlobach, S. & Heylen, D. K. J.Smile mimicry smoothens human-virtual human interactions (2023)[Contribution to conference › Poster] 2023 Social Cognition Workshop: From Self-Knowledge to Knowing Others. Fasya, E. L., Bos, E. v. d., Heylen, D. K. J. & Kret, M. E.Using Autoepistemic Logics for Understandable and Flexible User-Models (2023)In Proceedings of the 21st International Workshop on Non-Monotonic Reasoning (NMR'23) (pp. 133-136) (CEUR Workshop proceedings; Vol. 3464). Wolff, J., Boer, V. D., Heylen, D. & Riemsdijk, M. B. V.https://ceur-ws.org/Vol-3464/short5.pdf
2022
Immersive Virtual Reality Avatars for Embodiment Illusions in People With Mild to Borderline Intellectual Disability: User-Centered Development and Feasibility Study (2022)JMIR Serious Games, 10(4). Article e39966. Langener, S., Klaassen, R., VanDerNagel, J. & Heylen, D.https://doi.org/10.2196/preprints.39966Three Interactive Add-ons for Small Local Playgrounds: Towards Designing for Context-sensitive Play Activities (2022)In CHI PLAY 2022: Extended Abstracts of the 2022 Annual Symposium on Computer-Human Interaction in Play (pp. 357-363). Association for Computing Machinery. Van Delden, R., Reidsma, D., Weijdom, J. P. & Heylen, D.https://doi.org/10.1145/3505270.3558387Exploring the Interaction Space to Support Running Experiences with Drones (2022)[Contribution to conference › Other] Social Drones for Health and Well-being Workshop at NORDICHI 2022. Balasubramaniam, A., Reidsma, D. & Heylen, D. K. J.From Head Transform to Mind Transplant: Social Interactions in Mixed Reality (2022)[Thesis › PhD Thesis - Research UT, graduation UT]. Kolkmeier, J.https://doi.org/10.3990/1.9789036554510Machine Learning for Semi-Automated Scoping Reviews (2022)[Working paper › Preprint]. Social Science Research Network (SSRN). Mozgai, S., Kaurloto, C., Winn, J., Leeds, A., Heylen, D., Hartholt, A. & Scherer, S.https://doi.org/10.2139/ssrn.4218678The multimodal EchoBorg: Not as smart as it looks (2022)Journal on multimodal user interfaces, 16(3), 293-302. Falcone, S., Kolkmeier, J., Bruijnes, M. & Heylen, D.https://doi.org/10.1007/s12193-022-00389-zPupil diameter as implicit measure to estimate sense of embodiment (2022)In Proceedings of the 44th Annual Meeting of the Cognitive Science Society (Proceedings of the Annual Meeting of the Cognitive Science Society; Vol. 44). The Cognitive Science Society. Falcone, S., Zhang, L., Pradhan, S., Englebienne, G., Brouwer, A.-M., Cocu, I., Stuldreher, I., Heuvel, M., de Vries, P., Gijsbertse, K., Heylen, D. & van Erp, J.https://escholarship.org/uc/item/8rt963b9Consulting the Oracle: A Delphi study for determining parameters for a mental health user profile and personalization strategy for an online service to aid grieving older adults (2022)Internet interventions, 28. Article 100534. Brandl, L., Cabrita, M., Brodbeck, J., Heylen, D. K. J. & van Velsen, L.https://doi.org/10.1016/j.invent.2022.100534Flavor text generation for role-playing video games (2022)[Thesis › PhD Thesis - Research UT, graduation UT]. University of Twente. van Stegeren, J.https://doi.org/10.3990/1.9789036553100The Relative Contribution of Five Key Perceptual Cues and their Interaction to the Sense of Embodiment (2022)Technology, Mind, and Behavior, 3(1). Falcone, S., Brouwer, A.-M., Cocu, I., Gijsbertse, K., Heylen, D. K. J. & van Erp, J. B. F.https://doi.org/10.1037/tmb0000068Assessing the Pupil Dilation as Implicit Measure of the Sense of Embodiment in Two User Studies (2022)[Contribution to conference › Abstract] 12th International Conference on Methods and Techniques in Behavioral Research, MB 2022. Falcone, S., Englebienne, G., Heylen, D. K. J. & van Erp, J. B. F.Toward a Semi-Automated Scoping Review of Virtual Human Smiles (2022)In Smiling and Laughter Across Contexts and the Life-Span, SmiLa 2022 - as part of the International Conference on Language Resources and Evaluation, LREC 2022 - Proceedings (pp. 1-5). European Language Resources Association (ELRA). Mozgai, S., Winn, J., Kaurloto, C., Leeds, A., Heylen, D. & Hartholt, A.https://aclanthology.org/2022.smila-1.1Virtual Exposure Training Program for Public Speaking (2022)[Contribution to conference › Poster] 22nd ACM International Conference on Intelligent Virtual Agents, IVA 2022. Fasya, E. L., Bos, E. v. d., Kret, M. E. & Heylen, D. K. J.
2021
Personal and Personalized Conversations: Designing Agents Who Want To Connect With You (2021)[Thesis › PhD Thesis - Research UT, graduation UT]. University of Twente. van Waterschoot, J. B.https://doi.org/10.3990/1.9789036552691How Familiarity Influences the Frequency, Temporal Dynamics and Acoustics of Laughter (2021)In 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 1-8). Jansen, M.-P., Truong, K. P. & Heylen, D. K. J.https://doi.org/10.1109/ACII52823.2021.9597414User Requirements for Developing Robot-Assisted Interventions for Autistic Children (2021)In Social Robotics : 13th International Conference, ICSR 2021. Singapore, Singapore, November 10-13, 2021. Proceedings (pp. 629-639) (Lecture Notes in Artificial Intelligence; Vol. 13086). Springer. Schadenberg, B. R., Reidsma, D., Heylen, D. K. J. & Evers, V.https://doi.org/10.1007/978-3-030-90525-5_55“Go up in smoke”: Feasibility and initial acceptance of a virtual environment to measure tobacco craving in vulnerable individuals (2021)In 2021 IEEE 9th International Conference on Serious Games and Applications for Health(SeGAH) (IEEE International Conference on Serious Games and Applications for Health, SeGAH; Vol. 2021). IEEE. Langener, S., VanDerNagel, J., Klaassen, R., van der Valk, P. & Heylen, D.https://doi.org/10.1109/SEGAH52098.2021.9551854Agents United: An Open Platform for Multi-Agent Conversational Systems (2021)In IVA '21: Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents (pp. 17-24). Association for Computing Machinery. Beinema, T., Davison, D., Reidsma, D., Banos, O., Bruijnes, M., Donval, B., Fides Valero, Á., Heylen, D., Hofs, D., Huizing, G., Kantharaju, R. B., Klaassen, R., Kolkmeier, J., Konsolakis, K., Pease, A., Pelachaud, C., Simonetti, D., Snaith, M., Traver, V., … op den Akker, H.https://doi.org/10.1145/3472306.3478352Predictable Robots for Autistic Children — Variance in Robot Behaviour, Idiosyncrasies in Autistic Children’s Characteristics, and Child–Robot Engagement (2021)ACM Transactions on Computer-Human Interaction, 28(5). Article 3468849. Schadenberg, B. R., Reidsma, D., Evers, V., Davison, D. P., Li, J. J., Heylen, D. K. J., Neves, C., Alvito, P., Shen, J., Pantic, M., Schuller, B., Cummins, N., Olaru, V., Sminchisescu, C., Dimitrijevic, S. B., Petrovic, S., Baranger, A., Williams, A., Alcorn, A. M. & Pellicano, E.https://doi.org/10.1145/3468849Clinical Relevance of Immersive Virtual Reality in the Assessment and Treatment of Addictive Disorders: A Systematic Review and Future Perspective (2021)Journal of Clinical Medicine, 10(16). Article 3658. Langener, S., VanDerNagel, J., Manen, J. v., Markus, W., Dijkstra, B., DeFuentes-Merillas, L., Klaassen, R., Heitmann, J., Heylen, D. & Schellekens, A.https://doi.org/10.3390/jcm10163658Individuals with High Kinesthetic Intelligence Experience an Active Embodiment Illusion Assessed with Pupil Dilation (2021)[Contribution to conference › Abstract] 43rd Annual Meeting of the Cognitive Science Society, CogSci 2021. Falcone, S., Pradhan, S., van Erp, J. B. F. & Heylen, D. K. J.“I See What You Did There”: Understanding People’s Social Perception of a Robot and Its Predictability (2021)ACM Transactions on Computer-Human Interaction, 10(3). Article 28. Schadenberg, B. R., Reidsma, D., Heylen, D. K. J. & Evers, V.https://doi.org/10.1145/3461534Data underlying the article: "I See What You Did There": Understanding People's Social Perception of a Robot and its Predictability (2021)[Dataset Types › Dataset]. 4TU.Centre for Research Data. Schadenberg, B., Reidsma, D., Heylen, D. K. J. & Evers, V.https://doi.org/10.4121/14706063.v1Who Makes Your Heart Beat? What Makes You Sweat? Social Conflict in Virtual Reality for Educators (2021)Frontiers in psychology, 12. Article 628246. Lee, M., Kolkmeier, J., Heylen, D. & IJsselsteijn, W.https://doi.org/10.3389/fpsyg.2021.628246Robots for Autistic Children: Understanding and Facilitating Predictability for Engagement in Learning (2021)[Thesis › PhD Thesis - Research UT, graduation UT]. University of Twente. Schadenberg, B. R.https://doi.org/10.3990/1.9789036551649Data2Game: Towards an Integrated Demonstrator (2021)In Advances in Usability, User Experience, Wearable and Assistive Technology - Proceedings of the AHFE 2021 Virtual Conferences on Usability and User Experience, Human Factors and Wearable Technologies, Human Factors in Virtual Environments and Game Design, and Human Factors and Assistive Technology, 2021 (pp. 239-247) (Lecture Notes in Networks and Systems; Vol. 275). Springer. Steinrücke, J., Mavromoustakos-Blom, P., van Stegeren, J., Attema, Y., Bakkes, S., de Groot, T., de Heer, J., Heylen, D., Hrynkiewicz, R., de Jong, T., Oortwijn, T., Spronck, P., Theune, M. & Veldkamp, B.https://doi.org/10.1007/978-3-030-80091-8_28Designing Effective Dialogue Content for a Virtual Coaching Team Using the Interaction Process Analysis and Interpersonal Circumplex Models (2021)In Persuasive Technology: 16th International Conference, PERSUASIVE 2021, Virtual Event, April 12–14, 2021, Proceedings (pp. 19-32) (Lecture Notes in Computer Science; Vol. 12684). Springer. Huizing, G., Klaassen, R. & Heylen, D.https://doi.org/10.1007/978-3-030-79460-6_2
2020
The Sensory Interactive Table: Exploring the Social Space of Eating (2020)In ICMI 2020 - Proceedings of the 2020 International Conference on Multimodal Interaction (pp. 689-693). Association for Computing Machinery. De Vries, R. A. J., Haarman, J. A. M., Harmsen, E. C., Heylen, D. K. J. & Hermens, H. J.https://doi.org/10.1145/3382507.3418866Multi-Perspective Persuasion by a Council of Virtual Coaches (2020)In BCSS 2020: Behavior Change Support Systems: Proceedings of the Eighth International Workshop on Behavior Change Support Systems organized in conjunction with the 15th International Conference on Persuasive Technology 2020 (Persuasive 2020) (CEUR Workshop Proceedings; Vol. 2662). CEUR. Huizing, G. H., Klaassen, R. & Heylen, D. K. J.http://ceur-ws.org/Vol-2662/Go up in smoke: proof of concept study on tobacco craving in a VR environment (2020)[Contribution to conference › Abstract] Supporting Health by Technology 2020. Langener, S., van der Nagel, J., Klaassen, R., Brouwer, L. & Heylen, D.Introducing MULAI: A Multimodal Database of Laughter during Dyadic Interactions (2020)In Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020) (pp. 4333-4342). European Language Resources Association (ELRA). Jansen, M.-P., Truong, K. P., Nazareth, D. S. & Heylen, D. K. J.https://www.aclweb.org/anthology/2020.lrec-1.534/Sensory interactive table (SIT): Development of a measurement instrument to support healthy eating in a social dining setting (2020)Sensors (Switzerland), 20(9). Article 2636. Haarman, J. A. M., de Vries, R. A. J., Harmsen, E. C., Hermens, H. J. & Heylen, D. K. J.https://doi.org/10.3390/s20092636Helping Educators Monitor Autistic Children's Progress Across Sessions: A Needfinding Study (2020)In HRI '20: 15th Annual ACM/IEEE International Conference on Human-Robot Interaction, March 23-26, 2020 (pp. 436-438). Association for Computing Machinery. Schadenberg, B. R., Li, J. J., Petrovic, S., Reidsma, D., Heylen, D. K. J. & Evers, V.https://doi.org/10.1145/3371382.3378378Differences in Spontaneous Interactions of Autistic Children in an Interaction With an Adult and Humanoid Robot (2020)Frontiers in robotics and AI, 7, 1-19. Article 28. Schadenberg, B. R., Reidsma, D., Heylen, D. K. J. & Evers, V.https://doi.org/10.3389/frobt.2020.00028Alertness, movement, and affective behaviour of people with profound intellectual and multiple disabilities (PIMD) on introduction of a playful interactive product: Can we get your attention? (2020)Journal of Intellectual and Developmental Disability, 45(1), 66-77. van Delden, R. W., Wintels, S. C., van Oorsouw, W. M. W. J., Evers, V., Embregts, P. J. C. M., Heylen, D. K. J. & Reidsma, D.https://doi.org/10.3109/13668250.2018.1537845