CTIT University of Twente
Research Business & Innovation About CTIT Research Calls Looking for a job? Intranet

Workshop: The Virtuality Continuum Revisited


For the third time in succession the "Human Media Interaction" (HMI) research section of the CTIT succeeded to get a workshop proposal accepted at the most important international conference on Computer Human Interaction (CHI). In 2003 we organized such a workshop in Miami (on humor in the interface), in 2004 we organized a workshop in Vienna on the social aspects of ambient intelligence, and, now, in 2005, we will organize a workshop on the virtuality continuum (multi-party interaction between human and virtual agents in smart/ambient environments) at CHI 2005 in Portland. The workshop will be organized together with David Traum of the Institute of Creative Technologies in Marina del Rey, California, USA.

Preliminary announcement of this workshop:

More and more we see the need to model multi-party interaction in the interface. Interactions take place in environments rather than in the traditional human-computer interface and in these interactions different actors are involved, both human and non-human. Environments, e.g. a smart meeting room or a desktop storytelling environment, require the modeling of multimodal interaction: interactions between human users, the environments and objects represented in the environments, and (embodied) conversational agents that represent human users or that have been designed to play particular roles in the environment. Traditionally, an agent plays the role of an information or navigation agent, plays the role of a meeting assistant in a virtual meeting environment, or plays the role of an actor in a virtual storytelling environment. However, in many environments, rather than interacting with one particular user they need to interact with different human and also other synthetic agents, they need to know about the properties (personalities, intelligence, emotions, capabilities, etc.) of these different agents, they need to know who is aware of what they are saying or doing, and they need to maintain a model of the multi-party dialogue between the different agents.

The concept of the virtuality continuum was introduced in the literature by Milgram and Kishino in 1994. It describes a continuum from full reality to full virtuality. A real environment can have human inhabitants that interact with each other without computer-generated stimuli. At the other hand of the continuum we have an environment where all stimuli are computer-generated. In this workshop we investigate the modeling of multi-party interaction from the point of view of the virtuality continuum. For environments on this continuum, interaction modeling means multimodal (verbal and nonverbal) interaction modeling and it means modeling various humans and virtual humans interacting simultaneously. These virtual humans can either be fully synthesized and autonomous, they can real-time represent humans that remotely visit, meet or work in the environment, or they can be something in between.

This workshop aims to:


Identify HCI problems related to multi-party interacting in the virtuality continuum; shortcomings and necessary development of existing theories; role of the properties of the environment and its position on the continuum on interaction behavior of inhabitants, i.e., interacting with the environment and with other inhabitants.


Discuss the role of multi-party interaction modeling, assuming that objects, physical inhabitants, virtual inhabitants and (future) observers all play roles in the multi-party interaction. This research needs to take into account temporal and spatial characteristics of interaction in physical, augmented reality and virtual reality environments, including cross-modal reference resolution. In particular we need to discuss models that integrate theories from linguistics and nonverbal (gaze, turntaking, addressee detection) communication


Discuss problems related to the fusion and fission of information for multi-party interaction of virtual agents and their human partners in environments positioned on the continuum. What is the state-of-the-art in multi-modal interaction modeling, in particular human nonverbal interaction behavior and does it cover the needs of these environments?

We encourage participation from a wide range of disciplines including Human-Computer Interaction, Social Psychology, Computer Science, Artificial Intelligence and Natural Language Processing. The workshop will be limited to 16 participants. Please submit a 2-4-page position paper outlining your interest in this topic to anijholt@cs.utwente.nl. Position papers must be received by 15 January 2005. Participants will be notified of selection by 15 February 2005.

The workshop format will include a presentation by each participant and discussion. In addition each participant will lead a discussion on the issues raised by another participant's paper.