Operating a computer by gesture only

11 March 2010 

Operating computers without touching them, using only hand and arm gestures: it sounds futuristic, but it’s already possible. Researcher Wim Fikkert of the Centre for Telematics and Information Technology of the University of Twente investigated the gestures that people would naturally use to operate a computer in this way. Remarkably, most test subjects chose the same gestures of their own accord. Fikkert obtained his PhD for his research from the Faculty of Electrical Engineering, Mathematics and Computer Science on 11 March.

We are accustomed to operating computers with a keyboard and mouse, and touchscreens have been with us for several years now. The next step is that we will be able to operate computers without touching anything. Wim Fikkert of the University of Twente investigated how people could operate computers using just a few hand and arm gestures. The starting point of his research was not the technological aspect, but the gestures that people would use naturally. Remarkably, most test subjects chose the same gestures of their own accord. The users chose a separate gesture for each command, and also stuck to their choices. Another striking finding was that the users explicitly changed the shape of their hands at the beginning of a gesture and relaxed at the end of it.

Method
Fikkert used various different test designs for his experiments. In the simplest test the subjects thought they were controlling a computer with their gestures, when in fact someone else was doing so. In the most advanced test the subjects themselves interacted with a 4 x 1.5 metre screen at the University of Twente's Smart Experience Laboratory. They operated the system using wireless lasers on the backs of their hands for pointing and small buttons on their fingers for giving commands.

Note to editors
Wim Fikkert carried out his research at the Centre for Telematics and Information Technology research institute and the Faculty of Electrical Engineering, Mathematics and Computer Science. His tutors were Prof. Anton Nijholt, Prof. Gerrit van der Veer and Dr Paul van der Vet. The research was partly made possible by the Netherlands Bioinformatics Centre (NBIC). For more information, or a digital version of the doctoral thesis Gesture Interaction at a Distance, please contact science information officer Joost Bruysters (053 489 2773 / 06 1048 8228).