Collaborative partners | BCI projects

Brain-Computer Interfaces (or BCI) have become a hot topic in both science and popular culture. The basic notion of using one’s brain as direct input for both the control and exchange of information with a computer system sparks the imagination. Visions of keyboardless, computers and adaptive command centres often spring to mind. At the University of Twente’s BMS Lab, we work on realizing these visions. Together with our Partners at Thales, Artinis Medical Systems, Noldus Information Technologies, and ANT Neuro, we are setting up a testing ground within two projects and a development centre at the BMS Lab for both the fundamental science and BCI applications. Thanks to years of experience with the use of EEG equipment and the underlying science, the BMS Lab can facilitate, develop and support the BCI testbed.

EFRO Alert

It is of great importance to gain insight into the mental states of operators in complex system environments such as control rooms. Previous research has shown that mental states such as workload and alertness of operators have an influence on performance. And that these states can lead to human errors with all the consequences. At the moment, however, no sensors are being used that measure human functioning in interaction with high-tech systems. A light will illuminate if the system is not functioning. But no light goes on if the person or the team does not function properly and threatens to fail. For example, an operator is overloaded, and bored and as a result may be less alert or concentrated.

Read more
A promising innovation is to base the workload and alertness of a team of operators on brain activity, measured with a wearable Brain-Computer Interface (BCI). Another application concerns (online) learning situations in which the BCIs are used to indicate the mental state of students in order to improve education.
The aim of the ALERT project is to design a plug & play BCI system that can measure the workload and alertness of multiple operators (use case: Control rooms) and multiple students (use case: Learning environment) simultaneously. This system will collect neurophysiological data by means of sensors, and process this data using smart algorithms to determine characteristics of workload/alertness. These characteristics can be selected for each specific application.
The consortium consists of five partners: Thales Nederland, ANT International, Artinis Medical Systems, Noldus Information Technology and the University of Twente.
The experiments for this project are being conducted in the BCI testbed within the BMS Lab of the University of Twente.

Partners

In the Efro Alert project, we worked together with:

BCI Testbed

Work has started on realizing the potential of the BCI testbed. The research will consist of two main stages, namely, the sensing and utilization of brain signals and the development of applications with practical value. Researchers working for both the partners and the University of Twente are working together to tackle the challenges they face within the various underlying projects. The further development of the BCI testbed and its applications has become one of the long-term knowledge development projects within the scope of the BMS Lab and will feature in future projects.

The research project at the BMS Lab is defined by a focus on adaptability to the human state of mind. Three use cases serve as the basis for the research being conducted at the BCI testbed. All three use cases have a common theme based on the feedback from sensors, like those measuring brain activity (EEG), heart rate, or stress, to adapt and change what machines are showing or how processes are being run.

Stages within the project

The following three use cases form the basis of the BCI testbed:

1. Adapting To User Stress
Stress is a major factor in the workplace, especially in places where rapid action or decision-making is key, for example, a command centre of a ship. The first use case centres around identifying how people are coping with the workload and the associated stress and when it becomes overwhelming. The aim is to find a way to adapt the workload, task division, or presentation to the individual. An example could be to lighten the workload of radar operators when the system notices that they cannot keep up.
2. Adaptive Screen Technology
The statics, impersonal messages aimed at the masses that screens present daily present to us bring forward challenges as the needs and preferences of audiences differ. The BCI testbed focuses on methods to adapt screens automatically to the user based on their experiences and workload. The system could remove or add information based on the availability or scarcity of mental resources, for example, break room screens adapting to the individual to show less scheduling and work-related information to someone that is experiencing a lot of stress. Thus, the screens and the information they send become more relevant and in sync with their environment and audience.
3. Team And Machine Interaction
The final use case within the BCI testbed concerns the interaction between both team members and others and between team members and machines. One goal is to develop visualizations that illustrate team interactions and teamwork, which allows for more effective management based on team effectiveness. Amongst the factors included in these studies are trust in machines, trust in the team members and teamwork. The possible end result is an application that provides immediately applicable advice for teamwork based upon these measurements.

What Does the BMS Lab Offer To The BCI Testbed?

Equipment and software to measure and utilise brain and physical signals
Support in the use of the equipment
Peaceful lab space to execute the studies
Tools to analyse and process data and images (video)
Programming support (reserve far ahead)
Facility for R&D by external parties
Test subject pool

BCI Testbed in the media

The BCI testbed has gathered attention in the media due to its versatile uses and implications. From radio interviews to numerous news articles all inspired by the great potential for generating new knowledge as well as new findings and their connotations. Furthermore, additional articles have been written about the functionalities and benefits and promising projects of the BCI testbed, such as the ones by Noldus, HBA Lab and Artinis. Most significantly, master’s student Interaction Technology, Max Slutter’s project has been further covered UToday and Ad.nl. Additionally, the project was covered in Radio 538 interview and in UT’s newspaper.

PARTNERS

In the BCI Testbed project, we worked together with the following:

Publications

Apart from the media, our projects have inspired a broad body of research within the BMS Lab. Examples of the research conducted and published so far include:

Dolmans, T. C., Poel, M., van’t Klooster, J.W.J.R., & Veldkamp, B.P. (2021). Perceived Mental Workload Classification Using Intermediate Fusion Multimodal Deep Learning. Frontiers in Human Neuroscience. doi:10.3389/fnhum.2020.609096
Slutter MWJ, Thammasan N and Poel M (2021) Exploring the Brain Activity Related to Missing Penalty Kicks: An fNIRS Study. Front. Comput. Sci. 3:661466. doi: 10.3389/fcomp.2021.661466
Gouweleeuw, K. (2021). Using Neurophysiological Signals to Measure Social Exclusion Induced by a Language Barrier. (M.Sc). University of Twente, Enschede.
Groothaar, L. (2021) The personalized audio tour. (B.Sc.). University of Twente, Enschede.
Mul, Marissa (2021) Enhancing museum experience through augmented reality interaction. (B.SC.). University of Twente, Enschede.
Luiten, Simone (2021) Improving the experience and engagement of museum visitors by means of EEG and interactive screens. (B.Sc.). University of Twente, Enschede.
Slutters, M. (2020). Exploring the brain activity related to missing penalty kicks: a fNIRS study. (M.Sc.). University of Twente, Enschede.
Waardenburg, F.H. (2021) Mirror therapy in Virtual Reality by a brain-computer interface for amputees experiencing phantom limb pain. (B.Sc.). University of Twente, Enschede.
de With, L. (2020). Using Functional Near-Infrared Spectroscopy to Meausre a Fear of Heights Response to a Virtual Reality Environment. (M.Sc.). University of Twente, Enschede.