Master thesis

Human Factors
Master thesis



In a recent study, we compared the relevance of different measures derived from the EEG to measure the vigilant state of individuals. With these measures, the major idea was to determine what analysis method is most effective in predicting lapses of attention, which in for example driving conditions may lead to serious accidents. We employed ERPs, Fourier analyses, and ERD/ERS. The employed research paradigm, however, may not have been the most effective. Goal of the MA-project is to develop an improved paradigm, which might simply imply that more non-target stimuli are presented, and which may enable to use the recently developed LPS method (Van der Lubbe & Utzerath, 2013).



Some websites are more easily encoded in our memories than others. Part of this variability may be due to specific features of the website, but fluctuations in the state of our brains will also affect memorability. Measures derived from the EEG, especially power in the alpha and theta band are likely predictors of what will be remembered later on, and what not. An experiment will be performed with a variation of the so-called oddball paradigm in which a standard website, target websites, and deviant websites will be presented for 1,000 ms. EEG will be measured during this oddball session. In a second test, participants will be presented with several websites, half of them presented before, half of them new, and participants have to indicate whether they saw the websites before. Acquired EEG data may reveal whether brain states predict memorability of the websites in the test session.



Minimal invasive surgery (MIS, keyhole surgery) is one of the most important developments in surgery during the past decade. In the past and present, MIS training followed the apprenticeship model, where trainees watch a hundred or so procedures, before putting their own hands on a patient. With the emerge of virtual reality training simulators this is soon going to change. This new technology is promising as it provides safe and highly repeatable training opportunities, as well as assessment of a person’s capabilities.

Minimally invasive procedures are quite demanding:

-          there is no perception of depth
-          the procedure often require counter-intuitive movements
-          haptic feedback is limited

Research has shown that doctors differ in how fast they acquire the necessary  skills, and the proficiency they reach in the long term. Figure 1 shows three example learning curves.

These differences can partly be explained by individual differences between doctors, such as amount of training and cognitive capabilities, like visual-spatial processing ability. 

The research field has long been on the quest for assessing such abilities in reliable as to predict whether someone is likely to become a good MI surgeon.

MIS simulators also have great potential during the process of training. In every simulator session an abundance of performance variables are recorded (e.g., motion efficiency, see Fig 1). It is compelling to use such data to track the training progress of individual trainees. Furthermore, using learning curves, it may be possible to predict ahead of time, what performance level a trainee is going to reach (for example, participant 06 will reach a much better level than 03). In the future it may be possible to use learning curves from simulator data to select high potentials for a training program.

In this master project you will set up a suite of simulator tasks and run a learning experiment on novice participants. Based on the results you will draw conclusions on designing simulator based trainings and simulator-based assessment. During your project you will work as a team of two students in close collaboration with Marleen Groenier from Technical Medicine.




Mind reading refers to getting a grasp on what someone is currently thinking of, without asking the person directly.

Opposed to some common belief[1], psychologists are well able to read people’s minds. The key to mind reading is to use so-called implicit techniques (Robinson & Neighbors, 2005), in contrast to the prevalent self-report methods. The variety of implicit methods falls into two classes. In free association tasks (e.g., Schmettow & Keil, 2013), the response is free form (such as telling a brief story after viewing a picture), which is then interpreted by the researcher using some detailed scoring rules. In experimental tasks, direction of thought is usually inferred from differences in response times.

A rather novel paradigm to mind reading is a variant of the well-known Stroop task. The semantic priming Stroop task implicitly assesses the strength of association between a picture and a word. Participants first view a picture, followed by a word that is written in color. As usual in the Stroop task, the participant has to respond to the color as quick as possible. When the participant has a strong association between picture and word, this leads to a distraction from the color naming task and can be measured as a delay in response time. By using words out of several categories, one can determine the broad direction of thought, the participant experienced.

To give an example: Supposed, you want to find out whether someone knows the fairytale of “Red Riding Hood”. You would prepare a set of pictures that cover the themes of the fairytale, for example showing an old lady, a wolf or a basket with food. Another set of pictures is not associated to the fairytale, serving as a control condition. In the same way two sets of target words are created. During the experiment picture-word pairs are presented in two conditions: either both are associated through the fairytale (e.g., picture of a wolf, followed by the word grandma), or they are completely unassociated (e.g., picture of car followed by grandma). When the response time for associated pairs are delayed, you would conclude that the person knows the fairytale.

The Stroop semantic priming task has been used before to assess attitudes towards computers (Schmettow, Noordzij, & Mundt, 2013; Sparrow, Liu, & Wegner, 2011). While the experiment itself is unlikely suitable as a measure of attitude, it can well be used for cross validation of questionnaires which are co commonly used in Human Factors research.

Research question

The classic Stroop task is a well-established experimental paradigm in cognitive psychology and the Stroop effect has been replicated dozens, if not hundreds, of time. In contrast, the semantic priming variant has only been used twice to our knowledge, making it susceptible. In this thesis project, the promises are assessed in one of two possible scenarios:

  1. Best case scenario: does the task produce the expected results under optimal conditions, i.e. when the expected associations are very strong, for example knowledge of fairy tales.
  2. Replication scenario: can the pioneering results of Sparrow et al., 2011 be replicated?


In your thesis project you will:

  1. Do a literature study covering experimental priming paradigms and the Stroop task
  2. Create a scenario with stimuli set (words and pictures, for example, fairytales and novels)
  3. Program the experiment (OpenSesame, PsychoPy or PyGame)
  4. Run the experiment to test your hypothesis
  5. Conclude on whether the semantic priming Stroop task works and how it can be used in Human Factors research


Robinson, M. D., & Neighbors, C. (2005). Catching the mind in action: Implicit methods in personality research and assessment. In M. Eid & E. Diener (Eds.), Handbook of multimethod measurement in psychology (Vol. 7, pp. 115–125). Washington, DC, US: APA American Psychological Association.

Schmettow, M., & Keil, J. (2013). Development of an Implicit Picture Story Exercise Measuring Personal Motives for the Interaction with Technical Products. University of Twente.

Schmettow, M., Noordzij, M. L., & Mundt, M. (2013). An implicit test of UX: Individuals Differ in What They Associate with Computers. In CHI ’13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA ’13 (pp. 2039–2048). New York, New York, USA: ACM Press.

Sparrow, B., Liu, J., & Wegner, D. M. (2011). Google effects on memory: cognitive consequences of having information at our fingertips. Science (New York, N.Y.), 333(6043), 776–8.




Cars with combined automated functions are becoming commercially available to the public. These include combinations of lateral and longitudinal car functions. At his time, drivers are still required to keep their hand on the wheel when these systems are activated. However, higher automation systems that no longer require the driver to keep their hands on the wheel are under development. It is expected that these will enter the commercial market within the next few years. In these systems, the driver has to take back control from the car if the car can no longer function automatically. This may be due to system limitations or system failures. Studies have already shown that situations in which the drivers needs to take back control can induce high workload. Some studies have taken some external elements from the driving situation into account when studying driver take-overs (take-over time and workload) such as the influence of traffic density. Gold et al. (2016) and Radlmayr et al. (2014) showed increased take-over times for highways with higher traffic density. Lyu et al. (2017) found an increase in workload for take-overs on roads with more traffic signs. However, these studies only take a part of the whole driving situation into account. Also, almost all take-over research focused on highway scenarios until now.

The goal of the current study is to evaluate the effect of both traffic- and road complexity on take-overs in a partially automated car. Furthermore, this study will include not only highway but also urban scenarios. If we can identify an influence of complexity on take-over behaviour, we may adapt the car to driver communication to the complexity of the current driving situation. For example, a longer transition period or slower communication may be required.

We are looking for a motivated master student to set-up and conduct a driving simulator study to reach our research goal. You will guide the participants through the experiment, and perform an initial analysis on the results. The project can be adjusted to fit your specific learning goals. Basic programming skills in C++ are a plus but not required as the majority of the scenarios are already programmed. This study is a great opportunity to get hands-on experience with conducting a full experiment in a driving simulator.

Are you interested in performing this experiment? Please contact



In the I-CAVE project, University of Twente (UT), Delft University of Technology, TU Eindhoven and Radboud University are collaborating on the development of a fully functional Highly Automated vehicle. The UT works on the human factors side of this project, investigating driver’s subjective feelings of safety and, in general, trust in automation.

In this study we will test whether, during specific driving scenarios, there is a mismatch between driver’s trust and the objective reliability of the I-CAVE vehicle. Importantly, our goal is NOT to achieve the highest possible level of trust in the user, but to align driver’s trust with the objective reliability of the vehicle. This is known as trust calibration and is, in the automotive domain, one of the biggest human factors challenges that still need to be properly tackled.

Our study aims to answer the following research questions: is there a mismatch between user’s trust and vehicle reliability? If yes, why? Driver’s input will be used to tweak vehicle behaviour, before testing an “improved” version of the same driving scenarios.

The study will be performed in the new driving simulator of the University of Twente. You will be asked to build driving scenarios through SILAB software and, by using wearable physiological technology and questionnaires, measure driver’s trust in the simulated self-driving vehicle. To note, developing scenarios in SILAB does not require a programming background. Yet, a basic knowledge of programming languages such as Python or Matlab is recommended.

If you want to know more about this project, please send an email to Francesco:




Conversational interfaces and agents i.e. chatbots and voice interfaces can be used to support costumer experience with services etc. ( However, it is still hard to identify the real application and usefulness of CI to support information retrieval from the end-user point of view.

In particular the use of CI is often presented as a way to enable users to interact with Natural Process languages, nevertheless, people (as adaptive agents) could quickly learn how to minimize the wording during the conversation with chatbots by acquiring a basic set of “command lines” to speed up their tasks achievement.

This exploratory work will aim to:

  1. review the state-of-art on the quality of interaction assessment with chatbot tools
  2. test and gather initial data to evaluate this emerging type of interaction
  3. Compare performance and perceived usability in using conversational agents (CI) or Point-Click and Touchable (PCT) interaction.

We will not focus on how to improve or humanise interaction with conversational agents, but we will focus on the following three research questions:

  1. Are CI tools efficient, effective, and satisfactory tools to achieve a task in a GUI compared to when people use PCT modality of interaction?
  2. How we can assess satisfaction in using CI in a comparable way of GUI interaction with PCT?
  3. Are people adapting their behaviour during the interaction with CI?

In tune with the research questions, the following three expectations can be described:

  1. If CI are enhancing the way in which we interact with GUI: CI will enable end-users to perform tasks in GUI at least the same level of efficiency, effectiveness and satisfaction than when people completed the same task with a PCT modality.
  2. Literature analysis will inform the current use of satisfaction questionnaires and their reliability for assessing CI (Voice interface and Chabot).  A quick review of literature, so far, shows that there is not a validated questionnaire like (SUS or UMUX) to rapidly assess user satisfaction. Nor adapted validated questionnaires were proposed like in the case of auditory interfaces. Therefore we will attempt to develop an initial questionnaire and to validate it. Our expectations is that the score of the adapted questionnaire to measure the satisfaction of CI interaction, will be acceptably reliable, and it will correlate with the scores of UMUX for the PCT interaction.
  3. We know for literature that people behave differently during conversation with people or CI. In tune with that we may argue that people adapt their style of conversation to the CI modality. We will control this adaptation in terms of quantity of verbalisation or text during the interaction with CI. Our expectation is that end-users will reduce the verbalisation/text quantity over time regressing toward a sort of “command line” style. This regression will be affected by people previous expertise with CI, i.e., reduction of words will be more evident in people who do not have experience with CI.


This work is a qualitative and exploratory analysis to identify approaches to measure the experience in using chatbots and to identify hypotheses for future investigation.

The final outcome will be: i)  a comparative analysis of people performance using CI tools and PCT modality to achieve tasks online; ii) insights of user behaviour during interaction with CI and, iii) an adapted questionnaire (adapted from SUS and UMUX and previous research on CI) to assess the satisfaction in the usage of CI, and the its initial validation.

Questionnaire design

Previous studies will be explored to identify key variables of interaction with the CI tools e.g.,  Key aspects for the interaction with CI will be explored, and a list of potential items will be defined. The list of Items will be reviewed by experts (interview/online survey) and consensus on the items will be used to revised the questionnaire.


  • Coperich, K., Cudney, E., & Nembhard, H. Continuous Improvement Study of Chatbot Technologies using a Human Factors Methodology.
  • Duijst, D. (2017). Can we Improve the User Experience of Chatbots with Personalisation? MSc Information Studie, Amsterdam.  
  • Følstad, A., & Brandtzæg, P. B. (2017). Chatbots and the new world of HCI. interactions, 24(4), 38-42.
  • Hill, J., Ford, W. R., & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations. Computers in Human Behavior, 49, 245-250.
  • Kuligowska, K. (2015). Commercial Chatbot: Performance Evaluation, Usability Metrics and Quality Standards of Embodied Conversational Agents. Browser Download This Paper.
  • Xuetao, M., Bouchet, F., & Sansonnet, J.-P. (2009). Impact of agent’s answers variability on its believability and human-likeness and consequent chatbot improvements. Paper presented at the Proc. of AISB.
  • Kerly, A., Hall, P., & Bull, S. (2007). Bringing chatbots into education: Towards natural language negotiation of open learner models. Knowledge-Based Systems, 20(2), 177-185.
  • Hirschberg, J., & Manning, C. D. (2015). Advances in natural language processing. Science, 349(6245), 261-266.
  • Shawar, B. A., & Atwell, E. (2007). Chatbots: are they really useful?. In LDV Forum (Vol. 22, No. 1, pp. 29-49).
  • McTear, M., Callejas, Z., & Griol, D. (2016). The conversational interface. New York: Springer, 10, 978-3.
  • Agostaro, F., Augello, A., Pilato, G., Vassallo, G., & Gaglio, S. (2005, September). A conversational agent based on a conceptual interpretation of a data driven semantic space. In AI* IA (Vol. 3673, pp. 381-392).
  • Heller, B., Proctor, M., Mah, D., Jewell, L., & Cheung, B. (2005, June). Freudbot: An investigation of chatbot technology in distance education. In EdMedia: World Conference on Educational Media and Technology (pp. 3913-3918). Association for the Advancement of Computing in Education (AACE).



Think-aloud protocols are widely used in usability and user research to explore people interaction quality with devices, software, interfaces, websites, and  documents. In classic Concurrent think aloud (CTA) approach users are asked to complete a set of tasks with the artefact tested, and to constantly verbalise their thoughts while working on the tasks. In retrospective think-aloud users are asked to verbalised after the interaction by watching a video recordings of their interaction (Video-cued RTA).

RTA method are proved significantly more fruitful in revealing problems that were not observable, but could only be detected by means of verbalisation. CTA method is a more faithful representative of a strictly task-oriented usability test, while the RTA method is likely to yield a broader gamut of user reactions.

Moreover, today is also possible to cue with eye-tracking feedbacks both CTA and RTA. Empirical comparison showed that this may potentially add value to user research.


To design an experimental protocol to compare cued thinking aloud techniques, and to perform experiments on a range of products to compare the performances of the different techniques in terms of false alarm, and the usefulness of users’ insights.


  • Eger, N., Ball, L. J., Stevens, R., & Dodd, J. (2007). Cueing retrospective verbal reports in usability testing through eye-movement replay. Paper presented at the Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 1, University of Lancaster, United Kingdom.
  • Elbabour, F., Alhadreti, O., & Mayhew, P. (2017). Eye tracking in retrospective think-aloud usability testing: is there added value? Journal of Usability Studies, 12(3), 95-110.
  • Elling, S., Lentz, L., & Jong, M. d. (2011). Retrospective think-aloud method: using eye movements as an extra cue for participants' verbalizations. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada.
  • Ericsson, K. A., & Simon, H. A. (1998). How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity, 5(3), 178-186.
  • Fonteyn, M. E., Kuipers, B., & Grobe, S. J. (1993). A description of think aloud method and protocol analysis. Qualitative health research, 3(4), 430-441.
  • Ramey, J., Boren, T., Cuddihy, E., Dumas, J., Guan, Z., Van den Haak, M. J., & De Jong, M. D. (2006). Does think aloud work?: how do we know? Paper presented at the CHI'06 Extended Abstracts on Human Factors in Computing Systems.
  • van den Haak, M., De Jong, M., & Jan Schellens, P. (2003). Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue. Behaviour & Information Technology, 22(5), 339-351. doi:10.1080/0044929031000
  • Van Someren, M., Barnard, Y., & Sandberg, J. (1994). The think aloud method: a practical approach to modelling cognitive.



Every day people use multiple technologies to perform complex tasks, such as buying products online, informing their decision making, or supporting their work activities. Several independent evidences in literature converge on the idea that multiple elements affected people expectations toward the use of a technology, including individual attitudes, skills and capabilities and technology related aspects, such as: product’s aesthetics and usability perceived before the use, fluency, brand and price etc.

In many cases (high risk) processes are dependent on the technology to deliver the appropriate service. It is perhaps reasonable to assume that the implicit agreement of this technology-driven world is that: people trust technology they are using to perform task and decision making in terms of: performance, functionalities and reliability of outcomes. Trust toward technology does not happen immediately but rather is built throughout the relationship between user and artefacts. This is a set of beliefs about a product’s characteristics – i.e., functioning, reliability, safety, etc. And it results from the gained experience of people in the use of different technologies over time. User’s overall trust is, therefore, strongly related to the concept of user experience, i.e., experience with (and the exposition to) different products enable people to develop a set of general attitudes and beliefs toward those technology, including the overall trust.


Building on literature and previous data on explicit measure of trust this exploratory work will attempt to investigate the trust before the use of medical device by using eye-tracking technologies.

Correlation between implicit and explicit measures will be investigate to model:

  • the process of trust before the use;
  • the factors which affect this process;
  • how trust may be bias by design and information presentation.


  • Borsci, S., Lawson, G., Salanitri, D., & Jha, B. (2016). When simulated environments make the difference: the effectiveness of different types of training of car service procedures. Virtual Reality, 20(2), 83-99. doi: 10.1007/s10055-016-0286-8
  • Corbitt, B. J., Thanasankit, T., & Yi, H. (2003). Trust and e-commerce: a study of consumer perceptions. Electronic Commerce Research and Applications, 2(3), 203-215. doi:
  • Fruhling, A. L., & Lee, S. M. (2006). The influence of user interface usability on rural consumers' trust of e-health services. International Journal of Electronic Healthcare, 2(4), 305-321. doi: 10.1504/ijeh.2006.010424
  • Gefen, D. (2000). E-commerce: the role of familiarity and trust. Omega, 28(6), 725-737. doi:
  • Karat, C. M., Brodie, C., Karat, J., Vergo, J., & Alpert, S. R. (2003). Personalizing the user experience on IBM Syst. J., 42(4), 686-701. doi: 10.1147/sj.424.0686
  • Lankton, N. K., & McKnight, D. H. (2011). What does it mean to trust facebook?: examining technology and interpersonal trust beliefs. SIGMIS Database, 42(2), 32-54. doi: 10.1145/1989098.1989101
  • Lawson, G., Salanitri, D., & Waterfield, B. (2016). Future directions for the development of virtual reality within an automotive manufacturer. Applied Ergonomics, 53(Part B), 323-330. doi:
  • Lippert, S. K., & Swiercz, P. M. (2005). Human resource information systems (HRIS) and technology trust. Journal of Information Science, 31(5), 340-353. doi: 10.1177/0165551505055399
  • Marie Christine, R., Olivier, D., & Benoit, A. A. (2001). The impact of interface usability on trust in Web retailers. Internet Research, 11(5), 388-398. doi: 10.1108/10662240110410165
  • Mcknight, D. H., Carter, M., Thatcher, J. B., & Clay, P. F. (2011). Trust in a specific technology: An investigation of its components and measures. ACM Trans. Manage. Inf. Syst., 2(2), 1-25. doi: 10.1145/1985347.1985353
  • Montague, E. N. H., Winchester, W. W., & Kleiner, B. M. (2010). Trust in medical technology by patients and healthcare providers in obstetric work systems. Behaviour & Information Technology, 29(5), 541-554. doi: 10.1080/01449291003752914
  • Pennington, R., Wilcox, H. D., & Grover, V. (2003). The Role of System Trust in Business-to-Consumer Transactions. Journal of Management Information Systems, 20(3), 197-226. doi: 10.1080/07421222.2003.11045777
  • Salanitri, D., Hare, C., Borsci, S., Lawson, G., Sharples, S., & Waterfield, B. (2015). Relationship Between Trust and Usability in Virtual Environments: An Ongoing Study. In M. Kurosu (Ed.), Human-Computer Interaction: Design and Evaluation: 17th International Conference, HCI International 2015, Los Angeles, CA, USA, August 2-7, 2015, Proceedings, Part I (pp. 49-59). Cham: Springer International Publishing.
  • Salanitri, D., Lawson, G., & Waterfield, B. (2016). The Relationship Between Presence and Trust in Virtual Reality. Paper presented at the Proceedings of the European Conference on Cognitive Ergonomics, Nottingham, United Kingdom.
  • Shin, D.-H. (2013). User experience in social commerce: in friends we trust. Behaviour & Information Technology, 32(1), 52-67. doi: 10.1080/0144929x.2012.692167
  • Ziefle, M., Rocker, C., & Holzinger, A. (2011, 18-22 July 2011). Medical Technology in Smart Homes: Exploring the User's Perspective on Privacy, Intimacy and Trust. Paper presented at the 2011 IEEE 35th Annual Computer Software and Applications Conference Workshops.



In order to achieve sustained adaptability in the face of anomalous situations, teams adapt their communication structures depending on the task (Barth, Schraagen, & Schmettow, 2015) or their level of experience (Schraagen & Post, 2014). We found that the naval and medical teams that we studied adopted a 'scale-free' network structure (Schraagen, 2015). A scale-free network structure is a structure in which a few nodes have many connections, whereas most nodes have few connections to other nodes (much like the hub-and-spoke structure of the U.S. airports). Scale-free network structures have been identified in numerous studies (see Caldarelli, 2007, for a review). However, it is not clear whether the results obtained with hundreds or thousands of nodes are also applicable to small teams commonly found in the military (and, hence, whether our results obtained with relatively small teams are stable and generalizable). Also, results obtained with Social Network Analysis may not be easily communicated to teams or be accepted by teams, for their lack of content.

We have therefore focused on the actual order in which team members communicate with each other rather than who is the most central team member. In a study recently carried out (David & Schraagen, in press), we focused on communication patterns that played out in a matter of a few hours during the Air France 447 incident. We used Butts’ (2008) relational event model to examine the communication dynamics amongst the pilots in the cockpit of the flight AF447, to illustrate how communication patterns may be studied at longer time scales, at a ‘transaction level’ (Schraagen, 2017). The analysis of the communication transcript revealed patterned changes in some communication dynamics in the cockpit after the flight entered an unexpected situation, such that the communication between the pilots was increasingly determined by immediately preceding local communication events, making the system less responsive to external events. 

However, our analyses so far have been based on single teams under specific conditions. We do not know to what extent the results we obtained are generalizable to other teams. 

Technical objectives: Extend results from previous studies to multiple teams. What is required in this respect is a detailed comparison of various structural communication patterns in teams to unexpected disturbances. 

Technical approach: Based on availability and high face validity of data, you will study actual team communications prior to and immediately following an anomaly. You will study recorded communications of expert team members under real-life threatening situations. Our data set will largely be based on transcriptions of Cockpit Voice Recorders (CVR) retrieved after accidents. This will primarily be from the aviation domain; however, incidents from other domains will be studied as well. We will use CVR data from at least 20 accident investigation reports to obtain sufficient generalizability. 

Butts’ (2008) relational event framework, a statistical way of analysing sequences of relational events, will be used to capture transaction patterns across different nodes and extended time scales considering past and current communications. Further, relational event sequences can be investigated quantitatively, as can the tendency of the system to encourage or discourage some of them, without the need for specific content information.

Fundamental research content: (1) do communication patterns change when a system enters an unexpected situation? (2) can patterned communication deficiencies be discerned across a wide range of teams and situations?


·         Familiarity with R, and Bayesian statistics (model selection criteria; BIC)
·         Interest in quantitative approaches to modelling team communication


Barth, S., Schraagen, J.M.C., & Schmettow, M. (2015). Network measures for characterizing team adaptation processes. Ergonomics, 58(8), 1287-1302. DOI: 10.1080/00140139.2015.1009951

Butts, C.T. (2008). A relational event framework for social action. Sociological Methodology, 38(1), 155200. doi: 10.1111/j.1467-9531.2008.00203.x

Caldarelli, G. (2007). Scale-free networks: Complex webs in nature and technology. Oxford: Oxford University Press.

David, L.Z., & Schraagen, J.M.C. (in press). Analysing communication dynamics at the transaction level: The Case of Air France Flight AF447. Cognition, Technology & Work.

Schraagen, J.M.C., & Post, W.M. (2014). Characterizing naval team readiness through social network analysis. Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting (pp. 325-329), Chicago, Il., October 27-31, 2014. Santa Monica, CA: Human Factors and Ergonomics Society.

Schraagen, J.M.C. (2015). Resilience and networks. Proceedings of the 6th Resilience Engineering Association Symposium. Lisbon (Portugal), 22 – 25 June 2015. 

Schraagen, J.M.C. (2017). Beyond Macrocognition: The Transaction Level. In J. Gore & P. Ward (Eds.), NDM13 Naturalistic Decision Making and Uncertainty, Proceedings of the 13th bi-annual international conference on Naturalistic Decision Making, Bath, 20-23 June 2017 (pp. 182-188). Bath: The University of Bath.

MHF11 - Evaluating an app to facilitate group meetings


The SpeakCount App

The Challenge

SpeakCount is an application that assists self-moderated teams to increase their productivity during meetings. The developers of this app hope that SpeakCount can facilitate Engaging, Productive, Invigorative and Creative (E.P.I.C.) meetings. But can it really work as expected? Anything worth being added up? Now it’s your challenge to test it with experiments.

Background information

The application involves the concept of participatory sense-making[8] as the teams are requested to go through a process for which “meaning is generated and transformed in the interplay between the unfolding interaction process and the individuals engaged in it”.

More specifically: SpeakCount consists of an interactive puzzle with pieces in different shapes that are projected on a surface visible to all team members and an application installed on each member’s smartphone. The application is meant to be used during a team’s self-moderated meetings and it has the ability to track the voice of the smartphone users and count the time that each of them is talking during a meeting. It also provides a button for the individuals to evaluate positively each other’s contribution during a meeting. The meeting has to be initiated through a website and all the participants should be connected to it via their smartphones’ app. Once the meeting starts, the team screen appears an empty puzzle and each user receives 5 of the puzzle pieces on their smartphone interface.

During the meeting, the algorithm calculates the current percentage of each person’s participation in the overall meeting and automatically fills in th e corresponding puzzle pieces from the user’s smartphone. At the same time, whenever two or more users tap (like) the button on their smartphones when a third person is talking, an extra puzzle piece appears to the team puzzle screen. At the end of the meeting, the result is either a complete or a partly complete puzzle and the users can interpret the result on their own.

More information:

Research Question

The main research objective is to determine the effectiveness of the app. Up to which extent does the app improve the productivity of self-moderated meetings and increase the satisfaction of the participants regarding a meeting?

Two dimensions could be explored within the scope of the research:

·         Different kinds of meetings: For which type of meetings can the application be
          beneficial (brainstorming, decision making…)?
·         Longitudinal effectiveness: What are the effects for the teams that are using often the
          app in their meetings?

Initiators of SpeakCount:

Syn(+) Ergasia Team

Athina Kapousouz                

Cynthia Chen                        

Georgios Papanikolaou        

Jessica Ren Yueying            

 Relative literature

  1. Bersin, J., McDowell, T., Rahnema, A., & Van Durme, Y. (2017). The organization of the future: Arriving now. Global human capital trends 2017: Rewriting the rules for the digital age, 19-28.
  2. Duhigg, C. (2016). What Google learned from its quest to build the perfect team. The New York Times Magazine, 26, 2016.
  3. Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010). Evidence for a collective intelligence factor in the performance of human groups. science, 330(6004), 686-688.
  4. Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative science quarterly, 44(2), 350-383.
  5. Lehmann-Willenbrock, N., Allen, J. A., & Belyeu, D. (2016). Our love/hate relationship with meetings: Relating good and bad meeting behaviors to meeting outcomes, engagement, and exhaustion. Management Research Review, 39(10), 1293-1312.
  6. Buengeler, C., Klonek, F., Lehmann-Willenbrock, N., Morency, L. P., & Poppe, R. (2017). Killer Apps: Developing Novel Applications That Enhance Team Coordination, Communication, and Effectiveness. Small group research, 48(5), 591-620.
  7. Hamari, J., Koivisto, J., & Sarsa, H. (2014, January). Does gamification work?--a literature review of empirical studies on gamification. In System Sciences (HICSS), 2014 47th Hawaii International Conference on (pp. 3025-3034). IEEE.
  8. Jaegher, Hanne De, and Ezequiel Di Paolo. (2007) "Participatory Sense-making." Phenomenology and the Cognitive Sciences , 6(4) : 485-507.
  9. Fletcher-Watson, S., De Jaegher, H., van Dijk, J., Frauenberger, C., Magnée, M., & Ye, J. (2018). Diversity computing. Interactions, 25(5), 28–33. doi:10.1145/3243461

Master thesis + internship

Adaptive cognitive performance through personalized instruction




General context

Being adaptive is critical for successful performance in the 21st century in which individuals frequently need to learn new skills to better interact with complex, changing technology-based environments. Despite the urgency to become an adaptive expert, studies on training of adaptive performance are scarce. TNO is investigates the effects of a game-based intervention that aims to train adaptation skills. The study investigates the effects of instruction upon adaptive performance. First, the effect of training in critical reflective thinking is investigated. Furthermore, it is tested whether personalizing the instruction to the needs of the individual participant increases learning effects.

Internship (10 EC)

Intern-student will be involved in the following activities:

·         participate in the organization and execution of the above experimental lab-study.
·         provide instruction and guidance  to participants of the study
·         organizing and managing the data of the study; preparing the data for statistical
·         Conducting statistical analysis to address the research questions
·         Contribute to the report of the study (in a TNO-report, or in a scientific journal paper).

Master thesis (25 EC)

This research project involves multiple studies. Currently we are designing a second experiment, to be carried out after the internship, to study adaptive cognitive performance by means of the game “Space fortress”. The plan is to conduct this study early 2019. As a master thesis student you will help set up the study, conduct a literature study, which you will write up as the theoretical framework and method section of your master’s thesis. After data collection you will analyse the data and complete your master’s thesis by writing op the results and answering the research questions. 

More information

Karel van den Bosch

Heleen Pennings



T: +31 (0)88 866 58 42

T: +31(0)888663176

M: +31(0)611289472