PhD Defence Aranka Bijl | Game on or game over? | Using games to assess real-world skills

Game on or game over? | Using games to assess real-world skills

Aranka Bijl is a PhD student in the department Learning, Data-Analytics, Technology. (Co)Promotors are prof.dr.ir. B.P. Veldkamp and dr. S. de Klerk from the faculty of Behavioural, Management and Social Sciences (BMS), University of Twente and dr. S. Wools MSc from Cito.

Games have become a source of everyday leisure for many people. Digital games can elicit higher-order cognitive skills through immersive and complex environments, while simultaneously generating extensive performance data. Even though games might not be able to represent all domain-specific demands, the abstraction they provide may foster transferable skills useful in vocational contexts. Educational assessment similarly relies on the abstraction of reality in assessment to enhance standardization and support objectivity and fairness of assessment results. This raises the question whether performance in such environments can be used to support meaningful inferences beyond the game context, particularly in high-stakes assessment of higher-order vocational skills. Serious games offer a promising medium to optimize both validity and reliability by combining complex, interactive environments with the collection of rich, standardized performance data. Therefore, the central question of this dissertation is: How can the design principles of serious games be leveraged for summative game-based assessment to measure higher-order vocational skills and support valid and reliable inferences about candidate ability?

The four studies in this dissertation describe a design-based research project on the development and validation of serious games for the assessment of higher-order vocational skills. The research demonstrates that game-based assessment is a promising but complex approach to measuring higher-order vocational skills. The findings show that it is possible to design environments that capture meaningful behavior, differentiate between levels of expertise, and allow for reliable assessment criteria. At the same time, there is a tension between game design and summative assessment demands which could not be fully resolved.