See Events

PhD Defence Maaike Heitink

eliciting teachers' and students' technological competences - assessing technological skills in practice

Maaike Heitink is a PhD student in the Department of Research Methodology, Measurement and Data Analysis (OMD). Her supervisor is B.P. Veldkamp from the Faculty of Behavioural, Management and Social sciences (BMS).

Given the vast development of technological applications, education cannot ignore the use of technology in preparing students for participation in society. Technology use in education has become both a goal and a tool, implying that teachers need to be able to use technology as an effective tool in their teaching and students need to learn how to use technology appropriately in their day-to-day lives. In our current society, the use of technology for teaching and learning purposes seems to be taken for granted. However, several studies have shown that the appropriate use of technology in day-to-day practices is not obvious and that there is still much room for improvement in the technological competences of both students and teachers. Measuring or mapping students’ and teachers’ technological skills is necessary for effective education in which using technology is either a goal or a tool. However, research into the measurement of the actual technological skills of teachers and students is limited. Measuring technological skills is often done through indirect and inauthentic measures (self-report, pencil and paper tests), which measure an interpretation of skills rather than actual performance. The assessment of performance, however, depends on methods that successfully elicit the behavior that needs to be measured. Therefore, the overarching aim of this dissertation was to investigate how to elicit the actual technological skills of teachers and students for the assessment of these skills in authentic educational settings. The four studies presented in the previous chapters contributed to this overarching aim.

The first two studies presented in this dissertation were focused on ways to elicit the technological skills of teachers. These studies were based on the argument that teachers’ technological skills are displayed through their enactment of technology-infused pedagogical practices in conjunction with the professional reasoning on which their practices rely. In the first study, the following research question was answered: How do teachers reason about the use of technology in their pedagogical practice? Through video cases, 157 primary and secondary education teachers demonstrated their technology use in practice and commented on their actions. Most teachers used technology to support a learning activity. The underlying reasons that the teachers mentioned were related to making their teaching attractive, facilitating the learning process, and realizing educational goals. Most technology use was intended to strengthen either pedagogy and subject matter learning, or only pedagogy. The majority of teachers’ technology use in practice showed aspects of the knowledge transfer model of teaching and about half of the video cases showed good alignment between reasoning and practice.

The second study provided a more detailed view of how teachers reason about their use of technology in the pedagogical choices they make. The study focused on pedagogical strategies (e.g., activating learning, classroom management, dealing with diversity, fostering learning strategies) that teachers use in practice, as these reflect the teaching choices they make based upon their professional reasoning. The research question for this study was: How and why do teachers use technology to facilitate their enactment of pedagogical strategies in practice? Data from 29 video cases showed how primary teachers used technology to facilitate specific pedagogical strategies. The teachers who participated in this study showed their use of technology to support enactment of a variety of pedagogical strategies, namely, student involvement, a safe learning climate, classroom management, activation of student learning, clear instruction, adaptive teaching and for teaching learning-to-learn strategies. However, by far the most frequent technology use was for supporting pedagogical strategies to promote activation of student learning. Only a few teachers showed behavior supporting the enactment of the pedagogical strategies of supporting adaptive teaching and teaching learning-to-learn strategies, however, if they did show this behavior, they almost always used technology to facilitate these pedagogical strategies. Although many teachers reasoned about using technology to adapt their teaching to student needs, this was seldom observed in their teaching practices.

The third study examined the elicitation of students’ technological skills through a digital assessment environment in which students had to apply their technology skills in authentic tasks. An authentic digital assessment environment for eliciting students’ online information literacy skills was designed and developed. Both students’ responses to the assessment tasks and their search behaviors (use of search terms and search strategies) were investigated. This study was guided by two questions: (1) How do upper primary and lower secondary school students differ in their online information literacy skills? and (2) how do upper primary and lower secondary school students differ in their use of search terms and their online search strategies? A total of 1036 students from upper primary and lower secondary schools participated in this study. This study showed that students from different educational levels differed in their online information literacy skills and their search behaviors. Pre-vocational students were less competent with regard to skills related to online information literacy than primary or pre-university students and students from general secondary education. Most students searched for information predominantly with a whole sentence or question instead of using keywords, and often used ineffective search strategies.

Capturing teachers’ and students’ technological skills in authentic settings should eventually lead to more reliable and valid assessment of these skills as used in educational practice. The studies presented in this dissertation started from the assumptions that assessment of teachers’ and students’ technological skills should be based on practical and authentic situations and that it requires a personalized approach to assessment that emphasizes the learning process more than the eventual performance. Hence, an appropriate approach to assessing technological skills would be formative assessment. Therefore, the fourth study concerned a systematic literature review that focused on revealing the prerequisites that need to be considered when formative assessment with the potential to support learning is implemented in classroom practice. This review was guided by the following research question: What prerequisites regarding the teacher, student, assessment and context need to be considered when implementing Assessment for Learning (AfL, a formative assessment approach) in the classroom? An extensive literature search resulted in 25 relevant studies from primary and secondary education for this review. Results identified prerequisites regarding the teacher, student, assessment and context. Key conditions that were found were: teachers must be able to interpret assessment information on the spot, student engagement in the assessment process is vital; assessment should include substantial, constructive and focused feedback; and the school should have a school-wide culture that facilitates collaboration and encourages teacher autonomy. The prerequisites identified in this study are also relevant for the assessment of technological skills in classroom practice. Further research could focus on how best to formatively assess students’ technological skills and what conditions need to be in place to have students, teachers and schools engaged in the assessment. An important condition in this regard is also the need for teachers to have knowledge and skills regarding the use of hardware and software related to computer-based assessment (e.g., computer response systems for immediately collecting students’ responses to short questions).

When considering students’ and teachers’ technological skills, we should be aware of the rapidly changing nature of the digital world. Technological innovations often offer new possibilities for teaching and learning in the classroom, with the consequence that some skills disappear and new skills are required. Both teachers and students should be able to respond to these changes in society in order to adapt to and master the digital world.