supervisor: pantelis papadopoulos
Topic
The popularity of Audience Response Systems (ARSs – or simply quiz tools) is based, in part, on Mazur’s seminal work on the Peer Instruction (PI) paradigm (e.g., Mazur, 1997, 2009) according to which students first provide their initial answers to multiple-choice questions (aka “voting phase”), receive collective feedback based on class responses through the ARS and peer discussion, and then answer the same questions for a second time (aka “revoting phase”) before they receive the correct answers and participate in the class discussion that follows. Peer discussion allows students to get peers’ perspectives, but it also requires a significant amount of lecture time and it can be challenging for the teacher in the case of larger audiences. Without peer discussion, however, the students in most available ARSs are left with limited feedback, usually in the form of percentages showing classroom distribution among the different question choices. This is the case of widely popular tools such as Kahoot, Socrative, and Wooclap. Consequently, students may feel encouraged to focus more on probabilistic strategies or succumb to the conformity bias by changing their initial answer to the most popular one. In both cases, the students are deprived of meaningful engagement and reflection within the quiz.
To address this information gap, studies on online assessment and group awareness have suggested including additional feedback metrics that could better describe the characteristics of the population that voted each question choice. At the same time, literature on metacognition suggests that eliciting metacognitive judgments from students may have a positive impact on their understanding and metamemory. For example, asking students to explicitly state their level of confidence while answering a quiz can affect their performance and the way they will later accept or reject collective feedback.
This topic will focus on how personal characteristics (e.g., perceived confidence and level of preparation) affect students’ activity in cases of confirmatory and opposing collective feedback. The topic was part of a WSV project funded by UT’s Teaching Academy (https://www.4tu.nl/cee/innovation/project/132/self-assessment-and-group-awareness-with-audience-response-systems).
Method
The study will include a controlled experiment in which two (or more) treatment groups will receive different types of collective feedback (confirmatory/opposing at different levels). Students’ activity in the two phases of a quiz (e.g., scores, metacognitive judgments, etc.) will be the dependent variables of the study, while the feedback condition (e.g., weakly/strongly confirmatory, weakly/strongly opposing) will be the independent variable. To be able to control and manipulate the feedback the students of each group will receive, the study will be conducted in Qualtrics (https://utwentebs.eu.qualtrics.com/).
references
Mazur, E. (1997). Peer instruction: A user's manual series in educational innovation. Upper Saddle River, NJ: Prentice Hall.
Mazur, E. (2009). Farewell, lecture? Science, 323, 50-51.
Papadopoulos, P. M., Natsis, A., Obwegeser, N., & Weinberger, A. (2019). Enriching feedback in audience response systems: Analysis and implications of objective and subjective metrics on students’ performance and attitudes. Journal of Computer Assisted Learning, 35(2), 305–316. https://doi.org/10.1111/jcal.12332
Papadopoulos, P. M., Obwegeser, N., & Weinberger, A. (2021a). Concurrent and retrospective metacognitive judgements as feedback in audience response systems: Impact on performance and self-assessment accuracy. Computers and Education Open, 2, 2021, 100046, ISSN 2666-5573, https://doi.org/10.1016/j.caeo.2021.100046
Papadopoulos, P. M., Obwegeser, N., & Weinberger, A. (2021b). Let me explain! The effects of writing and reading short justifications on students' performance, confidence and opinions in audience response systems. Journal of Computer Assisted Learning, 1-11. https://doi.org/10.1111/jcal.12608