6. Online pedagogy | 4. Implement and evaluate | Technology Enhanced Learning & Teaching

After the design and development phase you can put the design into practice. During the implementation phase you gather data regarding the quality of the course/meeting, evaluate the students' reactions (was it a success?) and evaluate the students' learning.

A distinction is made between monitoring during the implementation and evaluating after the implementation. The monitoring phase is formative, this means that it gives you the opportunity to make improvements during the implementation. Evaluation is summative, this means you can draw hard conclusions regarding the quality of the course.

General
- Determine questions you would like to answer during (monitoring) and after (evaluating) the implementation phase:
  What elements were unclear or troublesome for students?
  What went well and what was troublesome for you?
  How did the students respond during the activities in the meetings?
  What elements went very well?
- Choose tools to monitor and evaluate the quality of the course/meeting.
- Analyze the results.
- Formulate actions for improvement.
- Analyze the test results and relate them to the intended learning outcomes.
- Formulate actions for improvement regarding the education, learning objectives and/or test(items).
Where to look for information?
The CELT toolbox includes information and tools to support you with the evaluation of your education.
If you need any help with evaluating your course, you can contact the educational specialist of your faculty.
Specific for online pedagogy
- Monitor closely. When it is one of your first designs and/or deliverances of an (online) meeting, it is important that you closely monitor the implementation phase. A lot is new for you as well as for the students, so there is a lot to learn. In case of a whole course it is advised to end each module and conference with a brief survey among the students (e.g. 2 multiple choice questions with room for comments) and a reflection by yourself.
- Determine specific questions regarding the design and implementation of the online meeting or course:
  Could I address all the questions in the chat during the live conference?
  How often did the students ask me questions during their subgroup work?
  What was the response of students to the poll questions?
  How was it for me and the students to use the whiteboard during the conference?
- Choose tools/methods to monitor and/or evaluate the quality of the course/meeting.
  Most of the tools you can use for evaluation are the same as the tools you can use to activate students in the Canvas course or online meeting, e.g. polls, Q&A, quiz, questionnaire, etc.
  You may also look into a feedback tool called Impact!, available at the UT.
  You can also record your meetings and save the chats to support your reflection on the live meetings. Another possibility is that you ask a peer to join the meeting or look at the recording and discuss the findings afterwards.