Expertise development for examiners

How can (new) lecturers develop their expertise?

There are different kinds of opportunities for examiners to develop their expertise (and all others involved in assessment tasks). Below is the main development opportunity that applies to all (new) lecturers starting work at UT described. But there are also opportunities for lecturers who want to gain specific knowledge (after their UTQ), often organized by a programme or faculty. A programme can for instance offer a specific workshop in the field of assessment or promote professional learning communities related to assessment themes. During a study day, assessment topics can be given extra attention. Special occasions can be organized in which teachers' knowledge and experience gained in the context of e.g. a SUTQ trajectory or WSV projects, can be shared. Etc.  
NB. If you, as an Examination Board - in cooperation with a programme director for instance - would be inspired to organize a workshop around a certain assessment theme, CELT/TELT may be able to assist you. Just ask.   


Development of Expertise - the UTQ trajectory 

All teaching staff of the University of Twente have to acquire their University Teaching Qualification (in Dutch Basiskwalificatie Onderwijs) certificate within 3 years after their employment. A UTQ qualification is nationally acknowledged. 
From the academic year 2023/24, starting teachers at the UT will obtain their UTQ based on partial certificates. The partial certificates correspond to the five distinct competencies teachers are required to demonstrate:

Teachers demonstrate their competences for the first 4 competences, by attending the related CELT course, combined with self-study and exercises, and by demonstrating through the results of targeted assignments that they possess the knowledge, skills, and attitude required for the particular competence.
For competence 5 the teachers take part in intervision meetings with peers. The UTQ trajectory concludes with a final interview, in which an independent assessor from CELT plus the relevant programme director assess together whether someone has successfully completed the trajectory. 
More information about the UTQ process can be found [here]. 

Partial certificate for Competence 3 Assessment + BKE/UEQ

One of the competences is related to assessment. Since Sept. 2023, the partial certificate is also designated as a (in Dutch) Basis Kwalificatie Examinering (BKE)* certificate. Which we have renamed for UT purposes and to stay consistent, as a University Examination Qualification. The advantage of designating this certificate as "BKE certificate" is that this certificate is in the Netherlands increasingly recognized as a certificate for assessment competence in higher education. Another reason is that Examination Boards now may decide to appoint examiners based on this certificate, for instance, if someone does not have to follow a UTQ trajectory due to having a small or temporary appointment at the UT.  The certificate is separately registered in AFAS/HR system.  

Learning objectives and content of the course 

In the course Testing & Assessment, participants will be provided with the knowledge, insight, and skills to be able to meet the requirements for competence 3. to demonstrate.  Below is a copy of the description of Competence 3 in the UTQ manual. Words and the way it is described differ slightly from the way we described the objectives for the course, see further below, but by and large, we're definitely focused on the same objectives. 

Related to Competence 5, it is also expected that participants reflect on what was learned during the UTQ trajectory; which implies that one also reflects on what was learned from working on Competence 3. A participant will relate this to his/her own teaching situation and to future professional development in teaching.
As a teacher, one may use, now or in the future, all kinds of assessment methods, like different kinds of written tests or assignments. In the sessions, therefore, attention is paid to written tests as well as assignments. This way, teachers will be well prepared for the future. To prove one’s competence in the field of assessment in the scope of UTQ, however, a UTQ candidate may choose one type of assessment and limit oneself to providing proof of competencies related to this kind of assessment. 
During the training, participants already will work on pieces of evidence. They can get peer feedback on some pieces of evidence during the sessions. So bit by bit one can work on gathering all the needed evidence to prove your competencies.

* The Basis Kwalificatie Examinering (Basic Qualification in Examination) is a certificate of competence in constructing high-quality assessments in Higher Education as recognised by all Dutch Universities of Applied Science. Since 2013, almost all Dutch Universities of Applied Science (hbo) institutions have offered a BKE. For this and in conjunction with the SKE, requirements were drawn up on national level in 2013, which were recalibrated in 2020 (Kwaliteiten van de toetsbekwame hbo-docent. Herijking van het programma van eisen voor de basis- en seniorkwalificatie examinering zoals opgesteld door de Expertgroep bke/ske. Van schilt e.a., 2020). Universities increasingly offer or considering introducing a BKE and SKE.

Information about the course (to obtain a partial certificate)

Target group:

lecturers / (assistant) professors / others solidly involved in assessment tasks
Requirements for taking part: To provide evidence, it is important that someone is closely involved in the assessment of a course (at the UT). The assessment should also have been administered so that assessment results are available. If this is not possible but someone would like or needs to participate in a certain period, one can contact the course coordinator to see if there is a possibility to prove the required knowledge and skills in another way.

Dates:

six  times a year (see Coursefinder)

Location:

On Campus, CELT, Citadel 

Max participants:  

12

Set-up:

The course will be scheduled during a 4-week period.  After the sessions, participants will have a 3-week period to submit evidence to demonstrate one’s competence in testing and assessment. In total, the course + gathering and submitting evidence will take about 30-40 hours, depending on what materials someone already has available and present knowledge and experience.

Course activities:

Preparation: self-study, preparatory assignments. In sessions: presentations of trainers; plenary discussions; exercises in small groups; discussion of homework assignments (peer feedback); discussion of questions. During the course, one already works on gathering and constructing evidence to show one's competence. 

Certificate:

To be eligible for a certificate, one is expected to attend all meetings, and actively participate in the meetings, prepare for the sessions through self-study and assignments. To show your competence one delivers proof by way of a report with supplements, in which all the relevant issues are addressed and which will convincingly show that someone knows how to assess students in a valid, transparent, and reliable way. The evidence provided should be of sufficient quality to qualify for the certificate.  

Trainers:

CELT-trainers: Angelique Assink, Sonja Borst, Marloes Luttikhuis, Helma Vlas (course coordinator).

Materials: 

Participants will get access to a Canvas site offering all the necessary content in a modular setting.

Aim and learning objectives:


The course is about assessing student learning in higher education. We will use a step-by-step plan to present all the points for attention for constructing and grading a test/assignment in a valid, reliable, and transparent way. One will also learn to analyze assessment results in order to gain insight into the quality of the assessment as well as the education provided. 
With this course, we aim to ensure that participants will have a good basis of knowledge, insights, and skills to fulfill the role of an examiner. The course helps also to construct evidence that meets the criteria for the University Teaching Qualification (UTQ) in relation to Competence 3 - Assessment.

Based on insightful evidence, a participant will demonstrate that he/she is able to:

  • formulate clear, meaningful, active learning objectives (ABCD formula) that guide teaching and assessment for a specific course
  • to design, construct, administer, assess, analyze, and evaluate the assessment for a specific course according to the phases of  the assessment cycle, in line with the learning objectives (constructive alignment) and purpose of your assessment
  • design the assessment process in such a way that the criteria for good assessment are demonstrably met to a high degree: validity, reliability, and transparency, whereby you can expertly argue the choices made at each stage in relation to the quality criteria
  • argue how your assessment (including formative assessment) has an effect on (stimulating) student learning
  • reflect on the entire assessment process and arrive at well-founded points for improvement for the next time.

In the course some more topics will be addressed, e.g. what kind of rules and regulations have to be taken into account. We expect the participants to know about these matters, but we won’t assess these topics separately.
A participant is expected to reflect on one’s learning experiences relating to the assessment process he/she described, the T&A course attended, and the process of arriving at the evidence. This reflection will be part of Competence 5 (professionalization) for the UTQ and assessed in the assessment interview to complete your UTQ trajectory.

Topics: 

Topics that will be addressed during the course, are among others: 

  • the purposes of (formative and summative) assessment
  • assessment policy, Rules and regulations of the Examination Board and the EER
  • factors involved in choosing a suitable assessment method
  • based on the assessment cycle, points of attention for designing, constructing, administering, and evaluating an assessment in such a way that your assessment will be valid, reliable, and transparent (quality criteria for assessment)
  • Constructing an assessment scheme and assessment specification table
  • construction of a reliable answer model (written tests) and constructing criteria or a rubric for assignments
  • caesura, ways to transform scores into a grade
  • ways to analyze student results and assessment results (on item/criteria level)
  • ways to provide effective and efficient feedback
  • evaluation of one's own assessment process