toolbox
Constructing questions
Starting point: test specification table
When you have decided to use a written test as an assessment method, your first step will be to construct a blueprint for your test. This can be done by creating a so-called "test specification table". With this, you can ensure that your learning objectives are covered and the questions are on the right level (of Bloom). It helps to construct a test that focuses on the key areas and weights different areas based on their importance. The table helps you make a resit test or a test next year similar to your original test. Under Design, you can find more explanation and a format.
Decide on question type
A decision that is also involved in constructing your specification table is the decision about question type. As an examiner, you have to choose the appropriate question type for what you want to assess. Validity is the main issue here. Do you really test what you want to test (your learning objectives)? Will the score and grade accurately indicate the extent to which the student has the intended level of knowledge and skills?
Efficiency may play a role in the choice of question format. Consideration can be given to what is the best possible choice given the circumstances. A combination of question types in one test is certainly possible. So you can offset (dis)advantages of one question form with another question form.
A distinction in types of questions that is often made:
Closed questions, where the answer options are provided, such as
- Multiple choice questions. A question (or partial statement) and multiple answer options are given. One answer is the correct answer or best answer.
- True/False questions. A question is followed by true/false answer options.
- Completion question. A statement or paragraph is given with one or more blank parts, which should be filled in (chosen) by the students.
- Matching questions. For instance, two rows of items are provided, and the student has to indicate how the stems or problems on one side are related to answers on the other side. They have to find relationships.
Open questions, for which students have to come up with their answers, such as:
- Completion of fill-in question. A statement or paragraph is given with one or more blank parts, which should be filled in by the students.
- Short answer questions. A prompt is given and it demands a written answer that varies in length but will be brief (just one word, a number, a short list, a few sentences).
- Essay questions. A prompt is given that requires an elaborate answer. Sometimes the question can be complex and might contain several sub-questions. The response asked for can be: extended (free writing) or restricted (requirements are given, such as: compare in your answer X and Y).
Within these main types, there are other subtypes and variations. For instance, one can use a case as the basis for the questions or provide data to which operations should be applied.
The different types of exam questions have their own advantages and disadvantages. Open questions require more attention and time for reviewing compared to closed questions, but creating good-quality MC questions is quite hard and takes a lot of time beforehand.
The following sites give a nice overview about the mentioned question types: Exam Questions: Types, Characteristics, and Suggestions | Centre for Teaching Excellence | University of Waterloo (uwaterloo.ca) and Advantages and Disadvantages of Different Types of Test Questions | UAMS Educational Development
Nb. People often think that closed questions can only be used for lower-order cognitive skills. To check if students can recall some factual knowledge, for instance. Closed questions can be used for higher-order thinking (application, analysis, evaluation) too. Although it is not easy to come up with good questions. This handout shows some examples. Under 'Tips & resources," further below, more resources for formulating higher-order closed questions are provided.
In "Assemble the test" we will provide more information about considerations when putting the test together. For instance, we address how many questions are needed to make a test reliable and how many questions can be put in a test when limited time is available.