The specification matrix, or 'test blueprint'

(By: Hilde ter Horst (Zoëzi) and Riet Martens)

Tests can have a range of different functions. Especially where they influence major decisions that affect a student (pass or fail), tests must meet certain quality requirements. For instance, it is important that tests measure what they are intended to measure (the learning objective), and that the aspects addressed are representative of the subject (validity). In reality, it is never possible to test every aspect of the subject matter. The point is that you should prepare a representative selection, one that tests all the learning objectives, with the key learning objectives having a greater weight in the final assessment.

A matrix of test specifications (also known as a test blueprint) is a very useful tool for composing representative tests. You can use this test blueprint to indicate how many test questions are to be allocated to subject matters and how many to skills. With closed-ended questions in particular, it is difficult to formulate questions that test more than just the ability to recall knowledge. A specification matrix can also help you to include questions at the level of insight and application. It is also a tool that you can use to refine the validity of your test.

A specification matrix can also be used to compose multiple tests, with comparable content, on the same subject matter.

Benefits of a test blueprint or specification matrix

It takes time to create a test blueprint, so in the case of simple tests you should consider whether this outweighs the benefits. However, in important tests (pass or fail), it is certainly worthwhile investing the time to achieve the best quality possible. Benefits of preparing a test blueprint are:

-

it helps ensure that you do not include too many questions or assignments that focus on the same subject matter or the same skills;

-

it makes it more likely that the test will reflect a representative sample of the learning objectives to be assessed;

-

when preparing two tests on the same subject matter, it helps improve their mutual equivalence;

-

it can be used to justify the contents of a test to others, such as colleagues, review committees etc.

How to create a test blueprint?

As the name suggests, a test blueprint acts as a blueprint for a test. You can see at a glance which subject matter has to be addressed by the test, and which skills the students will need to have mastered (what behaviour they will have to demonstrate). The table below gives a general impression of what a test blueprint looks like.

Learning objective

Subject matter

Behaviour

Total number of questions per learning objective

Recall

Production

1

A

 

 

4

2

B

 

 

3

3

C

 

 

4

4

D

 

 

1

 

Total

4

8

A total of 12 questions or a multiple thereof

The blueprint shows that the test should include four times as many questions about subject matter A than about subject matter D. This 4:1 ratio reflects a that one component is more important than the other. Exactly which skills need to be included in the test blueprint depends on the learning objectives involved. The example uses the behavioural aspects of recall and production, a distinction that is commonly encountered in professional practice. Where production is required, students must be able to refine, process, account for, summarize, explain or assess the information provided.

Specification matrix on Blackboard (not accessible to students)

Tip: Post the specification matrix on the Blackboard page of your course. In that way, you can use it again next year to compile a similarly representative test. You can do this in such a way that it cannot be accessed by students:

Step 1: Creating a group:

Go to your course on Blackboard and click “CONTROL PANEL”; under the COURSE TOOLS block click “Advanced Group Tool” and then “Create groups”; under option 2 “Group Options”, select “Group File Exchange Available”; click the SUBMIT button in the bottom right area of the screen.

Step 2: Assign rights:

Click the “Advanced Group Tool” and then “Manually Enroll People”; on the right side of the screen click the names of the lecturers who are to get rights to the new group; in the middle block, under Actions, use “Add selected users to group” to set up the group that you have just created and click the ADD button; click the SUBMIT button in the bottom right area of the screen.

Step 3: Switch the “Groups” button on (only if it had not already been switched on):

The group has been created and the rights assigned, now the “Groups” button must be turned on to enable all the instructors to quickly access the File Exchange, to post or to review documents. Switch the button on as follows: Go to “CONTROL PANEL” and then, under the COURSE OPTIONS block, click “Manage Course Menu”; behind the button labelled “Groups” click the “MODIFY” button and make sure the “Available for Student /Participant users” option has been selected. Click the SUBMIT button in the bottom right area of the screen.

Next, the “Groups” button appears in the left-hand menu of the box and, by clicking File Exchange, lecturers can drop documents into their own workplace where they are not accessible for students.

Sources:

http://toetswijzer.kennisnet.nl/html/toetsconstructie/toetsmatrijs.htm

## go to summary of job aids