Introduction
Curricula have changed significantly, and specialized software is part of many courses (e.g. programming, math). A project is not suitable for assessing all technical skills. Students should be able to carry out test assignments independently and under time constraints. A project does not offer those restrictions.
Until now, the University of Twente did not have an option to use specialized software during exams. The University of Twente does not have dedicated digital assessment facilities.
However, students do carry their own laptops where this software is installed. But student laptops also contain programmes that should not be available during an exam: e-mail, Dropbox, and other communication tools.
Solution
Digital assessment
A digital test can be more effective as a solution for the situation described above. The department of Advanced Technology (AT) and Applied Physics (TN) initially developed a solution (further developed by UNTESO) which allows the student laptops to run under a controlled operating environment by means of a USB stick.
Bring your own device (BYOD)
These laptops were started with a distributed boot, which limited their functionality. Only a few programmes were available, which were necessary for the test, such as MatLab and Libre Office Writer. The Internet connection was controlled, making it possible to deliver the results via the Internet, while making all other channels of communication (local and remote) inaccessible.
Pilot Set-up
We conducted pilots with this system in the 2015 and 2016 course.
Aspect | Description |
Training student assisstants | Student assisstants were trained to assist with any technical questions and to work as an invigilator / proctor during exams |
Test session | In the test session, all students got acquinted with the system and learned how to boot the test environment |
Actual test / Resit | Students executed the exam using their own devices |
Delivery portal | A delivery portal was created to safely transfer students’ work to the lecturers |
Logistics | The test was scheduled in a large room to accommodate all the students |
Spare laptops | Spare laptops were on site for students with technical issues and MAC users |
Hack test | A penetration / hack test was done to test the security of the system |
results
The numbers of the pilot are summarized in the following figure:
The Pilot has been surveyed amongst the students. The results are summarized in the figure below:
Lessons learned
During the pilot conducted in 2016, we had the following points of concern and lessons learned.
Points of concern
- It takes time for students to get used to the Linux GUI. For example copy / pasting materials is different.
- It is important to only give students with technical issues a spare laptop. When given the choice, students preferred to use a spare laptop. This resulted in a large number of spare laptops needed (30%).
- The system does not work on MacBooks.
- Spare laptops should be available when implemented campus wide.
- Logistics around spare laptops definitely need to be resolved.
- General organization and communication should not be underestimated.
Positive points
- Over 100 students were safely assessed with this method.
- The system is technically stable (apart from Matlab-related issues, which were mainly solved in the last session with updated OpenGL drivers).
- Lecturers are very satisfied with the solution.
- We recommend using one dedicated location for digital testing with sufficient power connections (Therm).
- The USB sticks’ bar codes are scanned. Logistics around this worked fine.
- Booting the USB stick did not cause trouble with students, thanks to a dedicated instruction and test session.
- The exam submission portal works well.