Digital testing, how’s it done?

===== June 2017. Pilot period has been concluded. We have concluded NOT to go forward with the solution as described below. We are steering our efforts in a different direction. See Digital testing by using Chromebooks. =====

Herbert Wormeester is associate professor in the Physics of Interface and Nanomaterials workgroup. In addition to his academic work, he is programme director for the Bachelor in Advanced Technology. Herbert and Jeroen Verschuur are the joint coordinators for the module 5 'Signals Models and Systems' in Advanced Technology and Applied Physics. This module made use of a new digital test method.

Individually and under time constraints

These days, we expect different skills from students than we did 30 years ago. Curricula have changed significantly and throughout education the computer has become an indispensable tool. Not only as a word processor, but chiefly to work out problems and make and optimise designs. ‘A project is not always suitable for assessing these skills,’ said Herbert. ‘Students should be able to carry out part of this work independently and under time constraints. A project does not offer those restrictions.’ Choosing a digital test was obvious.

Not collaborative

Students are accustomed to working with the many opportunities offered by the digital world. E-mails, Dropbox, WhatsApp… With an individual test this is obviously not the intention. That is the reason why the department of Advanced Technology (AT) and Applied Physics (TN) asked two students, Martijn Meefout and Riccardo Sneep, to further develop Jeroen Verschuur's solution. A laptop that can run under a controlled operating environment by means of a USB stick. This operating system can predetermine what software can be used, and which communication channels are open. A wireless mouse must continue to work, but chatting via bluetooth with other students, cannot. Also, access to its own hard drive and all non-approved internet sources needs to be denied. The students took on the job with their own company, UTENSO.

Own laptops

The beauty of this solution was that the students could use their own laptops. These laptops were started with a distributed boot, which limited their functionality. ‘There were only a few programmes available, which were necessary for the test, such as MatLab and Libre Office Writer,’ explains Herbert. ‘The Internet connection was controlled, so that it was possible to deliver the results via the Internet, while making all other channels of communication inaccessible.’ The test worked on all the laptops from the UT notebook project, which many students have. It didn't however work on Apple computers. ‘It is therefore important to have some laptops in reserve during the test.’

Trying out the test method

To avoid problems and awkward situations during the test itself, the faculty organised a mock test two weeks in advance. Students could come and try out the test settings with their own laptop. ‘But in reality it's quite a different story. You can have everything technically perfect, but it also requires some getting used to, by the students. They need to work in a slightly different way than they're used to.’ Matlab worked a little differently, LibreOffice Writer was used instead of Word because cutting and pasting pictures was slightly different. ‘Those who were headstrong and didn't follow the instructions exactly, ran into problems.’ According to Herbert, it is important to be understanding when there are technical problems. A number of students had certain problems during the test, which were taken into account during the assessment. ‘These were very diverse. For some students, Matlab was less stable and one student had the USB stick knocked out of the laptop. You obviously can't foresee everything in advance, no matter how well you test the system. Familiarisation was clearly better on the second attempt and only one student had a problem.’

Initial findings

The first practical application has shown that the system works well. Future improvements to the test will focus on better instructions for the participants, so that files are better protected in case of a computer crash. The biggest challenge is the human aspect: it shouldn’t matter to a student if the test is taken through their own notebook software, or the test environment which is offered by the digital test.