Remindo

Open Remindo (utwente.remindotoetsmanager.nl) and click on 'Questions', you will see a page with a part of the item bank. The red numbers in the screenshot of this page are explained below:

1. Questions is the button to open the item bank
2. Structure of item bank: For information about the structure of the item bank, see the Question bank structure above. Further data about the question is added through categories and properties. See the Using 'Question properties' section below.
3. Questions: Each question is shown in a two-line display that contains all the key details at a glance (ID, code, status, etc.). Please note: the question code is displayed in various reports (see for example our instructions for analysis and reporting), so we recommend entering a recognizable code for each question.
4. Question details: The bottom half of the screen contains all the question details and buttons / tabs where you can edit the question and meta data.
5. Correct answer: The correct answer is highlighted in green. The fact that the answers are not in alphabetical order indicates that this question will be randomized in the exam.
6. Add new questions: To add a new question, click Add question.

To add a new question, click on 'Add question'. You will see the following pop up screen (the red numbers are explained below):

1. Type of question: choose the question type. For more information about different question types in Remindo, see our practice test.

2. Code: enter a recognizable code for each question. The question code is displayed in various reports.

3. Available for use in paper exams: There is a 'print & scan' function in Remindo. When a question is available for use in paper exams, the question can also been used in the 'print & scan' functionality.

4. Available for practices and graded practices (formative)

5. Available for exams (summative)

6. Language: Select the language of the question

7. Calculator available: It is possible to select per question the availablity of a calculator (simple or scientific calculator)

Click on 'add'. In the following screen the question can be entered and the options can be set (the red numbers are explained below):

8. Add element: It is possible to add specific elements to the question.

For example: Add the element 'Case text block', to (re-)use or create a case in the question. Once a case-text is created in Remindo, it is possible to use these case in several questions.

Or compose a combined question, by adding another question type.

9. Edit question options: On the right side of the screen, you can edit the options.

By thicking the pencil, you can edit the question text or text in answer options.

10. Options: In the top section you can indicate whether the question has only one correct answer or multiple. You can also choose whether to shuffle (randomize) the answers. They will then be shown in a different order for each student.

11.  Scoring type: specify the scoring of the question. There are several types of scoring possible. For example score mapping, add different points (or negative points) per answer alternative.

For open questions an answer model (correction model) can be used. It is possible to specify the number of points per correction criteria. 

12.  Answer options: The correct answer alternative is checked here. If you enter 'fixed' for an individual answer, only that one will be fixed in that order while the rest is randomized.

Feedback can be entered for each answer option or for correct/incorrect answers. This is generally used for formative (practice) tests.

When the quesion is entered and the options are set, click on 'Save question'.

In Remindo, students can be asked to enter their answers into a worksheet. This looks like a basic Excel sheet. Useful features:

• The lecturer can prefill some cells, and these cannot be edited by the students
• The range of cells where student input is expected is coloured yellow
• For each cell, you can indicate the expected value
• It is also possible to add more advanced options, such as searching for a particular answer anywhere in the row/column

In the screenshot below, you will see a basic worksheet question:

1. In the question header you can include instructions, images etc.
2. You define the table with the number of rows/columns, and fill out the headers (and any other cells you need). Only the yellow cells are open to student input.
3. In the answer model, answers are defined for cells B2 - C3.
4. In cell B2, students have to fill out '43' as the answer. If you want to allow more answer options, you can use the button 'Add Cell value' to specify additional options. The answer can also be text.

It is possible to use case text in a question. Once a case text is created in Remindo, it is possible to use this case in several questions.

To add a case text to a question (edit an existing question or add a new question):

1. Add the element 'Case text block' to the question.

2a. To create a new case text in Remindo, click on Create new case text. Name the case text and click on save. You can now write (or paste) the case text. Once a case text is created in Remindo, it is possible to use this case in several questions (see 2b).

2b. To re-use a case text, click on the plus sign at the end of the row (Pick case text). The case text is now added to the question. If you want to preview the case text, select the specific case.

When you open a test matrix ('recipe' in Remindo) as created by the exam office, you will see a screen that looks something like this:

1. Tests: You open a recipe from the Tests menu;
2. Course: Choose the appropriate course;
3. Name of recipe: The exam office always includes the exam date in the recipe name, as well as the course name and additional details;
4. 'Rule': Each recipe contains a rule with only a placeholder question;
5. Add questions: To start adding questions, click Add selection.
6. Selects questions: All the questions which are added to the rule are selected. You can use this feature to randomize exams (for more information: 'Adding questions to a recipe - Randomization').
7. Randomization per rule - Pin or Unpin the rule: When more than one rule is used they can be randomized by 'pinning' or 'unipinning'. When a rule is 'pinned' (in the screenshot) the rules will be assigned to students in the given order (for more information: 'Adding questions to a recipe - Randomization').
8. Randomisation within or across a rule: The question order can be randomized within or across the rule(s) (for more information: 'Adding questions to a recipe - Randomization').

The process for adding questions to a recipe is explained in the video below.

Don't forget to save the recipe and to click 'Approve recipe' at the top. Only approved recipes can be scheduled in Remindo. If you want to preview the recipe, please see our Previewing an exam page.

To test the exam from student perspective, please follow the steps below:

1. Go to https://utwente.remindotoets.nl, log in with SURF-conext;

2. Click on ;

3. Select the course of the exam, at the left side of the page;

4. Select the exam you want to test. Click the magnifying glass icon to go to the test recipe details;

5. On the left of the page, click on 'Test this recipe' in the menu Options;

6. Now, you can make the exam from student perspective.

In the Preview you will see additional information about the test recipe in Remindo, such as the number of points, correct answers and correction criteria. Please follow the steps below to generate a preview of the exam in Remindo:

1. Go to https://utwente.remindotoets.nl, log in with SURF-conext;

2. Click on ;

3. Select the course of the exam, at the left side of the page;

4. Select the exam you want to test. Click the magnifying glass icon to go to the test recipe details;

5. On the left of the page, click on 'Preview of this recipe in the menu Options;

6. Now, you will see the detailed information of the exam.

You can still make some changes in the exam questions until the start time of the exam. Click on  to see the changes reflected immediately. For more information about editing questions, see 'Adding questions manually'

Please note: Changes in exam questions are made in the other environment of Remindo: https://utwente.remindotoetsmanager.nl.

A technical assistant will be available to set up the chromebooks and assist with any issues. Please note however that they are not qualified or intended to perform official exam surveillance (checking attendance lists, verifying which students are allowed extra time, monitoring for fraud, etc.). During the exam, the technical assistants will come to you with any issues they may see. For example, they will ask you to double-check a student's ID if there is a suspicious second login attempt. If you cannot be present yourself, please arrange for surveillance through your programme staff.  

Please point students to the student page of this instruction site and/or enter instructions into your Canvas course.  

A corrector can see the assigned correction work in Remindo Toets (log in by SURF-conext).

Follow the steps below to check open questions in Remindo. Once all the open questions are checked, the grades will be calculated automatically:

1. Click in the top right corner on Results - Check answers.

2. Select to check for a Test moment. 

3. Select the right test moment. Use the filter option to select an specific test moment.

4. To the right of the (filtered) recipe, the number of students with open correction work assigned to you is shown.

5. Using these buttons, you can choose to check all questions per student , or to check answers per question  

You can use the tabs View correction work and Check answers to switch between correction methods.

6. By clicking  you will start checking the answers of the selected student or question.

7. Per question (and per student) you will see the scoring criteria at the right of the page. By using '+' and '-' buttons next to the score, you can enter the score for that specific criterium.

TIP: You can also typ in the field in the middle for a custom grade.

Once all the scoring criteria of a question have been evaluated, the total score of that question will be shown.

It is possible to change the scoring criteria, see 'Changing grading criteria' for more information.

8. At the end of grading, grades need to be submitted. Use the check boxes to select the results of the correction round and click Submit correction.

To check the open questions, it is possible to set correction criteria in Remindo. If necessary, these correction criteria or score per correction criterium can be changed after the test moment. 

Follow the steps below to change the correction criteria, after the test moment:

1. Log in on Remindo toets, use SURF-conext.

2. Go to Results - Analysis, click the tab for Test-moment.

3. Select the right Test-moment. Use the filter option to select an specific test moment.

4. On the analyse-page at the tab Analyze items, you can click the 'Interaction analyse' icon next to an specific item to open the interaction analysis of that item.

5. Click on  to add or edit a correction criterium for that specific item. The following screen will open, the red numbers are explained below.

6. To which results would you apply the changes? Remindo will standard select the test moment and test recipe, which was selected in step 3.

7. Which correction do you want to make? It is recommended to make changes in the scoring criteria at first in concept.

8. When you have chosen 'by changing the response model' (at step 7), the current scoring criteria of the item are visable. It is possible to change these current criteria or to add a new scoring criterium.

9. Do you want to re-open correciton rounds, after changing the score model?

NOTE: When adding a new correction criterium, Remindo will alway re-open the related correction rounds.

Once a student's results have been submitted, they are no longer available through Results > Check answers. However, you can still access their results through the Analysis page. Thereby, for a course coordinator it is possible to edit the correction for an individual student.

If there is a risk of cooperation between students during the exam, or if they may have access to the internet, you can enable a plagiarism check in your Remindo exam. The examinations office can tell you more about this option and activate it when the exam is scheduled.

After the exam, you will see an additional button in the Remindo grading interface:

When you open this report, you will see the Turnitin plagiarism report. The reports can also be opened from the Analysis page:

From both locations, you will be taken to the Turnitin similarity report:

The system highlights the similarities found and the source (website or other candidates). If an overlap is found with another student, the report will indicate the name of the student in question (Student s6300128 in the example above). 

Good to know:

• The option for a plagiarism check must be activated at the time the exam is scheduled. Plagiarism reports cannot be created after the fact.
• Please include a notification on the exam cover sheet that the student's work will be checked for plagiarism. In Canvas an automatic warning is built in, but this is not (yet) the case in Remindo.
• You can use 'print to PDF' to export individual reports (no bulk download).
• The reports are generated immediately when each student hands in their work. If two students are cooperating, the first student will at first have a score of 0%. You will of course be able to tell because their name will appear in the report for the second student, but it may be confusing that the score does not occur in both reports. The scores are refreshed at a later point, so this may only occur at the beginning.
• When a group of more than two students have all handed in overlapping work, the system may only show the largest overlap in the report (and not show the other student names). Please be aware of this if you come across a situation like this. We are working with the software supplier to improve this functionality.

The Turnitin website has more information about reading and interpreting the similarity reports.
More information about Turnitin can be found following the link below:

-> help.turnitin.com

If the system flags any suspicious behavior, please consult with your Board of Examiners about current procedures in your programme / faculty.

By changing an answer key (scoring model) of a question, you can edit which answer is correct or incorrect. After changing the answer key, closed questions are automatically re-scored.

If you adjust the scoring on recipe level, you will do so retroactively on all administrations of this recipe / question. It can also influence ongoing tests if you have not yet adjusted the question to the correct version in the management environment. As soon as a new version of the question is administered, this answer model adjustment will no longer be included in the current administration.

If you adjust test moment level, it only applies to that question version at that test moment. This makes the impact of the adjustment somewhat smaller.

Its advised to always only change scoring keys at Tes moment level!

1. Go to Results - analysis
2. Go to Tab "Test moment" tab
   
   
   
   
3. Select the test recipe
4. Click on Analyze items
   
   
   
5. Click on Show interaction analysis  (at the end of the row of the item)
6. Click on the Correct scoring model

1. Change response model: Remindo supports these options to make alterations to the answer keys.
2. Status: Changes can be made in concept first and approved by another person or you can make them active right away.A preview of the new results (grades) is visible at the bottom of the page in Remindo*.
3. Change the response model: When at point 1 'By changing the response model' is selected, you can change the response model. By adding new options or scoring criteria to the answer key.

* Please note: a change in the answer key can also have a negative effect on the result (grade). If you have any questions, please contact e-assessment-ces@utwente.nl

By changing an answer key (scoring model) of a question, you can edit which answer is correct or incorrect. After changing the answer key, closed questions are automatically re-scored.

If you adjust the scoring on recipe level, you will do so retroactively on all administrations of this recipe / question. It can also influence ongoing tests if you have not yet adjusted the question to the correct version in the management environment. As soon as a new version of the question is administered, this answer model adjustment will no longer be included in the current administration.

If you adjust test moment level, it only applies to that question version at that test moment. This makes the impact of the adjustment somewhat smaller.

Its advised to always only change scoring keys at Tes moment level!

1. Go to Results - analysis
2. Go to Tab "Test moment" tab
   
   
   
   
3. Select the test recipe
4. Click on Analyze items
   
   
   
5. Click on Show interaction analysis  (at the end of the row of the item)
6. Click on the Correct scoring model

1. Change response model: Remindo supports these options to make alterations to the answer keys.
2. Status: Changes can be made in concept first and approved by another person or you can make them active right away.A preview of the new results (grades) is visible at the bottom of the page in Remindo*.
3. Change the response model: When at point 1 'By changing the response model' is selected, you can change the response model. By adding new options or scoring criteria to the answer key.

* Please note: a change in the answer key can also have a negative effect on the result (grade). If you have any questions, please contact e-assessment-ces@utwente.nl