Research evaluation occurs regularly. Governmental programs have ex-ante, midterm and final evaluations. Institutes, centres and people are evaluated. University research groups are assessed.
Research evaluation is important in terms of accountability, quality control, strategic decision making, and for identifying strengths, weaknesses, opportunities and unintended consequences of research. Research evaluation has also become important in assessing the appropriateness of research policy measures, and for identifying systems failures.
The challenge to professional evaluators, and to government and other officials responsible for research evaluation, is to design and conduct tailor-made evaluation exercises that serve policy needs and strategic aims. The course builds on the methods and experiences in various research evaluation practices to show that research evaluation can be done systematically. It offers a catalogue of research evaluation methods, techniques and rationales, as a firm base for designing, assessing and envisioning its multiple uses in policymaking.
The course includes lectures by experts in research evaluation, study of case material
in small groups and international exchange of experiences. The aim is to provide:
- introduction and overview of research evaluation frameworks, models and approaches;
- in-depth discussion of key issues and methods;
- systematic approaches to the design of research evaluation;
- insight in the use of research evaluation as a strategic policy tool.
WHO SHOULD ATTEND?
- Government officials who commission research evaluation studies, or are required to interpret the results of such studies.
- Actual or potential research evaluation professionals who want to improve their understanding and skills.
The course language is English.
Please bring your own laptop.
Course material includes the lecture notes, background publications and the book Learning from science and technology policy evaluation. Cases will be studied with the help of selected original data and documents. Data and documents from actual evaluation studies will be used to work in groups on design and methods of research evaluation
The international team of teachers provides access to state-of-the-art research evaluation in Britain, United States, Scandinavia, Germany, the Netherlands, and of European programs.
- Design of research evaluation, selection of methods, in relation to the policy problem and general context.
- Output identification and measurement: methods and problems. Scientific output and technological effects, economic and non-economic effects.
- Policy contexts of evaluation. Implementation of evaluation results and policy use of evaluation.
- Experiences with evaluation. Evaluation of universities; evaluation of institutes; evaluation of research programs.
- Development of research evaluation and corresponding changes in national and international research systems.
Data and documents from actual evaluation studies will be used to work in small groups on design and methods of research evaluation.
The groups will apply what they have learned in the lecture sessions, and will report their results, and receive comments from the lecturers.
The aim is to provide hands-on experience: this is training in methods, and a way of letting participants recognise the difficulties and pitfalls of research evaluation and stimulate them to reflect on it.
- Evolution of research evaluation
- Overview of evaluation methods and rationales
- Design of research evaluation, selection of methods in relation to the policy problem
and general context
- Exercise I on evaluation design: logic chart
- Output identification and measurement: methods and problems
- Scientific output and technological effects, economic and social impacts
- University rankings and other performance metrics
- Exercise II on evaluation of institutes and university research
- Evaluation of policy tools and research funders
- Ex-ante evaluation
- Evaluation of economic impacts of research and innovation
- Policy uses of evaluation
- Exercise III on communicating evaluation results and policymaking
- Course dinner
- Exercise IV on sharing personal experience
- Future of research evaluation
Informal interaction with the faculty and among participants with different backgrounds during coffee breaks, meals, and other opportunities, provides an important learning experience.