UTFacultiesBMSDept TPSSTEPSEventsSTEPS Research Evaluation Course

STEPS Research Evaluation Course 26 - 27- 28 September 2022

RESEARCH EVALUATION 26 - 27 - 28 september 2022
3 HALF DAY ONLINE COURSES
09:00 - 13:00 CEST TIMEZONE

3 consecutive half day online courses on new developments in research evaluations in context for research evaluatiors and government officials on 26 - 27 - 28 September 2022

please register by 12 September 2022
More details & registration

THE TOPIC

Research evaluations often take a standardized approach. With research policies linked to societal challenges and economic goals, new approaches in research evaluation have developed.
How can they help you in improving your research policy and make it more evidence-based?
Think of evaluations of policies that fund research to strengthen regional economic development.
Or peer reviews of international research programs in medical science. How to evaluate a research institute for transitions in agriculture? In such evaluations, questions of research quality and research management are connected to issues of impacts, effectiveness, policy logic, implementation and policy legitimacy.

THE OBJECTIVE

Explore the new developments in research evaluation beyond the standardized forms.
Using real-life cases, we bring together experts who work at the frontier of research policy, management, and evaluation theory & practice.

Course 1 - September 26: Designing research evaluations for policy-making
In course 1 we start off with a case related to a program for international collaboration in medical science. How to develop an evaluation that really is useful for the future of the program? We discuss the use of logic charts and program logics to connect policy objectives, research activities and outcomes - and discuss the limits of the approach

In the second part of course 1, we move to situations in which research is embedded in a mix of policies and the relations between policies, research efforts, and impacts are not straightforward. In such cases, the design of the evaluation often is part of the evaluation itself and a first contribution to the policy process. We also introduce the idea of ex ante evaluations, at the start of programs, that help to develop better policies and facilitate the policy use of future research evaluations.

Course 2 - September 27: New evaluation methods for research organizations and programs
Outcomes of research evaluations are increasingly used as evidence for policy makers and managements. Because of this use, there is a clear need for appropriate and responsible research evaluation methods. In Course 2 we explore the impact of new research policies focusing on transformation and how they might affect the more regular methods of research evaluation including bibliometrics and peer review.

In three lectures, experts show the limits of the mainstream evaluation methods, and discuss ways forward. While policy makers and scientists are still adapting to impact, the concept of impact is changing within the context of SDG’s and related transformation policies. This has profound implications for evaluating science and raises the question how we can track the contribution of research to transformation. Bibliometrics have a key role in research evaluation. Yet they are criticized for their effect on research management and research careers. Can they be useful in more advanced research evaluations? - Methods of peer review are well known in the evaluation of science. But how to manage peer panels for policy-oriented research evaluations?

Course 3 - September 28: Evaluating research for system interventions and social transformation
In Course 3 we discuss the evaluation of institutes, programs, policies with a mission to innovate systems. Engineering universities, agricultural research institutes, programs for public health: their aim is to strengthen regional innovation systems, push sustainable transitions in agriculture, and contribute to meeting SDGs locally, nationally and globally. How can we evaluate the impact of research on such overarching system goals.

In three lectures experts in research management and system evaluations present and discuss approaches for evaluating system effects of research programs and organizations, and their use for policy making. Can institute evaluations help to strengthen research impacts on sustainable transitions? How to capture economic impacts of research at regional level? How to asses the impact of research funding organizations on the research system? Questions beyond the usual forms of research evaluations on which our experts like to share their answers with you.

COURSE FACULTY

Erik Arnold, Technopolis Group & Royal Institute of technology (KTH)

Kate Barker, University of Manchester

Thed van Leeuwen, Leiden University

Mireille Matt, INRA National Institute of Agronomic Research

Philip Shapira, University of Manchester & Georgia Tech

Esther Turnhout, University of Twente

Barend van der Meulen & Gonzalo Ordóñez-Matamoros, course-directors, University of Twente