Let’s Talk About Impact is a series that explores various aspects of making impact. We strive to make a change for the better, to make an impact, but what lies beyond this buzzword? Today we will talk about capturing and evaluating impact. Of course, we cannot talk about 0.3 or 0.7 impact (or 2.3, or 100), but then how can we “measure” impact?
Let’s get this out of the way: the impact indicator does not exist. There are multiple reasons to make this bold claim: talking about impact varies per discipline, is time-bound, there can be unexpected impacts, and impact is such a broad term that one indicator would not cover that breadth.
Indicators
A key word when talking about measuring impact is “context-sensitive”. A one-size-fits-all measurement does not exist, but indicators may work within a shared discipline, or a shared type of project. Impact indicators should be aligned with the goal of a project. For instance: a health intervention may use “Quality-adjusted life year (QALY)” as an indicator for impact, whereas an educational innovation’s impact may lie in the number of schools that have adopted this innovation and used it for 5 years. These indicators both go beyond just outcome, so how many people used something, how many citations, how much money, but really try to capture the change that took place. To capture the impact.
The paragraph above presents a story about societal impact, but what about academic impact? Citations are limited, as they don’t capture if your work made a lasting change. They give an insight in the reach of your work, but not the significance. Efforts to include this relevance and field-specificness have led to new indicators like the Field-Weighted Citation Index (FWCI). The FWCI tries to place a citation in context, by showing its relevance in the field. This is a step in the right direction, but is weak on representing the thought of lasting change. Also when talking about academic impact one can reflect on the sustained effect of their work. Has your theory or model been adopted by relevant peers? Do they keep using it? Did you change a certain standard?
Narrative
One can notice how the impact measurement discussion is moving away from rigid indicators (e.g., the h-index is not preferred in research evaluations anymore), and moving towards a more narrative-based approach. You may have heard of a “narrative CV”, or of an Impact Case. At BMS there is increased attention to the latter. An impact case tries to capture the impact of your work, project, research line, in a well-underpinned narrative. As such, you can provide context while also providing pieces of evidence. The latter is important, a narrative still needs to be well-underpinned and explicit.
At BMS we offer Impact Case workshops. You will walk out the door with your draft impact case in your hand! New workshop moments are planned in quartile 3 on:
- Wednesday 12 March 14:00-16:30
- Thursday 20 March 9:30-12:00
This shift from only metrics to narrative is also seen in research evaluations. Curious how we approached this at BMS? Have a look.
Colophon
This series is composed by BMS Research Support. While most content applies to a wider array of disciplines, some may resonate more with a BMS audience. If you work at BMS and have any impact-related questions, reach out to Tom Boogerd. If you work at another faculty, you can still reach out and we can find a colleague of your faculty who can help.
This news item is also available as a page on our website.