How to measure research impact?

Researchers are likely to have to demonstrate for promotion and tenure that their work has an impact on others in their discipline as well as increasingly also on society at large.

Often quantifying methods are used to measure the success or “impact” of individual researchers. Many of the quantifying methods do not consider research data, but focus on publications (bibliometrics). However, other criteria such as the ability to secure third-party-funding are in place. Because of the strong focus of established bibliometric evaluation systems on articles and monographs, such as publication volume, publication prestige (publisher, Journal Impact Factor), citations (article level metrics), download statistics or the attention score in social media (altmetrics) in the past there are only few incentives for conducting open, reproducible research because alternative forms of research outcome (e.g. research data) were valued less. But the tide is changing and infrastructures are playing an important role in this process as they are promoting and facilitating alternative forms of publications (e.g. data publication, software publication) along with suitable metrics.

To gain a deeper understanding of established and alternative metrics, browse the following resources.

Watch Toma Tasovac (DARIAH-RS), Arjan van Hessen (CLARIAH) and Steven Krauwer (CLARIN ERIC) commenting on the adequacy of certain metrics for measuring impact.

Contents

To put it in a nutshell, Toma Tasovac likes outreach, but doesn’t like impact. Paraphrasing impact with “the accumulation of academic capital” he doubts the adequacy of bibliometric methods when it comes to the humanities. You want to know why? Watch the video!

Placeholder video 2

Arjan van Hessen points out the fact that universities tend to measure impact only by the number of publications. Find out what this means for the willingness of sharing data and watch the video!

Placeholder Video 3

Steven Krauwer explains how he is measuring his own impact. You want to know how? Watch the video!

Placeholder video 4


  1. IMPKT Tools: These tools have been developed within the RI Europeana. Especially noteworthy is the Impact Playbook, “a step by step approach to help you identify your impact”.
    • Fallon, Julia (October 2017): Impact tools & resources.

    • Verwayen, Harry / Fallon, Julia / Schellenberg, Julia / Kyrou, Panagiotis et al. for Europeana (October 2017): Impact Playbook. For Museums, Libraries, Archives and Galleries. PHASE I: Impact Design.

  2. Altmetrics: Read more about altmetrics in the following document:
    • European Commission Expert Group on Altmetrics (2017) Next-generation metrics: Responsible metrics and evaluation for open science. European Commission (Directorate-General for Research and Innovation). Available at: doi:10.2777/337729.

  3. Software citation: Learn more about software citation and consult the following slides or papers
  4. Bibliometrics: Some critical stance on bibliometrics as “a proxy for expert assessment” can be found in the following document:

Here you will find the link to the registration for the PARTHENOS Webinar “Impact?!” in February 2018 (held by Dr. Juliane Stiller and Klaus Thoden) and later the recording of this webinar and other materials.