Counting what really counts? Assessing the political impact of science
Résumé
The production of scientific knowledge is expected to benefit society in a variety of ways. However, and despite the many theoretical models available in the literature, there are few practical frameworks for assessing the dimensions of the effect of research on society—including its political impacts. As part of the ASIRPA approach, in this paper we propose an ordinal rating scale to allow assessment of the political impact of research, based on a review of the literature, qualitative evidence of political impact gleaned from a collection of case studies, and an expert panel. The resulting metric uses a 1–5 scale to evaluate the intensity of the political impact of research according to generic criteria associated to each rating level. Routine application of this scale in case study research is increasing, and is allowing robust, simple and consistent self-assessments of the political impacts across cases, to complement qualitative case study descriptions. The methodology used to design the rating scale prompted the panel experts to reveal their evaluation rationales and justify their judgments, increasing the transparency of the assessments. We believe that the benefits of assigning an ordinal measure to the political impact of research outweigh the risks of misuse of an impact number. The advantages include influencing political agenda-setting by showing what really matters, the opportunities it provides for scaling-up analyses of multidimensional impacts and identifying impact-generating mechanisms, and learning about and promoting discussion of the value systems reflected in the assessment.