Harmonizing the semantic interpretation and data collection of U-Multirank indicators: an inter-university initiative in Flanders

Abstract

Over the past decade, university rankings have become increasingly popular among a broad variety of end-users ranging from policymakers to researchers, students and their parents. Although ranking results presented by a single rank score seem to be easily interpretable, many caveats have been drawn regarding the indicators, their accompanying semantics as well as the methodologies used for data collection. In many cases, the semantic content of the indicators is poorly defined, leaving a grey zone that allows universities to collect data according to their own interpretation. This, together with differences in the local context of a university, results in a great variety in data that impedes a reliable comparison of universities. To overcome this variability, a Flemish initiative led by ECOOM-UHasselt, accommodated the semantic interpretation of the indicators used in U-Multirank. In consultation with U-Multirank, indicator definitions were scrutinized with regard to the specific interpretation of each of the four participating Flemish universities. Consequently, joint decisions with regard to the semantic interpretation and the data collection were taken and registered in a so-called ‘Flemish handbook for semantic comparable U-Multirank results’. This level of fine-tuning ensures an identical interpretation and a harmonized data collection resulting in truly semantically comparable ranking results.