Harmonising data semantics of rankings: Results from Italy and Flanders

Abstract

University rankings have become increasingly present in the world of higher education. Although ranking results seem to be easily interpretable, their meaning depends on which indicators ranking organisations select and how these indicators are calculated. Furthermore, the quality of ranking results is influenced by how universities worldwide interpret and collect the required data. That is, data collection is strongly affected by physical constraints such as the local context of a university and technical aspects of data storage but also by the degree to which the semantic content of the required data concepts is described. Poorly described data concepts allow universities to collect data according to their own interpretation and thus create data variability that prevents a reliable comparison of universities in rankings. In 2016, the Flemish Ranking Work Group initiative was initiated, a collaboration between four out of five universities in Flanders (Belgium) and led by ECOOM-Hasselt. This initiative accommodates the semantic interpretation of the ranking indicators and aligns data collection, in consultation with several well-known university rankings, in order to have semantically comparable rankings results for at least these universities. This process of accommodation and semantic alignment resulted in, subtle, yet important changes in the data collection of each of the participating universities. In 2018, the first results of this collaboration were published and the effect of changes in indicator semantics became visible. This presentation will focus on the results of semantic alignment of data collection of universities rankings and the effect it has on ranking results.