The expert group was asked to provide advice on, and propose solutions to, issues associated with university rankings in relation to the Recognition & Rewards programme. Ludo Waltman, professor of Quantitative Science Studies and deputy director of the Centre for Science and Technology Studies (CWTS) at Leiden University, was a member of the expert group. We spoke with him about the recommendation paper.
“We’re talking about rankings that claim to be able to sum up a university’s performance in the broadest sense in a single score, while lumping together the university’s various activities (including education and research) and disciplines. The best-known examples are the Times Higher Education World University Rankings, the QS World University Rankings and the Academic Ranking of World Universities (ARWU), also known as the Shanghai Ranking.”
“Rankings combine data from various sources, and data concerning publications and citations play an important part here. Surveys are also used, and universities are asked to supply data themselves as well, for example on the ratio of students to staff.”
“It seems that the main target group is made up of students (and their parents) researching which foreign university to study at. Rankings create the impression of being able to help students to find a good university to join. However, this is highly debatable, because the rankings have far more to do with research than education.”
In the summer of 2022, the Board of Universities of the Netherlands (UNL) set up an expert group and asked it to provide advice on, and propose solutions to, issues associated with global rankings in relation to the Recognition & Rewards programme. The group’s recommendation paper, entitled Ranking the University, was published in July 2023 and has received an administrative response from UNL.
“League tables are misleading. They make strong claims about the information they provide, for example by saying they showcase the best universities in the world, but they are completely incapable of delivering on those claims.”
“League tables are largely determined on the basis of publications and citations, whereas a heavy emphasis on publications and citations is precisely what the Recognition & Rewards programme wants to move away from. This is a significant area of tension. After all, universities aiming for good scores in league tables could easily be tempted to judge their researchers mainly on publications and citations, thus seriously undermining the Recognition & Rewards programme.”
“It’s difficult to say with certainty, but I expect it to have a negligible effect. If it ends up having any effect at all, it will in all probability be far less than the effect of all kinds of external factors over which Dutch universities have little influence anyway, such as, for example, the strong growth of the Chinese academic system.”
“The main change in culture will involve having to be far more honest about rankings. Some of the information that rankings provide can be fairly relevant in certain situations, but at the same time, we are all perfectly aware that rankings simply cannot deliver on their simplistic claims about what the world’s best universities are. We need to stop treating rankings opportunistically and stop accepting the problematic narrative of rankings in a wholly uncritical manner. Instead, we must dictate the narrative, by actively promoting what we ourselves see as the strengths of our universities.”
Utrecht University recently decided to stop supplying data for the Times Higher Education global ranking. As a result, Utrecht University was not included in their World University Rankings 2024. In a response, the university has stated that rankings put too much emphasis on scoring and competition, and the university actually wants to focus on cooperation and open science. Utrecht University is the first Dutch university not to participate in this global ranking. In taking that decision, Utrecht University was directly acting on the recommendations of the expert group, which advised Dutch universities to stop supplying data to non-transparent league tables in the long term – if there is international support for this.
“By adopting our recommendations, universities will show their staff that they are unequivocally opting for the new Recognition & Rewards programme. Staff will be able to rely on universities embracing the Recognition & Rewards programme not just in word but also in deed.”
“The work of communications advisers is about to become much more interesting. Instead of blindly following rankings and framing the rankings’ results as positively as possible, communications advisers will be able to start focusing on conveying the real story of their institution. What is the institution really proud of? What sets the institution apart from other institutions? And what does this mean in practice for the institution’s education and research? These are the questions that really matter. In answering such questions, communications advisers will be making a crucial contribution to a change in culture with regards to rankings.”
Ludo Waltman was a member of the rankings expert group set up by the UNL Board in the spring of 2022. He is professor of Quantitative Science Studies and deputy director of the Centre for Science and Technology Studies (CWTS) at Leiden University. His work focuses on infrastructures, algorithms and tools to support research assessment, science policy and scholarly communication. Waltman is also coordinator of the CWTS Leiden Ranking, a ranking that stands for a multidimensional perspective on university performance.