Agreement on Reforming Research Assessment

The European ‘Agreement on Reforming Research Assessment’ supports the national Recognition & Rewards programme a great deal. Through it, hundreds of organisations from Europe and beyond have jointly expressed the need to reform research assessment.

The Royal Netherlands Academy of Arts and Sciences (KNAW), the Dutch Research Council (NWO), ZonMw, Universities of the Netherlands (UNL) and 13 individual universities have signed it too. An agreement initiated by the European Commission, Science Europe and the European University Association (EUA), it contains arrangements about how research results must be assessed in order to enhance the quality and impact of the research, as well as to ensure integrity. In addition, attention is explicitly given to recognising and rewarding academics for the various tasks they perform.

Co-creation process

The Agreement on Reforming Research Assessment was initiated by the EUA (the European umbrella organisation of universities) and Science Europe (the European umbrella organisation of research funders), and is the result of a co-creation process that involved more than 350 interested organisations from over 40 countries. The European Commission provided the EUA and Science Europe with support during the process.

The agreement

The agreement contains principles and commitments regarding scientific integrity, quality and impact, the diversity of scientific activities and collaboration. One of its points of departure is that research must be assessed primarily on the basis of a qualitative evaluation, rather than on mere numbers of publications. Peer review plays a central role and is supported by the responsible application of quantitative indicators. The signatories commit to a timetable that stipulates that they will, after one year, share with each other how they are working on implementing the agreement and, after five years, evaluate their progress.

Taking steps

Many of the requested reforms have already been carried through in the Netherlands via the Recognition & Rewards programme. The fact that many changes are now also being implemented outside of the Netherlands is good news, since science is quintessentially international. The signatories are not confined to European organisations. With this agreement, scientific organisations throughout the world are sending a message that it is essential to reform the way in which research is assessed. In this way, all of these organisations are also taking steps to assess the wide variety of careers in the academic world in an appropriate way. Now it is up to the scientists to apply this agreement to their own working environment, appointments advisory committees and assessment committees.

The signatories will exchange practices and experiences internationally in order to learn from each other. This exchange is intended to contribute towards the adoption of internationally similar assessment practices, which will help ensure more equal career opportunities for scientists who are internationally active.

Principles

The agreement starts with principles that signatories commit to in terms of:

  • Overarching conditions:
    • Complying with rules and practices on ethics and integrity
    • Guaranteeing freedom of scientific research
    • Respecting the autonomy of research organisations
    • Ensuring independence and transparency when assessing research
  • Quality and impact:
    • Focusing research evaluation criteria on quality
    • Recognising contributions that advance the knowledge and (potential) impact of research
  • Diversity, inclusiveness and collaboration:
    • Respecting the diversity of scientific activities, practices and forms of outputs
    • Using assessment criteria that respect variety in scientific disciplines
    • Recognising and valuing diversity in research roles and careers, including outside of academia

This is followed by a number of commitments to which signatories promise to adhere. The core commitments are:

  1. Recognise the diversity of contributions to, and careers in, research in a manner appropriate to the nature of research
  2. When assessing research, stop the inappropriate use of journal- and publication-based statistics, in particular the Journal Impact Factor and h-index
  3. Base research assessment mainly on qualitative evaluation that focuses on peer review, supported by responsible use of quantitative indicators
  4. Avoid the use of research organisation rankings when reviewing research