The international playing field

The Recognition & Rewards programme is a unique joint programme. Universities, university medical centres, research institutes and research funders are working together on a culture change in order to create room for the diverse talents of academics. A frequently asked question is what other countries around us are doing. Although the Netherlands is an ‘early mover’, similar developments are taking place elsewhere in the world. What relevant developments related to Recognition & Rewards are there at international level? In this section, we highlight four cases from abroad: the Norwegian Career Assessment Matrix (NOR-CAM), the Coalition for Advancing Research Assessment (CoARA), the Declaration of Principles of the Latin American Forum on Research Assessment (CLACSO-FOLEC) and the Statement of Principles on Recognising and Rewarding Researchers of the Global Research Council (GRC).

The Norwegian Career Assessment Matrix (NOR-CAM) – The Norwegian Approach

In 2019, a working group appointed by Universities Norway (UHR) was mandated to recommend guiding principles for the assessment and evaluation of research(ers) in light of the transition to Open Science. Just over a year later, the group published a systematic framework: the Norwegian Career Assessment Matrix (NOR-CAM), a toolbox for recognition and rewards in academic careers.

The Norwegian Career Assessment Matrix (NOR-CAM) serves as a framework for assessing general academic activity (results and competencies). One of the main aims of NOR-CAM is for multiple areas of expertise to be assessed more systematically than is currently the case. Illustration: Universities Norway (UHR).

While the internal drivers established the need for action, the development of the Norwegian Career Assessment framework was also inspired by initiatives in other countries, such as the Dutch Recognition & Rewards programme. One of the main aims of NOR-CAM is for multiple areas of expertise to be assessed more systematically than is currently the case. It proposes a more flexible and holistic framework for recognition and rewards in academic research and adopts three core principles: more transparency; greater breadth; and comprehensive assessments as opposed to one-sided use of indicators.

Broad Evaluation

‘The aim is to facilitate interaction between the documentable and/or measurable quantities and the applicant’s qualitative assessment of these.’
The matrix consists of six areas of expertise to be assessed: research output; research process; service and leadership; research impact; teaching and supervising; and professional experience. In each category, relevant results, experience, competencies and activities can be described, documented and reflected on. The matrix contains examples of relevant competencies, activities and results for each of the categories in order to show what can be worthy of merit, but the list is not exhaustive. Components that are added under the different categories must be adapted to the context in which they are to be used. It’s important that the applicant gives a subjective assessment of their own results and competencies. The aim is to facilitate interaction between the documentable and/or measurable quantities and the applicant’s qualitative assessment of these.

The matrix can be combined for different purposes and needs. And because it is a matrix, the assessment can be adapted to emphasise different competencies for different tasks/positions/career stages depending on both the individual’s career and the institutional needs. Such an expanded research assessment approach aims to incentivise and reward a broader range of academic activities, and ultimately to improve academic culture and the quality of research. At this moment in time, Norwegian universities are implementing NOR-CAM locally.

In this video, Alexander Jensenius presents the toolbox and its motivation. Jensenius is professor at University of Oslo. He was member of the UHR working group.

The International Coalition for Advancing Research Assessment (CoARA)

Since the Agreement on Reforming Research Assessment was published in July 2022, more than 600 organisations in Europe and beyond have signed it. There is a broad international agreement from the research community on the urgent need to reform the existing research assessment practices. The Dutch approach shows strong alignment to the Agreement as it focuses more on the quality of all types of contributions and less on the number of publications.

Ten principles

Assessment processes relying predominantly on journal- and publication-based metrics can be a hurdle in the recognition of diverse contributions and may negatively affect the quality and impact of research. They also contribute to an unhealthy research culture and an unaffordable publication system. That is why the Agreement on Reforming Research Assessment establishes a common direction for research assessment reform, while respecting organisations’ autonomy.

The Agreement contains principles and commitments regarding scientific integrity, quality and impact, the diversity of scientific activities and collaboration. It is based on the following overarching principles:

  1. Comply with ethics and integrity rules and practices.
  2. Safeguard freedom of scientific research.
  3. Respect the autonomy of research organisations.
  4. Ensure independence and transparency of the data, infrastructure and criteria necessary for research assessment and for determining research impacts.
  5. Focus research assessment criteria on quality.
  6. Recognise the contributions that advance knowledge and the (potential) impact of research results.
  7. Recognise the diversity of research activities and practices, with a diversity of outputs, and reward early sharing and open collaboration.
  8. Use assessment criteria and processes that respect the variety of scientific disciplines, research types and career stages and that acknowledge multi-, inter- and trans-disciplinary as well as inter-sectoral approaches, when applicable.
  9. Acknowledge and valorise the diversity in research roles and careers, including roles outside academia.
  10. Ensure gender equality, equal opportunities and inclusiveness.

These principles can be summarised in four core commitments:

  • Recognise the diversity of contributions to, and careers in, research, in accordance with the needs and the nature of the research.
  • Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators.
  • Abandon the inappropriate uses in research assessment of journal- and publication-based metrics, in particular the inappropriate uses of Journal Impact Factor (JIF) and h-index.
  • Avoid the use of rankings of research organisations in research assessment.

The Coalition

Following this Agreement, the Coalition for Advancing Research Assessment (CoARA) was created for all signatory organisations that have subscribed to the Overarching Principles and Code of Conduct. The Coalition offers a space for its members to learn from each other’s experiences, to advance the process of research assessment reform in Europe and beyond. In line with the Agreement, it is the Coalition’s vision that the assessment of research, researchers and research organisations needs to recognise the diverse outputs, practices and activities that maximise the quality and impact of research. The signatories want to enable systemic reform of research assessment on the basis of common principles and commitments within an agreed timeframe, as set out in the Agreement. Through the exchange of information and mutual learning between all those willing to improve research assessment practices.

Ways in which interested parties can engage with CoARA

Interested organisations can sign the Agreement and become members of the Coalition For more information about joining CoARA, go to: https://coara.eu/sign/ https://coara.eu/agreement/faq/
Members and signatories commit to implementing the 10 core principles via an action plan for their organisations For more information about action plans and support guidelines, go to: https://coara.eu/agreement/action-plan/
CoARA members can join or propose Working Groups To browse and contact the first-wave CoARA Working Groups, go to: https://coara.eu/coalition/working-groups/ For updates regarding the second call for Working Groups, go to: https://coara.eu/news/second-brokerage-event-working-group-call-2023/
CoARA members can contribute to National Chapters, which facilitate the exchange of knowledge, mutual learning and conversations on CoARA-relevant issues specific to different types of organisations of a given country. To see an overview of the growing number of National Chapters, go to: https://coara.eu/coalition/national-chapters/
CoARA members participate in the decision-making processes of the Coalition at the General Assembly For more information, go to: https://coara.eu/news/save-the-date-coara-general-assembly/
Research assessment experts can join the CoARA Steering Board For more information, go to: https://coara.eu/news/call-for-candidates-for-coara-steering-board-members-incl-one-vice-chair-position/

The Coalition for Advancing Research Assessment summarised in numbers (reference date: October 9, 2023):

  • 637 organisations have signed the Agreement, 28 of them are from the Netherlands;
  • 63% of these organisations are universities;
  • Members come from 41 different countries.

Reshaping recognition and rewards from Latin American and the Caribbean 

A general discontent runs through global science, and it results from the various distortions existing in current research evaluation systems. Latin America and the Caribbean are no exception. CLACSO-FOLEC emphasises the need to recognise and reward open access scientific publication and participation in content evaluation.

Research assessment systems are highly fragmentated among countries and between scientific systems and higher education institutions, so there is a complex mixture of evaluation procedures, each with its own objectives, scope and assessment dynamics. In addition, mainstream journal-based metrics, such as the Journal Impact Factor (JIF), achieved an incremental centrality in the research assessment practices and methodologies used by funding agencies and national science councils. Moreover, academic publishing systems and infrastructure are publicly owned and non-commercial, but the Diamond Open Access publishing circuit is not adequately contemplated in evaluation systems. These trends limit the local autonomy of research agendas, discourage good open access practices and reduce the potential interaction with society, while making other contributions made by researchers invisible.

Research quality and socially relevant science

In this context, 2019 saw members of the Latin American Council of Social Sciences (CLACSO), a network that currently consists of 898 research centres in 56 countries in Latin America and other continents, promote the creation of the Latin American Forum for Research Assessment (in Spanish, FOLEC), conceived as a space for mobilising research evaluation transformation, conducting research on research, and monitoring reforms. In 2022, a set of principles and guidelines for research assessment reform were approved by CLACSO’s 27th General Assembly. The document, known as Declaration of Principles: A New Research Assessment towards a Socially Relevant Science in Latin America and the Caribbean, encompasses research quality and socially relevant science, multilingualism and interdisciplinarity and recovers principles of the San Francisco Declaration on Research Assessment (DORA) and open science. At present, CLACSO-FOLEC’s Declaration of Principles has around 300 adherents, half of which are universities, research centres and academies, and the rest are scientific journals, repositories and science councils and agencies.

Crucial factors

‘It is evident that there is now a growing momentum towards broader and more inclusive policies, methodologies, and processes for assessing research careers.’
In dialogue with the Dutch Recognition & Rewards programme, CLACSO-FOLEC endorses the assessment of different dimensions of researchers’ activities in the region. These include teaching, mentoring, public communication of science, citizen science and engagement with society, also called ‘extension’, and of a diversity of research outputs, for example books, chapters and reports. To better evaluate social relevance in social sciences and humanities, it encourages indicators of knowledge co-production with the community, fundamental research focused on use, social intervention and creation for social purposes, as well as basic research. CLACSO-FOLEC emphasises the need to recognise and reward open access scientific publication and participation in content evaluation in repositories and diamond journals, as well as other components of open science.

Growing momentum

It is evident that there is now a growing momentum towards broader and more inclusive policies, methodologies, and processes for assessing research careers. Cooperation and the exchange of different local design, experiences and learnings are crucial to achieve a comprehensive, articulated and incremental reform of evaluation systems and research careers. In this respect, both the Dutch Research Council (NWO) and CLACSO-FOLEC continue working to strengthen these spaces of dialogue in broader and more inclusive global conversations.

Laura Rovelli coordinates the Latin American Forum for Research Assessment (FOLEC) from the Latin American Council of Social Sciences (CLACSO) and is a member of the Executive Board of the San Francisco Declaration on Research Assessment (DORA). She has a PhD in Social Sciences and is a researcher at National Council for Scientific and Technical Research (CONICET) in Argentina.

New international agreement by science funders on rewarding researchers

Arfan Ikram speaks during the Global Research Council.
Research funders from around the world have signed the Statement of Principles on Recognising and Rewarding Researchers. The statement sets out principles of how funders will work together to achieve new ways of assessing research and researchers, appropriate to the current times and in line with today’s researcher. It also states that there is growing momentum for reform towards broader and more inclusive assessments of research, but that there are still challenges that organisations face to make such changes, too. To help alleviate these challenges and achieve new ways and means of assessing research worldwide, research funders have agreed on nine principles. The Statement of Principles was signed on 31 May in The Hague, during the annual meeting of the Global Research Council (GRC), the collaboration of national research councils and research funders around the world.