In 2019, a working group appointed by Universities Norway (UHR) was mandated to recommend guiding principles for the assessment and evaluation of research(ers) in light of the transition to Open Science. Just over a year later, the group published a systematic framework: the Norwegian Career Assessment Matrix (NOR-CAM), a toolbox for recognition and rewards in academic careers.
While the internal drivers established the need for action, the development of the Norwegian Career Assessment framework was also inspired by initiatives in other countries, such as the Dutch Recognition & Rewards programme. One of the main aims of NOR-CAM is for multiple areas of expertise to be assessed more systematically than is currently the case. It proposes a more flexible and holistic framework for recognition and rewards in academic research and adopts three core principles: more transparency; greater breadth; and comprehensive assessments as opposed to one-sided use of indicators.
The matrix can be combined for different purposes and needs. And because it is a matrix, the assessment can be adapted to emphasise different competencies for different tasks/positions/career stages depending on both the individual’s career and the institutional needs. Such an expanded research assessment approach aims to incentivise and reward a broader range of academic activities, and ultimately to improve academic culture and the quality of research. At this moment in time, Norwegian universities are implementing NOR-CAM locally.
Since the Agreement on Reforming Research Assessment was published in July 2022, more than 600 organisations in Europe and beyond have signed it. There is a broad international agreement from the research community on the urgent need to reform the existing research assessment practices. The Dutch approach shows strong alignment to the Agreement as it focuses more on the quality of all types of contributions and less on the number of publications.
Assessment processes relying predominantly on journal- and publication-based metrics can be a hurdle in the recognition of diverse contributions and may negatively affect the quality and impact of research. They also contribute to an unhealthy research culture and an unaffordable publication system. That is why the Agreement on Reforming Research Assessment establishes a common direction for research assessment reform, while respecting organisations’ autonomy.
The Agreement contains principles and commitments regarding scientific integrity, quality and impact, the diversity of scientific activities and collaboration. It is based on the following overarching principles:
These principles can be summarised in four core commitments:
Following this Agreement, the Coalition for Advancing Research Assessment (CoARA) was created for all signatory organisations that have subscribed to the Overarching Principles and Code of Conduct. The Coalition offers a space for its members to learn from each other’s experiences, to advance the process of research assessment reform in Europe and beyond. In line with the Agreement, it is the Coalition’s vision that the assessment of research, researchers and research organisations needs to recognise the diverse outputs, practices and activities that maximise the quality and impact of research. The signatories want to enable systemic reform of research assessment on the basis of common principles and commitments within an agreed timeframe, as set out in the Agreement. Through the exchange of information and mutual learning between all those willing to improve research assessment practices.
Interested organisations can sign the Agreement and become members of the Coalition | For more information about joining CoARA, go to: https://coara.eu/sign/ https://coara.eu/agreement/faq/ |
Members and signatories commit to implementing the 10 core principles via an action plan for their organisations | For more information about action plans and support guidelines, go to: https://coara.eu/agreement/action-plan/ |
CoARA members can join or propose Working Groups | To browse and contact the first-wave CoARA Working Groups, go to: https://coara.eu/coalition/working-groups/ For updates regarding the second call for Working Groups, go to: https://coara.eu/news/second-brokerage-event-working-group-call-2023/ |
CoARA members can contribute to National Chapters, which facilitate the exchange of knowledge, mutual learning and conversations on CoARA-relevant issues specific to different types of organisations of a given country. | To see an overview of the growing number of National Chapters, go to: https://coara.eu/coalition/national-chapters/ |
CoARA members participate in the decision-making processes of the Coalition at the General Assembly | For more information, go to: https://coara.eu/news/save-the-date-coara-general-assembly/ |
Research assessment experts can join the CoARA Steering Board | For more information, go to: https://coara.eu/news/call-for-candidates-for-coara-steering-board-members-incl-one-vice-chair-position/ |
The Coalition for Advancing Research Assessment summarised in numbers (reference date: October 9, 2023):
Research assessment systems are highly fragmentated among countries and between scientific systems and higher education institutions, so there is a complex mixture of evaluation procedures, each with its own objectives, scope and assessment dynamics. In addition, mainstream journal-based metrics, such as the Journal Impact Factor (JIF), achieved an incremental centrality in the research assessment practices and methodologies used by funding agencies and national science councils. Moreover, academic publishing systems and infrastructure are publicly owned and non-commercial, but the Diamond Open Access publishing circuit is not adequately contemplated in evaluation systems. These trends limit the local autonomy of research agendas, discourage good open access practices and reduce the potential interaction with society, while making other contributions made by researchers invisible.
In this context, 2019 saw members of the Latin American Council of Social Sciences (CLACSO), a network that currently consists of 898 research centres in 56 countries in Latin America and other continents, promote the creation of the Latin American Forum for Research Assessment (in Spanish, FOLEC), conceived as a space for mobilising research evaluation transformation, conducting research on research, and monitoring reforms. In 2022, a set of principles and guidelines for research assessment reform were approved by CLACSO’s 27th General Assembly. The document, known as Declaration of Principles: A New Research Assessment towards a Socially Relevant Science in Latin America and the Caribbean, encompasses research quality and socially relevant science, multilingualism and interdisciplinarity and recovers principles of the San Francisco Declaration on Research Assessment (DORA) and open science. At present, CLACSO-FOLEC’s Declaration of Principles has around 300 adherents, half of which are universities, research centres and academies, and the rest are scientific journals, repositories and science councils and agencies.
It is evident that there is now a growing momentum towards broader and more inclusive policies, methodologies, and processes for assessing research careers. Cooperation and the exchange of different local design, experiences and learnings are crucial to achieve a comprehensive, articulated and incremental reform of evaluation systems and research careers. In this respect, both the Dutch Research Council (NWO) and CLACSO-FOLEC continue working to strengthen these spaces of dialogue in broader and more inclusive global conversations.
Laura Rovelli coordinates the Latin American Forum for Research Assessment (FOLEC) from the Latin American Council of Social Sciences (CLACSO) and is a member of the Executive Board of the San Francisco Declaration on Research Assessment (DORA). She has a PhD in Social Sciences and is a researcher at National Council for Scientific and Technical Research (CONICET) in Argentina.