NRF Rating: thoughts

Author

Affiliation

Published

April 24, 2023

The South African National Research Foundation (NRF) rating system claims to evaluate and benchmark the research performance of individual researchers in the country. The system’s purpose is intended to:

Recognise and reward research excellence The system acknowledges researchers who produce high-quality research and contribute significantly to their respective fields. A favourable rating is supposed to increase recognition, both nationally and internationally, as well as improve funding opportunities.
Encourage research productivity By providing incentives and recognition for high-quality research, the NRF rating system aims to promote academic productivity and encourages continuous advancement.
Enhance research capacity It supposedly identifies academics with solid potential and supports the development of research capacity in South Africa by providing funding and other resources to rated researchers.
Facilitate collaboration The NRF rating system claims to facilitate scientific cooperation by enabling researchers, institutions, and funding agencies to identify potential partners based on their research expertise and performance.
Promote international competitiveness The NRF suggests that a robust research evaluation system helps to ensure that South African researchers remain competitive on the global stage, which is essential for attracting international funding, partnerships, and talent.
Inform decision-making NRF ratings inform institutional, national, and international decision-making regarding research priorities, funding allocations, and strategic planning, ensuring that resources are directed towards high-impact research.

There are alternatives to the NRF rating system. The H-index is a globally recognised rapid assessment of research impact, of which Google offers one implementation on their Google Scholar system. This H-index is consistently applied to researchers from any country or any academic discipline. The metric is based on citation data and provides a more objective and quantitative measure of research impact. Since the H-index is easily accessible and hassle-free, it is calculated on the fly using various citation databases, such as Google Scholar. This last point contrasts starkly with the NRF rating system, which is lengthy, and requires significant effort and time from both the applicants and reviewers.

Further comparisons of the NRF rating system to a metric such as Google Scholar’s H-index reveal other possible advantages. The NRF’s approach is a more integrated and robust assessment of research ‘prowess’ as the system considers multiple aspects of academic contributions. This includes not only the quality, impact, and significance of research output (similar to the H-index, but differ in how these are assessed) but also a broader contribution to academics’ research fields using assessments that are not based on publications, such as participation in various international bodies, panels and working groups. This integrated assessment leads to a more nuanced evaluation of academic performance that citation metrics, such as the H-index, cannot capture. It also acknowledges academics for their role in developing research capacity, which in South Africa is a critical role that all academics must play.

NRF ratings are determined through a rigorous peer-review process, which claims to ensure that the evaluations are fair and unbiased. However, despite the peer-review process, personal biases or conflicts of interest may still influence the ratings, and the system could be more objective. The system is also specific to South Africa, and the recognition that might stem from one’s NRF rating does not favour one as much as one would wish to think. This is true especially once international research funding becomes attractive and one is willing to enter more comprehensive international research consortia.

In the past, rated researchers were offered incentive funding. This system no longer exists, at least not in the format it was implemented in the early- to mid-2010s. Note, the ‘Competitive Support for Unrated Researchers (CSUR) - 2024 Funding Framework’ and ‘Competitive Programme for Rated Researchers (CPRR) – 2024 Funding Framework’ do take rating into account, but others, such as the ‘African Coelacanth Ecosystem Programme (ACEP) – South African Marine and Antarctic Research Strategy,’ do not. Similarly, international funders, where I will focus my attention in the future, also do not acknowledge NRF ratings.

Whether or not one maintains an NRF rating depends on personal research values. This should be decided on personal conviction and not dictated by the institution within which one is employed. I have yet to experience the NRF rating system to offer me any tangible advantage regarding recognition of research excellence, encouragement of productivity, enhancement of capacity, the facilitation of collaboration, or enhanced international competitiveness. The only benefit resulting from NRF ratings goes to the employers to inform institutional decision-making.

Reuse

CC BY 4.0

Citation

BibTeX citation:

@online{smit,_a._j.2023,
  author = {Smit, A. J.,},
  title = {NRF {Rating:} Thoughts},
  date = {2023-04-24},
  url = {http://tangledbank.netlify.app/pages/NRF_ratings.html},
  langid = {en}
}

For attribution, please cite this work as:

Smit, A. J. (2023) NRF Rating: thoughts. http://tangledbank.netlify.app/pages/NRF_ratings.html.

--- date: "2023-04-24" title: "NRF Rating: thoughts" number-sections: false format: html --- The South African National Research Foundation (NRF) rating system claims to evaluate and benchmark the research performance of individual researchers in the country. The system's purpose is intended to: 1. **Recognise and reward research excellence** The system acknowledges researchers who produce high-quality research and contribute significantly to their respective fields. A favourable rating is supposed to increase recognition, both nationally and internationally, as well as improve funding opportunities. 2. **Encourage research productivity** By providing incentives and recognition for high-quality research, the NRF rating system aims to promote academic productivity and encourages continuous advancement. 3. **Enhance research capacity** It supposedly identifies academics with solid potential and supports the development of research capacity in South Africa by providing funding and other resources to rated researchers. 4. **Facilitate collaboration** The NRF rating system claims to facilitate scientific cooperation by enabling researchers, institutions, and funding agencies to identify potential partners based on their research expertise and performance. 5. **Promote international competitiveness** The NRF suggests that a robust research evaluation system helps to ensure that South African researchers remain competitive on the global stage, which is essential for attracting international funding, partnerships, and talent. 6. **Inform decision-making** NRF ratings inform institutional, national, and international decision-making regarding research priorities, funding allocations, and strategic planning, ensuring that resources are directed towards high-impact research. There are alternatives to the NRF rating system. The H-index is a globally recognised rapid assessment of research impact, of which Google offers one implementation on their Google Scholar system. This H-index is consistently applied to researchers from any country or any academic discipline. The metric is based on citation data and provides a more objective and quantitative measure of research impact. Since the H-index is easily accessible and hassle-free, it is calculated on the fly using various citation databases, such as Google Scholar. This last point contrasts starkly with the NRF rating system, which is lengthy, and requires significant effort and time from both the applicants and reviewers. Further comparisons of the NRF rating system to a metric such as Google Scholar's H-index reveal other possible advantages. The NRF's approach is a more integrated and robust assessment of research 'prowess' as the system considers multiple aspects of academic contributions. This includes not only the quality, impact, and significance of research output (similar to the H-index, but differ in how these are assessed) but also a broader contribution to academics' research fields using assessments that are not based on publications, such as participation in various international bodies, panels and working groups. This integrated assessment leads to a more nuanced evaluation of academic performance that citation metrics, such as the H-index, cannot capture. It also acknowledges academics for their role in developing research capacity, which in South Africa is a critical role that all academics must play. NRF ratings are determined through a rigorous peer-review process, which claims to ensure that the evaluations are fair and unbiased. However, despite the peer-review process, personal biases or conflicts of interest may still influence the ratings, and the system could be more objective. The system is also specific to South Africa, and the recognition that might stem from one's NRF rating does not favour one as much as one would wish to think. This is true especially once international research funding becomes attractive and one is willing to enter more comprehensive international research consortia. In the past, rated researchers were offered incentive funding. This system no longer exists, at least not in the format it was implemented in the early- to mid-2010s. Note, the ‘Competitive Support for Unrated Researchers (CSUR) - 2024 Funding Framework’ and ‘Competitive Programme for Rated Researchers (CPRR) – 2024 Funding Framework’ do take rating into account, but others, such as the ‘African Coelacanth Ecosystem Programme (ACEP) – South African Marine and Antarctic Research Strategy,’ do not. Similarly, international funders, where I will focus my attention in the future, also do not acknowledge NRF ratings. Whether or not one maintains an NRF rating depends on personal research values. This should be decided on personal conviction and not dictated by the institution within which one is employed. I have yet to experience the NRF rating system to offer me any tangible advantage regarding recognition of research excellence, encouragement of productivity, enhancement of capacity, the facilitation of collaboration, or enhanced international competitiveness. The only benefit resulting from NRF ratings goes to the employers to inform institutional decision-making.