University staff see positive and negative impacts of Research Excellence Framework, study shows

University staff see positive and negative impacts of Research Excellence Framework, study shows

24 June 2019

A report published today by Research England, Cardiff University and the University of Sheffield has revealed that academics have a range of opinions about the Research Excellence Framework (REF), seeing it as having a positive impact but with room for improvement.

In surveys and interviews, academics and non-academics at four universities said the REF has positive impact in many ways. For example:

  • The REF encourages engagement outside of academia, such as with the community.
  • University departments often use supportive activities to motivate and guide academics toward better REF performance, through production of excellent research outputs and delivery of societal impact.
  • The focus on open access and open research practices in the REF has a consistently positive influence on both researchers’ own work and UK academic culture more widely.

Feelings were also generally positive about changes made for the current exercise (REF 2021) compared to the last (REF 2014). For example:

  • Research managers expect the greater consideration of equality and diversity considerations in REF 2021 will have a positive impact on the well-being of academics compared to REF 2014.
  • Survey respondents showed moderate agreement to the statement that REF 2021 will be more supportive than REF 2014.
  • Researchers viewed REF2021 as offering somewhat more flexibility than REF 2014.
  • Respondents were positive about REF2021’s even greater focus than REF 2014 on quality rather than quantity of research outputs.

But there are some issues that academics would like to see addressed in future research assessment. These are:

  • Some universities seek to implement additional strategies, beyond those required by the exercise, in order to influence the outcome; this impacts on creativity and other aspects of academic culture.
  • Just under a sixth of respondents reported their departments using potentially pressurising activities.
  • Potential negative impacts on the health and well-being of the research community.
  • While the vast majority of respondents had not been asked to change the focus of their research to accommodate the REF, 15 per cent said they had been asked to do this.

The study highlights the importance of higher education institutions exercising good practice when implementing REF 2021 and that academics are more likely to have a more positive experience when this is the case.

Participants were positive about the strengthening of the equality and diversity provisions within REF 2021 which are designed to ensure researchers are treated fairly. These provisions are central to the guidance that higher education institutions must follow when submitting research for assessment in REF 2021.

This research was a pilot study, carried out in order to obtain initial evidence on perceptions and attitudes about the REF. Research England will use its findings to inform the development of future research assessment approaches after REF 2021, alongside a range of evidence from the wider evaluation of the exercise and work considering the future of research assessment.

The REF 2021 exercise, which is already under way, will not be affected.

Steven Hill, Director of Research at Research England and Chair of the REF 2021 Steering Group, commented:

“Research England works closely with the other UK higher education funding bodies to ensure that the REF is always based on principles of equity, equality and transparency.

“Our aim with this study was to collect robust evidence that can inform the development of future research assessment.

“The findings give valuable understanding about the lived experiences of academics across our world-leading higher education sector with regard to the REF, and which aspects are viewed favourably or unfavourably.

“There is much in this report that reassures, but also some evidence that we can improve processes, so we will look to do so for future exercises.

“We also encourage higher education institutions to consider the report’s findings on good practice and the role it plays in supporting a positive research environment.”

Netta Weinstein, Senior lecturer of Motivation and Behaviour Change at Cardiff University and joint author of the report, commented:

“The REF exercise most likely influences academics’ behaviour and well-being in a multitude of ways, but little empirical evidence has been collected to understand its influence and the implications for research activity.

“Since we know that perceptions of the HE workplace are key to determining researchers’ well-being and productivity, it is crucial that we carefully evaluate how the REF is understood and received by researchers in different circumstances. In fact, it is especially important to research the REF - an exercise which fundamentally values evidence-based decision-making.

“In an initial large-scale attempt to do just that, the current study uncovered attitudes toward the REF and explored their correlates. Using complementary research methods, we identify both the critical and favourable views that academics hold toward the REF.”

James Wilsdon, Professor of Research Policy at the University of Sheffield and joint author of the report, commented:

“Too often, debates about the REF are dominated by those who shout the loudest. In this pilot study, we tried to dig deeper and uncover a more nuanced picture of the pros and cons of the REF as it’s experienced by frontline researchers and managers. And we wanted to study this in real-time, while the rules for REF 2021 are still bedding down.

“Our results – which draw on almost 600 survey responses and 21 in-depth interviews – highlight several positive developments linked to the latest rule changes, and other ways in which research assessment could be improved. I hope there’s further sector-wide analysis of these questions as REF 2021 runs its course, and that this is used to inform and improve future frameworks.”

Notes

  1. The Real-Time REF Review: A Pilot Study to Examine the Feasibility of a Longitudinal Evaluation of Perceptions and Attitudes Towards REF 2021’ will be published at 00:01 on Monday 24 June.
     
  2. The report is based on a survey of 598 academics and in-depth interviews with 21 research managers, all at Cardiff University, the University of Sheffield, the University of Sussex and University of Lincoln, between April 2018 and September 2018.
     
  3. The REF is the system for assessing the quality of research in UK higher education institutions. Research England manages the REF on behalf of the UK’s higher education funding bodies: Research England, the Scottish Funding Council (SFC), the Higher Education Funding Council for Wales (HEFCW), and the Department for the Economy, Northern Ireland (DfE). For more information, see the REF website.
     
  4. Research England shapes healthy, dynamic research and knowledge exchange in English universities. It distributes over £2.2bn to universities in England every year; works to understand their strategies, capabilities and capacity; and supports and challenges universities to create new knowledge, strengthen the economy, and enrich society. Research England is part of UK Research and Innovation alongside the seven Research Councils and Innovate UK.
    @ResEngland