Professor Darren Langdridge explains the Research Excellence Framework and its importance for academic research, including in the School of Psychology and Counselling.
If you have fond memories of playing the card game Top Trumps as a child, as I do, then the Research Excellence Framework – or REF – may well just be for you. I still remember the excitement of the Silver Surfer turning up in my hand to this day. Equally, if you love a league table – again, as I do – then you’ll be quite at home with the REF. It is after all one really big and expensive process to generate a university research league table, a bit like the Premier League but without the same salaries. Add the joke about Vice Chancellor pay yourselves.
So, what is the REF? The REF is a national research assessment exercise involving nearly all UK universities. It takes place every six years or so and is used by the four national funding bodies (Research England, the Scottish Funding Council, the Higher Education Funding Council for Wales, and the Department for the Economy, Northern Ireland) to assess the quality of research and determine the distribution of funding for research – around £2 billion per year – to UK universities. Yes, that’s right, the results of the REF determine the distribution of about £2 billion a year to UK universities!
Clearly, the REF matters to universities because it involves a lot of money. We are all competing to get the biggest possible share of the dosh allocated to our School/Department to support our research. The REF is however more than just about money. It is also about status and prestige, with many of the league tables ranking universities, in large part, on the basis of the quality of their research. The original idea behind the REF was one of accountability. The idea was that given the pot of money to support research is public money from taxes then there should be some process by which government can assure the public that this money is well spent. Whether the REF is a good way of doing this is hotly debated and I can tell you most academics are a little sceptical about the value of the REF in that regard. Indeed, many academics despise the REF and see it as an exercise in managerial control that distorts the nature of research itself. There is clearly a fair whack of truth in the criticism but REF is not going away, so we try to make the best of it.
So, I am sure you now want to know how they assess the quality of research in REF. Research is assessed within separate disciplinary units (called ‘units of assessment’), thirty-four units in total in the 2021 exercise. The unit most relevant to our School is the A4 unit which covers Psychology, Psychiatry and Neuroscience. This is actually a bit of an odd unit as most are single discipline. There’s a lot to be said about why psychology has been thrown in with psychiatry and neuroscience, it wasn’t always that way, but there is not the space here to discuss it in detail. Personally, like many other psychologists, I think it’s not the best situation as it leads to a problematic bias in favour of biological and cognitive psychology within our discipline.
Within each of these units of assessment – whether that is the psychology one (with those other disciplines), history, sociology, computing or chemistry (yep they all get their own unit) – there are sub-panels of senior academics appointed to carry out an expert peer review process. The sub-panels assess a set of information about the research in each subject area within an institution. My job as OU REF Chair for the A4 unit is to manage the process of putting together all that information for our submission, amongst other things. There are three types of information that are assessed to produce what is termed the ‘quality profile’, where percentage grades are assigned against each set of information. The three areas are:
- Research outputs – 60% of the overall quality profile
This is mostly journal article output for the A4 unit reporting the findings from our research but may be other forms of research ‘output’ like books. We have to submit a certain number of outputs for assessment by the panel (2.5 outputs per academic, with a minimum of 1 output per person in the unit). The idea is that we select our best work or – more accurately – the work we think will receive the highest grade. Sadly, these are not always the same thing.
- Impact – 25% weighting
Impact involves an assessment of quality based on a number of ‘case studies’ that seek to show how a research project or programme of research has had an impact beyond just other academics. The aim here is to reward research that can be shown to have a benefit for a non-academic user. A good example of this would be research leading to the production of a vaccine for COVID-19. If a working vaccine is produced and this is then taken by millions or even billions of people, saving endless lives, then this case study should score the highest grade for impact. Showing impact becomes more difficult outside the traditional sciences like vaccinology, however, as it is often difficult to show direct impact in other disciplines. The number of case studies submitted depends on the number of people in the unit.
- Environment – 15% weighting
This includes data on income from grant applications (a separate source of money to the REF), PhD student completions and a long document telling the reviewers about the ‘sustainability’ and ‘vitality’ of the research environment. The idea here is that a good research environment helps support high quality research and so should be further supported, a virtuous circle at best or at worse a way of entrenching financial support in the same old places.
The overall assessment of quality (the ‘quality profile’), as well as each sub-profile for outputs, impact and environment, is based on a 5-point scale (1* to 4* and Unclassified). Work is assessed as follows:
4 * Quality that is world-leading in terms of originality, significance and rigour.
3* Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence.
2* Quality that is recognised internationally in terms of originality, significance and rigour.
1* Quality that is recognised nationally in terms of originality, significance and rigour.
Unclassified Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment.
In the most recent exercise in 2014 funding was only attached to scores at the 3* and 4* levels (with 4* more heavily weighted than 3*). This will also continue to be the case for REF2021. The key for most institutions therefore is to maximise the percentage of 3* and 4* within each discipline quality profile.
As I mentioned above, the next assessment point is REF2021 and this involves a lot of work within UK universities for a lot of people. The deadline for university submissions to REF2021 is 31stMarch 2021, moved back from the original date as a result of the pandemic, with results now being released in April 2022. This is a big deal with very serious consequences for success and failure, jobs depend upon the results.
If you want further details about REF they are available on the REF website (https://www.ref.ac.uk/) and you can even while away the hours exploring the results of previous REF exercises if you are so inclined. The 2014 results are available here: https://www.ref.ac.uk/2014/. It’s compulsory reading for me!
That’s the Research Excellence Framework in a nutshell. Not the most exciting aspect of UK Higher Education by any means but nonetheless a very important process for the university sector.
Professor Darren Langdridge is the OU Chair of the A4 (Psychology, Psychiatry, Neuroscience) REF Unit. You can read more about his work here http://www.open.ac.uk/people/dl2688