The Times Higher has just published a very interesting and informative analysis of the REF (Research Excellence Framework), the replacement for the RAE:

Structural Adjustments – The REF Unscrambled

This detailed article is effectively the “story so far” of the REF, the next generation of one half of the dual support system. It’s important because the REF will be a major driver of UK university policy for the foreseeable future. Among the comments, opinions and statements of fact are the following nuggets:

• The REF at present appears to be largely similar to the RAE – “evolution not revolution” is the message which comes across clearly here. Like the RAE it is based heavily on peer review, with the addition of a “basket of metrics” readily available to inform judgements. As previously indicated, metrics will include research income and numbers of research students. Bibliometrics, intially a central feature of the REF and the source of much controversy, will now only feature in disciplines where they are found to be a robust indicator of research quality.
• Economic and social impact will feature much more prominently, though it is still not clear precisely how it will be measured. It’s likely that the case for impact will have to be made in the narrative which will feature in the REF, as in the RAE. David Sweeney (Director of Research at HEFCE) suggests that, rather than individual outputs having to demonstrate impact, the assessment will be made across a portfolio of submissions.
• HEFCE may be on a collision course with the Treasury if the REF ends up becoming simply RAE2008 “with numbers”. John Denham, DIUS secretary of state, has strongly argued that only a concentration of research funding in fewer universities delivers a world class HE system. This is what lay behind the Treasury’s initial desire for a metrics-only REF. However, RAE2008 identified and rewarded “pockets of excellence” in teaching-led institutions, effectively spreading the government’s investment more thinly and moving away from the concentrated funding model.
• Ironically, the government’s insistence on the inclusion of economic and social impact may well lead to a move further away from the concentration model: “You could find”, Bahram Bekhradnia (HEPI director) notes, “that different universities are scoring better on impact and utility of research than the research-intensive universities.”
• Staff selection is another issue which proved controversial in the last RAE, and it seems likely that the REF won’t force institutions to submit all staff to the exercise. The claim was that selection leads to “gamesplaying” by institutions, but, according to Geoffrey Crossick, warden at Goldsmiths: “To call it game-playing seems to me to somehow to miss the point. Any research evaluation system will shape the behaviours of institutions and departments. That is why we have it. The argument is how we make sure we shape the right kinds of behaviours.”

Interestingly, there was no mention of STEM (Science, Technology, Engineering and Maths) subjects getting preferential treatment through ringfencing of the research budget. Many in the sector feel that arts, humanities and (to some extent) social sciences subjects will have a harder time gaining funding in future because of the government’s commitment to maintaining funding for STEM despite cuts to HE funding more widely.

As far as the REF goes, practical issues still to be decided include: staff selection; accreditation of outputs to institutions or individual researchers; number of panels; number of outputs; frequency of assessments; and how bibliometrics is combined with expert review. HEFCE are planning to release the final proposals for consultation in autumn this year.