ARMA workshop – assessing impact strategies beyond REF2014

The Association of Research Managers and Administrators is to hold a one-day workshop to consider strategies for recording and maximising research impact beyond the Research Excellence Framework exercise in 2014.

The event, on 23 November, will feature speakers from the universities of Cambridge and York, which were involved in the 2010 REF pilot impact exercise. It will use case studies to assess toolkits and systems to manage impact.

Registration is via the ARMA website and is open for booking now.

Further Backing for Open Access Publishing of Research Data

Open Access RepositoryThe Association of Research Managers (ARMA) has teamed up with knowledge-transfer groups PraxisUnico and the Association for University Research and Industry Links (AURIL) to support open-access publishing of research data.

In a joint statement, the organisations say open access will lead to a more productive use of research results in both academic and non-academic settings; increase the ability of knowledge professionals to find, access and use the outputs and data generated by research for policy, social, cultural, health, environmental, as well as economic benefit, in the UK and internationally; and can potentially support innovation and the inventive process, helping to generate a faster translation of research into practice.

They also argue that research management, assessment and evaluation would be easier if research outputs are openly available, as this promotes innovation and competition
within and between research organisations. However, the three organisations – ARMA, PraxisUnico and AURIL – believe open access to data underpinning the results should not occur too early in the research process as it could  hinder collaborations with industry.

ARMA Conference 2011: Notes, Reflections and Analysis

This year’s annual Association of Research Managers and Administrators conference was subtitled “Good Advice for Hard Times”, referring of course to the recent and impending wave of government spending cuts as well as the HE White Paper, which is the biggest shake up of higher education for many years and is highly controversial.

In general the mood (with one notable exception) was one of acceptance of the cuts and the new reality that they represent, with some presentations focussing on ways of increasing efficiency in the service we provide and “getting more for less”. The content and topics discussed were similar to last year’s conference, although there was still a lot to be gained from hearing how other research support and development services improve bid quality and work with academic staff.

There was also a focus on the structure of research support in different institutions, with some favouring a centralised model while others devolve much of the day-to-day operational support to faculties, schools and departments. Many – perhaps the majority – fall somewhere in between these two extremes. There are strengths and weaknesses to both approaches of course, and one key message is that it’s vital to consult with “service users” (i.e. academic staff, researchers, heads of departments, research directors etc.) to determine where the optimal balance lies.

The ARMA conference continues to grow with over 400 delegates registered to attend this year’s event (compared to around 350 last year), and most HEIs were represented – many sent several people along to cover different parallel sessions. As well as universities, representatives from most of the major UK funders were also in attendance and there were workshops delivered by staff from the NIHR, Leverhulme, the British Academy, British Council, AHRC, ESRC and EPSRC.

What follows is a mixture of notes, reflections and analysis from the plenaries and parallel sessions I attended. I also have access to all the presentations given throughout the conference, and I’d be happy to share these with any member of staff who wants them. I’m pleased to discuss further with anyone interested, please contact me in the usual way.

Plenary 1: Prof. Anton Muscatelli – Academic and Economic Sustainability of Research

Prof. Muscatelli outlined the difficulties facing the HE sector, discussing the focussing of funding by research councils and others on early career researchers and outstanding senior individuals. He highlighted the “squeezed middle” in this picture – a popular way of critiquing the coalition government’s austerity policies more generally. He argued that the sector “needs to make the case for research and science within the next 4 years or we will face difficult times”. In the context of this remark, it was interesting to note that Prof. Muscatelli focused largely on economic impact as a way of promoting the benefits of research.

He also spoke about the demand management measures increasingly adopted by research councils to raise success rates and lower workload on admin staff. We’ve blogged about this a lot since the publication of the RCUK delivery plans at the start of the year.

Glasgow’s innovative Easy Access IP was mentioned, with Prof. Muscatelli arguing that as only a small proportion of IP can be exploited by a university (~5%), the rest should be offered for free to those who can demonstrate how it can be used to benefit society. This leads to better collaborations, more income streams, and increased reputation.

Research management was discussed: at Glasgow the whole process was mapped and it was found to involve 20 different groups and 15 handovers when looked at end-to-end. It’s critical to be aware of the different groups involved at an institutional level so you can identify problematic areas and resolve them.

Plenary 2: Ehsan Mahmood – How on earth did we get here and what happens next?

Cuts and concentration of funding dominated Ehsan’s downbeat but combative talk: “the world’s great powerhouse of knowledge creation is slowly losing confidence,” he declared portentously.

The talk was a direct challenge to government policy, which he attacked directly: “UK leaders have been slowly dismantling the system”. Specific policy measures were also critiqued: “The RAE encourages gameplaying and concentration, but discourages risk-taking,” which is probably the opposite to what you want a system of research funding to do. In the long run, he argued, scientific excellence and progress would slow down under a system which concentrates research funding.

This was an interesting and engaging polemic and it provided a refreshing change from the more “nuts and bolts” focus of some of the other sessions.

Session 101: Funding for health research (NIHR)

NIHR split their session into two. The first half focused on research career pathways funded by NIHR. In effect they outlined the opportunities available on their website, but it’s always worth being reminded of these in case anything new is announced or an old opportunity has been discontinued. Fellowships aim to build research capacity, including in social care research – several schemes were highlighted:

Tips on bid writing included the old chestnuts: read the instructions and follow the instructions. Remit is very important for different schemes.

Regarding peer review, the point was made that it is not always a subject specialist who reviews your application, so it’s important to make it understandable to a general audience. Try to answer the following questions in your application to increase chances of success: why this? why now? why here? why us? Track record of the PI is also critical.

The presenter repeatedly emphasised the importance of talking to the NIHR at an early stage to determine whether they are the right funder and this is the right award.

The second half was an outline of some opportunities available through the Research for Patient Benefit and related schemes:

  • i4i (Invention for Innovation): early stage/late stage product development and feasibility for the translation of healthcare technologies for increased patient benefit.
  • Programme Grants are so competitive and complex that NIHR have a pre-funding funding scheme – Programme Development Grants “to increase the rate and number of successful applications for a full Programme Grant”.
  • Full stage Programme Grants support applied health researchincluding health services research; public health research; behavioural research; economic evaluations; and modelling.

Mental health research is highly represented in funded projects, and recent reports indicate this will only increase.

Much of the question and answer session focused on the electronic submission system which NIHR have decided to use: CC GrantTracker, also used by charities such as the Leverhulme Trust. Here are some comments and tips on the system:

  • all co-applicants must approve their own involvement before bid can be submitted
  • all schemes are a two-stage process apart from RfPB
  • NHS service support costs, treatment costs and excess treatment costs are important
  • however, many present said that trusts don’t have a clear idea on overheads and service support costs

Session 205: Cognisable Caramboles: making your research and impacts clear

Disappointingly, the presenters didn’t really explain the title. However, this was a useful and engaging session on clarifying research and impact. It was defined as the ability to describe clearly what you do to the general public, patients and service users, and also policy-makers and politicians: “it all boils down to the conversation in the lift”.

The session focused primarily on writing a clear and coherent lay summary, which is a common section seen in most funding applications, but often left to last or copied/pasted from other sections of the bid. However, this may be the section that gives the reviewer a good (or otherwise) first impression of your project, therefore it’s very important to get it right. They’re also important because they can be used by researchers in different disciplines to find partners; research managers when reporting on grants; press offices etc.

There was a fascinating overview of different definitions of “lay” summary from different funders:

  • some pitch it at the level of a tabloid
  • some say “an interested 14 year old, pre-GCSE science”
  • some explain you should write for the “science pages of a major broadsheet”
  • NIHR expect lay summaries to be understandable to patient groups

This was followed by more tips on making your lay summaries clear:

  • Analogies can be useful
  • Titles with a full colon are more understandable
  • Think about the audience: e.g. will non-English speakers understand idioms such as “the lion’s share” or double-negatives “it is not inconceivable”?
  • Consider the layout and paragraph order – sometimes just consider separating the text into paragraphs!
  • Give to someone else to read and then watch their eyes – if they have to scan over part of it more than once then it could be difficult to understand

Session 305: Choosing and implementing an RMAS (Research Management and Administration System)

Fair warning: if you’re not a research administrator – or someone interested in the process of research management – this section will probably not be interesting to you. I attended because Lincoln, like many other universities, has been considering whether to invest in an electronic RMAS and which to choose.

Two different approaches were showcased: Newcastle developed their own in-house system (MyImpact), while Hull invested in an “off-the-shelf” RMAS (CONVERIS).

Jonathan Cant (Hull) presented some useful tips from his experience of buying in an RMAS system:

  • First define what you have at the moment
  • Then decided what price is worth paying and what savings are likely to be made: this is critical to making a business case to persuade your university to invest
  • Set up a project team with clear terms of reference – for an “off-the-shelf” system, there will likely be a 2 year cradle-to-grave project timescale
  • Identify your key drivers: e.g. monitoring, planning, reporting, REF, communication, enhancing visibility, facilitating collaboration
  • Set up a tender process
  • Watch out for project creep/scope creep
  • Roll out a “soft launch” first, then go to formal launch
  • Never assume anything, particularly when setting up the system

Jill Golightley’s (Newcastle) experience was rather different as they weren’t starting from scratch and had already partially developed a system called MyProfiles. The main learning experience Jill shared was around timescale slippage: at the start the project was 2 years, but it has so far taken 3 years and is still not fully developed. However, because of the existing system it was still felt to be the best option at the time.

Plenary 3: Ewart Woolridge – Building Leadership Capacity

Woolridge talked about the “emergent change” in the HE funding system: at the time there was no HE White Paper, and yet the funding system had already changed in terms of increased fees and shifting debt to students rather than the state. He used this uncertain state to emphasise the changing relationships within institutions. He distinguished between a couple of frameworks for staff working within HE:

  • ‘bounded’ professionals: where different roles have a specific function and territory (e.g. Finance and HR often see themselves in this way)
  • ‘unbounded’ / borderless professionals: where there is a shared space between professionals and academics (Research Support sometimes acts in this way)

Crossing borders and sectors is becoming increasingly important within HE and requires customised training. A paradox of HE is that it is at once both conservative and radical. All staff within HE, including research support staff, need an understanding of the wider context of HE.

Plenary 4: Ian Carter – Building a Professional Development Framework

Research management and administration (RMA) is maturing/has matured. We need a whole life-cycle approach when approaching research support – from conceptualisation to commercialisation, knowledge creation to knowledge appreciation.

RMA has been recognised as a career pathway. ARMA has partially driven this and has responded by consulting on a Professional Development Framework for RMAs. This is likely to provide accreditation a professional (not degree) level.

The intention of the Development Framework is to create “flexible professional communities” – the skills promoted will be useful across many jobs and disciplines, both within HE and elsewhere.

Session 406: The Repository Rollercoaster – JISC RePosit

Ellie James from the University of Keele, representing the JISC-funded RePosit project, explained the benefits of an institutional repository and explored some of the challenges of advocacy.

In short, depositing work on a repository can:

  • help to achieve impact for research by making research more widely known
  • promote collaboration within and between disciplines
  • act as a “shop window” for an institution’s research output, and even for an individual researcher
  • allow quick generation of web profiles and CVs, particularly when used in tandem with a Current Research Information System (CRIS)
  • enables researchers to meet funder guidelines

Keele had a publications database (part of their CRIS) before they adopted a repository, so there was an additional challenge in getting the two systems to work together. The biggest challenge though, was around encouraging academic staff to deposit their work. The only way to address this was found to be constant advocacy and promotion. An institutional mandate is also helpful, although this alone will not ensure deposit.

Session 503: Hand in hand up the Pathway: professionals working together

This session explored the differences between Knowledge Exchange and Research support. All universities structure their support services differently: some, as in Lincoln’s case, are joined together in a single department, with different staff taking responsibility for KE and Research; others delegate these functions to Schools and Faculties; still others have joined the two into a number of single roles which provide discipline-specific support.

There is very little research available to show which is the most effective approach, so what usually informs a particular institutional decision to choose one or the other system is what’s worked (or not worked) in the past. There was more of a “workshop” feel to this session (highlighted at least by the lack of notes I took), and we took part in a useful exercise to group typical KE and Research support functions around specific calls for proposals. So, for example, we thought about how KE and Research could support an application to the TSB as distinct from a Leverhulme Fellowship.

The increasingly prominent place of impact means that these two professional support areas are having to work more closely than ever before. Perhaps the key learning point was that a good relationship between these service areas is essential.

AHRC Demand Management

A few weeks ago I attended an ARMA/AHRC Study tour at Polaris House, the main home of the UK Research Councils,  in Swindon. Unlike the usual ARMA Study tours, which are intended primarily as an opportunity for research support staff from universities to increase their understanding of how funding bodies operate and to discuss topics of common interest, this Study Tour took the form of a Special Event and focused on ‘demand management.’

Demand management measures were agreed by all Research Councils as part of the recent Spending Review negotiations as a way of reducing wasted cost and effort in the system.

Research Councils have agreed to:

  • work in partnership with research organisations such that they self-manage demand and quality control;
  • use quality, impact/added-value and, where appropriate, Research Council strategy in delivering quality decision making;
  • consolidate and simplify/streamline funding schemes where possible;
  • maintain a range of funding models to deliver the objectives of each Research Council (e.g. for core business, capacity building, translational research etc);
  • use sift / triage processes robustly to reduce the burden on the peer review system;
  • share good practice and strive for continual improvement;
  • remain sensitive to the challenges of reviewing and supporting multidisciplinary and collaborative research;
  • discuss plans for demand management with stakeholders;
  • develop and share tools for demand management across Research Councils and research organisations;
  • maintain awareness of the effect of demand management on the wider community and relevant stakeholders.

The implementation of demand management, to comprise of a combination of new approaches to funding, good practice in HEIs and sanctions if necessary, is intended to be collaborative and led by Research Councils in partnership with Research Organisations. However each council will be adopting their own approach to reflect the differences in practices at each of the Research Councils. The ESRC’s approach to demand management has already been discussed on this blog and they will be deciding later this year whether to introduce sanctions. The EPSRC have gone down the route of individual sanctions, however the AHRC do not see the same patterns in submissions so this approach would not be appropriate. The AHRC have not ruled out implementing sanctions  however it is anticipated these will be institutional rather than individual.

Research Councils currently receive far more high quality applications than they are able to support and this is reflected in low success rates as Research Councils try to balance demand against available resources. There is also a concern that the high volume of proposals is placing an increasing burden on peer reviewers which, should that burden continue to grow, could have a negative affect on the quality of decisions being made.

Whilst Peer Review is designed to sift out the successful from the unsuccessful, a large number of proposals are submitted which have little chance of success. If these proposals could be removed from the system before they reach the Research Councils through a pre-submission ‘triage’ of applications the burden will be reduced and success rates should increase. Whilst Research Councils do have an internal sift or triage system to remove the proposals which are out of scope or ineligible, they want this process to move to the submitting institutions and are encouraging self-management of demand and quality control.

The AHRC’s current, and preferred, approach is to develop good practice guidelines for Research Organisations specifically in the operation of internal peer review and an internal institutional triage system whereby only the best applications are submitted to RC’s. About 20 different institutions were represented at the event and whilst there were differences in approach to peer review – ranging from processes whereby each application had to be reviewed by at least two internal academics and one external if the grant was significantly large, to a more informal expectation that proposals should be reviewed before submission to the Research Office for institutional approval – it was clear that no institution had a clearly defined approach to demand management. Approaches to mentoring of Early Career Researchers also varied widely across the sector, despite being seen as a vital part of their development. Whilst it is clear that these processes will become even more important to all institutions it was also recognised that measures to imbed demand management within institutions will take time and involve a cultural shift in the way researchers and institutions work.

Like all other Research Councils, the AHRC’s approach to demand management will also use more targeted schemes to include longer and larger awards with greater use of the Expression of Interest phase, and an increase in the number of ‘sandpit’ style workshops to limit the number of applications on specific schemes to those which have been invited. An internal review will also consider whether resubmission rates are high and consider stricter triage of applications prior to panel. It was stressed that sanctions would only be introduced as a last resort however they will be monitoring success rates as the basis for strategic discussions with Research Organisations and introducing sanctions if deemed necessary.

Tweeting #arma11

This week I and a couple of colleagues from Lincoln attended the 2011 Association of Research Managers and Administrators conference in a rather damp but very lively Glasgow. The theme of the conference was “Good Advice in Hard Times” and we’ll be posting some notes, analysis and reflections on the sessions we attended soon.

If you just can’t wait until then you can get some idea of  what went on at the conference by searching for the #arma11 hashtag on Twitter.

Unfortunately only a handful of people were tweeting at the conference – from over 400 participants – which means you won’t get the full range of ideas and opinions expressed at the event.

I think the lack of social media engagement at ARMA conferences is a real shame, especially given the wide sector representation at the event (virtually every UK HEI had at least one representative there) and the importance of the topics being discussed to higher education and beyond. Many other conferences in HE have a lively and active Twitter community which can stimulate debate and engagement in addition to the IRL chats over coffee. I met a few people at this conference who I may otherwise not have spoken to because we linked up on Twitter during the first day. What’s more, Twitter enables you to quickly and easily continue conversations once the conference is over.

A social media presence is something I hope can be promoted and encouraged more proactively at next year’s conference. In fact, I plan to email the ARMA conference committee to suggest just that.