• 1 Public research grants in the United Kingdom are distributed according to the ‘dual fund (...)

1In the United Kingdom, academic research is assessed every five to six or seven years through Research Assessment Exercises (RAEs). The first exercise was launched under Margaret Thatcher in 1986 as part of a new policy of quality-related funding linked to reduce public spending. It was managed by the University Grants Committee, replaced by the funding councils in 1992. The following exercises took place in 1992, 1996, 2001, 2008 and the next one, the Research Excellence Framework (REF), took place in 2014. Since 1986, the results of the RAEs have been used as a basis for calculating the recurrent research grants allocated each year by the funding councils as part of the global higher education public budget.1 The table below shows the mainstream quality-related (QR) research grants distributed by the two main funding councils, the Higher Funding Council for England (HEFCE) and the Scottish Funding Council (SFC), from 2011-12 to 2013-14 :

• 2  ˂http://www.sfc.ac.uk/funding/Fundingdecisions/FundingDecisions.aspx˃; ˂http://www.hefce.ac.uk/wha (...)

Source : Scottish Funding Council ; Higher Education Funding Council for England2

• 3 Lisa Lucas, The Research Game in Academic Life, Maidenhead: Open University Press & Socie (...)
• 4 Whereas teaching grants have been cut dramatically over the last three years, with the fu (...)
• 5 The analysis of the 2008 RAE and the 2014 REF results are beyond the scope of this (...)

2Even if the global higher education budget has dwindled since the late 1980s, the financial and symbolic impact of the RAEs is still high enough for British higher education institutions to “play the research game”,3 all the more so as research development is now at the heart of governmental priorities.4 This paper explores i) the extent to which research assessment campaigns can be said to reflect the State’s vision of the role and purpose of academic research in society, ii) the way RAE/REF research quality definitions and criteria have contributed to promote certain forms of research at the expense of others and influenced research practice accordingly.5

• 6 Louise Morley, Theorising Quality in Higher Education, London: Institute of Education, Un (...)

3The assessment criteria and frameworks defined for the purpose of the RAEs offer an illuminating perspective for examining the evolution of public funding policies in the United Kingdom. Indeed, the definition of quality in a particular field such as higher education can be said to reflect a certain number of values and power relations embedded in that field, which is the idea put forward by Louise Morley in her endeavour to “theorise quality in higher education”.6 She convincingly demonstrates that quality mechanisms in higher education, including institutional quality assessment campaigns such as the RAEs, can be regarded as a “political technology” i.e. a set of “structures and systems put in place to facilitate political ideologies” :

• 8 Ibid., 26.
• 9 “Academic capitalism” refers to “the way public research universities [respond] to neolib (...)

4In keeping with this approach, evaluative standards and criteria can be said to “represent an encodement of values and priorities”8 gradually imposed upon the academic world. Drawing on this theoretical framework which illuminates the context of British higher education, this paper analyses the extent to which RAE/REF definitions, working methods and criteria encapsulate public funding strategies in the United Kingdom. The 2008 RAE and the 2014 REF are most particularly examined as they represented a new stage in institutional research assessment, following the Roberts Review held in 2003 as well as the subsequent debates on the systematic introduction of metrics in 2014. A new criterion introduced in the REF, research impact, is a case in point : its institutional definition and scope illustrate the idea that taxpayer money should be invested only in research areas and research types that contribute significantly to national wealth and welfare, a form of “academic capitalism”9 which this paper analyses in the light of the British national political and cultural history.

• 10 Suzy Halimi, L’enseignement supérieur au Royaume-Uni, Paris : Ophrys-Ploton, 2004, 57-64 (...)
• 11 Suzy Halimi, op. cit., 111 ; Marie Scot, La London School of Economics and Political Sci (...)

5British higher education institutions are so diverse and have so many different historical backgrounds10 that it is difficult to speak of a unified higher education sector in the United Kingdom, even less so of a British higher education model. Nonetheless, ever since the ancient universities’ liberal model of education was challenged in the second half of the 19^th century, there has been a long tradition of cooperation between British universities on the one hand, and business and industry on the other,11 a tradition which, as this paper aims to show, has been drawn upon by governmental higher education authorities over the last decades to promote a certain idea of research worth.

• 12 Mary Scot, op. cit.
• 13 Committee on Higher Education, Higher Education, London: Her Majesty’s Stationary Office (...)
• 14 Inter alia, Sheldon Rothblatt (1976; 2000), Suzy Halimi (2004) and, more recently, Marie (...)
• 15 Quoted by Stefan Collini, "From Robbins to McKinsey", London Review of Books, (...)
• 16 Student numbers doubled between 1979 and 1996 (from 777,800 to 1,659,400), the (...)
• 17  ˂https://www.gov.uk/government/organisations/department-for-business-innovation-skills˃, (...)
• 18 In 2011-2012 for instance, part-time students represented more than 30% of the global st (...)

6In the late 19^th century, the first red-brick universities deliberately distanced themselves from the Oxbridge model by providing a more vocational education, in line with the scientific and technological developments of the time. In the same vein, a “new” university like the London School of Economics tried very early on to meet social and economic demands while developing a strong research agenda, thus putting forward a new, albeit much debated, model of higher education.12 In the second half of the 20^th century, the publication of the Robbins report in 196313 and the unification of the British higher education sector through the 1992 Further and Higher Education Act were decisive steps in the efforts to widen the scope of higher education and to draw it closer to society’s needs, as many historians of British higher education have shown.14 By stating that “courses of higher education should be available for all those who are qualified by ability and attainment to pursue them and who wish to do so”, Lord Robbins15 thus paved the way for a long-term policy of widening participation which resulted both in a bigger and more diverse population of students.16 The 1992 Further and Higher Education Act went one step further towards the widening of higher education by integrating vocational institutions into the select circle of universities. The subsequent higher education policies gradually confirmed the wish of political authorities to make sure that universities “produce highly skilled graduates and post graduates”,17 expecting them to meet the social and economic demands of their times and play their part in the economic development of the nation: the increasing number and variety of vocational short-term degrees like Foundation Degrees offered by higher education institutions as well as the rising proportion of part-time (and often mature) students in higher education18 are indicators of the efforts made by many British universities to meet these expectations and widen their scope. The following section focuses on two specific examples of the strong ties higher education institutions have developed with business and industry in the United Kingdom.

• 19 General Medical Council 2009, Tomorrow’s Doctors. Outcomes and standards f (...)
• 20 The involvement of PSRBs in universities varies from one discipline to the other (they (...)
• 21 For further analysis of the different regulatory and assessment mechanisms (...)

7Universities in the United Kingdom cooperate closely with the economic sector through the accreditation of their academic programmes by Professional, Statutory and Regulatory Bodies (PSRBs) i.e. the bodies in charge of supervising and regulating different professions and occupations in the country. For instance, in the field of medicine, the General Medical Council closely monitors medical studies and controls the registration of doctors in the United Kingdom: it checks regularly that the professional standards set out in its framework, Tomorrow’s doctors. Outcomes and standards for undergraduate medical education,19 are met by the medical schools and their undergraduate students. Similarly, the Institute of Physics (2010) has set standards in the field of physics by publishing The Physics Degree. Graduate Skills Base and the Core of Physics, a document which defines the knowledge and skills that British graduates in physics should possess. Most British universities work closely with PSRBs : accreditation appears as a mechanism which allows professional bodies to confirm that academic programmes meet the expected professional standards in their field of activity.20 Conversely, it is used by universities as a quality label to promote their programmes, as the almost systematic inclusion of PSRB logos in undergraduate and postgraduate brochures shows. Overall, even if academic and professional expectations sometimes clash, especially when academics refuse to comply with what they perceive as the unacceptable dictates of the professional bodies (which is not so frequent as PSRB members and faculty often interact, beyond the university walls), the British mechanism of accreditation works as a sort of external regulation of higher education standards that seems to satisfy all parties overall.21

• 22 REF, 2010, “Research Excellence Framework impact pilot exercise. Example case (...)

8Knowledge transfer is yet another example of the active links that British universities have developed with society: the term designates mutually beneficial collaborations between businesses and academic researchers. Initiated through the Labour governments’ Knowledge Transfer Policy in the late 1990s, it was put forward as a way of developing the economic potential of academic research, thus dynamizing business and industry through innovation. It has now become a third stream activity for universities today, beyond teaching and research. Knowledge transfer is implemented in various ways. For instance, Knowledge Transfer Partnerships (KTPs) are publicly-funded projects encouraging universities to transfer knowledge out to businesses and other organisations. They are often found in applied scientific fields like medicine, biology or even the social sciences, but some are also set up in fields where such initiatives are less common. In the humanities for instance, a KTP was set up between the history department at Kingston University and Historic Royal Palaces, a heritage institution in charge of managing non-residential Royal palaces in England. The aim was to devise a new program for visitors at Hampton Court Palace by drawing on cultural research into the life of Henry VIII. Entitled “Henry VIII: Heads and Hearts”, it included “new live interpretation, refurbished interiors, interactive displays, and multimedia elements, all inviting visitors to imagine themselves as courtiers attending the wedding of Henry VIII to Kateryn Parr at the Palace in 1543”.22

• 23 Ambassade de France au Royaume-Uni, Service Science et Technologie, “University (...)

9Similarly, spin-out (also called spin-off) companies allow some universities to develop outlets for the commercial exploitation of their research results. They have become part and parcel of universities’ activities, especially in the biggest research-led universities. The universities of Oxford and Cambridge, among others, have created their own spin-outs, in the form of knowledge transfer offices : ISIS Innovation Ltd was created by Sir David Cooksey as early as 1988 at Oxford University and Cambridge Enterprise Ltd was born in 2006 from earlier initiatives to support technology and knowledge transfer within the University of Cambridge. Both offices manage their universities’ intellectual property portfolios, from initial “seed” funding to R&D or legal and management support so as to enhance the social and financial benefits of the scientific research led by their researchers. They also rely on sophisticated networks of venture capital companies and business angels consortia to make sure that the initial support they have provided is sustained through more permanent forms of commercial development.23 University spin-outs thus exemplify yet another path higher education institutions have followed to extend their scope beyond academia.

• 24 Guardian (The). “Mandelson to set out role for universities in business super-ministry”, (...)
• 25 For an in-depth analysis of the British neoliberal approach to higher educatio (...)
• 26  Lisa Lucas, op. cit., 14.
• 27 Louise Morley, op. cit., 2.

10The idea that universities should contribute to the economic growth of the nation has been particularly actively promoted in the British political arena, most strikingly since 2009 when higher education was included in a new super-ministry – Business Innovation and Skills (BIS) – specifically aimed at fighting the effects of the 2008 crisis. The inclusion was interpreted at the time as “a move [which] placed universities and colleges at the heart of the government’s plans for economic recovery, sparking fears that their cultural and educational roles would be sidelined in the biggest department in Whitehall”.24 In other words, even though British universities’ ties with business and industry have gradually become a permanent feature of higher education in the United Kingdom, the wish of the various governments over the last decades has obviously been to draw them even closer to one another by i) defining politically the needs of the market25 through national priorities such as Science, Technology, Engineering and Mathematics (STEM) ii) making sure that higher education meets such needs through various programmes and financial incentives. One central tool has been to create a “pseudo-market system of funding”26 through research assessment exercises, illustrating the idea that “[q]uality combines with funding to provide a powerful metanarrative”.27

• 28  The Scottish Funding Council (SFC) ; the Higher Education Funding Council for Wales (...)
• 29 Lisa Lucas, op. cit., 31.
• 30 Louise Morley, op. cit., 2 ; 26.

11Following the principle according to which “he who has the gold has the rule”, the selective funding of research by the State, implemented since 1986 through Research Assessment Exercises (RAEs) and managed by the Higher Education Funding Council for England (HEFCE) for its Welsh, Scottish and Northern Ireland counterparts,28 has proved a very powerful tool for promoting a cost-effective, impact-driven research model whose purpose is to contribute to the economic growth of the nation, reflecting an “emphasis on harnessing higher education more tightly to economic concerns and demanding greater efficiency with fewer resources”.29 Drawing on L. Morley’s theory that quality-driven policies and their related institutional assessment mechanisms can be interpreted as “a regime and relay of power” translating certain “values and priorities”,30 the generic criteria, definitions and funding formulae of the various RAEs can be said to encapsulate certain ideals and standards imposed gradually upon the academic world by the funding councils. Even if they draw on large consultations with the academic community and rely on academic peer-review as the main mechanism, the RAEs still convey a certain message as to what public research should be for by concentrating public funding on certain types of research only. Their mechanisms and definitions illuminate their rationale, with the latest 2008 campaign as the main illustration.

• 31 About RAEs across the years, see M. Henkel, “The Modernisation of Research Evaluation  (...)
• 32 RAE, “Guidance on Submissions”, 2005, 35-36, <http://www.rae.ac.uk/pubs/2005/03/rae030 (...)
• 33 ˂http://www.rae.ac.uk/news/2004/fairer.htm˃, accessed 07/10/2013.

12The definitions, criteria and working methods used for the RAEs have evolved from one exercise to the next,31 yet the founding principle has always been that selective funding by the State should be based on research quality – defined by the funding councils for funding purposes as we shall see hereafter – and research volume – namely the number of researchers considered as “research active staff” by RAE standards, with major implications for academia as we shall see. Significant changes were introduced in 2008, after an official review was conducted by Sir Gareth Roberts and published as the “Roberts report” in 2003. First, a two-tier assessment system was introduced : in the 2008 RAE, reviewers were organized in 67 disciplinary subpanels (corresponding to 67 units of assessments or UOAs), supervised by 15 main panels32 : they mainly comprised academics but also included some non-academic members categorized as “research users”, i.e. most often members of business, industry or third sector organizations (both public and private). Second, the panels’ task was to draw an overall quality profile for each submitted file, a change specifically introduced so as to provide a higher degree of discrimination and a more accurate picture of research quality than the grades used in the previous RAEs. In Sir Howard Newby (then Chief Executive of HEFCE)’s own words :33

• 34  RAE 2005, op. cit., 34.
• 35 Most of these criteria for research quality are used in other research ass (...)
• 36 RAE 2005, op. cit., 32.

13For the purpose of the RAE, research was generically defined as “original investigation undertaken in order to gain knowledge and understanding”,34 a definition which was directly borrowed from the 2001 RAE (with minor syntactic changes). The definition aimed at allowing reviewers to identify the research elements in academic work unambiguously, thereby excluding other parts of scientific work which were not judged to contribute directly to the advancement of knowledge and understanding. Moreover, three dimensions of research were taken into account : research outputs (i.e. books, articles, patents, software, performances, or any other form of assessable output); research environment (i.e. both the infrastructures and the strategic policies implemented by departments to support and develop research); the third and last dimension was esteem indicators (i.e. elements showing researchers’ prestige and recognition mainly inside but also outside academia).35 Each of these three dimensions was awarded a grade from 1* to 4*, and a formula based on the weightings given to the different dimensions36 allowed each panel to define a general quality profile such as the one shown below, which is the profile for law (UoA 38) at the University of Bristol.

• 37 ˂http://www.rae.ac.uk/Results/qualityProfile.aspx?id=110&type=hei˃, accessed 07/10/201 (...)

Source37: RAE 2008. “Quality profiles. University of Bristol”.

• 38 e.g. RAE 2006g, “Panel Criteria and Working Methods. Panel G”, §32, <http://www.rae.ac.UK (...)
• 39 e.g. RAE 2006o, “Panel Criteria and Working Methods. Panel O”, §25, <http://www.rae.ac.Un (...)
• 40 e.g. RAE 2006f , “Panel Criteria and Working Methods. Panel F”, §26, <http://www.rae.ac.U (...)
• 41 e.g. RAE 2006g, op. cit., §11.
• 42 RAE 2006i, “Panel Criteria and Working Methods. Panel I”, §18, <http://www.rae.ac.UK/pubs (...)
• 43 e.g. RAE 2006h, “Panel Criteria and Working Methods. Panel H”, §18, <http://www.rae.ac.UK (...)
• 44  e.g. RAE 2006e, “Panel Criteria and Working Methods. Panel E”, §30, <http://www.rae.ac.u (...)
• 45  e.g. RAE 2006j, “Panel Criteria and Working Methods. Panel J”, §9, <http://www.rae.ac.United Kingdom/pubs/2006/01/docs/j.pdf>, </http> (...)
• 46  e.g. RAE 2006h, op. cit., §19.

15These generic criteria were interpreted or developed from a specific disciplinary perspective by the various panels who produced their own framework documents, entitled Panel Criteria and Working Methods. The study of these documents shows that overall, there was a consensus about the nature of academic research, with interpretations and developments focusing on specific dimensions of disciplinary culture and practice. For instance, some panels went beyond the generic definition of research and found it necessary to provide definitions for other types of research practiced in their fields, namely applied research,38 practice-based research39 or interdisciplinary research.40 Similarly, some panels provided their own interpretation or development of the three dimensions of research taken into account: some of them drew a full list of the types of research outputs they expected to find in the submissions41 while others indicated that they would not “regard any form of output as necessarily being of higher or lower quality than others”.42 Research environment was mostly understood as encompassing the infrastructures and equipment, professional development strategies or investment policies in the various research departments.43 Yet, some panels insisted on more specific elements like the ability for a team to devise a long-term strategy44 or to show that the targets set for the previous RAE had been met,45 therefore adopting either a prospective or a retrospective approach of the departments’ strategies. Human resources policies, particularly those targeting young researchers, were frequently presented as a key criterion for assessing the quality of the research environment. Last but not least, esteem indicators were mainly interpreted as the recognition of researchers’ excellence by the scientific community – as indicated for instance by the award of prestigious prizes, visiting professor statuses or fellowships in the United Kingdom and abroad – but also by society, through official expert status for governmental authorities or international organizations for instance.46 Most panels underlined the fact that they would not use metrics to assess scientific esteem, anticipating on the debates which were already taking place at the time about the massive introduction of metrics in the following assessment exercise.

• 47 RAE 2006g, op. cit., §29.
• 48 Inside most of the main panels, the subpanels adopted the same weightings across the boa (...)

16Beyond their interpretation of the generic definitions and criteria, the panels were allowed to choose from a range of weightings to reflect the relative importance of each of the defined dimensions of research in their disciplinary field: research output had to represent at least 50% of the overall profile, yet most panels decided to give it 70 to 80%, thus insisting on the importance of research output as the main dimension for assessing research worth. Panel G (engineering) was the only one to lower its weight to the minimum i.e. 50% while granting an exceptional 30% to esteem indicators (in which research income was included) to take into account the “importance of obtaining income to carry out significant amounts of engineering research [and] the intense competition to obtain such resources […]”.47 Apart from this isolated example though, the weightings for research environment and research esteem did not vary much and accounted respectively for 15% to 20% and 5% to 10% of the overall quality profile on average, as the table below shows:48

• 49 Ron Johnston 2008, op. cit., 131.

17Overall, the 2008 RAE panels were given sufficient leeway to adapt the generic framework to the specificity of research practice and research culture within their own disciplinary fields. Even if the level of interpretation was fairly limited in the end and amounted to little more than “one set of relatively vague descriptors [rewritten] into another”,49 these panel criteria and working methods still hold sway in academia.

• 50 Lisa Lucas 2006, op. cit., 80.
• 51 Inter alia, the socio-ethnographic survey led by Lisa Lucas’s survey in 2 univ (...)

18The financial and symbolic stakes attached to the RAEs are so high that most British universities and their academics devise strategies to get the best possible results in as many fields as possible.50 Different studies and surveys51 have shown that the generic definition of research and research quality as well as the disciplinary interpretations and developments suggested by the disciplinary review panels directly influence the research strategies that universities devise in order to maximize their gains and the excellence standards that researchers try to meet in various disciplinary fields :

• 53  One major impact, which is not developed in this paper, was on academics whose research (...)
• 54  The definitions of the highest quality levels in the previous RAEs were also b (...)
• 55  The research grants allocated to British universities in 2009-2010 by the different fund (...)
• 56 RAE 2005, op. cit., 31.

19Even if the influence varies from one disciplinary field to the other, some trends in RAE impact upon researchers’ practice have been identified.53 First, researchers were gradually encouraged to increase the international visibility of their work by specializing more and widening the international scope of their research. In keeping with the 2008 RAE quality scale,54 the three highest – and only funded55 – levels, from 2* to 4*, were characterized by the international scope of research, whether it was considered to be “world-leading”, “internationally excellent” or “internationally recognised”.56 One way of achieving this was for researchers to turn to more specialized – hence more easily identifiable – subject areas, raising concerns about their professional identity. In sociology for instance, Lucas reports that

• 58 Ibid., 105.

20Researchers were also led to raise the international profile of their work by attending international conferences or submitting papers to journals of high international standing, at the expense of other activities sometimes.58 In some cases, researchers were even excluded from their department’s submission because their international profile was deemed too low.

• 59 Ibid., 45.
• 60 Technopolis 2010, “Research Excellence Framework impact pilot exercise. Lesson (...)
• 61 John Halsey and Kenneth O’Brien, op. cit., 35.

22Ever since the RAEs were introduced, “the intrinsic creative value of research for academics has been replaced by a pressure to maximize the RAE value of their research”.59 As a consequence, profound behavioural changes in research practice have been brought about.60 Indeed, RAEs have worked as a powerful driver of research practices by deliberately valuing certain dimensions – short-term impact, measurable outcomes, international collaborations, etc. – at the expense of others. Funding councils’ policies, as translated through RAE quality criteria tied with funding formulae, have combined with research councils’ priorities to allow the state to favor certain forms and certain areas of research, thus illustrating what can be considered as a British form of neoliberalism in which the State plays an active role by “creating the appropriate market by providing the conditions, laws and institutions necessary for its operation”.61

• 62 REF 2011b, “Assessment Framework and Guidance on Submissions”, §14b, ˂http://www.ref.a (...)

24In many respects, the REF represents a major shift in the way publicly funded research is considered by governmental authorities. One of the introductory paragraphs in the “Assessment framework and guidance on submissions” reads : “The assessment provides accountability for public investment in research and produces evidence of the benefits of this investment”.62 In other words, the REF exemplifies a new stage in the move away from the public funding of research to the idea of public investment in research, implying that some kind of return on investment will be expected at some point by the investors, hence the introduction of “research impact” as a new criterion. Furthermore, the very name of the new campaign – it is no longer an exercise but a framework – as well as several changes in the mechanics of assessment show that the aim is for the State to “frame” or monitor academic research more closely as well as to standardize the assessment system as much as possible. REF guidelines, definitions and criteria provide an interesting insight into the rationale for these changes.

• 63  REF 2011b, op. cit., 48.

25First, the new definition of research for the purpose of the REF reflects a paradigm-shift in the way research is perceived and defined from a governmental perspective: it is indeed understood as “a process of investigation leading to new insights, effectively shared”.63 When compared with the definition of research in the two previous RAEs – “original investigation undertaken in order to gain knowledge and understanding” – a move away from universal scientific demands, namely the pursuit of knowledge and understanding, to outcome-driven types of research – research worth being equated with whether or not it is “effectively shared” outside academia – can be noticed. The underlying rationale is not whether researchers contribute to the advancement of knowledge – which is no longer presented as an end in itself – but whether they communicate and apply actively whatever progress they have made or insight they have gained in their research work, an approach confirmed by Sir Andrew Witty’s call for a “British invention revolution” in his 2013 “review of universities and growth” :

• 65 The decisions for the REF were made by the Higher Education Funding Council for England (...)
• 66  REF 2011a, “Decisions on assessing research impact”, §2c, 2d. ˂http://www.ref.ac.uk/medi (...)
• 67 REF 2011b, op. cit., §28.
• 68 The list of main panels and sub-panels for the REF can be found in annex B.

27Second, the leeway given to panels and sub-panels in the previous exercises in order to take disciplinary specificities into account was reduced significantly in that: the weightings for each research dimension were set65 generically for all units of assessment, namely 65% for research outputs, 20% for research impact and 15% for research environment;66 each of the four main panels provided “a single, consistent set of criteria [to be] applied by its group of sub-panels”, with very few sub-panel variations, as expected in the “Guidance document”:67 the number of main panels and sub-panels was reduced significantly, from 15 main panels to 4 and 67 sub-panels to 36,68 leading to even greater homogenization across disciplines. The new framework was thus characterized overall by greater standardization, as stated clearly in one of the very first paragraphs of the “Assessment Framework and Guidance on Submissions” (bold characters mine) :

• 70 Stefan Collini, 2012, op. cit., 107.

28The standardization obviously translates a wish by governmental authorities to subject research practice to non-academic regulation, which comes down to “translat[ing] complex and elusive human achievements into some kind of measurable ‘data’ that are comprehensible to a non-expert public”.70

• 72 ˂http://www.ref.ac.uk/panels/panelmembership/˃, accessed 14/10/2013.
• 73  The impact of research was already part of the 2008 RAE but only indirectly an (...)

30The list of panel and sub-panel members published in 2013 by the HEFCE72 shows that research users were included to a larger extent indeed: in the field of engineering for instance, main panel G in the 2008 RAE comprised no research users from the industry at all whereas main panel B in the REF, which included engineering (but also earth systems and environmental sciences, chemistry, physics, mathematical sciences and computer science and informatics), comprised four research users out of 15 panel members. The increased proportion of research users on the review panels stemmed directly from the definition of research impact for the purpose of the REF.73

• 74 REF 2011b, op. cit., 48.
• 75 REF 2011a, op. cit., §2c.
• 76 REF 2011b, op. cit., 44.
• 77 Technopolis 2010, op. cit., 6.
• 78 REF 2011b, op. cit., annex G.
• 79  The pilot exercise was organized in 2009-10 so as to test the feasibility of assessing (...)

31In the REF, impact was exclusively understood as impact outside the academic field and was defined broadly as “effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia”.74 The socio-economic impact of research was thus systematically and fully assessed for the first time and accounted for 20% of the overall assessment, a figure that is due to increase in the future.75 It was measured through its “reach and significance” according to a quality scale comprising five levels, from unclassified to 4*.76 As many academics seemingly viewed REF proposals “with a mixture of anxiety and antagonism”,77 HEFCE published various documents ahead of the 2014 exercise to provide extra-explanations for higher education institutions: a detailed “Impact case study template and guidance” was included in the final guidance document,78 and a wealth of case studies was provided through the pilot exercise report published in 2010.79

• 80 REF 2011b, op. cit., 48.

32As its official definition indicates,80 the scope of research impact is wide enough to cover a variety of applications :

• 81 Jeremy Bentham, An Introduction to the Principles of Morals and Legislation, O (...)
• 82 Technopolis 2010, op. cit., 2.
• 83 Ibid., 16.
• 84 Ibid., 1.
• 85 Stefan Collini 2012, op. cit., 120.

34Therefore, it goes beyond immediate economic impact and it could even be said to cover the benthamian concept of “utility”, namely “the property in any object, whereby it tends to produce benefit, advantage, pleasure, good, or happiness”.81 The impact pilot exercise report showed that the concept itself had not been particularly discussed, yet the methodology for illustrating research impact raised concern: “the biggest challenge was the need to acquire evidence of the reach and significance of a given impact”,82 all the more so as the “common menu of indicators” provided in the guidance mainly focused on “applied sciences and technological innovation leading to economic impacts”83 and was deemed “light on measures relevant to the social sciences and the humanities”.84 Indeed, if research impact can be considered as part and parcel of applied sciences, their results being often directly relevant and/or applicable to the needs of society, it is harder to identify, and even more difficult to dimension, in less applied subjects: as Stefan Collini put it, “[n]ot everything that counts can be counted”.85

• 86 Technopolis 2010, op. cit., 4.
• 87 Stefan Collini 2012, op. cit., 171.
• 88 The expression “beyond academia” included in the definition is developed three paragraph (...)
• 89 Stefan Collini 2012, op. cit., 164.
• 90 Technopolis 2010, op. cit., 28.

35The humanities are a case in point. The English departments participating in the pilot exercise did underline the fact that in their field, “impacts tended to be more conceptual than instrumental, so intrinsically difficult to convey and dimension”.86 This is confirmed by S. Collini who claims that research impact in the humanities is often diffuse and conceptual: he believes that in most cases, it can be described as “influence on the work of other scholars and influence on the content of teaching”,87 both of which have been excluded from the official definition of impact within the REF.88 Moreover, research in the humanities, as well as in other fields, is a form of “cumulative intellectual enquiry”,89 as Newton’s famous “standing on the shoulders of giants” metaphor powerfully reminds us. Some researchers participating in the impact pilot exercise thus “express[ed] unease at the idea of laying claim to social impacts that are almost certain to have arisen as a result of the efforts of many actors rather than one”.90

• 91 Martha C. Nussbaum, Not for profit : Why democracy needs the humanities, Princ (...)
• 92 Stefan Collini 2012, op. cit., 91.

36Overall, the various reactions triggered by the introduction of the socio-economic impact of research reflected centuries’ long debates over the very purpose of higher education, ranging from the ancient universities’ model of liberal education, still defended today by authors like S. Collini – whose latest book is tellingly entitled What are universities for ? – or Martha C. Nussbaum,91 to the neo-liberal vision conveyed by recent British governmental policies and their “economistic philistinism of insisting that the activities carried on in universities need to be justified, perhaps can only be justified, by demonstrating their contribution to the economy”.92 So, even if British academics can be said to have won a battle over the massive introduction of metrics in the REF, which were eventually turned down, they have certainly not won the war since the introduction of impact clearly shows that the State is striving to make universities contribute to economic growth and national welfare as directly and as efficiently as possible, a reflection of the current knowledge economy trends worldwide. Clearly, the definition of research as the open-ended search for deeper understanding, defended by some academics like Stefan Collini and undoubtedly many others, is not acceptable as such by current governmental standards.

• 93  Collini 2012, op. cit., 175.
• 94 Guy Neave, “Editorial”, European Journal of Education, 1987, Vol.22, N°2: 121- (...)

38First, the official definition of impact for the purpose of the REF implies that impact on academia – as assessed through research output and research environment – is not enough to prove research worth and that a necessary and sufficient condition for categorizing any other effect, change or benefit as worthy is that it should take place outside academia. In other words, it is assumed that “if the research in question can be shown to have affected a number of people who are categorized as ‘outside’, then it constitutes a social benefit of that research”.93 This rationale is highly debatable, to say the least, and it can also be interpreted as the translation of a profound mistrust in academia that finds its roots in the long power struggles that have opposed British universities to political authorities since the late 1980s.94 Research with no measurable or assessable benefits on society within the timescale of the assessment campaign will not be considered as quality research, therefore will not be financed by the State. Given the manifold, diffuse and long-run nature of research impact in many cases, universities are going to have to decide whether or not research that does not lead to short-term, measurable impacts – which is the case in the humanities for example but also in many fields where “blue skies” research is still considered as essential – is still worth funding or not, which might lead to further closures and layoffs in British universities, unless students’ demand for such subjects allows them to live on. The survival of some species of research is thus left to market forces.

• 95 REF 2011a, op. cit., §2c.
• 96  Research Councils UK 2012, Impact Report, ˂http://www.rcuk.ac.uk/Documents/publications/Impactrepo (...)

39Besides, the assessment of research impact in the REF has involved “research users” to a much greater extent than in the previous research assessment campaigns, down to the very definition of assessment criteria. As the weight given to impact in the assessment is due to increase in the future,95 the inclusion of non-academic assessors of research is also likely to become a permanent feature. The nature of research practice is undoubtedly going to be influenced by such a trend, away from more fundamental towards more applied types of research and short-term projects, all the more so as the research councils have also set up a "Pathways to Impact" policy encouraging researchers to consider the impact of their research throughout their research process.96 Along with the definition of impact “outside academia”, the inclusion of non-academics in greater proportions within review panels translates the idea that academic work is at its best when it engages with and is understood by non-specialist outsiders.

• 97 Technopolis 2010, op. cit., 3.
• 98  The Higher Education Statistics Agency (HESA) figures released in 2010 showed that the (...)

40Moreover, the collection and recording of evidence by higher education institutions to support their impact case studies has required the development of new information systems as well as specific administrative functions:97 non-academic communication experts have been called upon to write powerful and convincing research impact narratives, a move which could threaten an already fragile power balance between academic and non-academic staff within British universities in the long run.98

• 99 Stefan Collini, op. cit., 196.
• 100 Technopolis 2010, op. cit., 5.
• 101 REF 2011b, op. cit., 4.
• 102 Stefan Collini, op. cit., 196.
• 103 Ibid.

41Overall, the evolution of the definition of research and research quality in research assessment campaigns in the United Kingdom shows that British public policy has shifted from public funding to public investment in research, with funding bodies supporting research on the basis of cost-effectiveness and utility. What some perceive as an unacceptable level of “direct interference” by the State99 is likely to influence research practice even more powerfully, as anticipated by some academics who took part in the pilot exercise and expected “the behavioural changes to be as profound as those brought about by the Research Assessment Exercise, when it was first introduced”.100 Even if the funding bodies declare officially that their aim is “to assess all types of research without distorting the activity that it measures or encouraging or discouraging any particular type of research activity”,101 the introduction of research impact cannot be viewed merely, as “a modest and sensible attempt to demonstrate the wider ‘social value’ of research”.102 Following Collini, it can be interpreted instead as103

• 104 Stefan Collini 2012, op. cit., 197; Suzy Halimi 2004, op. cit., 117 ; Lisa Luc (...)

42The impact of the REF on researchers and institutions is yet to be analyzed but this new framework, just like the previous RAEs, could well widen the gap between research-intensive universities – the happy few who can afford to play the REF game and already concentrate public grants – and the other universities who will have no other choice but to focus on teaching, probably at the expense of a research-informed model of higher education in the long run.104 Whether this is going to strengthen the position of the United Kingdom in the global knowledge economy and to provide a model for other countries is yet to be seen but the obvious choice in the United Kingdom is for public policy to follow, and even amplify, market principles so as to regulate higher education, down to the very definition of research.