Skip to navigation – Site map

HomeIssues3.3Re-performance: a critical and re...

Re-performance: a critical and reparative methodology for everyday expertise and data practice in policy knowledge

Susan Oman
p. 291-313

Abstract

This paper introduces and develops an innovative re-performance methodology to reappraise tensions in evidence-based policy-making (EBPM). It conceptualises ‘re-performance’ as re-enacting and/or replicating aspects of another’s research, extending analyses of default, established methods of knowledge-for-policy. Re-performance is an under-utilised concept and this paper synthesises its assorted origins across humanities disciplines to demonstrate its conceptual utility for methods in research on research. Application of the re-performance methodology involves drawing from content analyses and ethnographic methods to understand the context of knowledge production as cultural production, and critical theory lenses to analyse these effects. The methodology was developed over two research projects on well-being and inequality metrics; both are presented to establish this approach as one that was developed in contestation over data and everyday knowledge (Case 1), that can be applied in a way that is reparative (Case 2), and consequently is of use to understanding current international data-policy controversies and crises. Overall, this article demonstrates re-performance as a methodology that recognises complexity in incorporating the social practices of everyday knowledge and expertise in a framework for policy studies in which the focus on data-in-research as performative reveals the effects in and on the cultures they describe.

Top of page

Full text

An early draft of this article was presented at the International Conference on Public Policy, Montreal, 2019. I thank the meeting’s organisers and editors of this Special Issue: Karel Cada and Anna Durnova. Thanks to Paul Martin and Richard Freeman for their encouragement to develop the ideas in this paper, and for reading previous drafts, alongside Anna Bull, Nigel Ford and Will Shankley, who also read previous drafts, and the anonymous reviewers for their generous and helpful comments and suggestions. I must acknowledge the research participants who made the empirical research so rich: the ONS for agreeing access to the free text data set, and my interviewees in the ONS and who provided contextual detail for the broader PhD research project. With ACE and the hundreds of people who donated their time in the individual arts organisations, I want to thank their generosity and openness to discovering how things might be improved. The PhD research project: All being well? Cultures of Participation and the Cult of Measurement (2017) was supported by the Arts and Humanities Research Council’s (AHRC) Connected Communities Large Project funding for ‘Understanding Everyday Participation: Articulating Cultural Values’, 2012–2017 [AH/J005401/1]. The research project: Social Mobility: the case of the arts was supported by two AHRC-funded Creative Economy and Engagement Fellowships: Data, Diversity and Inequality in the Creative Industries (or DDI) [AH/R013322/1]; What Constitutes 'Good Data' in the Creative Economy? Case studies in media and cultural industries [AH/S012109/1]. In addition to the funding, I would like to thank my PhD supervisors, Jackie Stacey and Andrew Miles and my fellowship mentors: Mark Taylor, Kate Oakley, Dave Beer and Helen Kennedy.

Introduction

1This article is interested in social science knowledge for policy that is generated from, and collected as, data about society. There are two problems in the social science literature that are ordinarily considered separately: first, the distinction between professionalised (expert) knowledge and worldly (everyday) knowledge; and second, the way that experts (the social sciences) describe people (the everyday). In looking at these issues together, it is possible to problematise not only data and descriptors, but the practices that generate and use them and therefore to interrogate not only the gap between different types of knowledge, but also the performativity of, and within, this gap.

2Policy Studies tends to focus on the gap between policy expertise and intellectual or technical expertise. They often conceptualise this dynamic in knowledge-for-policy as a boundary (Gieryn, 1983; Hoppe, 2010, Swedlow, 2017), tendentially overlooking the citizens that comprise this knowledge. This Special Issue aims to interrogate the culture-expertise boundary as an urgent concern for Critical and Interpretive Policy Studies. Rather than claiming to solve issues of the boundary, or the boundaries between knowledges, this article instead problematises the gap from multiple standpoints, foregrounding the performativity of data practices. It argues for a methodology that recognises the complexity of incorporating everyday knowledge, as well as the social practices of expertise, within a broader framework of data practices for evidence-based policy-making (EBPM).

3The article proposes a new methodology, re-performance, to better understand expertise as relations, and as an approach that accounts for the intersecting and overlapping issues and effects in layers across EBPM sites. The article begins by expanding on data and knowledge in EBPM as an urgent, yet longstanding, concern for Policy Studies. Contemporary governance is presented as evidence-driven, yet data are culture-driven, disrupting claims of rationality (Espeland & Sauder, 2007; Law et al., 2011; Campbell et al., 2017). Historicising data-for-policy foregrounds the cultural and political aspects of their production (Poovey, 1998; Porter, 1996; Savage, 2010). Relations across domains of policy, and expert and everyday knowledge, are differentiated and we lack empirical understanding of these dynamics ‘in-the-round’. Consequently, we could improve comprehension of how social science knowledge production and data practices operate in EBPM. The subtleties of necessary – yet mundane – interactions, relations, and shared understandings, as well as discomfort and lack-of-fit, are often an oversight in the macro-narratives offered by both Cultural Studies and Policy Studies. Therefore, the issue is both empirical and structural, and the re-performance methodology aims to incorporate these demands.

4The concept of re-performance is introduced here by way of a synthesis from across multiple lineages, reflecting its diverse, if sparse, usage across humanities disciplines. Re-performance is presented here as a conceptual development, embedded in the more familiar performativity literature. The intention is to demonstrate how the concept can be applied as a new methodology for understanding the role of data practices and their limits in policy knowledge. It describes how the methodology emerged to navigate and understand clashing logics of expertise and how it can contribute to Critical Policy Studies through revealing the social practices of data work in knowledge production for policy.

5The article presents two case studies of re-performance from the author’s own well-being research (Case 1) and inequality research in EBPM (Case 2). It demonstrates how the re-performance approach is able to engage empirically from different standpoints, using multiple methods, and remain robust across different policy domains and EBPM sites. The methodology brings a critical perspective to existing policy knowledge production. It problematises policy expertise by breaking down expert interactions with data and interpretations. Crucially, re-performance extends a traditional replication, as the synchronous incorporation of context and social practice into the analyses reveals choices made in data practices, as well as their social effects.

6The paper presents re-performance as a conceptual development and a methodological innovation that aims to address multiple EBPM concerns. It can reveal possibilities to bridge the gaps in professional and everyday knowledge, as well as showing how experts describe and engage with them. Furthermore, the gaps in accountability that enable established data practices for EBPM to go unchecked might be addressed by the knowledge that re-performance exists.

Understanding data and knowledge in policy

7EBPM as the use of social science in policy advice, has been conceived as ‘a moment where the bureaucrats are able to distance themselves from political decision-making’ (O’Brien 2016, p.127). This distance relies on the idea that policy-making is a rational process, with the aim of evidence overcoming emotion (or, indeed, corruption). It is also evoked by politicians both to defer responsibility and to allay fears relating to the outcome of such decisions. One pertinent example of this is the UK’s Prime Minister and senior ministers claiming to be ‘following the science’, or that ‘the science has changed’, regarding the COVID-19 pandemic, and the presentation of ‘the data’ in support of this narrative (Devlin & Boseley, 2020; BBC, 2021; Oman, 2021). Ideas of science and data are conflated for public consumption when broadcasting COVID-19 policy decisions – in reality to shield politicians from the social effects of policies – whether these effects are seen in discourse or in the negative impacts on the very numbers (data) cited.

8Of course, ‘the science’ has long had its limitations. In his seminal essay on how numbers ‘make people up’ by way of experts, philosopher and historian of science Ian Hacking contends that we should imagine two vectors:

one is the vector of labelling from above, from a community of experts who create a “reality” that some people make their own. Different from this is the vector of the autonomous behaviour of the person so labeled, which presses from below, creating a reality every expert must face ([1983] in Hacking, 2002, p.111).

9Policy evaluation in practice tendentially resolves to dissolve this conceptual space between the vectors. For example, this independent review of policy-making stated: “We need to bring the policy process closer to the real world, and bring the real world closer to the policy process” (IFG 2011, 101). While Hacking imagines what is at play at the boundaries of this conceptual dichotomy, it is clear that there are multiple dichotomies, boundaries and moments of further distancing.

10Policy Studies often describes the gap between knowledges as a boundary: something structural, diagnosed in domains and levels, inside which ‘boundary work’ happens, by a variety of policy experts (Hoppe, 2010). It is often conceived so as to understand how knowledge works instrumentally, such as the utilisation of academic knowledge, or ‘downstream impact’ (Hoppe, 2010). ‘Boundary work’ can describe innovations in how knowledge works in policy, including ‘knowledge brokers’ or ‘in-between’ actors, such as think tanks and consultants (Oman & Taylor, 2018). It also involves the protection of ideas of expertise, enforcing modes of distinction for the work of science from non-scientific or technical knowledge as policy (Gieryn, 1983). Thus, boundary work is conceptualised as working with and overcoming the boundary, whilst it also describes reinforcing bounded knowledges and the idea of expertise.

11Recently, what has become increasingly called ‘the new populism’ (Davies, 2018b), and the cultures of political discussion that invoke evidence, disinformation or ‘fake news’, have intensified the cultural aspects of data and evidence in policy and the public’s relationship with expertise. However, the cultural issues of policy and expertise do not suddenly appear, be it through rising populism, social media ‘bubbles’, fake news or ‘Big Data’ (ibid). Those interested in the politics of knowledge predicted various ‘crises’ of evolving data economies and practices, foreseen as the ‘coming crisis in sociology’ (Savage and Burrows 2007) and ‘the current crisis of positivist methods’ (Savage, 2013, p.3). The ‘crisis of science’s governance’ has forced questions such as ‘what is wrong with evidence-based policy?’ (Saltelli & Giampietro, 2007). Crucially, however, the nature of ‘the science’, its data, their role in policy (and how these issues intersect), is a key concern not only for policy-makers, but also for the media and the public. These are issues of rationality and trust as much as issues of culture and society.

12Policy Studies has variously engaged with the issues underlying EBPM and the cultural nature of expertise. Evidence is but one pressure amongst many in decision-making (Davies 2004). Policy-making is inherently cultural, including: the values of ministers, the influence of lobbyists and pressure groups, and media scrutiny (or lack thereof) of administrative practices and decisions (Davies, 2004). Prior limitations of EBPM equated ‘better’ to simply meaning ‘more’ evidence use, when the problem is often that there is too much evidence, or that the evidence doesn’t fit the evidence culture (i.e. it is too academic) thereby hindering ‘good’ decision-making (Stevens, 2011). Historicising data and social science research foregrounds the cultural aspects of their production (Poovey, 1998; Savage, 2010), compromising the neutrality of scientific methods and revealing their performative capacity for a ‘social life’ (Law, 1999). Despite existing insight into policy knowledge practices, and the fact that, empirically, policy-making has been found to be chaotic, civil servants are not realistic about this and nor are policy-makers transparent about this fact (IFG, 2011). Conversely, the process of EBPM and idealisation of the universally robust, which tends to accord with medical models, means it is not always fit for purpose (Cairney & Oliver, 2017), lacking cultural contextualisation (Greenhalgh, 2016). Thus, the culture of EBPM is disconnected from the reality of ‘good’ evidence for its limitations in capturing society.

13In sum, it can be seen that EBPM is a cultural practice (Bevir & Rhodes, 2010). Policy experts and their responses to urgent concerns should be understood in the contexts in which they navigate the institutional histories (Bevir & Rhodes, 2006). These contexts are often both chaotic (Stevens, 2011; Wilkinson, 2011) and mundane (Freeman & Sturdy, 2014). Similarly, the ‘states’ or ‘phases of knowledge’ (ibid), which inform policy as a cultural practice, demand attention and this must entail how knowledge is produced, inscribed, operationalised, practised, and ‘lived’. Thus, we need to move beyond a tendency to focus on and/or fix boundaries, be they between experts and policy, policy and the everyday, or people and experts. Instead, this article proposes to revisit policy knowledge as a relational phenomenon that is multi-layered, cross cultural and performative. The following section explains how asynchronous developments in theories of performativity present an under-developed concept of ‘re-performance’. It does so in order to demonstrate the utility of such a conceptual progression in order to understand EBPM with an innovative methodology.

Synthesising the conceptual lineages of re-performance

Reperformance: The repetition of an activity to observe whether the original activity was undertaken in accordance with established policies and procedures. Re-performance is a common audit test—for example, auditors may ... perform a walk-through test (O’Regan, 2004, p.222)

14The word ‘re-performance’, meaning to perform again or anew (Lexico, n.d.), has multiple lineages and diverse, if sparse, usage across humanities disciplines. This section presents the use of re-performance from across humanities disciplines as a way of engaging with knowledge as a relational phenomenon. It organises the re-performance literature into three key theoretical rationales, listed in Table 1.

Table 1: Key concepts and applications of re-performance

Key concepts

Applications

performativity and reproduction

understanding systems and effects

re-enactment

documenting absence and gaps

replication and re-evaluation

following the data/science to reveal misrepresentation

Source: The Author

15This section will demonstrate how different humanities’ literatures use performativity and re-performance in order to enable conceptual developments that help us to understand evidence for policy as a cultural product, as well as a cultural producer. Reframing the production of knowledge in these terms allows us to reveal tensions in data, policy and everyday expertise cultures. This reframing is also necessary to address the socio-cultural conditions of evidence that are re-produced and re-performed across policy settings, and how these logics translate to everyday settings.

Re-performance and performativity: capturing reproduction, systems and effects

  • 1 Troubling, as the act of destabilising supposedly fixed, known entities.

16The term ‘re-performance’ can be traced to early-17th century theology; for example, ‘[God] will perform, reperform, multiply performances thereof upon us’ (Donne [Parker, 1839], p. 212). Donne’s use of performance and re-performance alludes to the performativity of actions in that they are felt by their effects. Perhaps counter-intuitively, this echoes Butler’s development of ‘performativity’ ([1990] 1999; 1993), which destabilises gender as a neutral binary that categorises and organises; instead, it is expressed, performed, interpreted and reproduced. ‘Troubling’1 how such an established and organising concept is understood and has been constructed historically reinterprets the possibilities for how it can be productive and thus how it reproduces.

17Performativity can be seen to counter a certain kind of positivism and its delimited categories of gender, the state and the economy, and draw attention to the cultural aspects of these processes (Butler, 2010, p.147). The social sciences have borrowed this formulation ‘to supply an alternative to causal frameworks for thinking about effects’ (Butler, 2010, p.147). Problematising the assumptions of positivism is one of the aims of ‘the social life of methods’ (Savage, 2013) and is vital to understanding the performativity of methods themselves and the systems in which they are able to reproduce (Law, 2009). In the context of this article, ‘making methods the object of enquiry’ (Savage, 2013, p. 3) can reveal the ‘effects’ of what experts claim their knowledge does and what gets to ‘reproduce’ as knowledge that changes policy and obscures solutions which require more everyday knowledges.

18Butler’s (2010) analysis of the ‘performative agency’ of economics in the 2007/8 economic crisis demonstrates that the very mechanisms by which established forms of social science reproduce themselves, and perform as if valid, are the very same mechanisms of this culture of expertise that enabled the crash to go un-noticed. Method and rigour are inherited and too big to fail; as Mirowski (amongst others) demonstrates: the very cultural logic and technologies of expertise that created the 2007/8 crisis are naturalised as the only way out of it (2013). These relations of social science practices are performed (and re-performed) in such a way that the means and mechanism of these performances are ‘only made clear on the condition of breakdown or disruption’ (Butler, 2010, p.151). These relations are of both empirical and structural concern (ibid). Arguably, then, research which engages with what these relations do as an empirical and structural question might reveal what is awry in a system that obscures the faults and limits of its mechanisms in such a way that pre-empts crisis. Alternatively, on a more mundane scale, such an approach might have reparative effects, as will be demonstrated in the upcoming case studies.

Re-performance as reconstruction to document absence and gaps

19Re-performance, then, enables us to conceptualise how the shortcomings of technologies of expertise (be they disciplines or methods), and the wider mechanisms that allow them to reproduce, become visible. The ‘social life of methods’ frames this in terms which might be thought of as tracing the effects to appreciate the impact on populations or discourse. For example, John Law’s analysis of the Eurobarometer survey demonstrates that the social survey as a technology does not simply describe and enact European consumers’ views [in this case of farm-animal welfare], but enacts numerous other realities at different layers at the same time. He notes that it: ‘reproduces the individual act of consumption as a proper location for political action… enacts the EU as a neoliberal political site; performs Europe as an isomorphic population of individuals in a homogeneous, bounded, conceptual space.’ (Law, 2009, p. 250). The concept of re-performance and the proposed methodology also push this act of ‘tracing’ backwards for contextual detail on historical aspects of knowledge production. This enables an understanding of what – or whose – knowledge was overlooked in the process and is empirically possible through document analysis and interviews.

  • 2 Performance artist Marina Abramović impresses the importance of discussion on the politics and ethi (...)

20Social science research for policy-making is often inadequately documented in terms of research design, data collection and analyses (Oman, 2019b). Consequently, the data to interpret methods (and data practices) as objects in and of themselves are limited to what is decided to be recorded and presented in the public realm. The uses of re-performance in performance studies, and the history of performance art, address a lack of adequate record. These archives were, in part, inadequate because they were not valued as significant events within the ‘validated’ cultures of expertise at the time, and thus the only way to document these events and practices is to ‘re-perform them’2. This is because of the complexity of production and reproduction of these ‘events’, as much as the cultural differences in the contexts in which they might be interpreted.

21‘Scribal re-performance’ – a concept from poetics – accounts for multiple subjectivities in documentation and re-enactment (Watson, 1998). Crucially, scribal re-performance methodologically accommodates the tensions across differing accounts, documentations and interpretations. Accordingly, ‘the gaps’ become ‘the object of enquiry’, as much as those aspects which can be rendered more concrete. Addressing gaps and absence is, therefore, a further contribution to the benefits of conceptualising re-performance from across disciplinary uses and traditions.

Re-performance as replication and evaluation: ‘following the science’

22Re-performance acknowledges the subjectivities of individuals in the documentation of cultural production for example a live event, such as performance art in a way that can be applied to knowledge production, whether at the layer of each individual completing a Eurobarometer survey, or concerning how the survey designers asked the questions, or how the results were interpreted for policy action. The term is also used in accountancy to describe ‘a common audit test’ that can be as simple as a walk-through, or more complicated to replicate and test the workings to establish error or misrepresentation (O’Reagan, 2004, p. 222). It also describes ‘the repetition of an activity to observe whether the original activity was undertaken in accordance with established policies and procedures’ (ibid). For these reasons, this latter conceptualisation and usage of re-performance (that opens this section on synthesising) is particularly useful in understanding the production of knowledge-for-policy as relational; as something both socially constructed and bound by established protocols that afford claims to robustness.

23Critical studies of the ‘darker practices’ of market accounting firms revealed that ‘the audit review process cannot completely reperform the audit’ (Mitchell & Sikka, 2002, p. 18). As a consequence, ‘irregular audit practices rarely come to light’ (ibid). Re-performance is more than replication; it can extend such analyses not only checking the procedures that were abided by, but also to bringing those into question (Oman & Taylor, 2018). In short, re-performance allows understanding of the limitations of established procedures that enable inadequate, dangerous or reprehensible processes to be re-produced unnoticed or without criticism. This is illustrated in both the auditing practices of commercial organisations and in the financial sector (Mitchell & Sikka, 2002) as well as in the practice of economics which enabled the crash (Butler, 2010).

24The concept of re-performance has not previously been presented from across the humanities, and thus its varied utility as a tool for social science to understand expertise and its methods has not been recognised before. Synthesising these lineages of the concept of re-performance establishes precedence for a new re-performance methodology that is able to understand limits in various logics of expertise that tendentially inform policy-making through positivist claims to neutrality. It can be applied to the social sciences using the logic that, if we ‘follow the work of economics, examining the processes that make some facts possible and not others’ (Mitchell, 2005, p. 309), we can understand how knowledge is produced and reproduced. Furthermore, if we re-view this social science ‘in the wild’ (Mitchell, 2005, p. 318), we are able to understand the tensions between everyday contexts and data production for policy expertise in a way that can suggest ways forward. Having argued the benefits of re-performance as a conceptual development, the second half of the paper argues for it as a methodological innovation that aims to address these concerns.

Re-performance: the genealogy of a methodological solution to data contestation

25The case for a re-performance methodology laid out by this article is in its attention to the relation between the ‘policy expert’, often pitched as a homologous ‘knower’, and ‘what is known’. This Sociology of Knowledge approach reveals the limits in the existing logics of knowledge production for policy and how that might better incorporate everyday citizen understandings. Re-performance generates new data about how ‘the people’ feel about a policy issue (in this article, the case studies relate to well-being and inequality) and about the methods typically used to generate these data from the communities under study.

26The previous section’s presentation of re-performance shows that it can be applied to re-document or to reveal hitherto obscured detail, to check the workings of others, to analyse why results differ. It can also reveal the effects of how certain social science methodologies and processes can reproduce, despite their flaws, allowing risk to go unchecked. Thus, applying the concept to study knowledge-for-policy enables an exploration of what is happening as per a traditional multi-level framework. Furthermore, it goes beyond a hierarchical model, looking at this problem ‘in-the-round’, in layers of effects. This section presents how the methodology evolved to address this need.

27The re-performance methodology was developed through the author’s own empirical research from 2012-2017 on how well-being metrics were emerging as a tool for evidence-based policy-making. Advocacy for this reconfiguration of EBPM, internationally (Stiglitz et al., 2009; OECD, 2010), in UK government (Cameron, 2010), and by key members of the UK civil service (i.e. O’Donnell et al., 2014) suggests that it is worthy of its own epistemic community (Bache, 2013). In contrast, critique of the ‘promise’ of happiness and well-being (Ahmed, 2010) and the ‘political economy of unhappiness’ (Davies, 2011) emerged from the academy as a vital ideological counter-narrative to ‘the well-being agenda’. The drivers of this agenda have been problematised in Policy Studies as ideological, personal and political (i.e. Bache 2013; Bache & Reardon, 2013; Oman, 2017). Therefore, the well-being agenda as a project of politics, policy and knowledge production requires critical attention in a way that accommodates these interdependent interests (Oman, 2021).

28There were various reasons to start measuring well-being as a proxy for progress to re-frame political debate away from crises (mental health, ecological and economic) in a more solutions-focused way (Bache, 2013; Davies, 2015; Oman, 2017). The lack of universal agreement on the meaning of well-being – as acknowledged by the head of the Measuring National Well-being (MNW) programme (Allin, 2007, p. 46) and others (Dodge et al., 2012) – was tackled in the UK with a unique ‘participatory spirit’ (Kroll, 2011). Alongside interdisciplinary experts appraising what other nations were doing, a national debate was conducted by the ONS to establish ‘what matters to you?’ [the nation] in 2010-11. The outcomes of the public debate and expert advice are said to have informed what would be measured as well-being (ONS, 2012).

29Governments are said to ‘reach for the listening card precisely when they realize that their legitimacy is at a low ebb’ (Dobson, 2014, p.10). Dobson’s thesis that listening is performed as democracy suggests that the MNW Debate could rationalise the measuring of well-being to the populace, even though the media derogatorily described ‘Cameron’s Happiness Index’ as ‘woolly’ (BBC, 2010). Therefore, the debate could be interpreted as a public-engagement exercise to promote the idea that well-being can be measured in a way that is useful for governance. In other words, the debate was more a means of data communication, than of data collection, between the epistemic communities of well-being, policy expertise and ‘the people’.

30The research in question, which forms the focus of Case Study 1 (in the next section originally attempted to reanalyse data collected in the debate by way of a questionnaire in 2010-11. The ONS denied the author access to this data – a decision that was in tension with its ‘participatory spirit’. Details of the access issues, the counter-arguments and the decision’s reversal some months later, alongside results from alternative analyses, are presented elsewhere (Oman, 2015; 2017; 2019b; 2019c) and, consequently, they are not detailed here. The headline finding, however, was that the author’s alternative analyses had very different results when it came to what people prioritised for well-being.

31The MNW programme’s technical advisory groups were responsible for reviewing existing expertise from across psychology, economics and government; however, there is no publicly available evidence of methodological guidance on the qualitative methods of the debate, nor indeed on how one may ‘listen to a nation’ as the express purpose of the debate (Cameron, 2010; Matheson, 2011). Crucially, as the data produced by the debate were largely non-statistical, but rather qualitative, verbal or textual, methodological skills required for their generation and interpretation needed cultural processes that lay outside the institutional structures of the ONS – and arguably of their advisory groups. For example, participants in the questionnaire were not asked about data re-use and people who attended live events did not have their testimonies audio-recorded or noted (see Oman, 2019b and 2021 for details); therefore, how these data were analysed to form the basis of the outputs was unclear, to say the least.

32Re-performance was applied in Case Study 1 as an iterative, investigative methodology that first negotiated the denial of access to data and then the lack of documentation of data processes. Which forms of knowledge become ratified and able to reproduce themselves are both empirical and structural concerns (Butler, 2010, p. 151). It was, therefore, necessary not only to investigate how data produced by the debate were engaged with, but also how these data were interpreted and analysed – along with their effects. Re-performing the national debate, in order both to document it and to generate comparable well-being data, necessitated a preparatory data-gathering phase to understand the ONS’s approaches to data collection.

33In the absence of secondary data collected by the ONS, the research design evolved to reconstruct a scaled-down model of the MNW debate. This was a series of events that could, when analysed together, perform as if a sample of those generated by the national debate. The research design for primary data collection was constructed from ONS interviewee accounts, email responses and official policy documents.

34Re-performing the MNW Debate as a series of events enabled more detail to emerge on the ONS’s methodological approach to research design. As Judith Butler explains, in performance and re-performance, breakdown and disruption can be made clear (Butler, 2010, p. 151). Therefore, tracing the contradictions in documentation and the gaps in knowledge foregrounded the ONS’s methodologies as an example of well-being expertise and data practices. Furthermore, these could then be better examined in their own terms – and from different standpoints – from a critical distance, and from inside the process.

35Re-performance of the debate was necessary not only to collect comparable data, but also to document the events themselves and to create a record of what was previously missing – expressly because it had been undervalued in the first place. Crucially, the ethnographic work of re-performance also provided contextual information that evidenced a link between the lack of methodological detail in policy documents and the lack of methodical attention in research design, data collection and analysis. Consequently, re-performance can help to answer questions about how the claims made in the published findings, which are used to advocate what was decided, would be measured as well-being and the way in which well-being is being measured to evaluate policy.

36Thus, the re-performance methodology, here, re-enacted another’s methodology to check workings and reveal gaps in methodological rigour. Furthermore, in addition to revealing contradictions, the re-performance also provides new data, thereby generating comparable datasets. The possibilities for auditing (O’Reagan, 2004, p. 222; Mitchell & Sikka, 2002, p. 18) lie not only in a standard replication, but also in re-performing the data practices as social practices. Therefore, re-performance can re-view a specific example of knowledge production as a structural and empirical concern through the layers of methods at work in context.

37Crucially, then, re-performance also presents the opportunity for established and ratified methods and analyses to be extended. As in re-performing and recording one version of events, the differences or ‘gaps’ also become data for analysis (Oman & Bull, 2021). These stages of re-performance facilitate an interrogation of data practices so that the cultures of expertise that generate, process, disseminate and utilise such evidence - as fact - are reviewed. Re-performance enables understanding of how the methods used by experts to understand ‘the real world’ translate in those contexts. They also enable researchers to ‘stand in the gaps’ (ibid) and to advocate for change in policy and practice contexts.

A re-performance research design to understand cultures of well-being expertise: an example from the field of national statistics

38A re-performance methodology involves a research design that interrogates data and / or research practices that inform social science research for policy. It is also able to study the effects of these processes across different cultures of expertise. The approach is separated into ‘layers’ for conceptual and empirical purposes. Empirically, the research design accounts for how data about data practices will be collected, and explains what this means across boundaries and gaps between everyday, expert and policy knowledge.

39Conceptually, the use of the term ‘layers’ reflects Law’s attempts to capture layers of social effects of social research for policy. This is different from more typical Policy Studies’ approaches to multi-level frameworks, such as Hoppe’s ‘(inter)national institutional-cultural regime(s) or ‘landscapes’, policy or issue network(s), organisation(s), and project(s)’ (p179). It enables an extension of typical analyses of an expertise binary, whilst also reflecting the ‘top-down’ nature of a national research-for-policy project, to reveal what is happening across different layers in a project of social science research, such as a survey (Law, 2009).

40The ONS is a community of experts guided by historical principles set out by national statistical agencies and the demands of national and international policy-makers. It was claimed that the MNW Debate findings would inform the government’s final well-being metrics. Therefore, the questionnaire aimed to capture what well-being means to people and to gauge opinions on particular wellbeing metrics and statistical expressions.

41The MNW programme aimed to establish a working model of well-being metrics that could negotiate international well-being policy expertise and the institutional histories of the development of national statistics in the ONS, whilst also incorporating everyday responses to these changes. As suggested above, there are indications that the debate also presented an opportunity for public engagement to legitimate the idea that national well-being should and could be measured.

42Understanding the MNW Debate as an event of social science research for evidence-based policy is complex. It included 34,000 testimonies on what the government should do to improve the good life and therefore there were myriad considerations in the re-performance process. For example, how to include or reanalyse the testimonies (as access was later permitted)? What data were included, and what were missed, and how were these analysed and synthesised? How was it possible to conduct a national debate that would inform the redesigning of national accounts and policy-making, and not to account fully for the methods? What were the effects of these oversights and their implications for the policy expertise that was manifested in the debate’s findings? Interpreting these issues, whilst accounting for their complexity, required analyses across various intersecting societal contexts. This involved a research design that would try to emulate the processes and methods of the ONS, understanding them in broader contexts of well-being measurement, well-being studies and evidence-based policy. This research design, and its empirical and structural concerns, are outlined in Table 2.

Table 2: Re-performance framework applied to a research design to capture a complex context of policy expertise: ONS’s MNW Debate.

LAYER 1

What?

POLICY – EVIDENCE

Who?

Observation of well-being knowledge-producers and policy-makers

How?

Discourse analysis – policy documents and methodology / working papers

LAYER 2

What?

DATA PRACTITIONER / DOMAIN EXPERT IN CONTEXT

Who?

ONS employees with key roles in the well-being debate’s execution; analysis; dissemination

How?

Interviews; emails; telephone calls; meetings; replicating methods

LAYER 3

What?

EVERYDAY UNDERSTANDING

Who?

Self-selected to participate in the MNW Debate

How?

Free text field re-analysis

Who?

Social and community groups recruited through community organisations

How?

Focus groups in social / cultural contexts

Source: The Author

43Reproducing debate events aimed to reveal ‘the means and mechanisms’ at play in the performativity of methods that are ‘made clear on the condition of breakdown or disruption’ (Butler, 2010, p. 151). In the re-performance, 98 people participated in 14 events in familiar, and hence ‘naturalistic’, settings to allow ‘deep data from the local context’ (Thomas, 2008, p. 87) that was ‘culturally sensitive’ for marginalised communities, enabling access to ‘everyday interactions [to] reveal cultural norms and values’ (Liamputtong, 2011, p. 127). In the absence of a detailed description of how individual debate events had been facilitated, I began each event with two questions used in the MNW questionnaire: What matters to you?; What things in life matter most? This afforded the participant ‘the opportunity to define what is relevant and important to understand their experience’ (Liamputtong, 2011, p. 4). Participants answered the two questions as individuals and then reflected on these two questions as a group. The event was audio-recorded and analysed as a group discussion, generating a dataset that could be analysed in various ways: the key to an interpretive approach, avoiding the pitfalls of standardised categories and deductive models (Bevir & Rhodes; 2006, p. 2).

44This approach resulted in the debate being recreated on a smaller scale to collect data on everyday understandings of well-being. Alongside this, the methods used to undertake the debate, along with its documentation, analysis, data storage, ethics and dissemination of analysis, were also analysed. Consequently, the gap between what was documented and what was not became data in and of itself.

45Furthermore, to re-perform the MNW Debate was not only ‘the repetition of an activity to observe whether the original activity was undertaken in accordance with established policies and procedures’ (O’Regan, 2004, p. 222), it was also the interrogation of the established policies and procedures. Therefore, re-performance is an opportunity to understand how research is currently happening and how it could be improved in multiple ways: the way in which researchers listen to what problems require solving, and how they identify those solutions, as well as the way in which these processes are accountable when they directly impact policy.

46Through mainly inductive approaches across different layers of research, outlined in Table 2, a methodology emerged that is applicable to Critical Data Studies, Critical Policy Studies and the Sociology of Knowledge. Crucially, it is also applicable in research for policy contexts as a reformative approach to data and to research that informs policy. In its capacity to understand how different cultures of expertise might approach a policy issue, such as inequality, and a practical issue, such as data, and how these are engaged with and experienced in their own terms, the re-performance approach might be instrumentally applied to solve policy controversies by improving practices.

A re-performance research design to understand inequality data practices in context and to improve cultural policy

47This second case study of the re-performance approach is presented to demonstrate how a methodology devised in a site of contestation can be instrumentalised and applied in EBPM practice. The research was undertaken in partnership with Arts Council England (ACE) in 2018-19. ACE is a non-departmental public body (NDPB) and the largest funder of the arts in England. ACE wanted to introduce a measure of social mobility or class inequality to its data-monitoring processes. The author was asked to conduct research and to recommend an inequality metric.

48Inequality data – and the lack of positive change to diversity that these data make visible – are a site of contestation in the sector (Saha, 2020; Oman, 2019a). The cultural identity of the UK cultural sector is inextricably linked with the idea that it is committed to inclusion. However, datasets reveal, first, the failure to achieve diversity goals (Brook et al., 2020; Oman, 2019a) and, second, the amount of missing data from administrative processes (Oman, 2019a). There are, therefore, issues related to data practices and diversity practices in the sector that require attention at the same time.

49Despite the high symbolic value of inequality data in much cultural-sector discourse, the administrative practices that generate equality monitoring and diversity data are not valued on a day-to-day basis. They tend to be overlooked, as do issues which manifest as generalised complaints about these data-monitoring processes (Oman, 2019a). The project demanded a recommended metric which needed to be compatible with those used across government and the Civil Service, while any new practices or processes introduced to the arts sector needed to account for its working cultures. Therefore, it was necessary to embed the task of measuring social mobility, or establishing an approach to class measurement, in a broader project of understanding everyday data norms across the complex ecologies of expertise and working cultures that are the cultural sector.

50Inequality measurement happens at a number of levels in UK policy-making. The ONS holds responsibility for national survey-level data, which include demographic questions, such as in the Census, for example, together with other proxy measures, some of which appear in the well-being indicators described above. Other proxy indicators for socio-economic inequality include the kind of school a person attended and whether their parents hold degrees (as found in the ONS’s Labour Force Survey). These data dominate the quantitative work comprising the recent rise in class analyses of the cultural sector (e.g. Brook et al., 2020; Friedman et al., 2015).

51Additionally, there has been pressure on organisations and the public sector to collect workforce demographic data as a result of the Equality Act 2010 and the Equality and Human Rights Commission Employment Statutory Code of Practice (EHRC, 2015). This typically involves ‘Equal Opportunities’ forms that draw on the same questions as national surveys, although the formatting and wording may differ. In the cultural sector, equality of access to jobs and of access to commercial content, such as cinema visits, or publicly funded culture, such as the BBC, is ascertained using national-level survey data, consumer insight data and these mandatory monitoring processes. The BBC has, for example, added proxy questions to its data processes to understand the class of its workforce – in line with recent Civil Service developments (BBC, 2017; Cabinet Office, 2016).

Table 3: The benefits of a re-performance approach.

Data issue

Re-performance methodology

Understanding how data work in contexts improves understanding

Case 1

DATA: secondary qualitative data collected by ONS from public debate on well-being

Can reveal alternative findings, thus testing the limits of methods assumed robust

Can reveal how data practices are performative

DATA ISSUE 1: lack of access to secondary data

DATA ISSUE 2: lack of published detail on data and methodological practices

Can recreate missing data through re-performing data-collection methods and analyses

This improves understanding of the limits of data as evidence and their politics

REASON FOR ACCESS ISSUE 1: no permissions granted by participants;
ONS labour involved in cleaning data

Case 2

DATA: Class metrics for arts workforce

Can reveal how long-standing data practices work in specific contexts, revealing their limits, and the limits of the data produced

Can understand how to recommend solutions to overcome limits to existing data practices and improve various aspects of data practice in context

DATA ISSUE: Data does not exist, or is scant and inconsistent

Can reveal why missing data is such a problem in context

This improves the quality of the data and improves understanding of its possibilities and capacity for positive impact

REASONS FOR ACCESS ISSUE: Confusion in sector as to purpose of data.
Lack of guidance on best metrics

Source: The Author

52In order to address the issues of missing data more generally and the requirement for a class metric that was compatible with existing equality data and cultural-sector processes, it was necessary to understand these problems ‘in-the-round’. Crucially, policy expertise as data expertise is differentiated across the cultural sector (Oman & Taylor, 2018; Oman, 2019a). The layers of the re-performance framework in Case Study 1 were applied to Case Study 2 to capture this complex ecosystem of data collection that informs inequality policy using multiple methods. These included policy analyses of ACE Equality, Diversity and Inclusion policies – and non-ACE policies. They also included a review of academic and grey literature concerning arts-related and broader inequalities, together with the disciplinary histories of methodological approaches. Fieldwork generated most of the data; this included working inside ACE, sitting with the Research and Data teams and also attending various meetings relevant to diversity and inequality policy-making inside ACE and with the funded organisations. Alongside this, 15 organisations that are funded by ACE, called National Portfolio Organisations (NPOs), were sampled. Each NPO was chosen for a balanced distribution of geography, size of organisation, size of grant from ACE, discipline area (i.e., dance or art gallery) and social mission (i.e. reaching local working-class communities or working with disabled performers).

Table 4: Research design that applies the re-performance framework to diffuse policy expertise across the cultural sector.

LAYER 1

What?

POLICY – EVIDENCE

Who?

Observation of diversity & inequality in arts knowledge-producers and policy-makers

How?

Discourse analysis – policy documents and methodology / working papers; social media; inside ACE

LAYER 2

What?

DATA PRACTITIONER / DOMAIN EXPERT IN CONTEXT

Who?

Employees of NPOs with key roles related to data and / or diversity

How?

51 one-to-one interviews; emails; telephone calls; meetings

LAYER 3

What?

EVERYDAY UNDERSTANDING

Who?

Staff from across all functions of arts organisations, self-selected or nominated by NPO

How?

26 focus groups in NPOs

Source: The Author

53As with Case Study 1, the research design outlined in Table 4 divided the cultural sector into layers to interrogate tensions in data-policy expertise and in everyday experiences and perspectives of these processes. This also aimed to reveal the layers of effects, or the performativity of the data practices, in context. Two layers of research were undertaken in each of the sample organisations. One of these layers comprised 51 staff one-to-one interviews with people who held key responsibilities for diversity or data inside that organisation. One interviewee saw ACE’s demands for equality-monitoring data as being like a ‘death star’; others raised concern that the rationale for more data had not been communicated, and that people with these responsibilities across the sector ‘lacked confidence’ in the processes, even feeling uncomfortable asking their staff to participate (Oman, 2019a, p. 37-39). Consequently, these testimonies offered valuable insights into how data and diversity policy and practice are differentiated across the sector. Crucially, looking at data and diversity practice together in this way was a surprise for interviewees (ibid) who saw them as separate.

54In the other layer of research in NPOs, 26 focus groups were organised to gather the perspectives of workers in their teams across the many functions and pay-grades of each organisation. As with the ONS case, each of these group discussions centred around an elicitation activity, this time using ratified survey questions that were answered by people individually and then discussed as a group. In this instance, participants completed two composite questionnaires containing standard questions used to capture dimensions of inequality, but focusing on class and social mobility, and then reflected on the experience: the wording, the format, how they felt answering the questions and whether they felt confident answering them.

55The elicitation activity, using a questionnaire of approximately 50 questions and sub-questions, re-performed data collection in context, as people were completing Equality Monitoring forms in their place of work. Although class-related questions were new in these contexts, they have long-established methods with their own institutional histories. For example, one question has been used for decades in sociological measures of social mobility (Goldthorpe & Hope, 1972). This is a proxy question, asking for the occupation of the main wage earner in a household when the respondent was 14. It is considered a more accurate measure of class than income or self-identification (ibid; Brook et al., 2020). This question is part of a schema that informed the National Statistics Socio-Economic Classification (NS-SEC) system used for half-a-century (ONS, 2010). The schema identifies someone’s class origins by way of the school they attended, whether their parents attended higher education, and parental occupation at 14. While policy and data experts consider these questions most able to produce the most robust metric, the latter question was queried in the focus groups, who argued it had numerous limits in revealing class position and inequality in this context.

56Each focus group problematised this robust, established social science methodology that has long informed policy-making. Each group took either practical, political or personal objection to it; often all three (Oman, 2019a). The practical opposition included the very real issue that many felt unable to answer the question accurately. Furthermore, they understood that the question was trying to get at something, but were not sure what. In other words, they understood that it was a proxy question, but could not understand how it captured something useful. People wanted to understand how their personal data were going to be used to tackle a practical issue.

57The re-performance of these questions revealed how diffuse policy expertise is in a research exercise such as capturing demographic data to monitor and tackle inequality. People have a sense that something is happening, but do not see the value in these practices. They want to understand why this personal question about someone’s parents from their childhood is relevant to the workplace and to inequality. Without this understanding, this question is too personal and potentially unethical in a project that claims to be concerned with social justice. This ‘troubles’ (Butler [1990], 1999) how we categorise class which many see as being out of touch with everyday substantive realities. Furthermore, it calls into question the rigour of established research methods that generate inequality data when reviewed in context.

58Re-performance therefore reveals the limits to the logic of long-established, robust social science methods in context. These findings demonstrate that, no matter how established a process, the assumptions behind its applicability and its rigour may require revisiting. Furthermore, looking at cultures of expertise across different layers revealed gaps in understanding. The dominant logic of the policy expert in layer 1 wanted to apply the research method that they were told was robust; however, responses in layers 2 and 3 considered whether that question made sense in the asking, rather than whether the data made sense in terms of modelling. In essence, inequality data and data practices are performative at the point of collection and analysis and in EBPM and the re-performance methodology was, again, able to analyse these effects in a way that could contribute to critical studies of policy, data and knowledge, as well as to EBPM practice itself.

Discussion: re-performance to address cultures of expertise

59Issues surrounding social science knowledge and policy-making remain a concern for Critical Policy Studies, more recently manifesting as ‘culture wars’ (Davies, 2018a). The role of the expert is said to have emerged to manage the application of knowledge to society, whilst also presenting a barrier to meaningful citizen participation in these processes (Fischer, 2009, p.18-19) and the role of data and data practices in policy and public administration are an ongoing part of this concern (Davies, 2018b). While these professions are seen as ‘fostering important advances’, including quality of life, these interventions are seen as self-serving (Fischer, 2009, p.20). Yet the development of new technologies to deal with the knowledge-society problem – to evaluate political issues, understand the emotions, responses and behaviours of the everyday citizen – are also highly contentious, arguably reinforcing inequalities and structural ill-being (Davies, 2018a; O’Neill, 2016). Correspondingly, there is evidence that people care about these conditions of datafication less than the critical, academic experts do, who hold professional experts to account (Kennedy et al.; 2020). Understanding the possibilities of social science for EBPM, while at the same time holding such cultures, and the data practices that uphold them, to account, pose technical, practical and ideological challenges.

60Knowledge-for-policy and policy expertise are, of course, not universal objects for study. New forms of expertise lead to evolutions in evidence, its use and influence (Oman & Taylor, 2018), while many involved in producing knowledge-for-policy are overlooked in analyses (such as those responsible for equality-monitoring data). Yet the dichotomy presented as the expertise boundary often assumes that each side of the boundary is homologous. There is much criticism of the oversimplification of ‘the people’ or ‘citizens’ when describing policy controversies (i.e. Davies, 2018), with ‘policy-maker’ and ‘policy expert’ equally over-simplified. However, there has been less attention paid to appreciating this as complex and relational across the data-policy research dynamic.

61We therefore need new approaches to understanding data, knowledge and evidence in policy, which will account for the complexities of different knowledge cultures – and the cultural contexts in which evidence is generated. Furthermore, these methodologies need to account for various standpoints within these institutional logics (or indeed outside them). Performativity can be seen to counter a certain kind of positivism and its delimited categories (Butler, 2010, p. 147) whilst also describing the relations of power in which we are implicated, but which we oppose, turning them on themselves to produce alternatives (Butler, 1993, p. 241). The re-performance methodology enables these relations and mechanisms to be revealed through breaking them down within the system as an act of critique and reparation.

62The epistemology of re-performance foregrounds the relationship between the knower and the known. The methodology of re-performance emerged to solve my data problems as a researcher: lack of data, due to lack of access, which was arguably due to lack of expert status. This led to a series of revelations and then to an investigation of intersecting data problems as epistemic. The research design in the two presented cases aimed to account for this problem empirically and structurally. They take the very question of how a policy-maker ‘knows’ what ‘science to follow’ and organise their investigation as layers across logics of expertise. In each layer, process and practices are researched in context.

63Each case involved institutions that comprise a complex ecology of people across data, evidence, policy and research. Applying the re-performance methodology attempts to account for the ‘institutional logic’ of civil servants who ordinarily work within static, yet chaotic, architectures of knowledge production, distribution and delivery. It works across sub-settings, frameworks and logics to enable different standpoints on the larger process. As above, the context for re-packaging national accounts as ‘national well-being’ responds to international requirements and a demand for legitimacy, when such a crisis in legitimacy had been unlikely prior to the economic crash of 2007/8. The MNW Debate appealed to a logic of legitimacy across expertise cultures, yet struggled for legitimacy in ‘real world’ contexts because it failed to appeal to ‘everyday’ logics.

64In the context of the ACE inequality project, the NDPB was shifting in a context of an evolving new 10-year strategy and was under more scrutiny from the public it served (the sector). Members of the sector have increasingly imposed pressure to evidence equity (as the cultural identity of this policy sector), while being distrustful of data-collection processes as they are not logical to the person completing the Equality Monitoring form (Oman, 2019a). Demands from central government for performance-related data that include equality monitoring do not manifest themselves in more equal workplaces for those working in them.

65The re-performance framework can offer an articulation between the provisionally ready measure and the emerging organisational framework that can accommodate the uncertainties of the new data practices and the impact of these on the policy sector itself. It does this through re-performing processes that generate the measure in order to watch it work in the existing systems. If the data technology is not workable (i.e. no-one answers the question properly, data cannot be collected or wasn’t collected), then the overarching project has little utility. Re-performance helps to reveal these tensions to improve utility.

66Revealing tensions could be conceived as aggravating the ‘war on expertise’ (as in the title of this Special Issue), but as the two case studies show, if an example of institutional research is open to listening to ‘the public’, then in fact the re-performance framework has the capacity to be reparative. This is because it not only reveals how methods and measures which experts rely on as robust are not actually incontestable, but it also shows that the framework draws on ethnographic processes that reveal how the institutionalised data practices are sites of contestation in ways not ordinarily explored. They reveal how the public feels not only about the possibilities of having their happiness measured or about describing their life-course/identity/background, for example, but also about the real-life processes that comprise such a practice, whilst at the same time revealing anxieties of public sector workers under increasing pressures of datafication.

67Case Study 1 was retrospective. In that way, the performative utility of re-performance could only reflect on the methods and measures used and on the evidence that was presented as a comment on the evaluative claims made. To this end, a critical position was unlikely to be reparative; instead, it highlighted the clashing cultures of expertise – from the survey writer to the survey participant (Oman, 2015) – and the ‘selective traditions’ which prioritise inherited data and evidence formats and discourses (Oman, 2019c). Case Study 2, however, was undertaken in partnership with a policy institution. Although this policy institution was beholden to the demands of central government and to what was declared robust, the Case 2 institution was open to being told to do things very differently. To this end, the critical task was also solution-focused in that it endeavoured to be reparative: to understand the conflicting cultures of expertise, and to show how to overcome these.

68Finally, in each re-performance example, the actors responsible will vary according to the specific contexts and the challenges that need addressing. Re-performance could be an opportunity for trans-organisational expertise and participation in helping to prepare the processes that feed into EBPM. Presently, gaps in evidence are not revealed as knowledge moves, between actors and institutions, and thus inherited assumptions are allowed to reproduce unchecked. Crucially, this means that divides in cultures of expertise reproduce, rather than repair. Therefore, re-performance can be applied as a reparative methodology that retains its criticality.

Conclusion: theoretical and policy implications of the re-performance approach

69Critical Policy Studies needs to understand gaps in expert, everyday and policy knowledge: the relations and their breakdowns as part of EBPM and as part of the policy issues themselves. The proposed re-performance approach aims to reignite a sensitivity to the differentiated nature of expertise, feeling, and experience of social science methods and their effects in EBPM contexts. This article has shown how re-performing the methods of others in different policy contexts can reveal the cultural aspects of EBPM: methods are not neutral, but imbued with motives, misunderstandings and misrepresentations. The approach can also help to fill gaps in knowledge, to profile disregarded data and to foreground marginalised opinions, which are argued to be driving the crisis in expertise.

70The term ‘re-performance’ was grounded in its various theoretical lineages and applications to outline its methodological utility. The previous absence of re-performance from attempts to understand social science methods, EBPM or cultures of expertise is therefore a missed opportunity. I argue that this is an innovative, if ambitious, methodological framework that is intellectually robust, whist serving a need in EBPM processes, in Critical Data Studies and in Critical policy Studies.

71The paper has outlined an innovative, and conceptually driven methodology for understanding knowledge-making for policy. It shows how focusing on an element of an established method, and watching it work in context, reveals the weaknesses of a taken-for-granted approach. Furthermore, in revealing how analyses derived from unquestioned methods are performative and re-perform, binaries and boundaries of expertise and knowledge can be re-viewed from different standpoints across layers of research-for-policy.

72The article argues that to understand data practices in EBPM requires methodologies that capture contextual specificity, complexity and the mundane aspects of data and research policies, processes and practices – as experienced throughout society. The conceptual development and methodological innovation of re-performance interrogate the gap between different types of knowledge, as well as the performativity of, and within, this gap. These can be used to highlight issues in accountability that enable established data practices for EBPM to go unchecked and to bridge these divides. Retracing the original performance of research and data practices for EBPM through re-performance, then, ultimately aims for a more reflexive performance of social science research for policy.

Top of page

Bibliography

Ahmed, S. (2010). The Promise of Happiness. Durham, NC: Duke University Press.

Allin, P. (2007). Measuring societal wellbeing. Economic and Labour Market Review, 1(10), 46–52.

Bache, I. (2013). Measuring Quality of Life for Public Policy: An Idea Whose Time has Come? Agenda-Setting Dynamics in the European Union. Journal of European Public Policy, 20(1), 21–38.

Bache, I., & Reardon, L. (2013). An idea whose time has come? Explaining the rise of well-being in British politics. Political Studies, 61(4), 898–914.

BBC (2010, November 25). Plan to measure happiness 'not woolly' – Cameron. BBC News. Retrieved from:
https://www.bbc.com/news/uk-11833241

BBC (2021, February 17). Covid: Boris Johnson to focus on 'data, not dates' for lockdown easing. BBC News. Retrieved from:
https://www.bbc.co.uk/news/uk-56095552

BBC (2017). BBC Equality Information Report 2016/17. London: BBC

Bevir, M., & Rhodes, R. A. W. (2006). Governance Stories. London: Routledge.

Bevir, M., & Rhodes, R. A. W. (2010). The State as a Cultural Practice. Oxford: Oxford University Press.

Brook, O., O’Brien, D., & Taylor, M. (2020). Culture Is Bad for You: Inequality and the Creative Class. Manchester: Manchester University Press

Butler, J. (1999). Gender Trouble: Feminism and the Subversion of Identity. New York, NY: Routledge.

Butler, J (1993). Bodies that Matter: On the Discursive Limits of 'Sex'. London: Routledge.

Butler, J. (2010). Performative agency. Journal of Cultural Economy, 3(2), 147–161.

Cabinet Office (2016). Civil Service pilots new social mobility measures. Retrieved from:
https://www.gov.uk/government/news/civil-service-pilots-new-social-mobility-measures

Cairney, P., & Oliver, K. (2017). Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?. Health Research Policy Systems,15(35).
https://doi.org/10.1186/s12961-017-0192-x

Cameron, D. (2010). Prime Minister’s speech on wellbeing on 25 November 2010. Retrieved from:
https://www.gov.uk/government/speeches/pm-speech-on-wellbeing

Campbell, P., Cox, T., & O’Brien, D. (2017). The social life of measurement: how methods have shaped the idea of culture in urban regeneration. Journal of Cultural Economy, 10(1) 49–62.

Davies, P. (2004). Is evidence based government possible? Jerry Lee Lecture. Paper presented at the 4th annual Campbell Collaboration Colloquium. Washington, DC, 19 February 2004.Retrieved from:
https://webarchive.nationalarchives.gov.uk/20091013084422/http:/www.nationalschool.gov.uk/policyhub/downloads/JerryLeeLecture1202041.pdf

Davies, W. (2011, September) The political economy of unhappiness. New Left Review.

Davies, W. (2015). The Happiness Industry. London: Verso

Davies, W. (2018a). Nervous states: how feeling took over the world. London: Jonathan Cape.

Davies, W. (2018b, November 29). Why we stopped trusting elites. The Guardian, 2018. Retrieved from:
https://www.theguardian.com/news/2018/nov/29/why-we-stopped-trusting-elites-the-new-populism

Devlin, H. & Boseley, S. (2020, April 23) Scientists criticise UK government's 'following the science' claim. The Guardian.

Dobson, A. (2014). Listening for Democracy: Recognition, Representation, Reconciliation. Oxford: Oxford University Press.

Donne, J. & Parker, J. (1839). The Works of John Donne, D.D., Dean of Saint Paul's, 1621-1631, Vol. 3 of 6: With a Memoir of His Life. London: John W Parker

EHRC (2015). Employment Statutory Code of Practice. EHRC

Espeland, W. N., & Sauder, M. (2007). Rankings and reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1), 1–40.

Fischer, F. (2009). Democracy and Expertise: Reorienting Policy Inquiry. Oxford: Oxford University Press.

Freeman, R., & Sturdy, S. (2014). Introduction. In R. Freeman & S. Sturdy (Eds.), Knowledge in Policy: Embodied, Inscribed, Enacted (pp. 1-17). Bristol: Policy Press.

Friedman, S., Laurison, D., & Miles, A. (2015). Breaking the ‘Class’ Ceiling? Social Mobility into Britain’s Elite Occupations. The Sociological Review, 63(2), 259–289.

Gieryn, T. (1983). Boundary-Work and the Demarcation of Science from Non-Science: Strains and Interests in Professional Ideologies of Scientists. American Sociological Review, 48(6), 781–795.

Goldthorpe, J. H., Hope; K. (1972). Occupational grading and occupational prestige. Social Science Information, 11(5), 17–73.
https://doi.org/10.1177/053901847201100502

Greenhalgh, T. (2016). Cultural contexts of health: the use of narrative research in the health sector. Health Evidence Network Synthesis Report 49. Copenhagen: The Health Evidence Network/ WHO.

Hacking, I. (2002). Making up people. In I. Hacking, Historical Ontology (pp. 99-114). Cambridge: Harvard University Press.

Hoppe, R. (2010). “From ‘knowledge use’ towards ‘boundary work’: sketch of an emerging new agenda for inquiry into science-policy interaction”. In ‘t Veld, R. (Eds.), Knowledge Democracy (pp. 169-186). Berlin: Springer.

Hughes, L. (2020, June 10) Earlier lockdown could have halved UK deaths, says PM’s ex-adviser. Financial Times. Retrieved from:
https://www.ft.com/content/820761d1-8bb2-4386-9637-ce4280d7f6e3

Hallsworth, M., Parker, S. & Rutter, J. (2011). Policy making in the Real World. London: IFG.

Jackson, P. (2011) Governance by numbers: what have we learned over the past 30 years? Public Money and Management, 31(1), 13–26.
https://doi.org/10.1080/09540962.2011.545542

Kennedy, H. et al. (2020). Public understanding and perceptions of data practices: a review of existing research. University of Sheffield. Retrieved from:
http://livingwithdata.org/current-research/publications/

Kroll, C. (2011). Measuring progress and well-being: achievements and challenges of a new global movement. Berlin: Friedrich-Ebert-Stiftung

Laux, C., & Leuz, C. (2009). The crisis of fair-value accounting: making sense of the recent debate. Accounting, Organizations and Society, 34(6), 826–834.

Law, J. (2009). Seeing Like a Survey. Cultural Sociology, 3(2), 239–256.

Law , J., Ruppert, E., & Savage , M. (2011). The Double Social Life of Methods. Working Paper 95, CRESC, University of Manchester. Retrieved from:
http://hummedia.manchester.ac.uk/institutes/cresc/workingpapers/wp95.pdf

Leuz, C. (2018). Evidence-based policymaking: promise, challenges and opportunities for accounting and financial markets research. Accounting and Business Research, 48(5), 582–608.

Lexico (n.d.). Oxford University Press. Retrieved from:
https://www.lexico.com/definition/reperformance..

Liamputtong, P. (2011). Focus group methodology: principle and practice. London: SAGE Publications.

Matheson, J. (2011). National statistician’s reflections on the national debate on measuring national well-being. Newport: ONS.

Mitchell, A., & Sikka, P. (2002), Dirty Business — The Unchecked Power of Major Accountancy Firms. Basildon: Association for Accountancy and Business Affairs.

Mirowski, P. (2013). Never Let a Serious Crisis Go to Waste: How Neoliberalism Survived the Financial Meltdown. London: Verso.

Nutley, S. M., Walter, I. C., & Davies, H. T. O. (2007). Using Evidence: How Research Can Inform Public Services. Bristol: Policy Press.

O'Brien, D. (2016). What price evidence? The ethics of office and the ethics of social science in British cultural policy. Journal of Cultural Economy, 9(2), 127–140

O’Donnell, G. et al. (2014). Wellbeing and Policy. London: Legatum Institute.

O’Regan, D. (2004). Auditor’s dictionary: terms, concepts, processes, and regulations. Hoboken, NJ: John Wiley & Sons.

OECD (2010). A framework to measure the progress of societies - statistics directorate. Working paper no 34. Paris: Organisation for Economic Cooperation and Development.

Oman, S. (2017). All being well: Cultures of participation and the cult of measurement (Doctoral thesis, The University of Manchester).

Oman, S. (2019a). Improving Data Practices to Monitor Inequality and Introduce Social Mobility Measures: A Working Paper. Sheffield: The University of Sheffield.

Oman, S. (2019b). Re-ordering and re-performing: Re-placing cultural participation and re-viewing well-being measures. In B. Eriksson, C. Stage & B. Valtysson (Eds.), Cultures of participation – arts, digital media and cultural institutions (Chapter 12) London: Sage.

Oman, S. (2019c). The ‘selective tradition’ in well-being evidence for policy. Leisure Studies, 39(1), 11–25.
https://doi.org/10.1080/02614367.2019.1607536.

Oman, S. (2021). Understanding well-being data: improving social and cultural policy, practice and research. Basinkstoke: Palgrave MacMillan. https://doi.org/10.1007/978-3-030-72937-0.

Oman, S., & Bull, A. (2021). A feminist approach to joining up well-being and sexual misconduct policy-making in HE: Standing in the Gap'. The Sociological Review.
https://doi.org/10.1177/00380261211049024

Oman, S., & Taylor, M. (2018). The New Models of Subjective Well-being in Cultural Advocacy and the Politics of Research. Journal of Cultural Economy, 11(3).
https://doi.org/10.1080/17530350.2018.1435422

ONS (n.d.). Labour Force Survey.

Poovey, M. (1998). A history of the modern fact: problems of knowledge in the sciences of wealth and society. Chicago, IL: University of Chicago Press.

Porter, T. (1996). Trust in numbers: the pursuit of objectivity in science and public life. Princeton, NJ: Princeton University Press.

Saha, A. (2020). Why “diversity” in publishing is not enough. The New Statesman. Retrieved from:
https://www.newstatesman.com/culture/books/2020/06/publishing-diversity-black-lives-matter-authors-writers-colour

Saltelli, A., & Giampietro, M. (2007). What is wrong with evidence based policy, and how can it be improved? Futures, 91, August 2017, 62–71

Savage, M., & Burrows, R. (2007). The coming crisis of empirical sociology., Sociology, vol. 41(5), 885–899.

Savage, M. (2013). “The ‘Social Life of Methods”: A Critical Introduction. Theory, Culture & Society, 30(4), 3–21.

Savage, M. (2010). Identities and Social Change in Britain Since 1940: The Politics of Method. Oxford: OUP.

Stevens, A. (2011). Telling policy stories: An ethnographic study of the use of evidence in policy-making in the UK. Journal of Social Policy, 40, 237–256.

Stiglitz, J., Sen A., & Fitoussi J. (2009). Report by the Commission on the Measurement of Economic Performance and Social Progress. Organisation for Economic Cooperation and Development. Retrieved from:
http://www.stiglitzsenfitoussi.fr/documents/rapport_anglais.pdf

Thomas, A. (2008). Focus groups in qualitative research: culturally sensitive methodology for the Arabian Gulf?. International Journal of Research & Method in Education, 31(1), 77-88.

UK Government (2010). Equality Act 2010. Retrieved from:
https://www.legislation.gov.uk/ukpga/2010/15/contents.

van den Hengel, L. (2017). Archives of Affect: Performance, Reenactment, and the Becoming of Memory. In L.Muntean, L.Plate & A. Smelik (Eds.), Materializing Memory in Art and Popular Culture (pp.125-142). London: Routledge.

Watson, J. (1998). Affective Poetics and Scribal Reperformance in Lawman's "Brut": A Comparison of the Caligula and Otho Versions. Arthuriana, 8(3), 62–75.
https://doi.org/10.1353/art.1998.0041

Wilkinson, K. (2011). Organised Chaos: An Interpretive Approach to Evidence‐Based Policy. Making in Defra Political Studies, 59(4), 959–977

Top of page

Notes

1 Troubling, as the act of destabilising supposedly fixed, known entities.

2 Performance artist Marina Abramović impresses the importance of discussion on the politics and ethics of recording and conserving performance (van den Hengel, 2017). This includes confronting the lack of – and the quality of – record of original performance artworks (in the 1960s and 70s) as the incomplete archives rely on inferior photographs and inaccurate witness testimonies (ibid). ‘The only real way to document a performance art piece is to re-perform the piece itself’ states Abramović (2007, in van den Hengel, 2017, p. 131).

Top of page

References

Bibliographical reference

Susan Oman, Re-performance: a critical and reparative methodology for everyday expertise and data practice in policy knowledgeInternational Review of Public Policy, 3.3 | 2021, 291-313.

Electronic reference

Susan Oman, Re-performance: a critical and reparative methodology for everyday expertise and data practice in policy knowledgeInternational Review of Public Policy [Online], 3.3 | 2021, Online since 15 March 2021, connection on 16 April 2024. URL: http://journals.openedition.org/irpp/1833; DOI: https://doi.org/10.4000/irpp.1833

Top of page

About the author

Susan Oman

University of Sheffield, Information School
s.m.oman@sheffield.ac.uk

Top of page

Copyright

CC-BY-4.0

The text only may be used under licence CC BY 4.0. All other elements (illustrations, imported files) are “All rights reserved”, unless otherwise stated.

Top of page
Search OpenEdition Search

You will be redirected to OpenEdition Search