- 1 Here we follow Djelic and Sahlin-Andersson’s preference of the term ‘transnational’ versus ‘global (...)
1The production of global metrics by International Organisations (IOs) has had a large impact on transnational governance1. Metrics have infiltrated organisational cultures and the environments these organisations inhabit; crucially, they are reshaping the ways International Organisations work, compete and survive in an increasingly quantified, yet uncertain world. Although scholarship has devoted a lot of attention to statistical knowledge production by organisations like the OECD, the World Bank, UNESCO and many others, we know far less about parallel processes of construction of ‘non-knowledge’; that is, the ‘conscious or unconscious, concrete or theoretical… wilful ignorance or an inability-to-know’ (Beck, 2009; 123). Therefore, this article’s focus is on the enactment of ‘non-knowledge’ in the governance of education and well-being; or, in other words, the strategic prioritization of certain knowledge versus other.
- 2 METRO is receiving funding from the European Research Council (ERC) under the European Union's Hor (...)
2Building on political sociology, science and technology studies (STS), and using theoretical strands from critical accounting and organisational sociology, as well as the newly emerging field of the sociology of quantification, this paper builds on the findings of the research project ‘International Organisations and the Rise of a Global Metrological Field’2 (METRO). The project examines the production of statistical knowledge by International Organisations (IOs) in their efforts to find common solutions to some of the world’s biggest challenges, such as poverty, climate change and global health (Grek, 2020). Education and Development are the focal cases for our examination: in both policy areas, IOs have been central to processes of standardisation, de-contextualisation and performance management through numbers; as a result, they have been instrumental in commensurating, and therefore transforming both policy fields. In addition, Education and Development have been attracting larger policy significance, as they are increasingly considered central to both economic prosperity and social cohesion. The construction of the Sustainable Development Goals (SDGs), with monitoring processes that relate to both education and sustainability, has become a productive arena for an examination of how quantification impacts on the ways IOs shape the production of knowledge and non-knowledge.
3A central focus of our study are the moves of large IOs to establish partnerships in order to push for large transnational statistical agendas. The making of the SDGs is probably one of the largest and most ambitious collaborative statistical projects so far devised. We see the construction and validation of indicators in these collaborations by different IOs as a key moment when negotations and selections over the kinds of knowledge to be prioritized are made. Additionally, IOs such as the World Bank, UNESCO and the OECD, despite their increasing convergence over the ‘education-economic growth’ agenda (Resnik, 2006), still have quite substantial differences in their organizational and knowledge production cultures. The encoding and decoding of data processes and ways of working that these collaborative monitoring endeavours require (in order for consensus to be reached and data to be shared and co-produced), allows a comprehensive analysis of the workings and effects of the strategic production of knowledge in transnational governance. More specifically, through the examination of how country performance data is collected and interpreted, this paper uses two examples of the active construction of (non-)knowledge: these are first, the controversies over the making of indicators as part of the education-related SDG (SDG4); and second, the data visualisations of the OECD’s Better Life Index.
4METRO’s findings point us towards a closer investigation of the construction of consensus around the specifying and monitoring of the measurement goals. These meetings that bring together a range of actors, from the local to the international levels are, as we shall show below, those slow and convoluted processes that ‘wicked’ problems (Guy Peters, 2017) are discussed and a range of possible monitoring solutions agreed upon. As this paper shows, the process of collective problematisation requires not only the co-construction of knowledge by the relevant IOs, but also another significant function of theirs: that of the making of ignorance, or as we prefer to call it, non-knowledge. The social production of non-knowledge is a necessary precondition for reaching agreement about what kind of knowledge will be pursued in order to achieve a minimum consensus, so as to ensure ‘buy-in’ but also maintain actors’ own interests, values and positionings intact (Grek, 2020). The construction of non-knowledge is an essential part of the measurement process: rather than the opposite of knowledge however, or its reading as a binary, we like to view it as a symbiotic relationship, necessary for balancing out and achieving some kind of constant equilibrium – and hence movement – of the metrological field.
5Indeed, although the focus of sociological analysis on the making of ignorance seems to be relatively recent, philosophically and historically there is a certain symmetry in the making of both knowledge and non-knowledge; a process which has been political, strategic and thus, for the purposes of this paper, particularly productive in making sense of processes of quantification. The argument here is that, at least in the field of transnational performance measurement agendas, the making of any knowledge implies simultaneously the omission of other routes to knowledge; or, in other words, the active production of non-knowledge.
6Rather than suggest that the process of the fabrication of non-knowledge is an a-political process (ie the pre-requisite of the production of any knowledge, in that the latter is always selective and built on choices), this paper purports that it is precisely the opposite: it is political and represents governing at its most effective. This is because institutional orders can only be built on the basis of the selection of relevant, emergent ‘issues’ that turn into ‘problems’ through a continuous process of selecting, de-selecting, omitting and including parts (in METRO’s case usually datasets), so as to finally conclude in the construction of ‘the (policy) problem’, the solution to which usually lies in data already located and manipulated in order to ‘fit’ the problem in question. Although consensus over the ‘problem’ may never be truly reached, it is precisely the negotiations, conflicts and compromises that keep the metrological game going. Here we follow Rayner (2012), who suggests that for every ‘wicked problem’, some ‘clumsy solution’ is already available: or, in other words, there can be no ‘wicked problem’ unless it has already found its response in a ‘clumsy solution’. As Rayner suggests, the denial, dismissal, or active omission of ‘uncomfortable knowledge’ is an essential tool for organisations to coalesce around a minimum consensus. Non-knowledge is an integral element of the making of powerful technocracies, such as large IOs. It seems that due to the focus of analysis on the epistemic authority that technocratic rationality lends to their work, we have continued this analysis too long without acknowledging the role that strategic ‘unknowability’ might have in the measurement work of the actors that we observe.
7Empirically, as already suggested, the paper uses education and well-being as our ‘cases’; education policy, both in the global South and the global North, has increasingly been dependent on the measurement of its performance for the improvement of human capital. In addition, education can be a productive vantage point, since assessment and quantification of performance have a very long history in the field. It is a key element in the newly emergent well-being and ‘better life’ strategies that have prevailed the statistical governing project post financial crisis (Stieglitz et al, 2009). Education is closely congruent with the efforts to use ‘softer’ data sets for calculating the social. Last but not least, it is one of those policy areas that large IOs like UNESCO, the OECD, the European Commission and the World Bank have invested large amounts of data and expertise from the mid-20th century on.
8On the other hand, well-being has itself become an ‘international regime’: a concept constructed on the basis of the failure of past indicators, like the GDP, to measure economic growth. Hence, the concept of ‘well-being’ is now central to the work of many IOs in the Global South. In the following sections, I will first briefly review the growing literature on the politics and practices of quantification in governing, before moving on to a discussion of the article’s two empirical cases. The article will finish off with a discussion of the rise of ‘ignorance studies’ and the promise of political sociology as a theoretical and empirical frame in the fields of measurement and transnational governance.
9Scholarship on the role of numbers in governing societies has been abundant and has attracted multiple fields of study, including sociology, history, political science, geography, anthropology, philosophy, STS, and others. Prominent authors have written lucidly about the role of numbers in the making of modern states and the governing role of measurement regimes in various areas of public policy and social life (Alonso & Starr 1987; Hacking, 1990, 2007; Porter, 1995; Power, 1997; Desrosières, 1998; Rose, 1999; Espeland & Stevens, 2008). Similarly, anthropologies of numbers suggest that ‘our lives are increasingly governed by – and through – numbers, indicators, algorithms and audits and the ever-present concerns with the management of risk’ (Shore & Wright, 2015: 23; see also influential work by Merry 2011; Sauder & Espeland, 2009; Strathern, 2000). Further, important insights and perspectives on indicators in particular come from STS (Bowker & Star, 1999; Lampland & Star, 2009; Latour, 1987), including actor network theory (Latour, 2005). Finally, there is a growing body of studies relating to specific uses of indicators and quantification in transnational governance contexts (for example, Bogdandy, Dann & Goldmann 2008; Palan, 2006; Martens, 2007; Fougner, 2008; Bhuta 2012).
10Nonetheless, despite the burgeoning number of publications on the global ‘governing by numbers’, our understanding of the relationship of the politics of measurement and the making of transnational governance is less well-examined; as Djelic & Sahlin-Andersson (2006) suggest, due to the fluidity and complexity of the intense cross-boundary networks and soft regulation regimes that dominate the transnational space, transnational governance is a particularly productive field of enquiry on the role of numbers in governing. This lack of attention could be due to disciplinary boundaries; for example, scholars of IR have not paid much attention to the field so far, although there is a rise in some interesting literature of the role of numbers in global political economy (for example, Palan, 2006; Martens, 2007; Fougner, 2008).
11What are the properties of numbers that would suggest such a central role in the production of transnational governance? By contrasting numbers to language, Hansen & Porter (2012) suggest that, although it took scholars a long time to recognize the constitutive nature of discourse, we are now well aware of the role of language in shaping reality. However, they suggest that numbers are characterized by additional qualities that make their influence much more pervasive than words: these elements are order; mobility; stability; combinability; and precision. By using the example of the barcode, they lucidly illustrate ‘how numerical operations at different levels powerfully contribute to the ordering of the transnational activities of states, businesses and people’ (2012: 410). They suggest the need to focus not only on the nominal qualities of the numbers themselves but, according to Hacking, ‘the people classified, the experts who classify, study and help them, the institutions within which the experts and their subjects interact, and through which authorities control’ (2007: 295).
12It is precisely on the data experts that METRO focuses upon; following the literature on the capacities of numbers to both be stable yet travel fast and without borders, we need to cast light on what Latour called ‘the few obligatory passage points’ (1987: 245): in their movement, data go through successive reductions of complexity until they reach simplified enough state that can travel back ‘from the field to the laboratory, from a distant land to the map-maker’s table’ (Hansen & Porter, 2012: 412). IOs constitute such ‘centres of calculation’; this however, according to Merry, does not suggest that they are significant only in terms of their knowledge production capacities; as this article will show, it is equally pertinent to examine the choices they make in regard to what not to know. By examining specifically the role of indicators in transnational governance, Merry elucidates their governance effects (2011); consequently, if we consider IOs central in the production of both knowledge and non-knowledge, we can infer that their operation as large technocracies must have crucial governing impact. These effects empower IOs and set them in a complex and ever-evolving power game for influence and resources – through an examination of the interplay and interconnectedness of IOs’ data apparatuses, it is precisely this power game and its rules that we need to unravel. Indeed, Shore and Wright argue that:
13while numbers and “facts” have both knowledge effects and governance effects, it is also important to consider how these are produced, who designs them, what underlying assumptions about society shape the choice of what to measure, how they deal with missing data, and what interests they serve (2015: 433).
14The paper follows a ‘constructivist-institutionalist’ approach (Smith, 2009), as we work with Lagroye’s definition of governing as ‘a set of practices which participate in the organization and the orientation of social life’ (1997: 25). Thus, METRO builds on the premise that far from being a system composed uniquely of ‘national’ and ‘transnational’ bodies, governing the transnational is an ‘Institutional Order’ made up of all the actors who participate in the construction and institutionalization of global problems (Smith, 2009). In turn, transnational ‘governing’ is conceptualised as those ‘assemblages of apparatuses, processes and practices’ that make governing happen (Clarke & Ozga, 2011).
15As already suggested, a considerable body of research has already focused on the work of IOs in transnational governance. Yet this research has often seen them as monolithic institutions, or actors with similar interests in a similar context, without attention to the complex set of realities that bring them together and apart over time (with notable exceptions of course, see for example Cini 2008). IOs are often also seen as internally stable –this means that divisions of authority, institutionalised norms, expectations and values are thought to be commonly shared by all actors within an IO. Nevertheless, ‘most of the time, […] at least some of the actors within an IO will be seeking to change at least some of its institutions, whilst others will work to retain their stasis’ (Jullien & Smith, 2010: 4). The examination of actor alliance formation and mobilisation is hence vital in order to understand these relations –both upstream, i.e. the setting of rules and problem framing, as well as downstream, namely the application and maintenance of rules amongst the actors who are all engaged in competitive relationships (Jullien & Smith, 2010). Indeed, some of this actor mobilization and alliance-building is achieved not internally but through networking with other IOs.
16Thus, one of the key concepts that mobilises this research is the notion of ‘political work’ (Smith, 2009), as it is very rich at a number of levels relevant to our research agenda. When one studies political work, institutions themselves are not the objects of study per se; rather, the focus of the investigation is on the continual cycle of institutionalisation, deinstitutionalisation and reinstitutionalisation of ideas and values within the organisation in question and the external environment it is part of. The study of quantification as a policy instrument, can become a particularly fruitful context for such an analysis as one can examine ‘political work’ as those processes that engender the construction of new arguments and the activation of new alliances; subsequently, they either produce change or reproduce institutions, namely actors’ rules, norms and expectations (Jullien & Smith, 2010). In this context, the process of the construction of knowledge (and hence non-knowledge) is key in achieving some kind of consensus around what ‘the problem’ is and hence its solution. Empirically, and building on the rich literatures from STS, political and organisational sociology, and social studies of metrics, the paper operationalises the notion of political work through two lenses: first, looking specifically at actors and the work required to achieve consensus; and second, the data visualizations of ranking indexes and the inclusions and exclusions of knowledge required in the representation of country performance. Through a focus on the making of knowledge and non-knowledge in the context of the production of the education-related SDG (SDG4) and the OECD’s Better Life Index, the next sections will operationalize these conceptual tools, in order to give an illustration of what such processes of the making of non-knowledge entail.
17From a methodological point of view, the paper is largely informed by the Critical Discourse Analysis of a large volume of SDG4 related reports and documentation, as well as twenty interviews with key actors in the production of statistical data for Education 2030 (SDG4). Critical Discourse Analysis (CDA) (Chouliaraki & Fairclough, 1999) has been used in order to establish key topics discussed in indicator-related texts, such as external documents, minutes of meetings and conference presentations. The aim has been to use this method in order to go deeper into an investigation of the ways language contributes to the framing of new issues and the construction of new worldviews. On the other hand, the in-depth interviews have been used to elaborate on nuances and explain aspects of the actors’ work and experiences of meetings and events that are not easily depicted in text analysis. Although interview data has not been included in the analysis in this paper, it has still informed the case selection, since the topic of prioritizing certain data over other was one that emerged and re-emerged several times by the interviewees.
18Second, the paper examines one of the OECD’s initiatives to map country wellbeing data – the ‘Better Life Index’ (OECD, 2020), launched in 2011. It is an interactive data visualisation that allows people to compare countries' performance according to their own preferences in terms of what makes for a ‘better life’. It includes eleven performance dimensions of wellbeing, namely: housing, income, jobs, community, education, environment, governance, health, life satisfaction, safety, and work-life balance. The analysis of Better Life Index clearly illustrates the evolution of the ranking visualizations as OECD in the past was one of the key producers of ‘naming and shaming’ league tables, most notably their Programme for International Student Assessment (PISA) (see Grek, 2009). With the Better Life Index, the OECD appears to have shifted its approach to the communication of performance assessments. This is a response to calls – such as the authoritative report by Stiglitz, Sen and Fitousi (2009: 12) – that advocate for “measurement system[s] to shift emphasis from measuring economic production to measuring people’s wellbeing”. This softer take on the handling of country performance data emerges in the interactivity of their new ranking visualizations, which – as I will show – convey more multi-layered meanings compared to league tables.
19The analysis of the interplay of different visual elements – such as colours, icons, headings, and graphics – allows us to unpack the data visuals’ rhetorical functions (see Quattrone, 2017). Some of these rhetorical functions include their role in guiding the user through an interface, illustrate relationships between elements, provide context and tone, focus attention, and increase the impact of specific messages. From this perspective, visuals are value-laden materializations of specific visions of the world that make visible (or invisible) possible realities (Latour, 1986). Analysing data visually and discursively foregrounds how visualization is not a soulless depiction but the outcome of a process of work:
And it is the site for the construction of and depiction of social difference. To understand a visualisation is thus to inquire into its provenance and the social work it does. It is to note its principles of exclusion and inclusion, to […] decode the hierarchies and differences that it naturalises. And it is also to analyse the ways in which authorship is constructed or concealed and the sense of audience is realised (Fyfe & Law, 1992: 1; my emphasis).
20Thus, data visualizations are an ideal site to explore issues of epistemic inclusions and exclusions, as they are largely made quantitatively and denote arithmetic values whilst relying systematically on visual codes (through shapes, colours, and lines) that connote social, moral, and political values. The visual and discursive analysis of country performance data is an attempt to decode the constitutive elements of those data in order to make sense of the ways visual elements work metaphorically and evocatively in the making of new ways of seeing and (un)knowing the world.
21With these methodological considerations in mind, we will now proceed in the discussion of our first case of analysis, that is the construction of non-knowledge in the SDG4 and the making of the Better Life Index.
22In May 2015, a World Education Forum (WEF) was celebrated in Incheon (Republic of Korea) with the participation of over 1,500 people, including 120 Ministers of Education and representatives from a wide range of international governmental and non-governmental organizations. The gathering was organised and promoted as the successor or the Jomtien and Dakar meetings, which took place respectively in 1990 and 2000. Both of these meetings were widely acknowledged as milestones in the development and consolidation of the ‘Education for All’ movement. The main product of WEF 2015 was the so-called Incheon Declaration, along with the Framework for Action adopted by UNESCO Member States in November 2015. In conjunction, both documents established an ambitious and highly aspirational education agenda for the timeframe 2015-2030, condensed in the overarching goal to “ensure inclusive and equitable quality education and promote lifelong learning opportunities for all” and a number of associated targets (UNESCO, 2016).
23What is important here is that both documents were the product of a long, multi-layered and multi-sited negotiation process that involved numerous meetings and consultations, largely led by UNESCO under the auspices of the ‘Education for All’ (EFA) Steering Committee. At the same time, it should be noted that efforts towards the development of this agenda were in turn paralleled by the negotiation of the Sustainable Development Goals (SDGs)– the latter was one of the cornerstones of the 2030 Agenda adopted by the General Assembly of the United Nations in September 2015 and devised as a follow-up of the Millennium Development Goals. In fact, the EFA-led process and the debates facilitated by the UN Open Working Group on Sustainable Development Goals reinforced and informed one another through an intricate political process, eventually crystallizing in a single agenda conventionally known as SDG4/Education 2030 – a denomination reminiscent of the dual origins of the new set of goals.
24Education 2030/SDG4 represented simultaneously a form of continuity and a departure from previous instances of goal-setting such as EFA and the Millennium Development Goals. As a programmatic document oriented at nurturing and securing a form of collective commitment towards a shared set of aspirations, the new agenda builds on a well-established tradition of consulation and collaboration that has come to be recognized as a characteristic of the UN system. However, Education 2030 entails a certain discontinuity regarding education goal-setting practices – both in content and procedural terms.
25First, and as different scholars have noted, the new set of goals is characterized by an unprecedented degree of ambition, shifting away from the focus on primary education and gender equality that characterized the MDG era, but also expanding on the vision set up by the EFA program. It establishes a truly universal agenda that contrasts with the prior focus on developing countries (King, 2017; Unterhalter, 2019). Secondly, the very making of Education 2030 (and of the SDGs more in general) represents a path-breaking development in the long history of goal-setting practices and UN summitry. The open, inclusive and participatory nature of the consultative process facilitated by UNESCO and the EFA architecture was in many ways unprecedented, and the openly-negotiated and improvisatory character of the SDG debate contrasted with the technocratic origins of the MDGs (cf. Fukuda-Parr & McNeill, 2019).
26In many ways, it is precisely this open debate and the participatory nature of the SDG governing architecture that has allowed a plethora of contestations to unfold: one of most prominent ones is the large emphasis on some indicators (especially those that measure performance in literacy and mathematics) that comprise goal 4 versus others. The table below3 offers a useful overview of the different indicators in goal 4:
Figure 1.The SDG4 indicators are as follows: 4.1.1 on reading and maths proficiency; 4.2.1/2 on early childhood; 4.3.1 on VET; 4.4.1 on ICT skills; 4.5.1 on gender equality; 4.6.1 on adult literacy and numeracy; 4.7.1 on global citizenship and sustainable development
27Although the development of SDG4 has been described as ‘arguably the most inclusive process of consultation in the history of the United Nations’ (Naidoo, 2016), this was not matched by the making of the relevant indicators to measure the ambitions (Smith, 2019; McGrath & Nolan, 2016). The process became quite technical from the start; as it was expected, statisticians and their considerations for valid and robust data took hold of the process and most non-statistical knowledge was excluded. This was not however the only omission; perhaps the more significant one took place when there was an early decision upon some indicators which would be considered ‘global’ versus those that were relegated to the description of ‘thematic’ (Smith, 2019). This was a key moment, since,
28While global indicators are universally applied and expected to be reported by all countries, thematic indicators are considered voluntary. Therefore, the majority of resources in indicator creation, monitoring, reporting and state action will focus on the global indicators while thematic indicators are not taken into account in the UN’s annual SDG report (Smith, 2019:3).
29The pendulum had already swung. Although target 4.1 was promising that ‘by 2030, ensure that all girls and boys complete free, equitable and quality primary education leading to relevant and effective learning outcomes’, the 4.1.1 global indicator that came to be associated with it was much more limited in reporting on ‘quality’ only, whereas free and equitable education were downgraded to ‘thematic’, if they were even considered at all (King, 2017). In other words, the production of certain knowledge was privileged over others; this of course was done (and is always done) on the basis of the methodological robustness and validity of the exercise.
30Indeed, many of the interviewees that METRO examined, suggested that the fundamental problem of the SDGs lies in the fact that it began the process by setting the ambitions and establishing the goals, rather than checking whether there was enough data or the right methodologies to monitor them. Thus, the (limited arguably) resources that were put in the process, were invested in indicators that were already backed up with significant statistical evidence. The strategic choice to construct non-knowledge by emphasising some indicators versus others, become even more evident in the tensions that the negotiations around indicator 4.7 as created. As Antonia Wulff, from Education International, contends,
The expert group in charge of the SDG indicators rejected the proposed measurement strategy for target 4.7 on education for sustainable development, human rights and global citizenship… Education International is generally concerned about the slow progress made on key indicators and, importantly, the large disparity in the time, effort and resources put into developing 4.7 indicators as opposed to the learning outcomes under target 4.1. We are impatient to move forward. (Wulff, 2018)
31Although limitations of space in the present article do not allow for a more extensive empirical analysis of the privileging of certain kinds of knowledge over others, the above example serves as a useful illustration of the making of ‘non-knowlegde’; rather than simply an ‘inability-to-know’, we have seen how strategic decisions were made in relation to which knowledge was prioritized to be produced. One of the most significant repercussions of quantification has been the fact that whatever is quantified and measured becomes visible, in antithesis to aspects of social life less easy to count; although a collectively agreed ambition, indicator 4.7 on global citizenship, unless prioritized, measured and backed up with data soon, will always remain an unachieved utopia, rather than become a reality.
32Moving on to our second empirical example, the Better Life Index (OECD, 2020) is a core part of the OECD’s Better Life Initiative. It is an interactive report based on statistical data released every two to three years, that offers evidence on a range of well-being indicators and their variation over time, between population groups and across countries. The index is based on a multi-dimensional indicators framework that aggregates data provided by 37 OECD countries and 4 partner countries, reporting on more than 80 well-being indicators.
33The Better Life Index interface does not resemble other graphical representations of statistical indicators as we know them (figure 2). The Index resembles a meadow, full of colourful flowers, with numerous petals of different colours and sizes. The country names serve as the ‘stems’ of these flowers, as the default alphabetical order of the floral scene allows for the ‘meadow’ to almost acquire a dynamic character; the visual diversity of the petals and their positioning against the light background creates a sense of movement, proximity, and heterogeneity.
Figure 2. Homepage of Better Life Index: alphabetical view by default versus ranked view
34It is important to note that this is the default image. At the bottom right, the table allows to move from the alphabetical to a ranked visualization. Here, the details of the flowers become impossible to distinguish immediately as the flowers (standing for countries) are visualized in an upward performance trend, with little explanation. If we zoom on any flower this is what we see:
Figure 3. Close-up of the highest and lowest performing ‘flowers’ in the ranking
35Every colour-coded petal represents an individual wellbeing indicator, building on a 0-10 scale. In the ranked visual option, we can see how the aggregation of all the parameters creates a hierarchical ordering of countries by ‘well-being’ scores. Hence, we can see how these ‘beautiful pictures’ (Espeland & Stevens, 2008) do not only represent evidence, but they also seek to entice user participation. Whilst the flowery landscape is visually playful, there is an explicit invitation to the user: ‘Create Your Better Life Index’. One can adjust the significance that several aspects of social life carry according to their preferences regarding the importance they attribute to different elements of the ranking.
36The interactivity of the visual gives the impression that the user’s own preferences are the engines behind the construction (and the potential for multiple reconstructions) of their ‘ideal’ flower that is to be found in the field. Importantly, one can also decide to exclude one or more parameters of evaluation from the ranking, should they not be relevant to their evaluation processes. The room for personalization not only allows the OECD to gather user data but this visual ranking is also marketed as a tool for non-experts to identify a country that more meets their priorities and value systems, while learning potentially interesting information on their own country of origin or potential future destinations.
37The OECD does not conceal that this index is ultimately a wellbeing ranking. However, more emphasis is on how the index allows the (re-)construction of a combined perspective of country performance, users’ preferences, and multiple policy areas. Once one begins to ‘play’ with the tool, the interaction possibilities seem endless. The country-level statistical data are not lifeless; on the contrary, through interactive visual features, they encourage playful behaviour and a spirit of discovery deliberately appealing to the edutainment function of the ranking. These affordances allow the user to create a world of their own and orchestrate the hierarchical order of countries based on their personalised parameters of evaluation. Such an approach is especially surprising in the context of the OECD’s historical propensity to rely on league tables and naming and shaming as their key strategy (e.g. in the case of the PISA rankings – Grek, 2009). Contrariwise, in this case, the OECD adopts a much softer and indirect way of using rankings, by disguising the hierarchical ordering of performance behind a customizable visual interface. As Luigi Pirandello would say, the Better Life Index homepage suggests to its viewers nothing less than ‘It is so, if you think so’.
38Importantly, the visually intriguing features of the Index do not stop here as it allows for the exploration of the same data in a map format, which serves as a tool for contextualisation. The interactive elements of the map allow one to choose which different circles to explore; comparisons can also be drawn.
Figure 4. Mapping wellbeing – survey results
39The map offers important clues concerning the measurement of wellbeing in different world regions. Different regional comparisons appear; they allow the user to create their own comparisons with other regions, although the tool can by default select comparable regions by itself, too. By scrolling further down the page, direct comparisons, and ranked performance of the region against other regions of the same country is offered.
Figure 5. Country performance across different dimensions of well-being
40The Better Life Index, therefore, illustrates the key quality of interactive data visualisation in the global governance by numbers – they seek to come across as seemingly apolitical, yet they convey a political message. By offering the users the ability to create their own rankings – according to their values, interests, and priorities – the OECD ranks countries without providing a set (and more politically sensitive) list of winners and losers in the ‘soft’ policy area of wellbeing. Instead, it provides a unity of experience and reflects the move towards more equitable relationships between countries as equal participants in the global sustainability agenda. The strategic construction of knowledge in these data visualisations is prioritising certain ways of engaging with the data versus others: that is, instead of using the traditional format of league tables and the peer pressure that these would create, they instead promote a much flatter, multivocal and open way of engaging with the indicators. We observe therefore the strategic avoidance of the hierarchical ordering of performance (the non-knowledge) versus the tactical preference for constructing a plane field where country rankings are constructed in multiple ways and always according to the audience’s preferences. In this context, this relates to the propensity of newer performance monitoring agendas to create the conditions that increase trust in numbers through allowing reflexivity and even play.
41To sum up to the paper’s earlier discussion, recent years have seen the rise of the sociology of ignorance as a new field of studies that examines the other, less visible side of the politics of constructing knowledge: that is, the politics of the strategic construction of ‘non-knowledge’. Linsey McGoey has been one of the key advocates of the need for social science to examine ‘the mobilisation of ambiguity, the denial of unsettling facts, the realisation that knowing the least amount possible is often the most indispensable tool for managing risks’ (McGoey, 2012a: 3).
42The consideration of the symmetry of knowledge / non-knowledge is not of course new. Socrates insisted that his ‘wisdom’ was derived by his knowledge of what he didn’t know. Philosophically and historically the realisation of the limits of the human knowledge has always been present. Nevertheless, our over- emphasis on examining the political uses of knowledge in governing societies has resulted in not engaging nearly enough with the systematic and strategic construction of non-knowledge. Non-knowledge here is not seen as an impediment and obstacle to knowing, but as a productive force, that strengthens the role of knowledge and of the knowing subject. For scholars in the field of ignorance studies, we need to investigate non-knowledge as “regular” rather than “deviant” (Gross & McGoey, 2015: 4). Yet, to date these discussions lack a coherent, agreed-upon nomenclature (Smithson, 2008). Although some scholars use ignorance and non-knowledge interchangeably (e.g. Kleinman & Suryanarayanan, 2013: 495), others distinguish between the two (e.g. Gross, 2012), emphasising the need to avoid the negative connotations that the word ‘ignorance’ implies. Further, there are also scholars who have developed taxonomies of different types of ignorance and non-knowledge (e.g. Aradau, 2017; Beck & Wehling, 2012; Gross, 2016).
43A review of the literature in the growing field of ignorance studies would be beyond the scope of this article. However, the key message that most of this literature appears to agree upon, despite the differences in terminology, is that non-knowledge is productive and not just the negative side of knowledge. Actors may actively try to nurture and preserve ignorance to use it as a resource to advance their interests be it in claiming more funding, denial of responsibility, or assertion of expertise (McGoey, 2012b: 555). Importantly, McGoey emphasizes that such production and use of non-knowledge may be strategic and deliberate, but not necessarily conscious. Mallard and McGoey go further to propose an epistemological position ‘which asserts as a general maxim that ignorance can be an equally powerful political resource as knowledge’ (2018:3). They suggest that:
A second exploration by social scientists of how policymakers, experts and bureaucrats contribute to the production of soft forms of ignorance in international affairs… is the literature on the production of indicators, ratings, benchmarks which now circulate everywhere in the world of IOs and global media (Davis, Fisher, Kingsbury, and Merry 2012; Espeland and Sauder 2007; Espeland and Vannebo 2007). As scholars of transparency and auditing practices have long pointed out (cf. Strathern 2000; Power 1997), such indicators help to make policy decisions appear as if they belong to the realm of the certain and unquestionable even when policy options are based on the flimsiest set of factual observations. Most ‘global governance’ apologists who applaud the increasing use of benchmarking in policy research rarely acknowledge that the production of most indicators (like ‘rule of law’ indexes) is based upon fragile methodological foundations, and that the process of turning measurements into policy recommendations most often turns uncertainties and approximations into certainties… (Davis et al., 2012).
44Indeed, it is precisely the construction of the doxa of a governable, measurable world that paradoxically the production of non-knowledge results in: in such a world, actors that participate in its making, have to be selective and actively and purposefully ignore inconvenient data, or, as the empirical example of the SDG4 above illustrated, systematically disregard the development of some measurement tools versus others. On the other hand, the case of the OECD’s Better Life Index powerfully illustrates the masterful ability of IOs to strategically deploy data visualisations in order to create precisely the opposite effect; that is, instead of narrowing down the measurement field, they create enough interpretative flexibility so as to increase buy-in and engagement, rather than construct a field of ‘winners and losers’ (although these kinds of hierarchical representation of performance data are, of course, still very much present in the IOs’ performance monitoring arsenal). As recently one of the METRO interviewees emphatically suggested, ‘it is art, not science’. This art of assembling knowledge, while actively and strategically constructing non-knowledge, is necessary in order to leave the epistemic authority of the solutions uncompromised (no matter how ‘clumsy’ these may be), as well as having the door always open to the construction of new problems and solutions once the previous ones fail. The two empirical examples discussed in this paper powerfully illustrate that the pendulum does not always swing towards greater standardisation and the narrow selection of some quantitative measures over others. Although this seems to be the case with the SDG4, the Better Life Index representation of quantitative data is much ‘softer’ and pushes the other direction; this is the need for alignment around social principles and aims that, at least in the Global South, have been prevalent in the ontological and epistemological repertoires of indigenous and disadvantaged communities for centuries (for a more thorough analysis of data visualisations as alignment devices, please read Bandola-Gill, Grek & Ronzani, 2021).
45Although the field of ignorance studies has put a lot of emphasis on classifying kinds of non-knowledge, it has so far not achieved a coherent set of ideas about how to investigate the process of producing non-knowledge. The most notable exception to this is Scheel and Ustek-Spilda’s (2019) work; the latter use the notion of enactment from STS, whilst also making references to the concept of controversies, and in particular the examination of cases of non-transfer of knowledge – the moments of distortion, reinterpretation and loss that may occur when ‘data move between people, substates, organizations, or machines” (Edwards, Mayernik, Batcheller, Bowker & Borgman, 2011: 669). The paper contends that the attention to the particularities, representations and often visualisations (through graphs, maps and other visuals) that the enactment agenda allows could be seen a helpful way of investigating the tools and effects of the strategic production of (non)knowledge.
46Nonetheless, the empirical cases discussed above invite a frame of analysis that can also go beyond thinking of the production of knowledge and non-knowledge as simply an ‘enactment’ – as a performative event, of the kind that quantification practices are often seen as producing. Rather, they can be analysed through a political sociology theoretical framing, which can help make sense of the ways that it is the construction of policy problems (‘problematisation’) that often goes hand in hand with the production of non-knowledge. How is one to examine processes of problematisation? The promise of political sociology is a return to sociological analysis for interpreting political phenomena that span nations and localities. Although the focus of a number of (French almost exclusively) political sociologists is an analysis and interpretation of European integration, this paper is utilising some of their analytical concepts and empirical tools in order to cast light on the strategic production of non-knowledge.
47As suggested earlier, a focus on actors (as the case of the SDG4 showed) and the processes of achieving consensus (as the Better Life Index visuals portrayed) are the sine qua non components of the making of metrological realism. A focus on actors –and their involvement, shift, resistance or control- is nothing new; it has always been a significant tool for policy convergence theories. Nonetheless, through the triptych of ‘problematisation- institutionalization- legitimation’, political sociology offers the analytical and empirical toolkit to combine the study of formal institutions and informal practices with a variety of sociological indicators (social trajectories, academic background, careers and so on). Drawing on political sociology, this paper argues that focusing on the actors and processes involved in transnational monitoring agendas can take us beyond classical dichotomies, such as structure/agency, individual/collective, rational/unconscious, in order to understand what social agents involved in these processes think, say and do. Following Georgakakis and Weisbein,
our aim is to understand social phenomena as the product of an encounter (rarely conscious but played out in practice) between, on the one hand, (individual and collective) dispositions to act (habitus), which may be inherited, acquired through social and professional paths or offered by the position, and on the other hand, so-called relational contexts, which may be analyzed under various forms, in organizations, institutions and fields...Based on these biographies, the actors’ positions are established, not only in terms of membership (to a country, an institution, a unit within an organization and so on), but according to the structure of the social actors’ resources and experiences (2010: 6).
48To conclude, this article mobilized relevant literature and used two empirical examples in order to offer two propositions: first, that an investigation of metrological realism needs to focus on the social construction of non-knowledge as a vital component of studying the epistemic authority of transnational institutions; and second, that a political sociology agenda allows for a deeper understanding of the role of actors (and their strategic representations) in advancing their position in the transnational field by either pushing or pulling the construction of new problems and the relevant knowledge -or non-knowledge- to solve them. Perhaps a skeptical turn in the study of transnational regulation, performance measurement and monitoring must lead to an ‘un-settling’ of the classic studies of the political use of statistical knowledge, and offer the promise of a more creative, at times even inconvenient, analysis of the unaccounted and thus invisible processes of the construction of non-knowledge that the production of quantification necessitates.