Skip to navigation – Site map

HomeIssues XXVI-2A Brief Introduction to Complexit...

A Brief Introduction to Complexity Theory in Managing Public Services

Une introduction succincte à la gestion de la complexité dans les services publics
Nicholas Sowels

Abstracts

This article seeks to provide a brief overview of the development of complexity theory in public sector management. The article starts by reviewing the emergence of complexity theory, first in natural sciences and then in social sciences, as an attempt to analyse complex systems and phenomena which direct “Newtonian” causalities fail to explain fully. Next, it looks at how such complexity theory – which makes the distinction between complexity and chaos – has been used to examine public services. In particular, the article analyses how new public management (NPM) and post-NPM have led to far more complex public service networks and delivery systems than the bureaucratic government structures which existed previously. As a result, research into complex public service systems has itself contributed to the deepening of complexity theory. Finally, the article presents a series of cases in which complexity theory is applied to public sector management, and the management of common pool resources as analysed by Elinor Ostrom. It concludes that complexity theory is a powerful tool for challenging the standard frame of mainstream economics and NPM, but that its applicability is not easy.

Top of page

Full text

Introduction

1Work on this article began long before the Covid-19 pandemic, which exploded in early March 2020, and within a couple of weeks changed our lives fundamentally. It will affect us for a very long time. In some ways, what follows may seem trite, given the direct and indirect suffering the “coronacrisis” looks set to cause. Yet, this crisis is also a massive challenge for public policy as the coronavirus has swept the world: it has hugely reduced international travel and business; in spring 2020, it stretched health services so severely that countries across the globe went into lockdown to limit hospital patient numbers, at a staggering economic cost; governments have been scrambling to obtain simple and complex medical equipment and are once again talking about the importance of strategic industrial sectors; whole sectors of the economy have been on the brink of collapse, etc. Monetary and fiscal authorities are doing the unthinkable to fight the pandemic, to bail-out companies and support households. Confinement curtailed civil liberties and social distancing is set to disrupt our lives for a long time. In short, governments (along with households and companies) have been plunged into unprecedentedly complex crisis management. It is beyond my competencies to apply the ideas set out below to the coronacrisis. Yet some of them may contribute to readers’ own thinking about what we are living.

  • 1 The cost of the Hinkley Point power plant, for example could be more than £22 billion; Jillian Ambr (...)
  • 2 BBC News, “US oil prices turn negative as demand dries up”, business section, 21 April 2020, <https (...)

2My own interest in complexity theory has followed several intuitions. For several years, I have done some translations about the energy sector, an activity in which investments in infrastructure and plant are vast.1 Yet energy is also an area in which the global business environment changes very quickly. In the last 15 years, shale gas and oil have totally changed the energy balance of the USA; renewable energies have seen substantial price falls; nuclear power is much compromised since the Fukushima disaster in 2011, while gas is declining in Europe and coal is back: with the coronacrisis, the spot oil price was negative (sic) on Monday 20 April 2020.2 How are policy-makers investing billions (of euros, pounds or dollars) to react to such gyrations?

  • 3 The partition of India accompanying British withdrawal in 1947 led to more than one million people (...)
  • 4 David Colander and Roland Kuper note that, “in the course of the 20th century two things made the w (...)
  • 5 A widely-used paraphrase attributed to Douglas Jay, who in The Socialist Case (1937) wrote “in the (...)
  • 6 Philip Haynes argues that “IT systems have increased the complexity of public service organisations (...)

3The second intuition came from thinking about how Britain’s post-war Labour government did what it did. National debt after the war was around 250 percent of GDP and Britain desperately needed a loan from the United States (and Canada). Despite this, the Attlee government nationalised substantial parts of the UK economy, created the Welfare State (the NHS, social insurance and pensions), managed the British zone of occupied Germany, demobilised the armed forces... and departed ignominiously from India, leaving tragedy behind.3 How did they manage all this? And how did they do it without computers? Part of the answer is the far greater simplicity of goods and services at the time:4 welfare benefits were flat rate, consumer goods were rationed and limited in variety... and society was deferential to the “man in Whitehall who kn[ew] best”, etc.5 Part of the answer is that business and government were simpler, precisely because there were no computers, as bigger and more flexible IT systems themselves generate complexity (and project failures).6

  • 7 David Osborne and Ted Gaebler, Reinventing Government: How the Entrepreneurial Spirit is Transformi (...)

4The last intuition follows from participating in conferences in recent years, in which new public management (NPM) still figures largely, even though NPM is now quite old: Osborne and Gaebler’s landmark study on Reinventing Government, for example, was published in 1992.7 So, what has happened since? One answer to this question lies in the development of complexity theory and its application to public services over the last 30 years. Section 1 of this article attempts to summarise some of the key concepts in this field. The next section seeks to present how complexity analysis has evolved within public sector management. Finally, section 3 gives some examples of complexity theory applied to public services.

The varied origins of complexity theory

  • 8 Dani Rodrik, Economics Rules: The Rights and Wrongs of the Dismal Science, New York, W.W. Norton & (...)
  • 9 Paul Cilliers, “Boundaries, Hierarchies and Networks in Complex Systems”, International Journal of (...)

5The examination of the complexity of economic and political processes is actually not new. In the Wealth of Nations, Adam Smith provided both a historical analysis of economic development, and a detailed analysis of how economies worked, including his study of the division of labour, and his allusion to the “invisible hand” coordinating market activity. He thus laid the foundations of political economy as a discipline, and as a driver of public policy. Marx too analysed political economy issues from a historical, political and philosophical – yet critical – perspective. But later, the study of economics narrowed its field of enquiry to examine more specifically how markets operate, how “agents” make choices in allocating scarce resources and how prices are formed. This began in the latter half of the 19th century and was accompanied by the increasing formalisation of economic behaviour and the elaboration of graphical and mathematical models to explain such behaviour. This use of maths brought clarity and consistency to arguments. As Rodrik has noted, “[w]e still have endless debates today about what Karl Marx, John Maynard Keynes, or Joseph Schumpter really meant... By contrast, no ink has ever been spilled over what Paul Samuelson, Joe Stiglitz, or Ken Arrow had in mind”.8 But the development of economic modelling based on mathematical formalisation and statistical testing has both strengths and weaknesses. As Paul Cilliers, an early contemporary thinker of complexity, stated “[w]e cannot deal with reality in all its complexity. Our models have to reduce this complexity in order to generate some understanding. In the process something is obviously lost”.9 More specifically, the real problem is the way modelling seeks to be “positive” in its scientific approach, but invariably ends up being normative: reflecting the twofold meaning of the word “model” itself, as a simplification and as an ideal. This has become especially the case of mainstream economics, or what Colander and Kupers call the “standard frame”, and which provides the basis for neoliberal policy-making. While Rodrik shies away from complexity, suggesting that economics should be more willing to accept a plurality models, Colander and Kupers argue the answer lies in investigating the “complexity frame”.

  • 10 Elizabeth Anne Eppel & Mary Lee Rhodes, “Complexity theory and public management: a ‘becoming’ fiel (...)
  • 11 David Colander and Roland Kupers, ibid, p. 13.

6Looking at complexity, put very simply, means going beyond a “Newtonian” view of science, which “might crudely be summarized as (1) relationships between individual components of any system can be understood by isolating the interacting parts, (2) there is a predictability to the relationship among the parts, and (3) the result of interactions and the working whole might eventually be understood by simply summing the parts”.10 By contrast, complexity theory, or perhaps more accurately complexity theories seek to explain how systems evolve in varied and unpredictable ways. This entails looking both at the interconnectedness of the parts making up the systems and the parts themselves: “[i]n technical jargon, that means that dynamics and statics become blended, and the math becomes wickedly difficult”.11

  • 12 Ibid, p. 115.
  • 13 Cilliers, ibid.

7The inherent instability following the multiple interactions of parts in complexity systems is sometimes seen as chaotic, as often portrayed in the butterfly effect: “the idea that a butterfly flapping its wings in Brazil might precipitate a tornado in Texas”. Yet chaos theory is only a special case of complex systems, in which the changing state of a system has no order whatsoever.12 Instead, complexity theory seeks to identify more patterns of behaviour which, though complicated, also display elements of stability. For Cilliers, they have structures, which models seek to understand, although this is complicated as the boundaries of complex systems are hard to specify, while the “vitality” of a system “lies in its ability of transform [its] hierarchies”.13

8A key contribution to defining complexity has been made by Edgar Morin, who has given a broad epistemological overview of complexity theory, drawing much on natural sciences. He notes that “classical science” rejected complexity given its three fundamental explanatory principles: i) the principle of universal determinism, connecting past and future events; ii) the principle of reduction, by which knowledge of a composite can be achieved from knowing is constituent parts; and iii) the principle of disjunction, by which cognitive difficulties are dealt with by separating them into different disciplines. For Morin, the first de facto breach of classical science came with the second law of thermodynamics. Set out in the middle of the 19th century, it states that heat (energy) can only move from a more concentrated state to a less concentrated state (say from hot water to cold water). This principle has been more widely formulated as the idea that (in a closed system) order descends into disorder, or chaos. This is an irreversible process, and so could not be explained by the previously-existing laws of physics that were based on reversibility.

  • 14 Edgar Morin, “Restricted Complexity, General Complexity”, Worldviews, Science and Us, pp 5-29 (2007 (...)

9Complexity did then enter certain areas of mathematics and engineering in the 1940s and 1950s (information theory, cybernetics and general systems theory). But, the specific study of complexity in an inter-disciplinary way to address complex phenomena in physics, biology, technology and social sciences only began fully with the creation of the Santa Fe Institute in 1984.14 According to Colander and Kupers, the founders of the Institute were seeking specifically to understand complex phenomena by overcoming the separation of traditional academic disciplines, in order to address problems such as:

stock market booms and busts, financial crises, the reconfiguration of amino acids into the vastly more complicated molecules of living things, the sudden collapse of civilisations, the emergence of agriculture after nomadic hunter-gathering [which] all defied standard explanations, yet […] shared the common characteristics that involved the sudden seemingly spontaneous emergence of a new order.

  • 15 David Colander and Roland Kupers, ibid, pp.132-3.

10Significantly, one of the aims of the Institute was to use high-level mathematics along with ever-greater and ever-cheaper computer power to examine the non-linearities, abrupt transitions and interconnections between the parts making up systems. Significantly too, early funding for the Institute came from the Citicorp bank, to explore the relationship between complexity theory and economics, to understand macroeconomic phenomena in view of forecasting the future.15

  • 16 Edgar Morin gives the example of delinquent or criminal behaviour as excessive liberties inhibited (...)
  • 17 Ibid.

11The approach by the Santa Fe Institute of applying more complex mathematics and more computer power to understand complex systems, fits in with what Morin has called restricted complexity. This can be examined by “important advances in formalisation, in the possibilities of modelling”. But for Morin, this “still remains within the epistemology of classical science”, seeking to produce laws. He contrasts this with generalised complexity, which requires epistemological rethinking, “bearing on the organisation of knowledge itself”. What is notable here is that complexity requires understanding the relationship between the parts of a system and the whole, which affect each other, so that a knowledge loop is required to track such interaction. This may be complicated indeed, because new qualities or properties may emerge, just as a system – a whole – is more than the sum of its parts. Morin also states that the whole may also be less than the sum of its parts, when certain qualities and properties of the parts are inhibited by the organisation of the whole: what he calls subtractivity.16 Drawing on various disciplines, Morin further examines other key concepts of complexity such as the “self-organisation” of systems (which he actually terms “self-eco-organisation” because systems need energy and information from their environment), or even “metamorphosis” (like the sociological metamorphosis when prehistoric societies of a few hundred persons gave way to “enormous historical societies with cities, agriculture, army, civilisation, etc.”).17

Complexity theory and the management of public services

  • 18 Göktuğ Morçöl, “Complexity of public policy and administration: introduction to the special issue”, (...)
  • 19 Geert Teisman and Lasse Gerrits, “The Emergence of Complexity in the Art and Science of Governance” (...)

12For Göktuğ Morçöl, it is fair to ask whether “the concepts and methods of thermodynamics, atmospheric science, and the like [are] transferable to public policy and administration”. He also notes there are problems in ignoring the mathematical logic of complexity theory in the fields in which it has originated, to “apply it only metaphorically in the social sciences”. Yet Morçöl goes on to argue that such legitimate concerns should not rule out exploring “exciting theoretical and empirical possibilities complexity and related theories offer”.18 For their part, Teisman and Gerrits state two approaches more explicitly: the first attempts to replicate the original scientific approach in public services; the second uses the general ideas and concepts emanating from natural sciences but translates them into the target domain (of public management). The concepts of science can be powerful metaphors, but have to be applied carefully.19

  • 20 Friedrick v. Hayek, “The Use of Knowledge in Society”, The American Economic Review, Vol 35, No 4. (...)
  • 21 David Byrne and Gill Callaghan, ibid, p236.

13Given the inherent and increasing complexity of producing and delivering public services, it seems to me that complexity theory in this area has in fact developed its own, creative dynamics. The nature of many public services has always been more difficult to determine and measure than marketable goods, as has their provision. For goods and services sold in the market, measuring financial flows (turnover and profits, etc.) provide relatively simple criteria for evaluating the efficiency of producers, while bankruptcy sanctions failure. Moreover, prices summarise the necessary information in the market for producers to make choices in organising their business, as Hayek famously stated.20 For their part, consumers make choices between products according to their preferences and budgets. By contrast, policy-makers, public purchasers, public and/or private service providers and citizens as users/customers often make decisions with more opaque and complex information. What exactly is “a good education”? And how is it delivered? Or what exactly is “the right medical treatment”? And “can I trust this doctor’s advice”? are all questions nearly all households ask themselves at some point, and involve choices that are far more complex than the purchase of most goods and services. Yet complexity theory reveals even more how complicated such public services are. Byrne and Callaghan, for example, point out that health should be viewed as a set of intersecting systems involving: i) the individual human as a complex physiological system and eco-system with micro-organisms inhabiting the body; ii) the public health system of the population as a whole which has a massive intersection with the urban system; iii) the health care delivery system(s); and iv) the relationship of the human species with the global eco-system.21 (The myriad interactions of these complex health systems are painfully visible to us today with the coronavirus pandemic.)

  • 22 Philip Haynes, ibid, p1.

14Complexity in public services has also been much augmented by organisational change. The move away from the bureaucratic hierarchies which provided public services (and in fact many consumer goods) in the post-war world has made the organisation of producing, delivering and using/consuming public services ever more complex. As Haynes notes, it is important (today) “to understand better the major tensions in public service work, such as the contradictions between professional and managerial agendas and the differing strengths and weaknesses of public, private and non-governmental provision”.22

  • 23 Tom Christensen & Per Lægreid, “Complexity and Hybrid Public Administration—Theoretical and Empiric (...)

15Complexity theory, it seems to me, has been pushed forward precisely in public sector management as ever-more complex organisational structures and modes of governance have emerged. These have resulted substantially from the replacement of bureaucratic government (operating through command and control) by mechanisms of complex contracting and performance measurement linked to new public management (NPM), with all the associated reforms of de-regulation, managerialism, etc. The organisational complexity of public service systems has subsequently grown further with so-called post-NPM reforms which “introduced a combination of vertical integration via stronger control measures and greater capacity for the political executive, and more horizontal collaboration and coordination in the form of networks, teams and projects”.23 This has led to highly complex hybrid structures with varied providers responsible for servicing users/customers, who in turn expect ever-more personalised services.

16In Britain, for example, NPM emerged progressively during the 1980s, as successive Conservative governments sought to promote the economy, effectiveness and efficiency – the 3Es – of public services. This was done progressively through: increasing available information about the costs and outputs of public services; organisational reforms involving competitive tendering, outsourcing, deregulation and privatisation where possible; and the creation of quasi-markets in activities like education and health where services were (and still are) essentially free at the point of use. In the 1990s, the Major governments set up “citizen’s charters” for public services, with the aim of ensuring that citizens could expect more specifically-defined outputs, and have better recourse to complaint when necessary.

  • 24 Nehal Panchamia and Peter Thomas, "Public Service Agreements and the Prime Minister’s Delivery Unit (...)
  • 25 Roderick Rhodes, et al, Decentralizing the Civil Service: From unitary state to differential polity (...)

17This NPM reform programme was subsequently largely taken over by New Labour, when it came to power in 1997. But it was also amended and extended as Public Service Agreements (PSAs) were introduced in 1998, initially setting out 600 performance targets for Whitehall departments. Over time, these became a means for the government – especially the Prime Minister’s Delivery Unit working with the Treasury – to track more closely the achievement of specific public policy goals.24 At the same time, New Labour pursued a policy of “joined-up government” and the creation of networks within the NHS and local government, in order to strengthen cooperation alongside competition. For public policy specialists like Rhodes et al., this meant that “networks” had become a new primary organisational structure alongside “markets” and “hierarchies” in the delivery of services, with their own sets of cultures and interconnections.25

  • 26 Philip Haynes, ibid, p. 19.

18As a result of these on-going processes, the chains for designing, producing and delivering public services have become increasingly diverse and fragmented, involving multiple actors and agencies trying to meet the increasingly complex and diversified needs of the public, be they users or customers of public services. The bureaucratic organisations which created the Welfare State and managed the public sector in the decades after World War II have been superseded by dense networks of organisations, making up so-called complex adaptive systems (CASs). As a whole, these are more complex and often less-understood than their constituent parts. They have “soft boundaries” between organisations and other systems, leading to “entanglement”: i.e. “mutual dependence on each other and inability to maintain rigid separations and boundaries… increasingly witness[ed] when understanding the relationship between the private, public and non-governmental parts of society and the managerial and professional roles in these organisations”.26

  • 27 David Osborne and Ted Gaebler, ibid, pp. 1-24.
  • 28 Nehal Panchamia and Peter Thomas, ibid.
  • 29 The list here is taken from Philip Haynes, ibid, pp 24-50. In terms of attractors, David Colander a (...)

19The evolution of such complex adaptive systems is marked by multiple features which are now well documented in the literature. These include feedback that may be “reinforcing” or “balancing”; and emergence of new behaviours, typically by independent local actors who affect systems from the “bottom up” (which was in fact how Osborne and Gaebler described the non-ideological, non-methodical emergence of NPM in the USA during the 1970s and 1980s, at state and municipal government level).27 Significantly, unlike wholly chaotic processes, complex adaptive systems have elements of order “as defined by patterns of replicated behaviour for given periods”. These are known as attractors, and are often made up of values, beliefs and logics. They tend to be hierarchical, so that one attractor contributes more to social order than others, although such a hierarchy of attractors may vary over time (in the case of New Labour’s PSAs, for example, these became more cross-cutting during the 2000s, while output targets were reduced, although the culture of attainment remained entrenched28). Systems may also be subject to path dependency as they are shaped by national histories and institutions, which give change a certain momentum. But bifurcation points exist too, when organisations and systems are both on the brink of chaos, and subject to dramatic change. Finally, systems and organisations are also shaped by self-organisation, which involves the creation of new ideas and practices as local responses to system dysfunctions.29

  • 30 Philip Haynes, ibid, pp. 95-97.

20Given the array of factors and forces at play in delivering public services, Haynes warns explicitly about the temptation to reduce such complex situations and systems to simple assumptions of cause and effect, and indeed guards against searching for an “ultimate truth representation” of a particular activity set. Instead, by creating diagrams (or flow charts) seeking to represent complex public system, he proposes including the following set of considerations when analysing and reforming systems: i) what, for the purposes of representation, are considered to be the boundaries of any particular system? ii) who are the key actors involved, and where are they positioned in the system? iii) what are the core activities and where do they take place? iv) what secondary activities take place to assist the delivery of primary activities? v) where do outputs occur, and how do they relate to prior activities? vi) is there any logical order or connectivity of activities to be present diagrammatically? vii) what are the key points of connection, for communication and to link complex processes of interaction? viii) how do resources flow into inputs and activities?30 We shall see in the next section that Haynes and his colleagues at Brighton University have developed a toolkit to apply this schematic reforming policy, although this is far from amounting to a general approach to using complexity theory to implement reform.

Complexity theory in practice in public policy

  • 31 OECD Observatory of Public Sector Innovation, Working with Change: Systems approaches to public sec (...)

21So far for the theory, what about the practice? Cairney, among others, makes a key point by noting that “[t]he first difficulty with complexity theory is that it is difficult to pin down when we move from the conceptual to empirical analysis”. Perhaps given the very nature of complexity, policy prescriptions can only be tentative and piecemeal, as they are highly dependent on specific (local) circumstances. This at least seems to be how complexity analysis operates in practice, with case studies examining primarily local government issues, and specific attempts to solve “wicked” problems (i.e. problems that cannot be solved by “partial or transactional solutions, but require concerted, adaptive and carefully stewarded approaches”, and for which no optimum solutions exist).31 This section starts by presenting the Toolkit developed by Haynes and colleagues at the University of Brighton to apply complexity theory to public service management in the United Kingdom. It then moves on to summarise a number of other case studies and applications of complexity.

  • 32 The framework was developed by members of the Brighton Systems and Complex Systems Knowledge Exchan (...)
  • 33 The Brighton team draw here on a well-known management process developed by Donella Meadows.
  • 34 Philip Haynes, ibid, p. 70.

22The Brighton Systems and Complex Systems Toolkit Framework sets out a number of steps for applying complexity theory to public policy issues.32 Policy-makers are invited first to decide whether the situation or problem they are addressing is “simple”, “complicated”, “complex” or “chaotic”, with each category being broken down further (for example, the “complex” category has “unpredictable changes”, “changing interactions” and “identify and use patterns” as sub-categories). The next step in the toolkit involves “considering action and intervention”, and this is done by inverting the sequence of steps in the traditional management approach indicated in the toolkit, which runs from: resources and their use > identify types of change > use of information > setting rules > empowering self-organisation > directions of purpose > radical change.33 In the alternative radical ‘public value’ approach pioneered at Brighton, the first step for bringing about change in a complex system is the “radical change” step, which involves “building core and primary values that enable a critical sense of the priority issues that need action”, and going through the sequence to end with “resources and their use”. Hayes has noted, for example, that the reassertion of values of care has become important in hospital management, where services had become too “depersonalised and instrumental”.34

  • 35 Mary Darking, Philip Haynes and Julia Stroud, “Developing public service knowledge and learning abo (...)

23In an article presenting the application of their methodology and published in 2018, members of the Brighton team related experiences from a wide range of case studies, including “macro policy issues like the economy and public finances, and national energy policy, but also local examples [...about how] regional and local government could be support changing local dynamics in tourism, how social workers best respond to risk in partially closed family communities, etc.”. The team stressed the importance of beginning with “cultural interventions from the ‘bottom up’ [and b]uilding team and organisational cultures and making them resilient through adaptability [which is] argued to be at the core of a management practice that uses the insights of complexity theory”. This requires constant interaction and communication between managers/leaders and organisation staff. Devising diagrams explaining the use of stocks and the flow of resources (including human resources), and supplementing them with the use of “Post-it” notes to map out interventions are often key visual aids in conceiving and designing change of complex systems. Similarly, they stressed the importance of encouraging “helpful and functional self-organisation” in parts of a complex system, by providing “reinforcing information about [them] in [other] parts of the system”, etc. At the same time, this summary article by the Brighton team once again stresses the difficulty of taking the theory of complexity and actually applying it practically to changing the operation of public services. It concludes, however, by suggesting that “an appreciation of complexity theory and its concepts” by practitioners “drives a change in perspective”, within an environment that is inevitability uncertain, given “our dynamic, interactive, and innovative society”.35

  • 36 Erik-Hans Klijn, “Managing complexity: Achieving the impossible?”, in Critical Policy Studies, 2007 (...)

24The application of complexity theory to public policy has also been much researched at the Erasmus University of Rotterdam. An early case study by Klijn examined the construction of a railway tunnel through the city of Delft, which was first mooted in the late 1980s. However, given the involvement of many actors due to joint financing requirements – notably the municipality and local private investors, the national railway company and central government (with input from several ministries) and parliament – the decision-making process was long and complex. At one point changing national priorities led to the project’s outright cancellation, although the tunnel was finally opened in early 2015. Klijn acknowledges the apparently chaotic nature of the decision-making process, but also identifies “stabilising factors” (i.e. attractors) during the planning process, including: resource dependencies between actors, interaction patterns between them, rules and regulations in networks, and trust relations between actors. In the light of the uncertainties thrown up by the process, he notes that political and media commentary shifted towards favouring strong and decisive leadership capable of making clear decisions without being sucked into myriad negotiations between complex networks of actors. But he concludes that such a strong leadership approach is unlikely to solve the increasing complexity of society.36

  • 37 Jan Arpe, “Globalisation and its Complexity: Challenges to Economic Policy, Bertelsmann Stiftung, “ (...)
  • 38 Peter Ho, Coping with Complexity, McKinsey & Company, 2012, <https://www.mckinsey.com/industries/pu (...)
  • 39 Peter Ho, “The future-of-government approach is important, but not easy to execute”, extract of an (...)

25Globalisation is also a contributing factor leading to additional complexity challenges which public services have to face, and has been accelerating since the end of the Cold War. Indeed, it can well be argued that the resulting global interdependence of political, economic and social systems has produced unprecedented complexity. However, according to Arpe, the way individual decisions are made in given situations has not kept up with such increased complexity as humans’ brain structure only evolves very slowly, while institutional decisions are rooted in social systems such as organisations or cultures: traditional economics failed, for example, to predict and ultimately explain the global financial crisis, due to its cognitive biases. With prescience, Arpe notes that “it is virtually unimaginable what a global pandemic might mean”.37 These themes have also been developed by Ho (a civil servant from Singapore) in a McKinsey web article. Ho points out that the “most vexing wicked problems today – such as climate change, energy security, global pandemics, sustainable development, and cyberthreats – have causes and influencing factors that are not easily determined ex ante”. Formulating policies to deal with such issues requires integrating diverse views and expertise, creating mechanisms to share information and strengthen collective action, and essentially adopting a “whole-of-government approach”, as Singapore has done for economic policy, since the early 1990s, and subsequently extended to risk management (the so-called “whole-of-government integrated risk management” framework).38 Yet the process of overcoming the silos of traditional bureaucratic government, Ho has later admitted, is not easy.39

  • 40 Elinor Ostrom, “Beyond Markets and States Polycentric Governance of Complex Economic Systems”, Nobl (...)

26Lastly, it is more than fitting to recall the application of complexity thinking by Elinor Ostrom, the first woman to win a Nobel Prize in economics in 2009. Her work has challenged many of the basic tenets of mainstream economics. In her Nobel lecture, she testifies to the almost spontaneous tendency for “positive economics” to be normative, noting that “in the mid-twentieth century, the dominant scholarly effort was to try to fit the world into simple models and to criticise institutional arrangements that did not fit”. Her work – deeply grounded in empirical observation of municipal and local institutions – challenges notably the dichotomy of mainstream economics that goods are either private and so excludable (a person is excluded from use if they do not pay) or public, and therefore non-excludable (say, like national defence from which no member of a society can be excluded). Ostrom observed that so-called “toll goods” also exist, which are provided by small-scale public and private associations, such as theatres, private clubs and day-care centres. Their “subtractability of use” (i.e. the ability to exclude users for non-payment) is low, as tolls are low. Conversely and in particular, Ostrom identified the existence of goods she called “common-pool resources” (like groundwater basins, lakes, fisheries, forests, etc.). In this case, there is a structural difficulty in excluding members of a community from accessing the resources, but subtractibility is high, as non-members are excluded. Based on the study of such local organisations in many societies, the research she conducted (with colleagues) led to the formulation of general design principles for managing common-pool resources, including: boundaries between users and non-users; appropriation and provision rules that are congruent with local social and environmental conditions; collective decision-making arrangements for members of the pool; monitoring, sanctions and conflict resolution mechanisms; acceptance of the common-pool management by the local community and government. In short, Ostrom identified and analysed a form or common resource management which lies totally outside the usual market-state dichotomy, and which reflects real, existing, complex, bottom-up institutions.40

Conclusions and discussion

  • 41 Cilliers, ibid.
  • 42 This point was made and explained to me by Daniel Charrier, a quantum physicist by training, and co (...)
  • 43 Michael Veseth, “What is International Political Economy?”, <http://www2.ups.edu/ipe/whatis.pdf [1 (...)
  • 44 Nils Kvilvang, Erik Bjurström and Roland Almqvist, “Making sense of complexity in governance: the c (...)
  • 45 Tom Christensen & Per Lægreid, ibid.

27Given the diversity of ideas and approaches presented in this succinct summary of complexity theory, it should come as no surprise that, as Cilliers observed, there is “[n]o general model [that] can capture [the] singularities” of the multitude of contingent factors, specific conditions, contexts and times.41 For persons partial to analogies with natural sciences, this should not really be a surprise, as even physics – that hardest of all sciences – does not have a general theory with can explain both gravity and quantum mechanics.42 If I may make my own inter-disciplinary comparison with international political economy, I would venture to say that complexity theory in public sector management is “the study of a problématique, or set of related problems”,43 using a set of ideas and concepts from different schools of thought. They can be used for analysis and understanding, but provide no over-arching theory to shape policy. Thus the numerous concepts we have seen provide broad insights into the working of public services, yet remain hard to use as operational tools in terms of clear, direct policy actions. Or as Kvilvang, Bjurström and Almqvist put it more simply, “complexity theory is no panacea for unlocking the difficulties of public sector challenges”. But in can be used as a sense-making framework in fostering problem-solving capacities in complex organisations, and so contribute to the legitimacy of governance.44 Expressed in another way, complexity theory can be used to achieve the appropriate balance between autonomy and control in the increasingly hybrid and complex organisational structures through which public services are designed, produced and delivered.45

  • 46 David Colander and Roland Kupers, ibid, pp. 41-3.
  • 47 Ibid p. 9.

28Two of the main works cited here conclude with very different approaches to these issues. For Colander and Kupers, recourse to complexity theory should be channelled to fostering “laissez faire activism”. They draw on Hayek’s idea of knowledge coordination via the price mechanism to restate the need for governments to create the appropriate eco-system in which “people’s true social goals can emerge... In a bottom-up policy, the social goal emerges from the process. People are free to choose both their individual and collective goals, and are also free to choose how to achieve those goals”. But Colander and Kupers specifically claim that complexity theory helps move beyond the government-market dichotomy of the standard frame and passive laissez-faire (anti-government) policy. “Instead, complexity policy supports a policy that treats government and private enterprise as partners from which new blended institutional forms may evolve”.46 They acknowledge however that “a complex system works only if individuals self-regulate,... that they do not push their freedom too far, and that they make reasonable compromises about benefiting themselves and benefitting society”.47

  • 48 Philip Haynes, ibid, p. 57.

29In contrast, if Philip Haynes, and the team working on complexity at the University of Brighton have one clear lesson to put forward from their work on complexity, then it is the importance of values as an attractor in implementing policy change. As complexity theory raises “some serious questions about the ability of strategic managing and planning...because of high levels of uncertainty... [b]uilding a strong and resilient organisational culture that is founded on shared values becomes central to the strategy of a public service organisation”.48

  • 49 Paul Cairney and Robert Geyer, ibid.

30In their introductory and summary chapter of the handbook on complexity and public policy they edited, Cairney and Geyer are generally somewhat more circumspect. For them, lessons in one context may not be applicable to another, and as policy-making systems change quickly, making them difficult to predict, policy-makers need to adapt rapidly too. Given the limits to our knowledge of policy-making systems, and limits to our ability to control them, this often produces bottom-up or local approaches to policy advice, with arguments including: less reliance on central government to drive targets, in favour of adaptive local organisations; the use of trial-and-error projects to deal with uncertainty and change; treating “errors” as sources of learning not failure to be punished; and encouraging greater understanding in the public sector of emergence and feedback loops within complex systems.49

31Cairney and Geyer also briefly touch on the political implications of such a bottom-up, complexity approach which runs through much of what has been said here. They point out that policy which is shaped from a complexity perspective challenges the mechanical, “state in control” approach of much democratic politics. While central governments necessarily have to go through failure and learning processes when general simplistic policies fail, they are under a certain democratic constraints to do so, notably in states with a “Westminster model” of central government, based on national accountability and responsibility.

  • 50 The expression was coined by H.L. Mencken, an American journalist and essayist. I first read it and (...)

32Indeed, simplicity – and optimism – in political campaigning work well. “Take back control” was essential to the Leave vote in the 2016 referendum, as was “Get Brexit Done” to the December 2019 election. But as the saying goes “for every complex problem there is an answer that is clear, simple and wrong”.50 And few political decisions illustrate the clash between political simplicity and policy complexity more starkly than Brexit. However, maybe public sector policy change also requires the simplistic ideas to drive the political process. The ensuing simple solutions, in a complex environment, will necessarily and logically be wrong. But perhaps they may also trigger balancing forces, or be part of a sort of ongoing, dialectical process, as successive policy waves unfold over time?

33What is sure, for Britain however, is that Brexit is a bifurcation point in history, which is already developing its own path dependency. Following the large election victory of Boris Johnson and the Conservatives in December 2019, the Brexit process is hardening, with Britain distancing itself more and more from the European Union, and moving more and more away from regulatory integration with its European partners. And it seems unlikely that the coronavirus pandemic will affect this much.

34At the same time, the pandemic itself is a colossal moment of bifurcation in the global economy and world society. The direct impact of the health crisis is already leading to a withering of international links, and sharpening superpower antagonism between the United States and China especially. The indirect economic impact could lead to profound changes in the functioning of the global economy and in the economic and political system which has emerged since the end of the Cold War at the end of the 1980s. We will surely be living in interesting times, and the notions of complexity and systems interconnectedness should provide the reader with some feeling for how change is likely to be profound, planetary and problematic.

Top of page

Bibliography

Ambrose, Jillian, “Hinkley Point nuclear plant building costs rise by up to £2.9 bn”, The Guardian, 25 September 2019.

Arpe, Jan, “Globalisation and its Complexity: Challenges to Economic Policy, Bertelsmann Stiftung, “Shaping Sustainable Economies” program, <https://jan-arpe.com/wp-content/uploads/2017/06/Globalization_Complexity.pdf> [30 April 2020].

Brighton Systems and Complex Systems Knowledge Exchange, hosted by the School of Applied Social Science and the Community University Partnership Project (CUPP), <https://www.brighton.ac.uk/_pdf/research/ssparc/toolkitframework.pdf> [30 April 2020].

Byrne, David & Callaghan, Gill, Complexity Theory and the Social Sciences: the State of the Art (London, Routledge, 2014).

Cairney, Paul and Geyer, Robert, “Introduction” in Paul Cairney and Robert Geyer (eds), Handbook on Complexity and Public Policy (Edward Elgar, 2015) pp 1-15.

Christensen Tom & Lægreid, Per, “Complexity and Hybrid Public Administration—Theoretical and Empirical Challenges”, Public Organization Review, (2011) 11, pp. 407–423, DOI 10.1007/s11115-010-0141-4.

Cilliers, Paul, “Boundaries, Hierarchies and Networks in Complex Systems”, International Journal of Innovation Management, Vol 5, No 2 (June 2001), pp. 135-147.

Colander, David and Kupers, Roland, Complexity and the art of public policy: solving society’s problems from the bottom up (Princeton University press) 2014.

Darking, Mary, Haynes, Philip and Stroud, Julia, “Developing public service knowledge and learning about complex systems: using a community of practice to integrate theory and practice”, Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, University of Bamberg Press, pp. 46-58 <https://ubp.uni-bamberg.de/ojs/index.php/cgn/article/view/45> [30 April 2020].

El-Ghalayini, Yousif, “Complexity Theory: A New Way to Look at New Public Management”, Network and Complex Systems, Vol 7, No 1, 2017.

Eppel, Elizabeth Anne & Rhodes, Mary Lee, “Complexity theory and public management: a ‘becoming’ field”, Editorial, Public Management Review, 2018, Vol 20, No. 7, pp. 949-959.

Hayek, Friedrick v., “The Use of Knowledge in Society”, The American Economic Review, Vol 35, No 4. (Sep., 1945), pp. 519-530.

Haynes, Philip, Managing Complexity in the Public Services (London, Routledge, 2015).

Ho, Peter, “The future-of-government approach is important, but not easy to execute”, extract of an IPS-Nathan Lecture, reported in Today, 5 April 2017, <https://www.todayonline.com/singapore/future-government-approach-important-not-easy-execute> [30 April 2020].

Ho, Peter, Coping with Complexity, McKinsey & Company, 2012, <https://www.mckinsey.com/industries/public-sector/our-insights/coping-with-complexity#> [30 April 2020].

Klijn, Erik-Hans, “Managing complexity: Achieving the impossible?”, in Critical Policy Studies, 2007, Vol. 1, no. 3, pp. 252-277.

Kvilvang, Nils, Bjurström, Erik and Almqvist, Roland, “Making sense of complexity in governance: the case of local public management in the City of Stockholm”, Policy Studies, 2019, DOI: 10.1080/01442872.2019.1577372

Matei, Ani and Antonie, Cătălina, “The New Public Management within the Complexity Model”, Procedia – Social and Behavioural Sciences 109 (2014), pp. 1125-1129;

Morçöl, Göktuğ, “Complexity of public policy and administration: introduction to the special issue”, Public Administration Quarterly, Vol 32, No 3 (Fall 2008), pp. 305-313.

Morin, Edgar, “Restricted Complexity, General Complexity”, Worldviews, Science and Us, pp 5-29 (2007), doi.org/10.1142/9789812707420_0002 [20 April 2020].

Ness, Lawrence, R., “Exploring Information Technology Flexibility and Increased Complexity: Is Management Prepared?”, International Journal of Applied Management and Technology, Vol 6, Num 3, 2008.

OECD Observatory of Public Sector Innovation, Working with Change: Systems approaches to public sector challenges, GOV/PGC(2017)2, p. 14.

Osborne David and Gaebler, Ted, Reinventing Government : How the Entrepreneurial Spirit is Transforming the Public Sector, Reading, Mass., Addison-Wesley, 1992.

Ostrom, Elinor, “Beyond Markets and States Polycentric Governance of Complex Economic Systems”, Noble Prize Lecture, December 8, 2009, <https://www.nobelprize.org/prizes/economic-sciences/2009/ostrom/lecture/> [1 May 2020].

Panchamia, Nehal and Thomas, Peter, “Public Service Agreements and the Prime Minister’s Delivery Unit”, Institute for Government, 2014, <https://www.instituteforgovernment.org.uk/sites/default/files/case%20study%20psas.pdf> [27 April 2020].

Rhodes, Roderick, et al, Decentralizing the Civil Service: From unitary state to differential polity in the United Kingdom, Open University Press, 2003

Rodrik, Dani, Economics Rules: The Rights and Wrongs of the Dismal Science, New York, W.W. Norton & Company, 2015.

Stoker, Gerry, “New Localism, Progressive Politics and Democracy, in Andrew Gamble and Tony Wright (eds), Restating the State?, a special edition of The Political Quarterly, 2004, pp. 117-129.

Teisman, Geert and Gerrits, Lasse, “The Emergence of Complexity in the Art and Science of Governance”, Complexity, Governance and Networks, (2017), pp. 17-28.

Tharoor, Shashi, Inglorious Empire: What the British did to India, London, Hurst & Company.

Tom Christensen & Per Lægreid, “Complexity and Hybrid Public Administration—Theoretical and Empirical Challenges”, Public Organization Revue (2011) 11, pp. 407–423, DOI 10.1007/s11115-010-0141-4

Veseth, Michael, “What is International Political Economy?”, <http://www2.ups.edu/ipe/whatis.pdf> [1 May 2020].

Williamson, David J., “Assessing the relationships among information technology project complexity, complication, and success”. Paper presented at PMI® Research and Education Conference, Limerick, Munster, Ireland. Newtown Square, PA: Project Management Institute, 2012.

Top of page

Notes

1 The cost of the Hinkley Point power plant, for example could be more than £22 billion; Jillian Ambrose, “Hinkley Point nuclear plant building costs rise by up to £2.9 bn”, The Guardian, 25 September 2019.

2 BBC News, “US oil prices turn negative as demand dries up”, business section, 21 April 2020, <https://www.bbc.com/news/business-52350082> [4 September 2020].

3 The partition of India accompanying British withdrawal in 1947 led to more than one million people dying and some 17 million people being displaced: Shashi Tharoor, Inglorious Empire: What the British did to India, London, Hurst & Company, pp. 142-8.

4 David Colander and Roland Kuper note that, “in the course of the 20th century two things made the world more complex: our wealth increased and there were more of us. Increased wealth also meant more sophisticated products and services, with greater interdependence”, Complexity and the art of public policy: solving society’s problems from the bottom up: Princeton University press, 2014, p. 47.

5 A widely-used paraphrase attributed to Douglas Jay, who in The Socialist Case (1937) wrote “in the case of nutrition and health, just as in the case of education, the gentleman in Whitehall really does know better what is good for people than the people know themselves”; see <https://en.wikipedia.org/wiki/Douglas_Jay> [28 April 2020]. This view actually gained currency during World War II, but was later much criticised by the “New Right” around Thatcher in response to the crisis of the 1970s.

6 Philip Haynes argues that “IT systems have increased the complexity of public service organisations rather than simplifying procedures and practices”, Managing Complexity in the Public Services, London, Routledge, 2015, p. 98. Other studies have shown that: greater technological complexity and IT flexibility go hand-in-hand, but that such IT flexibility itself produces more complexity (for example, Lawrence R. Ness, “Exploring Information Technology Flexibility and Increased Complexity: Is Management Prepared?”, International Journal of Applied Management and Technology, Vol 6, No 3, 2008); and that IT projects are less likely to succeed when they are complicated (large in scale) or complex (as objectives many not be clear, requirements may change, operating environments subject to outside influences, etc. (e.g. David J. Williamson, “Assessing the relationships among information technology project complexity, complication, and success”. Paper presented at PMI® Research and Education Conference, Limerick, Munster, Ireland. Newtown Square, PA: Project Management Institute, 2012).

7 David Osborne and Ted Gaebler, Reinventing Government: How the Entrepreneurial Spirit is Transforming the Public Sector, Reading, Mass., Addison-Wesley, 1992.

8 Dani Rodrik, Economics Rules: The Rights and Wrongs of the Dismal Science, New York, W.W. Norton & Company, 2015, p. 31.

9 Paul Cilliers, “Boundaries, Hierarchies and Networks in Complex Systems”, International Journal of Innovation Management, Vol 5, No 2 (June 2001), pp. 135-147.

10 Elizabeth Anne Eppel & Mary Lee Rhodes, “Complexity theory and public management: a ‘becoming’ field”, Editorial, Public Management Review, 2018, Vol 20, N°. 7, pp. 949-959.

11 David Colander and Roland Kupers, ibid, p. 13.

12 Ibid, p. 115.

13 Cilliers, ibid.

14 Edgar Morin, “Restricted Complexity, General Complexity”, Worldviews, Science and Us, pp 5-29 (2007), doi.org/10.1142/9789812707420_0002. See also David Byrne & Gill Callaghan, Complexity Theory and the Social Sciences: the State of the Art, London, Routledge, 2014, Chapter 2 “Restricted complexity and general complexity: an outline of the arguments”, pp. 39-56.

15 David Colander and Roland Kupers, ibid, pp.132-3.

16 Edgar Morin gives the example of delinquent or criminal behaviour as excessive liberties inhibited by law.

17 Ibid.

18 Göktuğ Morçöl, “Complexity of public policy and administration: introduction to the special issue”, Public Administration Quarterly, Vol 32, No 3 (Fall 2008), pp. 305-313.

19 Geert Teisman and Lasse Gerrits, “The Emergence of Complexity in the Art and Science of Governance”, Complexity, Governance and Networks, (2017), pp. 17-28.

20 Friedrick v. Hayek, “The Use of Knowledge in Society”, The American Economic Review, Vol 35, No 4. (Sep., 1945), pp. 519-530.

21 David Byrne and Gill Callaghan, ibid, p236.

22 Philip Haynes, ibid, p1.

23 Tom Christensen & Per Lægreid, “Complexity and Hybrid Public Administration—Theoretical and Empirical Challenges”, Public Organization Review, (2011) 11, pp. 407–423, DOI 10.1007/s11115-010-0141-4.

24 Nehal Panchamia and Peter Thomas, "Public Service Agreements and the Prime Minister’s Delivery Unit", Institute for Government, 2014, <https://www.instituteforgovernment.org.uk/sites/default/files/case%20study%20psas.pdf> [27 April 2020].

25 Roderick Rhodes, et al, Decentralizing the Civil Service: From unitary state to differential polity in the United Kingdom, Open University Press, 2003.

26 Philip Haynes, ibid, p. 19.

27 David Osborne and Ted Gaebler, ibid, pp. 1-24.

28 Nehal Panchamia and Peter Thomas, ibid.

29 The list here is taken from Philip Haynes, ibid, pp 24-50. In terms of attractors, David Colander and Roland Kupers speak of “basins of attractions” that replace the concept of equilibrium, and which can be quite sticky, ibid, p. 54. These characteristics are mentioned almost ubiquitously in texts on complexity and public policy, including in, for example: Ani Matei and Cătălina Antonie, “The New Public Management within the Complexity Model”, Procedia – Social and Behavioural Sciences 109 (2014), pp. 1125-1129; Paul Cairney and Robert Geyer, “Introduction” in Paul Cairney and Robert Geyer (eds), Handbook on Complexity and Public Policy, Edward Elgar, 2015, pp. 1-15; Yousif El-Ghalayini, “Complexity Theory: A New Way to Look at New Public Management”, Network and Complex Systems, Vol 7, No 1, 2017.

30 Philip Haynes, ibid, pp. 95-97.

31 OECD Observatory of Public Sector Innovation, Working with Change: Systems approaches to public sector challenges, GOV/PGC(2017)2, p. 14.

32 The framework was developed by members of the Brighton Systems and Complex Systems Knowledge Exchange, hosted by the School of Applied Social Science and the Community University Partnership Project (CUPP), <https://www.brighton.ac.uk/_pdf/research/ssparc/toolkitframework.pdf> [30 April 2020].

33 The Brighton team draw here on a well-known management process developed by Donella Meadows.

34 Philip Haynes, ibid, p. 70.

35 Mary Darking, Philip Haynes and Julia Stroud, “Developing public service knowledge and learning about complex systems: using a community of practice to integrate theory and practice”, Complexity, Governance & Networks – Vol. 4, No 1 (2018) Special Issue: Teaching Complexity, University of Bamberg Press, pp. 46-58 <https://ubp.uni-bamberg.de/ojs/index.php/cgn/article/view/45> [30 April 2020].

36 Erik-Hans Klijn, “Managing complexity: Achieving the impossible?”, in Critical Policy Studies, 2007, Vol. 1, no. 3, pp. 252-277.

37 Jan Arpe, “Globalisation and its Complexity: Challenges to Economic Policy, Bertelsmann Stiftung, “Shaping Sustainable Economies” program, <https://jan-arpe.com/wp-content/uploads/2017/06/Globalization_Complexity.pdf> [30 April 2020].

38 Peter Ho, Coping with Complexity, McKinsey & Company, 2012, <https://www.mckinsey.com/industries/public-sector/our-insights/coping-with-complexity#> [30 April 2020].

39 Peter Ho, “The future-of-government approach is important, but not easy to execute”, extract of an IPS-Nathan Lecture, reported in Today, 5 April 2017, <https://www.todayonline.com/singapore/future-government-approach-important-not-easy-execute> [30 April 2020].

40 Elinor Ostrom, “Beyond Markets and States Polycentric Governance of Complex Economic Systems”, Noble Prize Lecture, December 8, 2009, <https://www.nobelprize.org/prizes/economic-sciences/2009/ostrom/lecture/> [1 May 2020].

41 Cilliers, ibid.

42 This point was made and explained to me by Daniel Charrier, a quantum physicist by training, and colleague at the Lycée Pierre-Gilles de Gennes-ENCPB, Paris.

43 Michael Veseth, “What is International Political Economy?”, <http://www2.ups.edu/ipe/whatis.pdf> [1 May 2020].

44 Nils Kvilvang, Erik Bjurström and Roland Almqvist, “Making sense of complexity in governance: the case of local public management in the City of Stockholm”, Policy Studies, 2019, DOI: 10.1080/01442872.2019.1577372

45 Tom Christensen & Per Lægreid, ibid.

46 David Colander and Roland Kupers, ibid, pp. 41-3.

47 Ibid p. 9.

48 Philip Haynes, ibid, p. 57.

49 Paul Cairney and Robert Geyer, ibid.

50 The expression was coined by H.L. Mencken, an American journalist and essayist. I first read it and specifically recall it from a text by Gerry Stoker, a specialist in British local government, “New Localism, Progressive Politics and Democracy, in Andrew Gamble and Tony Wright (eds), Restating the State?, a special edition of The Political Quarterly, 2004, pp. 117-129. Stoker uses the expression when talking about complexity.

Top of page

References

Electronic reference

Nicholas Sowels, “A Brief Introduction to Complexity Theory in Managing Public Services”Revue Française de Civilisation Britannique [Online], XXVI-2 | 2021, Online since 05 January 2021, connection on 02 November 2024. URL: http://journals.openedition.org/rfcb/8103; DOI: https://doi.org/10.4000/rfcb.8103

Top of page

About the author

Nicholas Sowels

Université Paris 1 Panthéon-Sorbonne

Nicholas Sowels is a Senior lecturer in English for economics at the Université Paris 1 Panthéon-Sorbonne. His present research interests include UK public services, Brexit and financial services, and inequalities in the UK.

By this author

Top of page

Copyright

CC-BY-NC-ND-4.0

The text only may be used under licence CC BY-NC-ND 4.0. All other elements (illustrations, imported files) are “All rights reserved”, unless otherwise stated.

Top of page
Search OpenEdition Search

You will be redirected to OpenEdition Search