A draft of this paper was presented during the panel “Empirical constitutionalism: Bruno Dente’s intellectual legacy” at the 2022 meeting of the Società Italiana di Scienza Politica in Rome. The author would like to thank the chairs, Simona Piattoni and Claudio Radaelli, for organizing the panel.
1In a speech given when he was awarded a degree honoris causa by the Universität Autònoma de Barcelona on 7 October 2019, Bruno Dente defined his 'first and only law’ of policymaking thus:
"In order to solve a complex policy problem, the policymaking process must exhibit the same amount of complexity – at least in the sense that the number and diversity of actors involved should reflect the types of interests affected by the problem or by the solution. In other words, trying to solve complex policy problems by drastically simplifying them is stupid and dangerous" (Dente, 2019, p. 29).
2These words were spoken by an Italian scholar who, throughout his long academic career, never contented himself merely with research, but also interacted constantly with policymakers in different and increasingly important roles: as a member of technical committees, as an advisor, and as a director of training courses for public managers at a regional, national, and European level. Above all, as he carried out these appointments, Bruno adopted a reflexive posture towards his intense interactions with policymakers and policy takers, and used their intricacies, contradictions, and failures to form a better definition of his analytical framework with a view to advancing policy evaluation in and for public institutions.
3Despite this fact, his work – and that of other Italian policy scholars in general – has had a very limited impact on the policy-analytical capacities (Wu et al., 2017) of Italian institutions, as the fourth section explains in greater detail. As a recent example of our failure, we might cite the words of Prime Minister Mario Draghi in his response to the Senate while debating the vote of confidence on 20 July 2022: "I have also said what I needed to say on the subject of citizens' basic income: basic income is a good thing, but if it doesn't work, it's a bad thing"1.
4The objective of the pages that follow is to confront, and explain, this apparent contradiction by analyzing it from three different perspectives:
-
by providing a historical reconstruction of the early steps taken in Italy by the first students of policy analysis and evaluation, with a focus on the theoretical dissonances with the established academic organization in the field of political science. The purpose here is to offer evidence that will not only recall, but also allow us to understand, the strengths and weaknesses of a relationship that has not yet been fully harmonized;
-
by placing the Italian case into a broader comparative perspective in order to gain a better understanding of the dynamics of the institutionalization of policy evaluation; and
-
- 2 Course at the Collège de France, 1977-1978.
by offering a critical and self-critical reflection on the limits and responsibilities of the epistemic community of public policy analysts as a result of its ineffective impact on Italian governmentality (Foucault, 20042).
5We will begin with a premise: my brief historical reconstruction is clearly not an impartial one as I spent my early years of study and research with Bruno Dente. This was the 1970s, and at first, as in other countries, there were changes taking place that challenged traditional interpretations of Italy’s political, institutional, and administrative structures. From a political standpoint, the Italian Communist Party's victory in the 1975 local elections, and its strong showing in the general election the following year, made it possible that it would ‘overtake’ the Christian Democrats in the future. But there is also a different figure which gives an idea of the distance between now and then: those voting for the Senate made up 93.39 per cent of people who were entitled to vote; those voting for the Senate in the general election on 28 September 2022 made up 63.90 per cent. At an institutional level, the creation of the ordinary statute regions in 1970 was followed by significant innovations in their planning and research structures. Locally, the movements in the 1960s in favor of the right to housing and social services had given rise to experiments in participation in districts that, in some cases, had become examples of participatory decentralization. In the economic planning sector, the passage from the first Five Year Plan (1996-1970) to the second (1971-1975) was marked by the publication of Progetto '80 (Ministero del bilancio e della programmazione economica, 1969), a document that sought to provide a new direction for planning policy. This document had revealed itself to be substantially ineffective. However, it had not been implemented in practice in any way.
6All these changes brought to light a series of limitations in the traditional categories used by historians, sociologists, legal scholars, and political analysts to study the Italian situation, because it made the distinction between politics and administration, between expert and political bodies, and between expressions of spontaneous and regulated participation vaguer and more complicated.
7A new generation of scholars now attempted to decipher these changes. These scholars came from atypical educational backgrounds, marked by a multidisciplinary approach. In the early 1970s, in fact, a number of institutions, from local entities to regions and foundations, had begun training programs in social sciences which, to a certain extent, loosened the monopoly held by university education. In these subject areas, the traditional training, based on a close connection between assistant and full professors, was replaced by schools that anticipated the structure of PhD courses to some degree, but which were also places for interdisciplinary training and socialization.
- 3 Both of us were students at the Catholic University of Milan.
- 4 The author obtained a degree in philosophy, defending a thesis on the negative dialectics of the Fr (...)
8Bruno Dente’s first education pathway (1969-1973), after his undergraduate degree in law3, was the Istituto per la Scienza dell'Amministrazione Pubblica (ISAP), established in Milan in 1959 on the initiative of the City Council and the Province, and dedicated to the study of the transformations underway in public administrations. While it adopted a predominantly legal approach, it was also open to contributions from historians and sociologists. My own education continued at the Comitato per le Scienze Politiche e Sociali (CoSPoS) in Turin, under the directorship of Norberto Bobbio. This was one of the four Italian programs financed by the Social Science Research Council of New York and the Ford Foundation, with support from the Fondazione Adriano Olivetti. In this case, too, the students accepted for the courses with scholarships came from various faculties4, and the seminars covered a broad range of subjects and methods. These contexts gave us great freedom to choose the paradigms that we needed, and we used this to the fullest.
9When he looked back over his research path, Bruno Dente recalled another important fact: "One must also remember that people of my age were students in 1968" (2019, p. 18). The author can only confirm the importance of this experience, which for us translated into viewing knowledge as a responsibility that has to lead to socially desirable consequences: "It was this foundational multidisciplinarity, coupled with the imperative to produce what Lindblom called 'usable knowledge', that shaped how we were thinking and doing research" (Dente, 2019, p. 18-19).
10As Bruno wrote, the freedom of research that we enjoyed led us to discover the study of public policy, which was a 'Made in the USA' field of research: "We were looking for new ways of making sense of the processes we were interested in. With public policy analysis we found a new unit of analysis, the policy, an analytical space within which it was possible to arrange the different elements that we were gradually discovering to be relevant in such a way as to identify causal connections and give reasonable explanations of outcomes" (Dente, 2019, p. 20). This text continues with quotations from scholars such as Harold Lasswell, Aaron Wildavsky, Charles Lindblom, Theodore Lowi, and Graham Allison (as well as Jean-Luc Godard and Leonard Cohen).
- 5 Among the most important programs, we should recall: The Council for European Studies (https://coun (...)
- 6 In 1975, the American Political Science Association (APSA) institutionalised this network with the (...)
11To give a better context to the dynamics of this process of intellectual import, we need to recall that, at the beginning of the 1970s, Italy was at the center of extensive contacts with the American social sciences through scholarships, post-doc programs, and research funding. In the first place, it was international political reasons that lay behind the major resources dedicated to these relationships. In Italy, the recent fascist regime (1922-1943) had had a profound effect on the organization of political science faculties, and on their teaching, and it was necessary to strengthen the common analytical framework so as to avoid a repetition of this drift. Italy was also the country with the strongest Communist Party among all the NATO countries. Although this fact caused concern on the one hand, it also attracted numerous political scientists (such as Sidney Tarrow, Joseph La Palombara, and Robert Putnam) who were interested in understanding the decisions taken by local governments and parliamentary groups on the left. For young researchers in the social sciences, there were also numerous opportunities to spend time in the United States5. The generation from a few years earlier had already made extensive use of these resources, and now continued to keep the contacts between the two scientific communities alive. Some, such as Franco Ferraresi, Paolo Farneti, Giorgio Freddi, Alberto Martinelli, and Gianfranco Pasquino, had returned to work in Italian universities, while others, including Giuseppe Di Palma, had remained in the United States or gone back there after returning to Italy (Giovanni Sartori)6. Particularly in the Turin-Milan-Bologna triangle, this ensured that our group of recent graduates had a series of resources that was indispensable in deepening our understanding of American political science, with extremely up-to-date libraries and frequent encounters with visiting professors.
12It may seem strange that, at a time when political debate on the left seemed to take only two alternatives into consideration – European social democracies or communist systems – two researchers with progressive ideas should have looked across the Atlantic, to a country where "… civil war and authoritarian rule have become frightening possibilities, an undeclared war in Vietnam that violates the moral conscience of the world, these are continuing conditions entirely unpredicted by political science" (Easton, 1969, p. 1053). By studying urban movements to regenerate districts and the War on Poverty, however, we discovered the policy sciences movement, and were struck by it because it defined itself as "an emerging interdisciplinary movement simultaneously serving democratic values, academic theory-building, and governmental needs" (Lasswell, 1951, p. 14).
- 7 "Whereas the distinction between politics and policy is easy to make in English, it is much more di (...)
13Although the rare translations initially obscured, rather than clarified, the originality of policy studies7, later, after a more precise reading of the English texts, these became the aspects that most inspired our analytical curiosity:
-
the interdisciplinarity: "Policy analysis is an applied subfield whose content cannot be determined by disciplinary boundaries but by whatever appears appropriate to the circumstances of the time and the nature of the problem" (Wildavsky, 1979, p. 15);
-
the idea that knowledge means having a responsibility to act: "In holding that to know is to bear a responsibility for acting, post-behavioralism joins a venerable tradition inherited from such diverse sources as Greek classical philosophy, Karl Marx, John Dewey, and modern existentialism" (Easton, 1969, p. 1059);
-
the clear reference to democracy and human dignity: "In a word, the special emphasis is upon the policy sciences of democracy, in which the ultimate goal is the realization of human dignity in theory and practice" (Lasswell, 1951, p. 15)
-
the active role of analysts: "The policy sciences are concerned with knowledge of and in the decision processes of the public and civic order" (Lasswell, 1971, p. l);
-
the support for citizen participation: "Whatever else policy analysts may be, therefore, I believe they should be advocates of citizen participation (..). Designing policies that facilitate intelligent and effective participation is an essential task of policy analysis" (Wildavsky, 1979, p. 255);
-
the right to speak out: "The task of policy analysis, therefore, is the weighty and ancient one of speaking truth to power" (Wildavsky, 1979, p. 126); and
-
the right to go against the grain: "The capitalistic assumption that capitalistic processes produce their own social solutions is sheer mythology" (Lowi, 1969, p. 18).
14In brief, what struck us was the careful mix of civil passion and scientific knowledge of which Giorgio Freddi (2009) wrote in his analysis of Aaron Wildavsky's contribution.
15All things considered, it was possible to imagine that these subjects would capture the interest of two researchers who had come into contact with the 1968 movements: "This is a foundational element of our work: the idea that we are trying to improve the theory and practice of public policymaking, by better understanding the reality of the policy process so that we can suggest ways of overcoming the obstacles that often make collective problems intractable" (Dente, 2019, p. 20).
- 8 In the course of 50 years of friendship, many arguments were discussed during informal meetings, bu (...)
16This orientation was – and remains – non-political in the partisan politics sense8. Both of us worked for administrations with different political majorities with the same strong commitment. Political positions were also not significant when it came to choosing the authors from whom we drew our inspiration. The criticisms we sought not to remove were those included in what Lindblom defined as ‘radical theory’ in his Presidential Address to the American Political Science Association in 1981, as opposed to mainstream, conventional political science (Lindblom, 1982). Our approach attempted to consider the most important contributions from both these perspectives.
17Our reading of Lasswell, Lindblom, and Wildavsky also picked some of the significant epistemological implications in their thoughts, such as the contingent and contextual nature of knowledge and the fluidity of the distinction between facts and values and between methods and results. These assumptions were already familiar to us thanks to our knowledge of the Frankfurt School's critical theory of society. When we discovered that other scholars were also acknowledging a common thread between policy sciences and the thoughts of Habermas and Offe, we found confirmation of our 'syncretism': "Policy analysis that recognizes the centrality of 'interaction' must therefore rest upon a powerful theory of action, and that theory of action is the heart of the critical theorist's concern." (Forester, 1982, p. 155).
18In this way, our contact with policy sciences unlocked an awareness of the 'epistemological revolution' underway in the United States (Rosenau, 1991), which – depending on its various roots – was being defined by the use of terms such as post-positivist, post-behavioralist, and post-modern. I will refer to this as 'post'.
- 9 The author was also heavily involved with the trade unions as part of the university ''movimento de (...)
19In every field of knowledge, the academic anchoring of a new discipline involves some degree of resistance on the part of the existing study subjects. In Italy, there was also a degree of added friction in the relationship between mainstream political science and the emerging policy sciences, and so before we examine the scientific reasons behind the misalignments, we should also mention the practical aspects of the relationship. In the previous paragraph, we highlighted the importance of an open educational context, one that has no subject-related boundaries, one that favors hybridization and is detached from the logics of academic reproduction. Once the study grants had been exhausted, however, the weak ties to the proper epistemic community became a handicap, not only to the institutionalization of policy sciences but also to the development of individual employment prospects, because it became necessary at that point to find a place 'in someone else's home'. Being a participant in the 1968 movement did not make things any easier9, in part because of the terrorist deviation of a section of the movement.
20It was the topic of interdisciplinarity that connected practical and theoretical questions, as Bruno Dente recalled: "Not having had any formal training in the discipline – actually, we were the very first to teach it in our universities – we assembled different elements coming not only from the American tradition, dating since Lasswell, but also from public administration, organizational sociology, law, public and development economics, game theory, urban planning, etc." (Dente, 2019, p. 18).
- 10 The Rivista Italiana di Scienza Politica was first published in 1971, and the Società Italiana di S (...)
21In turn, however, Italian political science in the 1970s and 1980s was in search of independent institutional recognition, with full emancipation from both the law and the other social sciences10.
22Because of this, public policy scholars were forced to steer their academic careers and research projects between different tensions: not only their membership of the epistemic community of political scientists, but also the indispensable search for collaborations in other areas of knowledge; also, respect for the standards of scientific production combined with the need for verification in the field through a direct involvement in initiatives managed by the public administrations (Dente et al., 1978).
- 11 For our Italian readers, we would recall Valerio Onida, Franco Bassanini, Gustavo Zagrebelsky, and (...)
- 12 In 1984, Bruno edited the first publication of Vincent Ostrom, Robert Bish, and Elinor Ostrom’s 198 (...)
- 13 British Academy Wolfson Fellow, Institute of Local Government Studies, University of Birmingham (19 (...)
- 14 The author played some role in this institutionalization when, in 1981, the National University Cou (...)
23Also, once our time as post-docs had come to an end, the resources at our disposal (such as libraries and seminars) became totally inadequate for the purposes of moving on from being students to being teachers and managers of research projects with a broad international scope. To access funding and grants for American universities, we needed references we did not have because our underlying training had its roots in subject areas (such as law and philosophy) from which the young Italian political science was intended to differentiate itself. In addition, as we will see shortly, the processes we wanted to study in depth were very different from the central themes of politological research in Italy: the instability of coalition governments, the numerous parties, their internal dynamics, and clientelism, for example. To escape this impasse, Bruno Dente turned to his great ability to build informal networks to circumvent formal obstacles. He was aided in his attempts to secure his first university appointments and publications by the support of a highly cohesive intellectual community of young jurists and political historians with a progressive position, who were in turn interested in an interdisciplinary approach to the study of public administration and local government11. Furthermore, during a period characterized by material obstacles that are unthinkable today, such as the long waiting times for airmail and the weight of luggage filled to the brim with photocopies, he developed a network of ‘homemade’ international relationships, paying for trips out of his own pocket and sleeping on the sofas of young colleagues and relatives who had emigrated. In this way, he was able to follow The Workshop in Political Theory and Policy founded by Vincent and Elinor Ostrom at Indiana University12, take part in international conventions, and establish close relationships with European local government scholars. This strategy, which he planned and carried out completely independently – guided only by his opinion of the scientific value of the relationships he formed – saw significant results as early as the 1980s, as proved by the book he published in Italy with Renate Mayntz and Laurence Sharpe (Mayntz et al., 1977), the grants he obtained from major international research centers13, and the first articles accepted by international journals (Dente & Regonini, 1980). Above all, it is proved by the commencement of the first courses on the Analysis of Public Policies at the University of Bologna (1982), on the initiative of Giorgio Freddi14.
24Following this biographical (and autobiographical) parenthesis, we will now examine some disagreements of greater theoretical significance. In our case, in addition to a new subject for analysis, the 'newcomers' also proposed an innovative epistemological approach that had profound consequences for the choice of research methods and the validation of results.
25In other countries, the policy sciences paradigm was able to count on some form of continuity with the theoretical research being carried out in other areas of the social sciences. As we have seen, it was possible in both the United States and Germany to trace links to original epistemological approaches that had already been acknowledged as authoritative. In the United States, the obligatory reference was to the pragmatism of John Dewey: "The policy sciences are a contemporary policy adaptation of the general approach to public policy that was recommended by John Dewey and his colleagues in the development of American pragmatism" (Lasswell, 1971, xiv). In the German Federal Republic, the development of a policy approach aware of the limits of the "intellectual tradition of value-free empirical research" (Scharpf, 1978, p. 349) was able to use the path opened by the broad debate in the 1960s that involved the social and political sciences, and which is often referred to as the 'positivism dispute' (Saretzki, 2015).
26In the 1970s and 1980s, it was hard to find comparable analytical interests in Italian political science. As we know, at that time Italian professors were placing the study of political parties at the heart of their research and teaching, as it was believed that they constituted the 'main argument' of the subject (Morlino, 1991). Besides, the singularity of the Italian party system, compared with the other European democracies, justified an investment in this direction. As von Beyme noted: "The specific nature of the 'Italian case' is covered above all in the theory of the parties" (1986, p. 95).
27The dominant theory among scholars of the Italian political system associated the relevant anomalies and incongruences of the Italian party system with the relevant anomalies and incongruences of its governments (meaning, in the narrow sense of constitutional law, the structure of the executive branch described in Article 92 of the Constitution) as part of a stringent cause-effect relationship. Accordingly, phenomena such as coalitions in perennial conflict, constant government crises, and vague – and in any event elusive – program commitments were simply viewed as being indicators of 'non-government' (Allum, 1974; Di Palma, 1978).
- 15 Dente had discovered the works of Theodore Lowi by studying his initial research on local governmen (...)
28The young policy scholars had read Theodore Lowi15, however, and had been struck by the consistency of his theory, summarized by the term 'policy determines politics', marking a clear overturning of research that saw policies as the direct dependent variable of politics. In Lowi's opinion, as we know, it is the characteristics of the problems on the table that structure the relationships between policymakers: "… a political relationship is determined by the type of policy at stake, so that for every type of policy there is likely to be a distinctive type of political relationship" (Lowi, 1964, p. 688).
29In addition, Lowi's clear typology – distributive, regulatory, redistributive, and constituent policies – opened the door to a new way of locating the Italian case within a comparative perspective. For example, the 'sharing-out government' (Amato, 1976) ceased to be an Italian prerogative when compared with the space distributive policies and pork barrel practices applied in the U.S. Congress to secure political benefits of one kind or another. Our methodological suggestion (which we expressed ambiguously, if truth be told) was at least to keep analyses of the dynamics of politics separate from the reconstructions of the decision-making processes in the various policy arenas: "The fact that representatives of the political parties are among the most active and recurrent policymakers does not imply that the strategies, interactions, and coalitions by which they determine substantive policies coincide with the strategies, interactions, and coalitions that these same actors pursue for questions concerning relations between parties or party factions. On the contrary, in the Italian case there seems to be a marked divergence between the choices made in these two arenas, a divergence that also raises a series of issues of some theoretical importance" (Dente & Regonini, 1989, p. 55).
30Giorgio Freddi, who must be credited with a large part of the mediation process, including at a theoretical level, elegantly summarized the resolution of the conflict in these words: "In Italy, too, Lowi's deliberately rhetorical ultimatum led to lively exchanges: on the one hand, there were some manifestations of arrogant zeal on the part of the young neophytes of the new subjects, while on the other there was a somewhat peevish reaction from the political science 'establishment'. However, in the face of the increasing high-quality production of volumes by policy analysts that clearly showed the close connections with the traditions of political theory, the conflict ended before it exploded" (Freddi, 2002, p. 7).
31Although, from a practical standpoint, these misalignments have been settled to some extent, it has been (and remains) impossible to find an acceptable mediation in the particular Methodenstreit that saw the masters of 'normal' political science in conflict with the young scholars who were beginning to face the involvement, difficulties, and contradictions of evaluating policies. The problem was going beyond an analysis of the processes – 'the knowledge of' – and moving on to 'the knowledge in' (Lasswell, 1971, p. l):
"The orientation is twofold. In part it is directed toward the policy process, and in part toward the intelligent needs of policy. The first task, which is the development of a science of policy forming and execution, uses the methods of social and psychological inquiry. The second task, which is the improving of the concrete content of the information and the interpretations available to policymakers, typically goes outside the boundaries of social sciences and psychology." (Lasswell, 1951, p. 3).
32This process of 'going outside' was clearly rejected by the most eminent Italian political scientist, Giovanni Sartori: "I would say immediately that whoever supports the theory of evaluative science supports an unsupportable principle (...). Evaluative science is a contradiction in terms, a form of knowledge that is never born or that self-destructs" (Sartori, 1979, p. 46).
33Now, however, decades later, it must be emphasized that this position was far from being a defense of science as an exclusively theoretical activity locked in an ivory tower. In fact, it was anything but: in a harsh criticism of American political science at the end of the century, Sartori wrote: "I take the view that mainstream political science has adopted an unsuited model of science (drawn from the hard, exact sciences) and has failed to establish its own identity (as a soft science) by failing to establish its own, distinctive method (Sartori, 2004, p. 785). Sartori did not define the principle of value-free knowledge as an absolute constraint, but rather as a principle intended to guarantee the impartiality and honesty of research. Moreover, much of his writing was inspired by a civil commitment to improving the way in which the Italian political system operated.
34Where does the problem lie, then? The crux of the matter is that, in Sartori's opinion, the extension of theory to practice is defined as the mere application of the theory to actual situations: "An operative or applicable theory is a theory which is translated into practice in a confirming manner, i.e., as anticipated and established by the theoretic design. Applicability is, then, the correspondence of result and intent, of outcome and prediction. More vividly put, applicability is the application which 'succeeds', not the application which fails, which attains unforeseen and unwanted results" (Sartori, 1974, p. 142).
35However, both the works arriving from the United States and the lessons learned from the first practical evaluations, especially at a local level, led us to believe that the type of usable knowledge produced by these exercises went well beyond the simple application of the theory to the actual situations that an analyst investigates: "The necessary strategies were therefore much more complex and diverse and implied the ability to substantially modify the policy process" (Dente, 2019, p. 24).
36In the first place, it is hard to find general theories in the social sciences that can act as a foundation for specific applications:
"The learning agent – the teacher or the consultant, for example – (…) cannot assert with confidence that in 'comparable circumstances' others who behaved similarly would achieve comparable effects" (Schön, 1971, p. 233).
"The study of social knowledge in social change calls for a study of amorphous inquiry, probing, investigation, or search as practiced by many kinds of people in various roles. The specialist contributions of those who engage in professional scientific discovery and testing have a place in such processes, but only a limited one" (Lindblom, 1990, p. 8).
"In the study of policies, where knowledge is endogenous, evaluation is a research and discovery method that is a part of the decision-making processes together with ordinary knowledge, with an awareness that social problems are often resolved by social interaction or social learning rather than by scientific knowledge" (Radaelli & Dente, 1995, p. 172).
37These quotations and those in the first paragraph all converge in the same direction: "Public policy analysis is about understanding and managing complexity in the public sphere" (Dente, 2019, p. 24).
38Here, 'complexity' is no longer a generic term to indicate a situation that is unclear, complicated, or hard to decipher; rather, it is a concept with broad epistemological implications that, as we will see shortly, invest the relationship between evaluator and evaluated (Kurtz & Snowden, 2003).
39Of course, it is not possible to summarize the extensive literature on the complexity of public policies and their evaluation in just a few lines. We should recall, however, that:
-
the problems are often poorly defined, closely interwoven, and perverse (Rittel & Webber, 1974); the rules are constantly being redefined: "Non-linearity means that the act of playing the game has a way of changing the rules" (Gleick, 1987, p. 24);
-
the processes have undefined borders, and the actors' networks are tangled: "We will look at the policymaking as an extremely complex process without beginning or end and whose boundaries remain most uncertain" (Lindblom, 1980, p. 5);
-
when most of the work seems to have been done and the decisions have been taken, implementation reveals its chaotic features: "To discover why something that seemed simple actually was so convoluted, we wrote a book on Implementation to show how the complexity of joint action – multitudes of agencies, innumerable regulations, stacked-up levels of government – made it difficult to move" (Wildavsky, 1979, p. 4);
-
the comparison between the original objectives and the results obtained is frequently no more than a pious hope: "We know that objectives invariably may be distinguished by three outstanding qualities: they are multiple, conflicting, and vague. They mirror, in other words, the complexity and ambivalence of human social behavior" (Wildavsky, 1979, p. 215); and
-
the solutions quickly become new problems: "Past solutions create future problems faster than present troubles can be left behind" (Wildavsky, 1979, p. 70); "What is normally considered the dependent variable (policy output) is also an independent variable. … Policy invariably builds on policy, either in moving forward what has been inherited, or amending it, or repudiating it" (Heclo, 1974, p. 315).
40In short, "… more is involved than multiple causes; there are multiple effects as well, and more important, there are patterns of interaction in which it is impossible to distinguish between cause and effect" (Lasswell & Kaplan, 1950, p. xvii).
41This is 'only' the first aspect of the complexity involved in evaluating public policies, however: the one that looks at the particular characteristics of the phenomena to be evaluated. The second has yet broader implications, because it entails the contention that the mere intervention of the evaluator can modify the observed policies. Bruno Dente explained this second source of complexity and precariousness of evaluations very clearly:
"In the first place, I know that we do not have a general theory of policymaking and implementation, and I suspect that it is also impossible to build one. The reason lies not only in the contextual nature of the policy game but also, and more fundamentally, in the fact that our own work modifies the object of our study. Actors learn and adapt their behavior not only in light of their own experience but also in light of the research that we do. Through education and training, the cognitive endowment of politicians, civil servants, representatives of interests groups, and, obviously, experts has been progressively modified and enriched. As a consequence, they have changed their strategies and preferred courses of actions, which makes our previously valid interpretations and advice less useful" (Dente, 2019, p. 27).
42By analogy, the circular nature of the relationship between evaluator and evaluated recalls an astonishing work by John Dewey (1859-1952) and Arthur Bentley (1870-1957), author of The Process of Government (1908). The book, entitled Knowing and the Known, was written in 1949, when the two authors were very elderly, and contains a harsh criticism of the theory of knowledge underlying logical positivism, based on a clear separation between the observer and the observed. As well as the European epistemological debate, Bentley "was also fascinated by contemporary physics, where he saw parallels important to his own evolving social philosophy. Quantum mechanics, for example, replaces the 'billiard ball' interaction of discrete particles with an integral 'field' whereby the 'reality' observed depends upon the instruments employed and the interests of the observer" (Ryan, 1997, p. 778).
43Dewey and Bentley wrote: "… In our general procedure of inquiry no radical separation is made between that which is observed and the observer in the way which is common in the epistemologies and in standard psychologies and psychological constructions. Instead, observer and observed are held in close organization" (Dewey & Bentley, 1949, p. 103-104).
44In actual fact, much of the work of authors such as Lasswell, Lindblom, and Wildavsky is based on this type of approach. This explains the permeability of American policy sciences to those we have termed the 'post' approaches (Rein, 1983). However, Dewey and Bentley's transactional method also built a small but solid bridge between policy evaluation and American political science as early as the beginning of the 1950s.
45Instead, in Italy, because of both ancient and recent traditions in political studies, the epistemological discord in approaches to evaluation was, to some extent, more marked. And with this, we have completed our discussion of the past because, from this point on, Dente’s published works speak for themselves.
46Today, several decades after the birth of the new discipline, there is no lack of retrospective analyses, both internationally and in Italy. These are not reassuring.
47There can be no doubt that policy studies in a broad sense have gained space in universities, above all in the United States, but also in Italy. The educational profiles are now better structured and more specialized, partly because more advanced quantitative research techniques have been adopted. In the United States above all (Radin, 2013), although in Italy as well to a certain extent, the policy analyst profession is better defined and more recognized. The number of clients has increased and these are more clearly differentiated, too, both in institutions and in the non-profit sector, in think tanks, private enterprises, and the media.
48However, "Growth in academic public policy programs and ready employment are not complete measures of success or salience" (deLeon & Steelman, 2001, p. 163). In fact, if we consider the promise offered by policy analysis and evaluation between the 1950s and the 1980s, the outcome changes considerably. Hopes of a positive impact on the quality and range of democracy have not materialized; on the contrary, there has been a constant decline in citizens' participation in Western countries over the past 30 years, in both the political arena – with a reduction in the percentage of voters – and in the social arena, with a contraction in the forms of mobilization from below that were experimented with in the 1960s and 1970s (Jörke, 2016; Parvin, 2018). Above all, the space for policymaking based on some form of informed convergence on the definition of problems and solutions has been reduced, while the post-fact world (Perl et al., 2018) and the polarization of political competition have spread rapidly:
"The United States unfortunately has become the leader in a movement away from careful use of evidence and analysis in policymaking toward a more politicized style of policy. While all political actors have had their own opinions as long as there has been politics, now everyone has their own facts. The failure to accept evidence, even coming from reputable scientific sources (climate change, for example) makes effective policymaking all the more difficult" (Peters, 2018, pp. 347-348).
49As we know, the reasons behind this situation are in turn extremely complex, and the epistemic community of policy analysts is a long way from being able to control it, even if some scholars have called into question the tendency of universities to be self-referential: "The incentive structures for faculty, departments, and universities encourage atomistic thinking and research to the detriment of more common interest efforts" (deLeon & Steelman, 2001, p. 169; see also Radin, 2018, p. 48). In fact, little remains of the spirit that led Lasswell and Wildavsky to create schools for new 'professional training in the public interest'. The trend towards 'speaking truth to colleagues’ sometimes seems to prevail over the suggestions offered by Wildavsky. There is also a tendency for some inconvenient issues, which revolve around the distribution of resources and power, to remain unexplored: "One of the central concepts of public policy analysis as I know it – the resources of the actors – is totally ignored by the vast majority of textbooks on public policy" (Dente, 2019, p. 19). In general, the trend toward an improved technical quality of research and more precise analytical methods has been accompanied by a clear attenuation of the potential for critical theory. In many ways, the original approach of the policy science movement, the methods and objectives of which we described in the first paragraph, has faded away. In the United States, the gap between the aspirations of the nascent stage and the actual results achieved is more marked, and has been more studied, not only currently (Ascher, 1986; Schön & Rein, 1994, p. xvi). The attempt to 'relaunch' the methodological complexity on more solid epistemological foundations has not provided the desired results. On the contrary, in some respects, it has made the subject more abstruse, and has given up the production of useful knowledge: "Some critical approaches have generally abandoned the idea of democracy and instead focus much of their attention on discrediting the continued use of positivist and neo-positivist methodologies" (Ingram et al., 2016, p 145).
- 16 These are the subjects of the two most recent reports prepared by the author for the Lombardy Regio (...)
50The dismal outcome we have recounted changes radically if, instead of taking into account only the training and supply of skills, on the one hand, and the quality of democracy on the other, we modify our perspective decisively and look at what lies in the middle: in other words, if we consider how policy analysis and evaluation have taken root in institutions. From this standpoint, it is undoubtedly possible to make the claim that this paradigm, this way of viewing what governments do and what they should do, has become a core component of good governance in a large majority of democratic systems. In the first place, the institutionalization of these tools now involves countries and continents that had previously been excluded from it (Furubo et al., 2002; Jacob et al., 2015; Stockmann et al., 2020). Second, the criteria that have been adopted go beyond a mere verification of the monetary costs and efficiency, and put effectiveness, relevance, and the equity of measures front and center (OECD, 2020). In addition, policy evaluation is being extended to new sectors, from food waste to the environmental impact of cattle farming16. Partly as a result of this, research teams are seeing a constant expansion of disciplinary skills (Jacob et al., 2015, p. 15), with the addition not only of IT experts, but also climatologists, geographers, and biologists. Finally, in many countries, starting with the United States, numerous legislative initiatives have been adopted to regulate the interaction between policy evaluation and other forms of audit, from controlling public accounts to evaluating the performance of public administrations.
- 17 Bruno Dente did not have a great deal of confidence in the evaluation of results, preferring to foc (...)
51The overall judgments on this major expansion differ according to the epistemic community to which the observer offering the judgment belongs. For the learned societies that represent the point of view of evaluation professionals and those who train them (the Associazione Italiana di Valutazione in Italy, and the American Evaluation Association in the United States), the enormous growth in the evaluation machinery brings with it a series of risks with regard to the reliability of the assessment tools, the transparency of relationships with those who commissioned the work, and the larger impact on society (Dahler-Larsen, 2012; Furubo & Stame, 2018)17.
52Here, on the other hand, we take another position, that of political science, and so we use a more 'technical' concept of institutionalization of policy evaluation in order to explore the dynamics activated by the increase in policy-analytical capacity in governments, legislatures, and independent agencies (Regonini, 2012; Jacob et al., 2015).
53Having established our particular observation point, we need to make an immediate statement. Scholars who study the policy sciences of democracy have never experienced an age of innocence, because ever since the 1950s, they have stressed one fact: evaluations carried out in and for institutions have an ineradicable political connotation: "Evaluation, which criticizes certain programs and proposes to replace them with others, is manifestly a political activity. If evaluation is not political in the sense of party partisanship, it is political in the sense of policy advocacy" (Wildavsky, 1979, pp. 228-229). In fact, evaluations "redistribute power and authority from politicians, interest groups and citizens to civil servants with the most analytical capacity" (Andersen, 2020, p. 270). There is also no guarantee that democracy will gain in the end. However, this fundamental awareness has not prevented the policy sciences from identifying and investigating a series of functions of policy evaluation that are essential for democratic institutions, and are generally defined in terms of accountability, learning, and enlightenment.
54Bovens has given us the most common definition of accountability: "... a relationship between an actor and a forum, in which the actor has an obligation to explain and to justify his or her conduct, the forum can pose questions and pose judgment, and the actor may face consequences" (Bovens, 2007, p. 470). Within institutions, this requirement means that evaluation must be a systematic exercise carried out by independent structures, often known as watchdogs or independent oversight institutions.
55If the function of accountability is to report what has happened in the past, the function of learning is to facilitate changes to future choices by "the updating of beliefs based on lived or witnessed experiences, analysis or social interaction" (Dunlop & Radaelli, 2013, p. 599). It is often social interaction, rather than analysis, that gives rise to a reorientation of preferences: "When a Regional Council discusses an annual evaluation report, development and learning derive from the fact that it is the results of policies that are being discussed, and not from the fact that some 'expert-demiurge' presents 'the' result of the evaluation" (Radaelli & Dente, 1995, p. 177).
56Finally, among the functions of policy evaluation institutions, that of enlightenment, of general orientation, of broadening the vision of the contexts, the problems, and the solutions is often cited: "Policymakers count on social science more for enlightenment than for engineering." (Lindblom, 1990, p. 273); "Social science can give officials a background of ideas, concepts and information that increase their understanding of the policy terrain" (Weiss, 1995, p. 146).
57To conclude, "It can be argued then, that the institutionalization (of evaluation research) – via professional policy fora and appropriate governmental institutions – can make the conflictual dimension of public policies more manageable, policy learning across coalitions more feasible, and ultimately policy change more likely" (Radaelli & Dente, 1996, p. 60). The quality of the public debate, and the failure of policy changes in many countries, would seem to contradict this relationship. However, the institution of independent judges cannot stop homicides, just as financial audits of supreme audit institutions do not put an end to corruption. The relationship between the learning, accountability, and enlightenment functions of policy evaluation and the quality of the democracy therefore requires a precise analysis of the specific conditions that impact the relations between knowledge and power in the various institutional contexts.
58To gain a better understanding of the functions of policy evaluation institutions, we need to look at a 'mature' example: the United States' Government Accountability Office (GAO), the watchdog of the U.S. Congress. There are two reasons for this choice. In the first place, the United States has been the most important, if not the only, example of the institutionalization of policy analysis for several decades:
"This evaluation praxis, which some European countries started to build already in the 1960s and 1970s, was built on an import from the United States rather than on earlier European traditions. (…) When we today, in Europe and beyond, talk about evaluation in the context of government, governance, international development and so on, it is about something associated with specific US-based notions about politics, policy development and implementation (…)" (Furubo, 2018, p. 191).
59The reasons why it is worth focusing attention on Congress's watchdog, the General Accounting Office, which was renamed the Government Accountability Office in 2004, were very well explained by Wildavsky in 1979: "Perhaps the General Accounting Office (GAO), which is beginning to do analytic studies, will provide a model of an independent governmental organization devoted to evaluation. Because it has a steady income from its auditing work, it can afford to form, break up, and recreate teams of evaluators. Its independence from the Executive Branch (the Accountant General is responsible to Congress and serves a 15-year term) might facilitate objective analysis" (Wildavsky, 1979, p. 228).
60The GAO was created in 1921 to audit public expenditure. Its functions were similar to those of a Court of Auditors, but it has been the subject of laws that have gradually expanded its area of competence. In order to achieve the practical implementation of the 1993 Government Performance and Results Act (GPRA), amended in 2010, the GAO produced precise methodologies to distinguish the performance evaluation of individual offices from the evaluation of policies and programs, while at the same time taking the interdependence between the two types of evaluation into consideration.
61So did everything work out well? No, it did not. Wildavsky's appreciation of the GAO continues thus: "But the independence of GAO has been maintained because it eschews involvement in controversial matters" (Wildavsky, 1979, p. 228). In effect, 40 years after those words were written, the increased political polarization in society and Congress makes it even more difficult to carry out an independent evaluation: "Policy analysis in the Congress is now being conducted in a much more partisan environment and this makes it much more difficult for the producers of these analyses, and for the public which may seek to learn something from them" (Joyce, 2018, p. 168). In addition, while on the one hand its strong institutional position acts as a shield, on the other it can reduce the space available for 'speaking truth to power' (Stephenson et al., 2019, p. 668).
62In order to offer an assessment of almost 50 years of commitment to the development of policy sciences in Italy, we will start out from a study that attempted to verify whether there was some form of correlation between the cultural and institutional embedding of policy evaluation and one of the principal components of good governance: the perception of effective policies for combating corruption (Dahler-Larsen & Boodhoo, 2019). In fact, albeit with all the warnings highlighted by the authors, the data confirm this correlation. This is good news, but a dismal picture emerges from the comparisons as far as Italy’s position with regard to both indices is concerned.
Figure 1. Evaluation culture and transparency
Evaluation culture (and transparency) takes up the results of the research carried out by Jacob et al. (2015) and includes not only strictly institutional aspects, such as the embeddedness of evaluation in legislatures, executives, and supreme audit institutions, but also the breadth of the professional organizations and the space given to the topic of evaluation in the public debate.
CPI refers to the Corruption Perception Index prepared and calculated by Transparency International (2011). It matches the perceived effectiveness of measures taken to fight corruption.
Source: Dahler-Larsen & Boodhoo, 2019, p. 274.
Such poor results make an urgent case for carrying out an in-depth study to identify the reasons behind this failure. The list should at least include the following:
-
the extremely rigorous controls carried out by jurists as gatekeepers of access to all forms of evaluation and advice regarding the functioning of institutions;
-
the influence of party membership on appointments to the various expert bodies;
-
the noose with which the perverse aims of performance evaluation in Italy – giving pay increases to managers – removed any evaluation of public policies (Regonini, 2018; Marra, 2021).
63We will now seek to present a brief comparison with France. The French Supreme Audit Institution has legal competencies very similar to the Italian one, but France occupies a far higher position than our country in the figure above.
64For many years, on those occasions when we attempted to promote the evaluation of public policies as a tool of legislative oversight, the parties involved (public administration managers, politicians, constitutionalists, etc.) regularly raised the same objection: the incompatibility of the proposal with Italy's institutional structure because the successful examples offered (the United States, Great Britain, Australia, etc.) were all common law countries with particular types of Supreme Audit Institution based on Comptrollers-General appointed by Parliament. In these contexts, which can be traced back to the Westminster Model, parliamentary oversight over the work of the executive evolved in a more decisive manner. It was claimed that countries in Europe (France, Italy, Spain, Belgium, etc.), which used the Judicial or Napoleonic Model, are different because they are characterized by the founding role played by public law, and by the fact that they have a Court of Auditors made up of independent judges who occupy an equidistant position between executive and legislature.
65However, the relevance of this distinction conflicts with the results of the most recent comparisons which, as the figure above shows, place France in a decidedly more advanced position than Italy with regard to the 'maturity' of the evaluation culture in institutions (Jacob et al., 2015, p. 12). Since 2008, in fact, France has been one of the countries that, like Switzerland and Sweden, cite in their Constitutions the evaluation of public policies as a tool for controlling parliaments with respect to governments. Article 24 of the new text reads: "Parliament shall pass statutes. It shall monitor the action of the Government. It shall assess public policies."
66The implementation of the reform, which is still underway, has seen the involvement of three different presidents from three different parties and all the principal French institutions. Recent analyses of its impact seem to be substantially positive: "Follow-up and evaluation have thus become a natural task for parliamentary committees that can rely on multiple sources of information and documentation. Beyond the access to governmental documents and reports, parliamentary committees have developed their own evaluation and research capacities. This is complemented by the interaction with other administrative units of the parliament and the cooperation with the Cour des Comptes in France" (De Vrieze, 2020, p. 432).
67Other observers have expressed more critical opinions. Above all, the risk relates to the role of Parliament which remains a weak institution in France compared with other countries: "A major stake for the future thus lies in the capacity of the French Parliament to grasp policy analysis while continuing to offer what is expected of Parliament: pluralist debate" (Rozenberg & Surel, 2018, p. 151).
- 18 Council of State, inaugural conference in the cycle of conferences on "The Evaluation of Public Pol (...)
68To tackle this problem, but more generally to reflect on the new relations between institutions after the constitutional reform, the Council of State dedicated its 2020 annual study to the topic of 'Carrying out and sharing evaluations of public policies'. The reason for this is clear: "The Council of State is not only a judge, and not only a legal advisor: it is also an institution that, because of its institutional position and composition, can – and therefore must, because every power is also a responsibility – illuminate and nourish the public debate in order to enable public officials to take decisions with full knowledge of the facts"18. This initiative aims not only to update the tools of government, but also to carry out a real redesign of French governmentality aimed at making policy evaluation a fundamental science of the public sphere (Regonini, 2017), with a precise impact on the relations between power and knowledge: "Article 24 of the Constitution defines the tasks with which it (Parliament) is entrusted (…). These three functions are complementary. There can be no good legislation and no in-depth monitoring without a pertinent evaluation of public policies. Accordingly, the evaluation of public policies (...) is the matrix of the two fundamental functions performed by Parliament" (Conseil d’État, 2020, from the speech by Gérard Larcher, President of the Senate, p. 162).
- 19 Consejo de Ministros, 19/04/2022, El Gobierno aprueba el anteproyecto de Ley de Evaluación de Polít (...)
69Following our brief summary of the situation in France, and after the reforms put in place by Spain, which included the institutionalization of public policy evaluation in its Recovery, Transformation and Resilience Plan19, we can say with certainty that the 'unsustainable backwardness' (Di Carpegna Brivio, 2021) of the Italian Parliament with regard to the evaluation of public policies, which makes our country the last of the great European states with a 19th-century form of governance structure, is not the fault of Napoleon and his Court of Auditors: it relates to precise national responsibilities. In the view of Elena Di Carpegna Brivio, the situation in Italy is the result of a series of missed opportunities:
-
the manner in which the OECD's recommendations on the quality of regulation were received and distorted;
-
the lack of attention to the innovations at a regional level in the early years of this century20;
-
the convoluted constitutional review project of 2016, rejected by the electorate in the referendum of 4 December 2016; and
-
the creation and subsequent abolition of the Senate’s Impact Evaluation Office [Ufficio Valutazione Impatto (UVI)].
- 21 See Valutare le politiche pubbliche – Francia: rilancio del Parlamento? March 2009 https://www.sena (...)
70In this brief section, we will look only at the last issue, which played out in the branch of the Italian Parliament that was best informed on the design and implementation of the French constitutional reform, as demonstrated by the dossier prepared by its Department for the Quality of Legislative Acts [Servizio per la qualità degli atti normativi]21.
71In 2015, after a series of studies and meetings with the Conference of Presidents of Regional Legislative Assemblies [Conferenza dei Presidenti dei Consigli Regionali], the Istituto per la Ricerca Valutativa sulle Politiche Pubbliche (IRVAPP) of Trento, the Associazione per lo Sviluppo della Valutazione e l’Analisi delle Politiche Pubbliche (Asvapp) of Turin, and individual researchers, the Senate organized a Master’s degree course in the Analysis and Evaluation of Public Policies at Ca' Foscari University in Venice to train a group of researchers who would be able to evaluate public policies in an institutional context, recruiting its students from among young graduates both inside and outside the Senate's research structure. When the course ended, in the summer of 2016, the Senate passed an internal decree creating the Ufficio Valutazione Impatto (UVI), led by its President, Pietro Grasso: "Its analyses are based on factual findings of the risks, costs and benefits, and costs and efficiency of public policies, and respond to criteria of validity, practicality, competence, impartiality, transparency, clarity, and comprehensibility. They are prepared by a working group expert in the analysis and evaluation of public policies, made up of competent and expert Senate staff and collaborators from renowned and respected institutions."22
72In the first years of its life, the UVI published numerous studies that analyzed and evaluated the health, education and research, productive activities, justice, taxation, and defense sectors. The reports were based on updated data, and the results were presented in a clear format. The network of international contacts was highly qualified. The review of foreign institutions with similar functions is still an important knowledge base today23.
73In June 2018, the new President of the Senate, Maria Elisabetta Alberti Casellati, proudly presented the UVI to the meeting of the Association of European Senates: "In recent years, the Italian Senate (...) has adopted significant initiatives to strengthen and structure its ability to analyze and evaluate public policies, making sage use of the most advanced scientific evidence and quantitative methods (...). It has created the Impact Evaluation Office, of which I have the honor of being the President, which continuously publishes in-depth analytical documents on specific areas of intervention on the Senate website"24.
- 25 Senate of the Republic, Impact Assessment Office, The Uncompleted. Evaluation of Legislative Acts i (...)
74The UVI's highly qualified activities gave rise to great expectations: "The hope is that, including by means of timely interventions to reform the Senate rules, the new Committee will in future be able to operate not only as a tool for strengthening the documentary and analytical capacity of the Senate administration, but also as a driving force for expanding and strengthening the oversight activities of the executive during the phase after a law has come into force" (Griglio, 2019, p. 220). When this article was published, however, the UVI was no longer in existence, even though it still appears in international comparisons of legislative oversight as one of the rare examples of renewal shown by the Italian Parliament (De Vrieze, 2020). One of the UVI's final dossiers, which bore the prophetic title 'The Uncompleted'25 (April 2018), included a detailed analysis of the distortions that have undermined the usefulness of policy evaluation in Italian institutions. However, there is no point in seeking an explanation for its disappearance in the other pages of the Senate. In all likelihood, although on the one hand the 'light' solution adopted to legalize and structure the implementation of the UVI, with an external committee independent of both the political bodies of the Senate and the structures that serve it, facilitated the quantitative and qualitative expansion of its activities, on the other hand, at the time (2018) of the transition from one legislature to another, with consequent new political equilibria, it revealed its fragility (Mazzantini & Tafani, 2020, p. 145).
75This fact exposes the total lack of proportion between the quality and commitment of the researchers and the network that supported them, on one side, and the political and administrative engagement that should have promoted its formal institutionalization on the other. As the French example demonstrates, to establish this new paradigm in an environment actively controlled by other authoritative disciplines such as law, public finance, and public management, is a long and challenging process that requires the involvement of numerous different institutions and epistemic communities (Regonini, 2017). In the absence of this 'conversion', the Impact Evaluation Office was unable to sustain itself.
76In Italy, in the documents accompanying the planning and implementation of the National Recovery and Resilience Plan (NRRP), the concept of 'reaching objectives' is only defined from two perspectives:
-
the legal perspective, with the adoption of decrees aimed at guaranteeing the formal appropriateness of spending and avoiding fraud and corruption; and
-
the public finance perspective, with measures directed toward ensuring compliance with deadlines in the use of funds, avoiding delays and diseconomies.
77These two planning criteria are absolutely necessary because:
-
they are forcefully requested by the European Union;
-
during the initial phase, compliance with them makes it possible to secure the legislative and financial 'stock' that is indispensable for moving on to the successive stages; and
-
given Italy's extreme deficiencies with regard to these two parameters in previous utilization of European funds, strong formal restrictions and careful monitoring of the expenditure procedures are decisive.
78Are these planning tools sufficient, however? No, absolutely not. If the concept of 'reaching objectives' is only defined based on these two criteria, the likelihood of actually obtaining results that at least bring about an attenuation of – if not a solution to – the problems underlying interventions is extremely small, given the almost total absence of evidence, data, and indicators to verify and evaluate an effective relationship between the planned interventions and the expected impact.
79In this case, too, a spot comparison with France is illuminating. The limitations of the Italian structure emerge clearly, even from a comparison of extremely brief, standardized documents such as the forms used by European Commission experts to verify the attainment of milestones and targets and to proceed with periodic disbursement of allocated funds to Member States.
80The two tables below report the 'verification mechanisms' used by France and Italy to measure something that is particularly controversial in our country: incentives to promote energy efficiency in residential buildings.
Table 1. Verification mechanisms – France
Related Measure (Reform or Investment)
|
Verification mechanism
|
C1.I1 – Buildings renovation / Energy renovation of private buildings – Number of MPRs validated
|
Summary document duly justifying how the target was satisfactorily fulfilled, with appropriate links to the underlying evidence.
The document shall in particular specify the number of “MaPrimeRenov” granted by ANAH (National Housing Agency) to households committing to carry out energy renovation works in their home. It shall also specify the number of beneficiaries benefitting from the bonus to undertake renovation works to bring their home out of the status of energy sieves, and the number of “MaPrimeRenov copropriétés” granted. The document shall include data extracted from the data system of ANAH, categorizing each MaPrimeRenov granted by a) type of beneficiaries (individual housing, copropriétés, also providing information on which beneficiaries benefit from the bonus to undertake renovation works to bring their home out of the status of energy sieves), b) whether it supports low-income, middle-income or high-income households, c) aid intensity, and d) location.
|
Source: ANNEX I Milestones, targets and related indicators – non-repayable support, 29/11/2021 https://commission.europa.eu/system/files/2021-11/fr_oa_annex.pdf (bold added by the author)
Table 2. Verification mechanisms – Italy
Related Measure (Reform or Investment)
|
Verification mechanism
|
Investment 2.1- Strengthening
of the Ecobonus and Sismabonus for energy efficiency and building safety
|
Copy of the publication in the Official Journal for primary legislation that is critical for achieving the objectives described in the CID and reference to the relevant provisions indicating the entry into force, accompanied by a document duly justifying how the milestone, including all the constitutive elements, was satisfactorily fulfilled.
|
Source: ANNEX I Milestones, targets and related indicators – non-repayable support, 22/12/2021 https://commission.europa.eu/system/files/2021-12/countersigned_-_italy_-_rrf_-_oa_1.pdf (bold added by the author)
81In the Italian document, publication of a decree in the Official Gazette appears about 100 times as the only verification mechanism of results in relation to various objectives.
82Based on this approach, therefore, there is a very high risk of an outcome that falls below expectations because the method used to manage these initial phases is exactly the same as the one that was at the root of the failure of other previous reforms, as has been diagnosed in numerous international comparisons and by the European Commission itself: "The quality of policy definition in Italy has not changed and the indicators highlight low levels of engagement and of evidence-based policy. (...) The evaluation through RIA (Regulatory Impact Analysis) is often more formal than substantial and mainly based on economic indicators" (European Commission, 2018, p. 528).
83Probably (and hopefully), the danger is not that 'the taps will be turned off' when it comes to the remaining payments for the NRRP, because monetary transfers to Italy are too big to fail and political agreement will prevail over expert verifications. In reality, the risks are that the powerful Italian distortions in the area of industrial, energy, environmental, healthcare, social, and educational policies will continue. In addition, once the results have been defined in terms of laws published in the Official Gazette, even accountability becomes an elusive criterion, particularly in the case of measures passed by a very large majority.
- 26 Bruno Dente did not agree with this approach: "We learn very little by studying policy failures; th (...)
84The most serious consequence of this approach, however, is that it is impossible to learn from the errors that have been committed. According to John Dewey's pragmatic logic, embraced by policy sciences, errors are precious events because they enable us to understand the need for learning and the direction it should take26:
"When we as citizens are deprived of our errors we also lose our capacity for self-correction, for self-improvement by moral development" (Wildavsky, 1979, p. 393).
"In this model, citizens, functionaries, social scientists, and other experts do what they have learned and then learn what they have done" (Lindblom, 1990, p. 219).
85As we have seen, the evaluation of public policies is a fundamental analytical perspective for learning from errors. Without this tool for reflection, all public programs are transformed into a bureaucratic system of organization: "We shall describe as a ‘bureaucratic system of organization’ any system of organization where the feedback process (…) does not function well, and where consequently there cannot be any quick readjustment of the programs of action in view of the errors committed" (Crozier, 1964, pp. 186–87).
86It is important to underline the fact that, in the case of policy sciences, learning is not simply a comparison between an original plan and the results obtained: rather, it is an exercise that makes it possible to modify the initial preferences themselves: "Implementation is no longer solely about getting what you once wanted but what you have since learned to prefer until, of course, you change your mind again" (Wildavsky, 1979b, p. 176). In other words, learning is a tool for coming to grips with the irreducible complexity of public policies. For this reason, "The role of governance is to surround learning with institutional structures and processes that reduce its errors and exploit its strengths" (March & Olsen, 1995, p. 223). If, as has happened in Italy, this investment is not made, institutions can only offer judgments like that offered by Mario Draghi, cited at the beginning of this article, which only acquires meaning when one moves from the policy arena to the political arena.
87Bruno Dente wrote: "If we were quickly forced to abandon our utopian hopes, we nevertheless maintained the idea that modern philosophers had to try to change the world" (Dente, 2019, p. 19). Compared with the specific objective of institutionalizing policy sciences, when we look at the results I have just presented, we plainly see that Bruno's, Luigi Bobbio's, and my generation in Italy has fallen far short of the anticipated results, notwithstanding the frequent interactions with public decision-makers. The silence that accompanied the disappearance of the UVI is the clearest evidence of this.
88The reasons for this might be attributed to a decidedly hostile context, and there is much still to be written on this subject. However, it would be better to use the last few lines to draw attention to two levers that we have been unable to activate with sufficient determination.
89The first relates to the ability to keep together two aims of the policy sciences that appear to be incompatible, to the extent that a kind of 'theorem of impossibility' has been generated for this type of knowledge. The reference is, of course, to Kenneth Arrow's theorem: "If we exclude the possibility of interpersonal comparisons of utility, then the only methods of passing from individual tastes to social preferences which will be satisfactory and which will be defined for a wide range of sets of individual orderings are either imposed or dictatorial" (Arrow, 1950, p. 342). This does not delegitimize democratic institutions, however, even if their procedures are based on a systematic violation of these ‘fairness’ criteria.
90A kind of 'theorem of impossibility' in policy sciences is based on the fact that no research method exists that is able to contemporaneously satisfy the need for a representation of the 'radical' complexity of policies but also has the ability to translate it into useful knowledge for policymakers and citizens. In certain respects, these two requirements, which are impossible to reconcile, recall Wildavsky's contrast between intellectual cogitation and social interaction. The attempts at institutionalization also increase this impossibility somewhat because they require linear procedures – whereas policy processes tend to be circular because there is a tendency to simplify the alternatives at play when the problems are perverse – and because of the requirement for evidence-based solutions when data interpretation allows numerous different readings (Stame, 2004). As we have seen, references to the risks of institutionalization frequently arrive from within the discipline:
"Wisely or not, a special social science subject labeled policy analysis attends to the needs of functionaries, not of citizens" (Lindblom, 1990, p. 189).
"… Once the policy sciences mechanisms became one of the hallmarks of contemporary governmental processes in the United States, their practice has been formidably distanced from their proscribed democratic ideals and origins" (DeLeon, 1997, p. ix).
91And yet, at different points, Lasswell, Lindblom, and Wildavsky emphasized the pragmatic need to achieve some kind of epistemological mutual adjustment in the initiatives undertaken by policy scientists in order to attempt to save both requirements, at least in part:
"The purport of inquiry is not necessarily 'theoretical' rather than 'practical': both manipulative and contemplative standpoints may be adopted" (Lasswell & Kaplan, 1950, p. xxv).
"Once a problem is posed, you may attempt to solve it by replacing interaction with cogitation, cogitation with interaction, cogitation with cogitation…" (Wildavsky, 1979, p. 283).
"All analysis is incomplete, and all incomplete analysis may fail to grasp what turns out to be critical to good policy. But – and this is a ‘but’ that must be given a prominent seat in the halls of controversy over incrementalism – that means that, for complex problems, all attempts at synopsis are incomplete. The choice between synopsis and disjointed incrementalism – or between synopsis and any form of strategic analysis – is simply between ill-considered, often accidental incompleteness on one hand, and deliberate, designed incompleteness on the other" (Lindblom, 1979, p. 519).
"If the reader will allow me my preference for two-thirds politics and one-third planning, this hybrid of social interaction and intellectual cogitation may be called policy analysis" (Wildavsky, 1979, p. 124).
92In the end, Dewey's (1927) inquiry concept was based precisely on the search for a form of contingent equilibrium between these two requirements.
93In other countries, the institutionalization of policy sciences was not born out of initiatives undertaken by institutions, but was rather the result of a public commitment by the most eminent members of the epistemic community of public policy scholars, who followed Laswell’s recommendation: “The aim is to subordinate the particular interests of a profession to the discovery and encouragement of public interest. This implies direct community participation as well as client service" (Lasswell, 1971, p. 119).
94In Italy, we predicated this hierarchy of priorities a great deal, but did not put it into practice very much. To apply the binomial used by Giorgio Freddi (2004) to summarize Aaron Wildavsky's contribution, we favored scientific knowledge far more than we did civil commitment. A brief review of the direction taken by the generations of scholars that have followed ours seems to reveal an even greater imbalance. But in this area, is it yet possible to increase scientific knowledge without any practical field verification? And what are the social costs of the lack of equilibrium? As analysts, we know perfectly well that the institutional embeddedness of evaluation of public policies is not brought to us by Father Christmas.
- 27 The reference here is to the debate I had at the National University Council with Luigi Firpo, emin (...)
95There is a degree of historical irony in this development of the discipline: when it first appeared in Italy in 1981, it was viewed with suspicion because it was seen as a form of Kathedersozialisten, of academic socialism27 (Lindenfeld, 1997). Forty years on, the concerns lie in the opposite direction.
96The second lever we have been unable to activate with sufficient determination relates to interdisciplinarity. As we have stressed on several occasions, we are referring here to a constitutive element of policy sciences, defined by a plural noun since the subject was first created: "A policy orientation has been developing that cuts across the existing specializations" (Lasswell, 1951, p. 3).
97As we know, disciplinary fragmentation is a tendency that relates to various areas of knowledge. It is associated with the system of incentives tied in with career progress and the allocation of research funds in universities. However, this only partly mitigates the responsibility of our epistemic community. I am thinking here of the limited interactions with even the Italian Evaluation Association, founded in 1997 on the initiative of a group of sociologists with a project which – at least in theory – was aimed not only at advancing knowledge, but also at consolidating the professional figure of the evaluator through lessons learned outside the classroom, in close contact with public institutions. Overall, the separation between disciplinary areas seems to have weakened the ability of both to 'speak truth to power', at least with regard to the Italian issues that we described in the previous paragraph.
98And yet, today more than ever before, and in Italy as in the rest of the world, interdisciplinarity is not only much in demand – it is also much used. Examples can be found in the teams that work on the United Nations' Sustainable Development Goals (SDG) (Montero & Le Blanc, 2019), the recent pandemic, and the climate emergency (Auld et al., 2021; van den Berg et al., 2021). Clearly, these problems – and governments' responses to them – are of great analytical interest, because by studying them we may be able to rewrite the traditional classifications of the quality of governance. Above all, however, they have an enormous need for the usable knowledge that flows from skills that transcend the limits of disciplinary fragmentation. One indicator of the new disciplinary contamination is the concept of ‘green governmentality’ or 'environmentality', used by geographers, biologists, and climatologists "to interrogate the intersections between nature, power, and society" (Rutherford, 2007, p. 292) in relation to policies implemented from Senegal to Ecuador, and from Ghana to Thailand and China.
99I will conclude with some quotations that successfully combine and acknowledge both the dramatic nature of the challenges before us and also the prospect of hope, in the spirit of John Dewey:
"Hope – the ability to believe that the future will be unspecifiably different from, and unspecifiably freer than, the past – is the condition of growth. That sort of hope was all that Dewey himself offered us, and by offering it he became our century's Philosopher of Democracy" (Rorty, 1999, p. 120).
"The expertise of policy researchers has potential for the new brave world of big questions that faces us in a post-pandemic future. (..) It is a return to the original vision of the founding figures of public policy. We need to reconnect to where we are coming from, and look at the future with more ambition and mindfulness of who we are and how our expertise contributes to the policy process and the quality of governance" (Dunlop & Radaelli, 2021, pp. 21-22)
"This, I think, is the most general lesson that can be drawn from the practice of Public Policy Analysis. It teaches us humility and patience. It reminds us that all the solutions we are likely to find are only partial. And that the complexity of modern societies is not something we must be afraid of" (Dente, 2019, p. 29).
100It is not much, but it is enough to allow us to move forward.