1The 2019 Nobel Prize in Economics awarded to Esther Duflo, Abhijit Banerjee and Michael Kremer for their experimental work in development economics has confirmed the mastery of randomized control trials (RCTs) as an essential method for impact evaluation. Since the 2000’s, RCTs’ use in development program is rocketing. This acknowledgement has elevated it to the rank of ‘gold standard’ (Martin Ravallion, chapter one). However, some criticisms are emerging about the scope of RCTs and their ability to report on the effectiveness of a policy. Yet, no serious academic discussion between “randomistas”, Nobel Prize winners (see Introduction of Angus Deaton and Epilogue of James J. Heckman) and other development economists has been launched to define the quality of randomisation tools for impact assessment.

2At the initiative of the European Development Network conference on ‘Malaise dans l’évaluation’ and AFD (Agence française de développement), Florent Bédécarrats (economist at AFD), Isabelle Guérin (socio-economist at IRD) and François Roubaud (economist at IRD) decided to coordinate a work which aims at opening a dialogue about the use of RCTs in development. This book is the result of a multi-disciplinary and collective work which tries to raise a debate about the legitimacy, strengths and weaknesses of RCTs’ methodology in the field of development. In an effort to create a controversy around the use of randomized control trials, the authors of this publication produce an anthology.

3The desire for comprehensiveness is achieved through twelve independent chapters, plus two introductory parts and a set of interviews. Each of which addresses a dimension of RCTs that should be subject to debate: methodology, statistical accuracy, effectiveness in the health field, ethics (non-exhaustive list). Regrettably, nothing is said about the potential long-term impacts of the use of RCTs in countries dependent of international public aid.

4The issues to be addressed are raised in the editors’ introduction. The innovative aspect of the book’s approach consists in questioning the effectiveness of RCTs beyond the theory. This means that their functioning in the field is analyzed beyond the initial theoretical assumptions. One of the major claims is the intention to raise awareness about RCTs’ pretended superiority and consider randomization as an efficient method among others. To do so, some alternative methodology and ways of improvement are proposed.

5Angus Deaton, 2015 Nobel Prize in economics, offers an introduction around eleven recurrent questions about RCTs. The objective of the chapter is to study the accuracy of strong assumptions of RCTs. He outlines the fact that randomisation have limits and is not always best solution. Each of the topics introduced here would be treated in details in the following chapters of the book.

6The first chapters deal with the methodological aspects of RCTs. Chapters one and two, by Martin Ravallion and Lant Pritchett respectively, address the dangers of claiming superiority of RCTs over any other method. Ravallion discusses their exponential use, and points out their inability to deal with large scale policies. Pritchett agrees. He highlights the “madness” around RCTs and argues for “kinky development”. To him, RCTs can be an excellent mean of guiding targeted programs led by “charity or philanthropic” (his words) and NGOs. Public policies need to be based on a broader knowledge of population needs and should be backed up with more big data.

7Chapters three and four highlight the good uses that can be made of RCTs. Jonathan Morduch acknowledges the above limitations but points out that all impact evaluation methods have limitations. RCTs should be used in conjunction with other tools. He also invites us to consider randomization in several ways, in particular from an exploratory angle, as a learning lever enabling us to confront presuppositions with reality, and then for the evaluation of projects pre-calibrated on the real needs previously observed. The ex-nihilo aspect of the use of RCTs is a recurrent criticism. Timothy Ogden puts forward the same idea and invites us to consider randomisation in the field of development as an “emerging technology” (p. 149), which is exciting, has its limits but is constantly being improved. Randomisation methods are still evolving, but can already make a significant contribution, particularly through reflection on program design and innovation in the implementation of anti-poverty programs.

8Chapters five, six and seven discuss the role of RCTs in the health, water, sanitation, hygiene (WASH) and microcredit sectors. Andres Garchitorena et al. offer a review of the history of RCTs in health and medicine and how randomization have been awarded a central role in clinical research. A parallel is made with RCTs for health purpose in development. They join Ravallion and Pritchett arguments on the fact that RCTs are not suitable for sector-wide approaches. However, given their strong internal validity, they seem well-suited for studies on the adoption of specific medical technology or treatment of diseases.

9Dean Spears, Radu Ban and Oliver Cumming draw the same conclusion, argue for more interdisciplinarity and outline the need for systematic “careful thoughtful research” (p. 184). Bédécarrats, Guérin and Roubaud address the role of RCTs in microcredit policies via a publication in the American Economic Journal that brought together six RCTs on microcredit. Based on the different results, they test critics/qualities acknowledged to RCTs (selection bias, data quality, internal and external validity, heterogeneous effects, measurement errors, program set up [not exhaustive]). Having themselves carried out a randomization in Morocco, the remarks are made in the light of their own results and allow them to measure all the field limitations. To François Roubaud, the practice of RCTs does not allow for effective inference. In a recent article, he also questions what is really measured in a randomisation program: the effectiveness of the treatment or the adherence to the treatment?

10In chapter eight, Agnès Labrousse makes a review of randomistas rhetoric through the analysis of “Poor economics”. She raises a critic of the systematic use of logos, pathos and use of figure of speech to hide lack of scientific rigor. In chapter nine, Robert Picciotto offers an historic return on the field of development policy evaluation. He concludes that RCTs prove efficacy in identifying attribution needs but fails to deal with other evaluation criteria such as sustainability (in time) or relevance in instable environments. As they do not fulfil all core criterion of impact evaluation, to him, RCTs can not be considered evaluations.

11Chapter ten (Michel Abramowicz & Ariane Szafarz) proposes a reflection about ethics and RCTs. Even if all authors mentioned this specific issue, a new terminology is brought here: the need for equipoise. It means that current RCT experiments are not well equilibrated and fail to measure potential risk nor potential benefits people are exposed to during field experiments. Eleventh chapter (Eva Vivalt) proposes the use of priors to improve study design and results analysis. She explains that it could help to avoid p-hacking and allow to explain some null results.

12In the epilogue, James Heckman proposes his 1992 paper “Randomization and Social Policy Evaluation” where he considers the benefits and limitations of randomized experiments. He draws the conclusion that randomization is suited for evaluation of social policy programs if some strong assumptions hold and if the evaluation focuses in estimating the mean impact of treatment. This conclusion, drawn on the early 90’s and papers published on the following years had already shed light on the main critical issues concerning RCTs today. To complete the precedent thoughts, a set of three interviews of policy makers can be found at the end of the book.

13The reader has in his or her hands a work that is self-sufficient and that will give him or her necessary knowledge to understand the issues related to RCTs. A wide range of their dimensions are critically addressed (but always argued). The different essays can be read independently. It can sometimes seem repetitive as each of the co-author offers a variation around RCTs. However, this allows us to keep in mind recurrent weaknesses and acknowledged strength of randomization in the field of development.