1They say that recognizing that we have a problem is half the battle and, if this is true, Anthony Bryk is definitely halfway home. He argues that educational reform efforts over the past two decades have failed to meet expectations for a variety of reasons including: hasty implementations, inadequate attention to how the reforms should be put into practice, failure to supply essential resources, and the emergence of unintended and possibly perverse consequences of the reform itself. Most of these problems have been acknowledged before, but effective solutions have yet to be seen. Bryk, as a result, turns to the quality improvement movement in healthcare for a way forward. Medicine, he reports, has greatly benefited from applying an “improvement science” (p. 469) approach and suggests that similar benefits might accrue to programs designed to advance educational practice. His AERA lecture is short on details, but more about what such an approach would entail can be found in Bryk, Gomez, Grunow and LaMahieu, (2015).
2I am drawn to several aspects of Bryk’s proposal. First, he asserts that we need to start developing “practice-based” (p. 473) warrants, warrants based on the details classroom work processes. He provocatively characterizes the movement for “evidence-based” improvements as something passé, as a carry-over from the “last decade” (p. 469). This at a time in which evidence-based methods are still considered the gold standard by many educational researchers and policy makers. Second, Bryk is striving to make instructional science into a true discovery science; he calls for an approach that “uses disciplined, analytic, and systematic methods to develop and test changes that achieve reliable improvements” (p. 475). Finally, Bryk advocates according more agency to teachers. He writes, “Practitioners need to engage fully with researchers and others in developing, testing, and enhancing the clinical work of schooling” (p. 473). All three of these goals seem important and worthy of pursuit.
3Chapter 3 of Bryk et al. (2015) is entitled, “We Cannot Improve at Scale What we Cannot Measure.” Achieving the goals listed above, therefore, hinges upon developing new kinds of measurement instruments, instruments that are attentive to the details of how instruction actually gets done within the instructional moment. As an example of an instrument designed to measure practical features of work practice Bryk and his colleagues (2015) describe the Developing Language and Literacy (DLLT) system, an observational metric for coding reading instruction in the early school years. Early literacy instruction in the U.S. is conventionally organized round six “macro” activities (interactive read aloud, shared reading, guided reading, etc.) each of which can be further decomposed into “micro processes” or periods (before class, before reading, during reading, etc.). Byrk et al. have developed special scoring rubrics for each of these intervals. As an example, they provide the “after-reading” period for the “guided reading” activity. Within this activity a score of 1 corresponds to “Does not engage students in discussions of the text” whereas a score of 4 corresponds to “Engages children in a rich discussion of the meaning of the text that is evident in the students’ comments about their thinking” (p. 95). This rubric is designed to direct attention to critical features of interaction between teachers and students. When applied to a particular encounter we are essentially scoring the meaning-making that is being achieved.
- 1 The issue goes beyond educational research and has broad implications for all the socia (...)
4The curriculum itself, as it happens, also has to do with meaning-making. The reading curriculum in elementary education has to do with engaging students in recovering meaning in texts. So, in the case of this example, we encounter meaning-making on two levels, both as a curricular matter and as the interactional processes through which the curriculum is being enacted. The score produced through the DLLT observational metric represents a judgment on the part of the auditor of the meaning-making that was achieved in the activity. The practical work of meaning-making, however, is inextricably tied to the situation in which it is produced. In reducing this work to a simple numeric score, the connections to the local setting are lost. So this is the conundrum: we have a critical need to understand how meaning-making is accomplished in the nonce, but, in attempting to measure it, essential features of the phenomenon elude us.1
- 2 The quotation may be apocryphal. One of Thorndike’s biographers suggests that the actual quot (...)
5Since the days of Thorndike, measurement has been a touchstone for educational research. The aphorism “Things are real to the extent that they are measureable” is often attributed to him.2 Given the difficulties in measuring meaning-making practice, some might be inclined to abandon the effort and return to educational psychology’s true forte—developing technologies for assessing learning outcomes. Ultimately, however, instruction is something produced within interaction and bound to a particular time and place. The price we pay by averting our eyes from the instructional moment is that we risk creating a science of instruction that has no tools for actually studying instruction.
6So Bryk’s efforts to train our eyes on work processes are timely and much appreciated. But to build an ‘improvement science’ for instruction that meets the objectives he has proposed, we must find a way of studying meaning-making that remains true to the nature of the underlying phenomenon. To do so will require finding answers to a number of vexing questions: If anything we learn about meaning-making is essentially bound to local circumstances, can there be any generalization? And, if not, is it possible to have a science without generalization? Would giving up measurement mean forgoing a claim to being a science? And the most practical of question of all: if we relinquish generalization and measurement, what will we have left to offer practitioners? Until answers are found a true ‘improvement science’ of instructional practice will continue to elude us.