Navigation – Plan du site

AccueilNuméros11-3Poursuite des discussions à propo...The Quest for Measurement

Poursuite des discussions à propos du dossier « pratiques fondées sur la preuve, preuves fondées sur la pratique ? »

The Quest for Measurement

(and Meaning-Making’s Escape)
Timothy Koschmann
p. 143-146

Entrées d’index

Dossiers et débats :

Pratiques et preuves
Haut de page

Texte intégral

1They say that recognizing that we have a problem is half the battle and, if this is true, Anthony Bryk is definitely halfway home. He argues that educational reform efforts over the past two decades have failed to meet expectations for a variety of reasons including: hasty implementations, inadequate attention to how the reforms should be put into practice, failure to supply essential resources, and the emergence of unintended and possibly perverse consequences of the reform itself. Most of these problems have been acknowledged before, but effective solutions have yet to be seen. Bryk, as a result, turns to the quality improvement movement in healthcare for a way forward. Medicine, he reports, has greatly benefited from applying an “improvement science” (p. 469) approach and suggests that similar benefits might accrue to programs designed to advance educational practice. His AERA lecture is short on details, but more about what such an approach would entail can be found in Bryk, Gomez, Grunow and LaMahieu, (2015).

2I am drawn to several aspects of Bryk’s proposal. First, he asserts that we need to start developing “practice-based” (p. 473) warrants, warrants based on the details classroom work processes. He provocatively characterizes the movement for “evidence-based” improvements as something passé, as a carry-over from the “last decade” (p. 469). This at a time in which evidence-based methods are still considered the gold standard by many educational researchers and policy makers. Second, Bryk is striving to make instructional science into a true discovery science; he calls for an approach that “uses disciplined, analytic, and systematic methods to develop and test changes that achieve reliable improvements” (p. 475). Finally, Bryk advocates according more agency to teachers. He writes, “Practitioners need to engage fully with researchers and others in developing, testing, and enhancing the clinical work of schooling” (p. 473). All three of these goals seem important and worthy of pursuit.

3Chapter 3 of Bryk et al. (2015) is entitled, “We Cannot Improve at Scale What we Cannot Measure.” Achieving the goals listed above, therefore, hinges upon developing new kinds of measurement instruments, instruments that are attentive to the details of how instruction actually gets done within the instructional moment. As an example of an instrument designed to measure practical features of work practice Bryk and his colleagues (2015) describe the Developing Language and Literacy (DLLT) system, an observational metric for coding reading instruction in the early school years. Early literacy instruction in the U.S. is conventionally organized round six “macro” activities (interactive read aloud, shared reading, guided reading, etc.) each of which can be further decomposed into “micro processes” or periods (before class, before reading, during reading, etc.). Byrk et al. have developed special scoring rubrics for each of these intervals. As an example, they provide the “after-reading” period for the “guided reading” activity. Within this activity a score of 1 corresponds to “Does not engage students in discussions of the text” whereas a score of 4 corresponds to “Engages children in a rich discussion of the meaning of the text that is evident in the students’ comments about their thinking” (p. 95). This rubric is designed to direct attention to critical features of interaction between teachers and students. When applied to a particular encounter we are essentially scoring the meaning-making that is being achieved.

  • 1 The issue goes beyond educational research and has broad implications for all the socia (...)

4The curriculum itself, as it happens, also has to do with meaning-making. The reading curriculum in elementary education has to do with engaging students in recovering meaning in texts. So, in the case of this example, we encounter meaning-making on two levels, both as a curricular matter and as the interactional processes through which the curriculum is being enacted. The score produced through the DLLT observational metric represents a judgment on the part of the auditor of the meaning-making that was achieved in the activity. The practical work of meaning-making, however, is inextricably tied to the situation in which it is produced. In reducing this work to a simple numeric score, the connections to the local setting are lost. So this is the conundrum: we have a critical need to understand how meaning-making is accomplished in the nonce, but, in attempting to measure it, essential features of the phenomenon elude us.1

  • 2 The quotation may be apocryphal. One of Thorndike’s biographers suggests that the actual quot (...)

5Since the days of Thorndike, measurement has been a touchstone for educational research. The aphorism “Things are real to the extent that they are measureable” is often attributed to him.2 Given the difficulties in measuring meaning-making practice, some might be inclined to abandon the effort and return to educational psychology’s true forte—developing technologies for assessing learning outcomes. Ultimately, however, instruction is something produced within interaction and bound to a particular time and place. The price we pay by averting our eyes from the instructional moment is that we risk creating a science of instruction that has no tools for actually studying instruction.

6So Bryk’s efforts to train our eyes on work processes are timely and much appreciated. But to build an ‘improvement science’ for instruction that meets the objectives he has proposed, we must find a way of studying meaning-making that remains true to the nature of the underlying phenomenon. To do so will require finding answers to a number of vexing questions: If anything we learn about meaning-making is essentially bound to local circumstances, can there be any generalization? And, if not, is it possible to have a science without generalization? Would giving up measurement mean forgoing a claim to being a science? And the most practical of question of all: if we relinquish generalization and measurement, what will we have left to offer practitioners? Until answers are found a true ‘improvement science’ of instructional practice will continue to elude us.

Haut de page

Bibliographie

Bryk, A. S., Gomez, L. M., Grunow, A., & LaMahieu, P. (2015). Learning to improve: How American schools can get better at getting better. Cambridge, MA: Harvard Education Press.

Clifford, G. J. (1984). Edward L. Thorndike: The sane positivist. Middletown, CN: Wesleyan University Press.

Garfinkel, H. (1967). Studies in ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall.

Garfinkel, H. (2007). Recherches en ethnométhodologie (trans. By M. Barthélemy, B. Dupret, J.-M. d. Queiroz, & L. Quéré). Paris: Presses universitaires de France.

Haut de page

Notes

1 The issue goes beyond educational research and has broad implications for all the social sciences. See, for example, Garfinkel (1967).

2 The quotation may be apocryphal. One of Thorndike’s biographers suggests that the actual quote might have been, “Whatever exits at all exists in some amount. To know it thoroughly involves knowing its quantity and its quality” (Clifford, 1984, p. 283, FN1). Though more wordy, the message would seem to be the same.

Haut de page

Pour citer cet article

Référence papier

Timothy Koschmann, « The Quest for Measurement »Éducation et didactique, 11-3 | 2017, 143-146.

Référence électronique

Timothy Koschmann, « The Quest for Measurement »Éducation et didactique [En ligne], 11-3 | 2017, mis en ligne le 31 décembre 2017, consulté le 22 mai 2025. URL : http://journals.openedition.org/educationdidactique/2873 ; DOI : https://doi.org/10.4000/educationdidactique.2873

Haut de page

Auteur

Timothy Koschmann

PhD, Department of Medical Education
Southern Illinois University

Articles du même auteur

Haut de page

Droits d’auteur

CC-BY-NC-ND-4.0

Le texte seul est utilisable sous licence CC BY-NC-ND 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.

Haut de page
Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search