Navigation – Plan du site

AccueilNuméros30-2Undone Computer ScienceMachine Learning, Understanding, ...

Undone Computer Science

Machine Learning, Understanding, and Interaction

Pierre Saint-Germier, Benjamin Matuszewski et Frédéric Bevilacqua
p. 17-39

Résumés

Les modèles d’apprentissage automatique performants souffrent généralement d’une opacité qui questionne la possibilité pour des humains de comprendre leur fonctionnement. À travers l’étude de CoMo, un système d’apprentissage interactif permettant aux utilisateurs d’associer sons et gestes, nous montrons qu’une forme de compréhension du modèle peut émerger des interactions utilisateurs-modèle. Cette approche interactive propose une alternative aux épistémologies fondées sur l’explication ou l’interprétation. Bien que les systèmes d’apprentissage profond, gourmands en données et en calcul, se prêtent difficilement à cette forme d’interactivité, les modèles d’apprentissage superficiel offrent des avantages épistémologiques sous-estimés dans les contextes où l’apprentissage automatique peut être rendu interactif.

Haut de page

Texte intégral

1 Introduction

1Machine learning (ML) techniques confer to computational systems the capacity to build and use rich representations from raw data sets, through specific training procedures. Epistemological questions surrounding this process are currently attracting growing attention from philosophers of science and epistemologists [Grote, Genin et al. 2024]. When human agents (e.g., policy-makers, medical doctors, or scientists) rely on ML algorithms to form beliefs and decisions about a target system, epistemological issues revolving around the notion of justification are inevitably raised. How can such beliefs be justified, given that ML algorithms play a decisive role in their formation? A common default assumption is that statistical models generated by machine learning from data are justified to the extent that they are predictively accurate. Anything that threatens predictive accuracy at the level of model development (e.g., because conditions for accuracy required by statistical learning theory are not met) or model deployment (e.g., because the environment is too unstable) downplays the justification of the beliefs and decisions acquired by ML.

2The focus on predictive accuracy, however, obscures other fundamental epistemological issues surrounding the use of ML systems. First, it is well known that a machine-learning system may be predictively accurate without providing explanations for each prediction or giving users a sense of understanding of what the system is doing. A commonly observed feature of ML techniques is that performance improvements are generally not accompanied by improvements in understanding, and sometimes even lead to greater opacity [Sullivan 2022]. This is particularly the case in Deep Learning (DL), where data are typically encoded into latent spaces of high dimensionality for which no intelligible interpretation is available. This problem, which is now being tackled head-on by the field of Explainable AI, raises fundamental conceptual questions about what it means for a model built by machine learning from data to be understandable at all. Second, whether machine-learning systems provide understanding to their users depends on a larger number of factors, including the technology of data production and collection, the knowledge independently possessed by users about the target domain, and the social arrangements in which technology and knowledge are embedded. For instance, the epistemic value of ML algorithms for the classification of medical imagery cannot be assessed independently from the way medical knowledge and decision-making are socialized through practices such as seeking a second opinion from a peer [Cai, Winter et al. 2019].

  • 1 Following common usage [Chollet 2021, 7], we denote by “Shallow Learning models” the complement of (...)

3The goal of the present paper is to suggest that those two blindspots of accuracy-based epistemologies of ML are not entirely independent and may be corrected simultaneously by considering the role of interactivity in the epistemology of ML models. To do so, we propose a case study of the Interactive Machine Learning application CoMo [Matuszewski 2020], allowing users to define sound-gesture mappings, for uses in various contexts such as artistic practice, education, or motor rehabilitation therapy. Although the ML architecture behind CoMo involves only Shallow Learning (SL) models, which typically allow for some limited degree of human interpretability, significant issues of understanding are nonetheless raised by the misalignment between the representations of gestures generated and exploited by motion sensors, digital signal processing algorithms and ML algorithms on the one hand, and the socially constructed, meaningful representations of gestures that human users may form from first-person and third-person perspectives.1

4Our original suggestion, based on the case study, is that an improved understanding of ML models may be obtained primarily by dynamically interacting with them, in addition to merely explaining or interpreting them, as the growing research in Explainable and Interpretable AI often assumes. The paper is structured as follows. In section 3, we introduce the CoMo framework as an instance of the Interactive Machine Learning (IML) paradigm, before focusing, in section 3, on the challenges for understanding raised by the distortions between the user-centric gesture representations and those processed within CoMo. Section 4 explains how patterns of interaction between users and the models induce understanding by progressively aligning human and machine representations of gestures. The question of whether, and to what extent, the resulting account of Interactive Understanding applies to DL architectures is addressed in section 5. We conclude that some SL techniques at least have an enduring epistemological potential, if one cares to look beyond accuracy and take understanding seriously in the epistemology of ML.

2 The CoMo framework

5CoMo is a software framework that aims at providing users with IML [Dudley & Kristensson 2018] to design sound/gesture mapping. In a nutshell, the function of CoMo is to enable users to ostensibly define gesture categories and associate them with specific sounds, so that when the system later recognizes instances of that category, it triggers the corresponding sound.

6IML was introduced by Fails & Olsen [2003], several years before the DL wave, in a task where users could improve the performance of a classification algorithm by periodically correcting its predictions, and later formalized by Amershi, Cakmak et al. [2014]. In their review, Dudley & Kristensson [2018] define IML as “distinct from classical machine learning in that human intelligence is applied through iterative teaching and model refinement in a relatively tight loop of ‘set-and-check’”. IML is also related to the concept of “Human-Centered Machine Learning” [Gillies, Fiebrink et al. 2016], which introduces practices and approaches from the field of Human-Machine Interaction into ML and stresses the role of context and user co-adaptation in ML processes.

7The design of CoMo follows these principles, enabling users to intervene periodically in the ML process to define the categories they want the system to learn, check that the gestures they perform are correctly classified, and change their definitions or performance strategies accordingly. More precisely, the use of CoMo alternates between training and performance phases. In the training phases, the user provides the system with instances of a gesture category they want to associate with a given sound, through demonstrations of the gesture recorded by means of motion sensors on a mobile device (typically a smartphone). From this recording, an HMM of the gesture category is constructed, taking processed motion descriptors as parameters. This process may be repeated for each gesture category the user wants to associate with a sound. In the performance phase, the system analyzes in real time the processed data coming from motion sensors and compares them with the model to pick the most likely category given the incoming data, and plays the corresponding sound file. An important challenge is to facilitate the ML procedure so that the system can be trained with very few examples, allowing the user to test the system in a rapid loop of trial and error, as depicted in Fig. 1. It is crucial that users can always return to the training phase to add new instances, remove old ones, or define new categories. In typical uses, the user is expected to move back and forth between the two phases, rather than train the system once and for all, before performing with it.

8The system is entirely based on Web technologies, which makes it accessible from any device running a Web browser, such as smartphones equipped with motion sensors (i.e., accelerometer and gyroscope). The possibility of using smartphones also allows for setting up and distributing the system quickly in a large variety of contexts. Moreover, CoMo was designed to work in collaborative and co-located settings, and is usable by a wide range of users without any knowledge of ML.

9The first prototype of the system was implemented and presented in March 2016, to respond to a use case from the company Orbe, which wanted to recognize whether the user was walking, running, or standing still. This prototype was then further developed during the PhD research project of M. Voillot, intended to create tangible and digital educational tools for early childhood [Voillot, Matuszewski et al. 2024]. The central use case was to create embodied storytelling where teachers could record movements that could be associated with sounds and later be triggered by pupils when “re-enacting” the story. Several other scenarios and use cases also served as test beds, such as workshops with artists, designers, and researchers. From these initial prototypes, the application has been developed further through its use in many different contexts: in artistic settings such as musical creation [Magalhaes, Matuszewski et al. 2020], as an experiment system in scientific settings such as the study of movement group coordination with children learning music (EmoDemos project), or in clinical contexts for post-stroke rehabilitation [Peyre 2022].

Figure 1: This figure presents the overall data flow within CoMo, from production of gesture data, to audio feedback through gesture recognition by the ML system. It also highlights (with dotted arrows) the feedback loop arising from the user’s trial-and-error interaction with the ML model.

Figure 1: This figure presents the overall data flow within CoMo, from production of gesture data, to audio feedback through gesture recognition by the ML system. It also highlights (with dotted arrows) the feedback loop arising from the user’s trial-and-error interaction with the ML model.

10Whether particular applications derived from the CoMo framework actually perform the function they are designed to perform depends on the ability of the ML algorithm to construct an accurate model of the gesture categories as defined by the user. Without such an alignment, the classifications of the system may be systematically misunderstood, and the system will not be usable. In order to get a clear diagnosis of this issue, we need to look into some detail into the way gestures are represented in the CoMo ML pipeline.

3 Machine representations of gestures

3.1 Defining gestures

11The precise definition of gestures varies between research fields. While it is generally associated with notions of meaning and intentionality, it has been modelled and described differently depending on contexts, such as studies in communication [Kendon 2004], [McNeill 2005], sign languages [Goldin-Meadow & Brentari 2017], human-computer interaction [Carfi & Mastrogiovanni 2021], or performing arts [Godøy & Leman 2009]. In this paper, we call gestures the body movements or postures that the users can perform themselves and classify into meaningful gesture categories. Although it may seem circular, this definition only adopts a user-centric approach to gesture as it takes for granted that users have an antecedent general notion of gesture and antecedent notions of specific gesture categories. Any performable movement that is recognized as an instance of a gesture category by the user will count as a gesture for us. This definition may also seem extremely wide, but it is limited by two important constraints. First, there is a performability constraint: only movements that are performable by a user count as gestures. Second, there is a recognizability constraint: what makes a performed movement an instance of a given gesture category is that the user recognizes the gesture as an instance of that category. What unites all instances of a gesture category is thus ultimately the common meaning that the user ascribes to them. Even though the commonsense notion of gesture links the meaning and identity of gestures with their kinetic profile, it is important to keep in mind that the meaning and identity of gestures are both socially constructed, through culture-mediated conventional associations, and capable of individual idiosyncratic variation.

12This user-centric definition raises a difficulty for the collaborative uses of the framework, where a user defines a gesture category not only for herself but also for other users. In such cases, the performability and recognitional constraints need to be understood distributively across all intended users: gesture categories need to be performable by each intended user and recognizable by all intended users. This means that performability and recognizability involve both first-person and third-person representations of bodily movements. While there is no conceptual difficulty in intending a gesture demonstration to define a category of gestures performable and recognizable by other users, it does place constraints and potential limitations on the collaborative use of CoMo, since the existence of aligned first-person and third-person representations of bodily representations cannot be taken for granted. So there may be conflicting views across a group of users as to whether a particular gesture belongs to a specific gesture category [Fdili Alaoui, Francoise et al. 2017]. In any case, what makes a movement a gesture, and a particular movement an instance of a specific gesture category, ultimately depends on the way users intend those movements to be performed and recognized.

3.2 Representing gestures

13The ML algorithm used in CoMo [Francoise, Caramiaux et al. 2012, Francoise & Bevilacqua 2018] is designed to classify in real time the incoming data into these user-defined categories. The fact that those categories are flexible and open-ended, because they are freely definable and revisable by the users themselves, is not in itself an issue, as it is precisely the point of ML techniques to deal with flexible and open-ended categories, only defined by examples in the absence of explicit, rule-based definitions. Yet, when one considers the details in the way gesture demonstrations are processed in the CoMo framework, it becomes clear that the representations of gestures in the system are at least highly idealized, and perhaps distorted to such an extent that the representational relation between the encoding of the categories by the model and the gesture categories defined by the user may seem broken.

Figure 2: The four layers of movement and gesture representations defined in CoMo.

Figure 2: The four layers of movement and gesture representations defined in CoMo.
  • 2 To be precise, the accelerometer device reports a complex raw signal that depends on two components (...)

14As shown in Fig. 2, four layers of gesture representations may be distinguished within the CoMo framework. First of all, motion sensors provide a raw signal, in the form of a time series of vectors taken from the 3D accelerometer (x, y, and z axis) and the gyroscope (yaw, pitch, and roll) of the mobile device, giving instantaneous raw accelerometer data2 and angular velocities along 3 axes, sampled between 50 and 100 Hz. This signal is continuously generated as soon as a CoMo client is connected. Intended as a raw representation of an individual gesture or of a series of gestures, such a sequence already leaves out several features usually taken to be essential to many relevant types of human gestures. First, gestures are typically thought to have temporal boundaries. For this reason, the stream of data points from motion sensors does not yet provide a representation of gestures as such, but at best, a representation of movements underlying gestures that only represent gestures after an appropriate segmentation. Second, in the typical use case where a user only uses a single mobile device that they hold in one hand, or attach to some body part, this stream of values only tracks the motion of a single part of the user’s body, leaving out the rich biomechanical dimensions of performing gestures. For instance, for a user who attaches their device to their wrist, moving their hand upwards with a straight or a broken arm might constitute distinct gestures, but their representation will be treated as equivalent if the accelerations measured at the level of the wrist are sufficiently similar. Third, these specific sensors (chosen in virtue of their wide availability in consumer-grade hardware) systematically neglect possibly important gesture parameters such as the absolute spatial coordinates of the body. This means, for example, that jiggling your hand above your head or below your knees will be represented in the same way by the sensors, despite possible semantic differences.

15The raw signal generated by the sensors thus only provides very basic and partial information about motion. Indeed, the characterization and classification of human motion typically use more refined descriptors, such as joint angles and their dynamics. For this reason, a second layer of Digital Signal Processing (DSP) is necessary for the computation of motion data in real-time. The vector associated with each sample is thus extended with information about 3D linear and compressed acceleration (without gravity), average movement intensity regardless of the direction, orientation, and angular velocities. This representation still takes the form of a stream of values, but these refined descriptors provide a better basis for the identification and recognition of gestures. This being said, the information contained at this level of description supervenes on the information contained in the raw signal. This means that the biases induced by the kind of data captured by motion sensors and their typical use (e.g., one mobile device per user) are not corrected.

16In order to obtain a representation of a gesture from this stream of processed data, a segmentation is required. In the learning phase, the user is responsible for this segmentation when they press “start” and “stop” on the interface, before and after demonstrating an instance of a gesture category to be mapped to a given sound. The system then records the values of the processed data time series between those two temporal boundaries and stores them in a memory slot associated with the sound file. When the user associates several gesture demonstrations with the same word, the system treats them as several instances of the same category. This constitutes the third layer of representation, which aims to represent individual gestures, organized into classes corresponding to user-defined categories. Again, since these categories are defined in terms of biased observations, those biases are inevitably inherited in the representation of gesture instances.

3.3 Learning user-defined gesture categories

17The task of the ML algorithm is then to learn, for each category, a general description in terms of which segments of the incoming data stream may be classified in the performance phase as instances of specific gesture categories among those defined by the user in the learning phase. In CoMo, this description takes the form of a Hidden Markov Model (HMM): a gesture category is represented by a series of states that are assumed to describe the probability distribution generating the data. This is the fourth layer of representation, aiming to represent gesture categories, as opposed to isolated instances.

18Such models embody the Markov assumption, according to which one only needs to consider the current state to predict the next one. This systematically privileges the representation of local relationships over global patterns and long-term dependencies. However, because gestures are typically rather short events, and because the designers allow the model to use sufficiently many hidden states to model their development, it provides in the case of CoMo a limited distortion of gesture categories. A more problematic bias concerns the way the system ultimately provides classifications of gesture instances in the performance phase. As explained above, the CoMo classification algorithm monitors the processed data and selects, at each point in time, the category that is most likely given the processed data vector. In cases where the user has defined only a small number of categories, and in particular, where no null category corresponds to rest, the system will continuously recognize gestures (and play sounds), even when none are intended by the user. Even if this bias can be corrected in practice by the addition of a null category or processed on the output likelihood, it remains that the mechanism consisting of inferring at each new time sample the best match between observed movements and available categories incurs a bias towards overgenerating classifications, namely classifying movements as gestures (and emitting a sound as a consequence), even when no gesture is intended by the user.

  • 3 Concrete examples can be found in the following articles and videos: [Voillot, Matuszewski et al. 2 (...)

19To sum up, attention to the details of the way representations of gesture instances and gesture categories are constructed within the CoMo architecture reveals that it includes considerable amounts of idealization and even distortions that threaten the users’ understanding of the gesture recognition system, and ultimately the reliability of the device3. For instance, if a user’s intended gesture category essentially involves some of the features that are within the blindspots of the motion sensors or of ML model, its intended instantiations during the performance phase may be systematically misclassified.

20A fair assessment of these limitations, however, commands to recall that those distortions result from difficult tradeoffs between representational accuracy and usability, given the motion sensor technology present in easily available mobile devices such as smartphones. Furthermore, the fact that genuine threats to understanding are made possible by the biases induced by such tradeoffs does not mean that understanding is not possible, under the right conditions. To answer this question, we need to get a better grip on the potential reasons for understanding failure in ML systems like CoMo, and more importantly adopt a more dynamic and pragmatic point of view, paying attention to the way users interact with the model over time.

4 Interactive understanding

4.1 Idealization failure

21Following the analysis of the way gestures are processed by the CoMo system, it is natural to explain the difficulty of understanding that users typically experience, at least in part, by the distorting idealizations that this processing involves. According to this hypothesis, users have difficulties understanding how the system works, because the system introduces serious distortions in the way it represents and learns the users’ gestures. Some features of gestures (straight or broken arm) that are difference-makers for users’ categorizations are treated as equivalent (if gestures are recorded from a single mobile phone connected to the forearm). Conversely, some features of gestures that are not difference-makers for users’ categorizations (e.g., movements performed in the short time lapse after the user presses “record” and before they start performing their intended gesture) are represented as difference-makers by the system. According to some accounts of idealization [Strevens 2016], this counts as an idealization failure. The problem is not that the system systematically misrepresents some features of gestures. This is inevitable given the necessary tradeoff between representational accuracy and usability. The problem is that those distortions are not limited to non-difference-makers and therefore obscure the categorization of users’ gestures. For this reason, it may be tempting to attribute the lack of understanding to an idealization failure.

22This view may be reminiscent of the Idealization Failure Hypothesis proposed by Sullivan [2024], according to which “complex or opaque ML models fail to enable understanding of real-world phenomena when there is ML idealization failure” [2024, 1446]. A significant difference, though, is that Sullivan’s Idealization Failure Hypothesis is concerned with the understanding of target phenomena in the world by means of ML models, whereas we are primarily concerned with the understanding of the CoMo model by users. However, part of the reason why users fail to understand how the model works in the case of CoMo is that they fail to understand how it learns a model of target phenomena, since its proper function is to learn on the fly gesture categories. Therefore, the question of model understanding is dependent on the question of target phenomena understanding, in the case of CoMo at least. Whatever diminishes target phenomena understanding also obscures model understanding, to the extent that understanding how the model works involves understanding how it learns about target phenomena.

23Idealization failure, however, cannot be the whole story in the case of CoMo. Given that the distortions caused by the motion sensors are hard-wired into the system and that they propagate to all other representational layers, idealization failure is a massive and unavoidable problem. This view leads to an excessively pessimistic view of the prospects for understanding gestures, unless radical changes are made to the motion-sensing infrastructure of the system. However, the whole CoMo framework was designed to be easily used with commonly available mobile devices equipped with motion sensors. Taking those socio-technical constraints as given, the focus on idealization failure suggests that the possibilities for understanding and reliably using CoMo in practice are severely limited. Experience with the way users actually deal with CoMo shows, on the contrary, that some non-trivial degree of understanding is achievable.

4.2 Understanding CoMo in the wild

  • 4 The workshops held by the Movement Analysis Network “movA” https://mova.one/ (accessed February 11t (...)

24In order to see this, it is necessary to adopt a more dynamic standpoint, taking into account how the understanding of CoMo by users evolves over time. In holding interdisciplinary workshops4 where users such as dancers were introduced to the framework and had the opportunity to extensively test it, we could witness how participants explore the systems through trial and error, and improve their understanding of the gesture classifications by the system. This process is greatly facilitated by the immediate auditory feedback triggered by the recognition results. While the workshops usually start with explaining basic facts about the types of sensors used in the system, it is only through experimenting with the system that users acquire an embodied and situated knowledge about the recognition procedure. Let us describe in greater detail some striking examples observed on such occasions.

25First, after recording a few gesture categories, users usually check whether the system recognizes the gestures well. Interestingly, dancers tend to spend a long time exploring all the different ways a gesture can be performed, while still leading to the expected recognition results evidenced by the sounds. They might spend time trying to test whether a parameter such as the height of the smartphone is important or not. Similarly, they might be surprised that pointing to the left and to the right in the sagittal plane would not be differentiated. Such explorations provide the participants with a first sense of what is actually captured and what is not, and allow them to get a personal “sense” of the information in the raw and processed data. Interestingly, the users first realize “what’s missing”, such as the insensitivity to height or absolute directionality, rather than the specific characteristics of the motion sensors and descriptors used in the system.

26Second, an interesting challenge for understanding is linked to the way the system handles gesture segmentation and its limits. As described above, the start and end of gestures are defined in the recording using the Start/Stop button. This forces users to pay attention specifically to how they initiate and end their demonstrations, in particular if the movement begins from a state of rest and ends with a return to the initial state of rest. Generally, users tend to assume that the system will neglect the initial and final phases of stillness and only record their gestures when they move. In particular, if they wait too long before starting to move, the model will consider this phase of stillness as essential to the category and will not recognize instances if this initial phase of stillness is not respected. The experience of such difficulties forces users to formalize precisely how their gestures should start and stop, and more generally to identify what needs to be clearly demonstrated for their gestures to be correctly classified.

27Third, a common difficulty has to do with the sharing of gesture categories across users. The recognition is generally efficient when the same person does the recording of the examples and the performance. In this case, all the movement’s idiosyncrasies appear as advantages to perform the recognition, just like one would better recognize handwritten words from examples coming from the same writer. Indeed, difficulties arise when participants try to use the gesture recognizer trained by another person, implicitly testing how the system generalizes and whether it is prone to overfitting. Accuracy then highly depends on the way the gesture categories are transmitted to other users. Verbal descriptions of gestures are largely insufficient since it might be difficult to communicate speed and dynamic movement articulations. Learning through imitation then remains the most practical approach. In particular, the real-time recognition feedback of the CoMo system can facilitate such transmission, and the users can literally use this feedback to learn the gestures and how to make the system work well (such feedback is generally called “knowledge of results” in sensori-motor learning).

28It is worth noting that when the system does not respond to what the user expects, there is often an ambiguity as to whether this is due to a model failure or an imprecision in performing the gesture. Such ambiguity also invites the users to try again and make changes in order to obtain the expected results, which might lead to overwriting all the initial gestures to obtain the expected sounds. Such a human adaptation, imposed by the model limitations, can still lead to a predictable and thus usable system.

29In contrast with the explanation based primarily on idealization failure, the initial difficulties experienced by CoMo users suggest an alternative explanation in terms of alignment failure: users’ understanding is impaired when the categories that are relevant to them are systematically misaligned with the categories that are made salient by the ML pipeline. What drives understanding in the case of CoMo is the constant feedback between the training phase and the performance phase, which gradually helps the two systems of categorization to align. The endemic distortions in the motion-sensing infrastructure do not need to be corrected for a better understanding to be obtained. They only need to be circumvented by focusing on gesture categories that are minimally distorted by those biases, e.g., because their definition is insensitive to the relative location of the body part to which the device is attached, just like the sensors. This alignment process is bi-directional in the sense that the users adjust the categories of gestures they define and their demonstrations to get closer to what the machine can identify, so that in return the classifications of the machine are closer to the user-centric categories. It is essential to adopt this dynamic point of view to see that this alignment process is in fact compatible with constant idealization failure. Over time, the endemic idealization failure is compensated by a gradual, bi-directional process of alignment. As long as this process converges toward robust gesture categories (insofar as the gestures that users expect the machine to recognize are indeed predictably recognized) some understanding of the machine-learning model is obtained.

4.3 Interactive understanding

30The original picture that emerges from the case of CoMo is thus one in which understanding a ML model is a dynamic and interactive achievement, consisting of a continual bi-directional adjustment between the user-defined and ML-constructed categories through situated and embodied interaction. What drives this achievement is the existence of a feedback loop connecting the performance phase to the learning phase. Performance errors can be corrected by immediately updating the model with new demonstrations of gestures. While users may gain some understanding by exploring the behavior of the model during the performance phase alone, the specific type of understanding that CoMo users have gained crucially involves the capacity to intervene back on the training of the model. Without this feedback on the training phases, possibilities for alignment remain limited.

31Generalizing from the case of CoMo, a user may come to understand, or better understand, how a ML model works when there is a bidirectional interaction between the user and the model by which (1) the model is successively retrained with new examples provided by the user, and (2) the user adapts over time their examples to the constrained generalizing capacity of the model, in such a way that the user’s intended categorization of the target phenomena aligns with the classifications of the model. Users come to understand how the system works because the way they think about the target phenomena to be modeled gradually aligns with the generalizations of the model, and they experience this alignment by interacting with the model. When this occurs, we shall say that users gain an interactive understanding of the model they have been using.

32Interactive understanding is an original epistemic achievement that shares many features commonly associated with understanding in the recent philosophical literature [e.g., Hannon 2021]. First, understanding is often described in terms of original mental abilities such as grasping causal or explanatory connections between variables. In the case of interactive understanding, CoMo users may also be described as gradually grasping why some examples are better learned by the model than others. Second, understanding does not reduce to a pure psychological experience (akin to the “aha” effect), but includes a normative component. In the case of interactive understanding, that normative component is tied to the successful use of the ML model. A normatively correct interactive understanding is an interactive understanding that does lead to the successful alignment of user and model categorizations and ultimately to the successful use of the system. Third, understanding, unlike (generic) knowledge, may vary in degrees. This is an essential component of interactive understanding, which develops gradually over time. Finally, understanding is often thought to be connected with new behavioral abilities (e.g., understanding a mathematical proof generates the ability to prove distinct but similar mathematical propositions). Interactive understanding, as exemplified by CoMo, also comes with extended abilities to better record gestures and better trigger sounds with them.

33The interactive understanding of ML models is also distinct both from interpretation and explanation, as those terms are commonly applied to ML models. As a matter of fact, SL models are often taken to be interpretable in the sense that they involve parameters that can be mapped to meaningful features and have been selected by the designers of the ML pipeline precisely because they are known or assumed to be relevant to the task at hand. In the case of CoMo, the hidden states of the HMM models, for instance, are intuitively interpretable as short stages through which the gesture passes over time. Gesture categories may be characterized as transitions between such states modulated by probabilities. For instance, and with considerable simplification, a wave-like gesture will be modeled by sequences of upward and downward motion states. Nevertheless, even with such an interpretation in hand, it remains difficult to guess how to create examples that will be reliably classified by the system. This is in fact due to two decisive factors: the choice of the gesture categories and the precision of gesture performance. Those two factors can vary wildly within and between groups of users, with the use case, and can only be evaluated through first-hand interaction with the model. In other words, having an interpretation of the underlying ML model without interacting with it does not suffice for understanding how ML works in CoMo. Conversely, interactive understanding is achievable from the perspective of users who ignore the technical details of the model and, a fortiori, cannot provide an intuitive meaning to its internal parameters. In that sense, one may interactively understand how the model works without having an interpretation of it.

  • 5 We do not mean to take a stance on the question whether understanding in general ultimately reduces (...)
  • 6 Embodiment is essential in the case of CoMo because CoMo deals with gestures, but it does not follo (...)

34The epistemic achievement obtained by interactive understanding is not reducible to explanation either, at least not in the way the latter is understood in the field of Explainable AI.5 Even though CoMo users, after having interacted in the right way with the model, may grasp some explanatory connection between features of the gestures and the particular generalization capacities of the model, the grasping is closer to an embodied and situated know-how than to a full-fledged propositional knowledge of the features that are likely responsible for the model’s behavior.6

35It is important to keep in mind that several enabling conditions need to be satisfied for this kind of interactive understanding to arise. First, there must be a feedback loop connecting the inference phase back to the training phase of the model. In other words, it has to be possible to easily and quickly update the model after inference. Second, this feedback loop needs to be accessible from the user’s perspective. The user herself needs to be able to intervene in the re-training of the model, for the right sort of two-way interaction to take place. Third, and as a consequence, the update of the model should not require a lot of data to be significant. In other words, it should be possible to reorient the behavior of the model with a small number of examples. While those conditions are satisfied in the case of CoMo, they place a limitation to the kind of ML models that may come to be interactively understood. This, in particular, may cast doubt over the possibility of gaining an interactive understanding of more complex and data-hungry DL models, to which we now turn.

5 Interactive understanding and deep learning

36The account of interactive understanding we have developed so far was suggested by the example of CoMo, and seems tailored for SL systems. Whether it applies to DL models, including for instance modern Large Language Models (LLMs), is far from obvious. First, to start with LLMs, it is well known that some of their general users, being unaware of the basic principles behind the transformer architecture, tend to develop misleading conceptions about the nature of chatbots (e.g., that they are conscious) and about the character of their relationship with them (e.g., an enduring personal relationship) after interacting with them. Such phenomena may appear as prima facie counterexamples to the interactive understanding account that we have fleshed out in the previous section, since more interaction gives rise to less understanding.

5.1 Interacting without understanding?

37However, on a second look, it could be argued that such misconceptions originating from interactions with LLMs are not counterexamples, precisely because the required feedback loop between model inference and model training is not instantiated in this case. At the inference phase, the interaction between users and the model is unidirectional, in the sense that the user can only prompt the model and explore how it responds to various inputs, but cannot immediately intervene on its training in return.

  • 7 We are grateful to an anonymous reviewer for raising this potential counterexample.

38It might be objected, then, that Reinforcement Learning by Human Feedback (RLHF), an essential part of the training phase of current LLMs, does involve an interaction between humans and the model that affects the training of the model. But the humans, in such cases, are not really users of the model, only informants. So the interaction remains unidirectional, in the reverse direction: the informant reacts to the model’s proposal, but obtains nothing in return. What is missing is a closed loop between those two interactions and the possibility of easy and fluid feedback between them, which is the crucial ingredient that leads to interactive understanding in the case of CoMo. Its absence can therefore explain why the same kind of understanding may not be achieved by interacting with LLMs.7

  • 8 We are grateful to the same anonymous reviewer for raising this potential counterexample.

39Another possible objection would be that the interaction between users and the model may be considered bi-directional in the inference phase if one takes seriously the phenomenon of in-context learning (ICL), whereby the models display learning behavior after being given new examples within a prompt [Brown, Mann et al. 2020]. In particular, it has been shown that, for some tasks, in certain conditions at least, ICL is functionally equivalent to performing gradient updates on the same examples [Dai, Sun et al. 2023]. Such results, however, are limited to linear tasks (e.g., linear regressions), and it remains unclear how general the equivalence is. So we cannot safely conclude that in general, or even most of the time, the kind of interaction observed in the real-life deployment of LLMs, gives rise to the level of bi-directional adjustment required for the relevant sort of interactive understanding. To that extent, the misunderstandings induced by interacting with LLMs cannot decisively be taken as counterexamples to our account, at least until more general functional equivalence results are firmly established.8 These considerations raise nonetheless the possibility that interactive understanding may occur locally in DL architectures, to which we now turn.

5.2 Interactive understanding within deep learning

  • 9 For a sense of this diversity in applications related to sound and music, including sound-gesture m (...)

40If the misconceptions instilled by using LLMs are not counterexamples to our account, the question of whether the account applies to DL systems at all becomes more pressing. Given the diversity of ways DL techniques may be used in full-fledged, ready-to-use systems, it is useful to distinguish several cases.9

41First, many ML systems involve complex pipelines where heterogeneous techniques may be used in combination. For instance, latent diffusion models typically integrate transformers within diffusion architectures [Rombach, Blattmann et al. 2022]. Similarly, it may be conceivable that DL and SL may coexist within a single architecture, offering both the high performance of DL and the bidirectional interactivity of SL. In the case of a system like CoMo, one may rely on DL techniques to optimize the algorithms that take care of the DSP part of the pipeline, and leave the gesture learning part to SL algorithms. If the DL algorithm is sufficiently fast at inference time, such a system may support the bidirectional interactivity necessary for the relevant sort of understanding. DL in this kind of scenario would however play at best an auxiliary role. The putative DL-driven DSP algorithm that may be used in such a CoMo-like architecture would remain frozen when it operates within the pipeline, and thus would not play any decisive role in the ML process. The system that users may come to understand interactively remains primarily an SL system, qua ML system, even though it may involve components that were antecedently shaped by DL techniques.

42Second, turning to architectures where DL plays a decisive role in the learning process, such as LLMs, it is common to observe, as we pointed out above, a clear division of labor between the unidirectional adaptation of the model to human informants during the training phase (to the extent that it involves RLHF) and the converse adaptation of the user to the model during the inference phase. This is related to deep socio-technological facts concerning LLMs: high accuracy requires very large datasets, which in turn necessitate long and costly training phases, and do not permit the relevant sort of bi-directional adaptation process that drives interactive understanding in the case of CoMo. Even though data collected from the behavior of users may be collected when a LLM is deployed and then used to update the model, such loops are considerably indirect and cannot be noticed, understood nor exploited by general users. The sort of user-centric interactive understanding that we are interested in cannot occur in this way.

43Third, and more interestingly, the progress of few-shot learning techniques in DL suggests however that it is conceivable to obtain interactive understanding in at least some classes of DL architectures. We already mentioned ICL in LLMs, which, for some tasks at least, seems to allow the bi-directional dynamics required for interactional understanding. To take another example closer to CoMo, Sanchez, Caramiaux et al. [2021] used transfer learning to adapt a pre-trained Deep Neural Network (DNN) for image classification to an interactive learning setting for the classification of users’ drawings along user-defined categories (much in the same way that CoMo classifies users’ gestures along user-defined categories). The transfer learning consists of using the stacked layers of the DNN as an encoder, providing lightweight embeddings from which a second, much smaller DNN (i.e., a simple multilayer perceptron) can be trained efficiently with few examples. In this way, Sanchez, Caramiaux et al. [2021]’s DL pipeline could meet the pre-conditions for interactive understanding. Moreover, they report that users who were ignorant of ML concepts were able, after a few trials, to gain an understanding of the system in this way.

  • 10 Other techniques involving Model Agnostic Meta Learning (MAML) have been used recently to achieve f (...)

44It might be objected that small multilayer perceptrons are not so different from SL models. If so, this case would not be so different from the first case, where data taken from the outputs of a frozen DNN are used to train a smaller SL model. In practice, this does not make a big difference, especially from the point of view of users who do not have prior knowledge of ML concepts. In any case, this example shows that techniques of transfer learning are available to help some DL models meet the requirements for interactive understanding.10

45To sum up, while the interactive understanding account may cover some DL systems specifically designed to be able to learn from few data points in real time, it is fair to say that interactive understanding remains out of reach for most DL models. This may seem as a limitation of the interactive understanding account, but also as a new insight into the opacity of large DL models. Large DL models remain opaque, not just because they are not interpretable, or because their outputs are hard to explain, but also because, unlike SL models or DL models capable of few-shot learning, they do not easily lend themselves to the sort of interactive understanding that we have been highlighting in the case of CoMo. This comparison between SL and DL thus helps isolate the conditions under which the epistemic benefits of interactive understanding hold, and says something significant about the epistemic limitations of the large DL models that currently attract most of the attention.

6 Conclusion

46Making ML models understandable by their users is one of the central epistemological challenges of ML. What the detailed examination of the IML framework of CoMo has taught us is that making ML more explainable or interpretable is not the only way to make progress on this front. Making ML models more interactive is another approach that deserves more attention. At least in some cases, where ML involves small datasets, fast training, and fast inference, users can easily intervene on the training and the evaluation of the ML model. In such cases, understanding can be gained by interacting with models rather than explaining or interpreting them.

47As we saw, there are at the same time significant limitations to the scope of interactive understanding. Our case study focused on a SL system, where the conditions for interactivity are the most favorable. The fact that DL models vastly outperform SL models in accuracy on many tasks with standardized benchmarks does not mean that DL models outperform SL in all epistemically relevant criteria. We have isolated one, interactive understanding, that is more easily obtained by SL models (or small DL models) than by large DL models, for a number of reasons that include the socio-technical context in which the systems are deployed. In the case of CoMo, both the challenges to understanding and the opportunities for interactivity are related to the availability of accelerometers and gyroscopes in everyday objects like smartphones, as well as the fact that it can be run on laptops, and operated in various contexts for a variety of users, such as elementary school pupils, dancers, or post-stroke patients.

  • 11 We thank four anonymous reviewers for extremely valuable comments that led to significant improveme (...)

48This should encourage us to reconsider the dominance of accuracy in the evaluation of ML methods. There are undoubtedly good reasons to devote resources to improving DL techniques to solve well-defined problems for which highly accurate solutions are required. As a result, however, the systematic investigation of the epistemic virtues of ML models beyond accuracy largely remains an undone science. There are also good reasons to strive for DL models that are smaller, faster, and require smaller datasets to generalize: one of them is to make them more understandable through interaction. A fortiori, those are also reasons not to abandon SL models with high interactive potential in contexts where they are accurate enough. The widespread view that large DL models made SL obsolete, or that the bigger the better when it comes to DNNs, may only reflect the widespread acceptance of a narrow accuracy-based epistemology of ML. Whenever understanding matters more than high accuracy, Interactive Machine Learning has a lot to offer.11

Haut de page

Bibliographie

Amershi, Saleema, Cakmak, Maya, et al. [2014], Power to the people: The role of humans in interactive machine learning, AI Magazine, 35(4), 105–120, doi: 10.1609/aimag.v35i4.2513.

Brown, Tom, Mann, Benjamin, et al. [2020], Language models are few-shot learners, Advances in Neural Information Processing Systems, 33, 1877–1901.

Cai, Carrie J., Winter, Samantha, et al. [2019], “Hello AI”: Uncovering the onboarding needs of medical practitioners for Human-AI collaborative decision-making, Proc. ACM Hum.-Comput. Interact., 3(CSCW), doi: 10.1145/3359206.

Carfì, Alessandro & Mastrogiovanni, Fulvio [2021], Gesture-based human–machine interaction: Taxonomy, problem definition, and analysis, IEEE Transactions on Cybernetics, 53(1), 497–513, doi: 10.1109/TCYB.2021.3129119.

Chollet, François [2021], Deep Learning with Python, Shelter Island: Manning, 2nd ed., www.manning.com/books/deep-learning-with-python.

Dai, Damai, Sun, Yutao, et al. [2023], Why can GPT learn in-context? Language models secretly perform gradient descent as meta-optimizers, in: Findings of the Association for Computational Linguistics: ACL 2023, 4005–4019, doi: 10.18653/v1/2023.findings-acl.247.

Dai, Penglin, Zhou, Junfei, et al. [2025], Meta-transfer learning-based cross-domain gesture recognition using WiFi channel state information, IEEE Transactions on Consumer Electronics, 71(2), 2530–2543, doi: 10.1109/TCE.2025.3552827.

Dubos, Anne, Bevilacqua, Frederic, et al. [2017], Designing gestures for interactive systems: Towards multicultural perspectives, in: Proceedings of the Interact’17 Conference (extended abstract), Springer, Lecture Notes in Computer Science, vol. IV, 524–526, doi: 10.1007/978-3-319-68059-0.

Dudley, John J. & Kristensson, Per Ola [2018], A review of user interface design for interactive machine learning, ACM Transactions on Interactive Intelligent Systems (TiiS), 8(2), 1–37, doi: 10.1145/3185517.

Fails, Jerry Alan & Olsen, Dan R. [2003], Interactive machine learning, in: Proceedings of the 8th International Conference on Intelligent User Interfaces, 39–45, doi: 10.1145/604050.604056.

Fdili Alaoui, Sarah, Françoise, Jules, et al. [2017], Seeing, sensing and recognizing Laban movement qualities, in: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, New York: Association for Computing Machinery, CHI’17, 4009–4020, doi: 10.1145/3025453.3025530.

Françoise, Jules & Bevilacqua, Frederic [2018], Motion-sound mapping through interaction: An approach to user-centered design of auditory feedback using machine learning, ACM Transactions on Interactive Intelligent Systems (TiiS), 8(2), 1–30, doi: 10.1145/3211826.

Françoise, Jules, Caramiaux, Baptiste, et al. [2012], A hierarchical approach for the design of gesture-to-sound mappings, in: Proceedings of the International Conference on Sound and Music Computing, Copenhagen, 233–240.

Gillies, Marco, Fiebrink, Rebecca, et al. [2016], Human-centred machine learning, in: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, ACM, 3558–3565, doi: 10.1145/2851581.2856492.

Godøy, Rolf Inge & Leman, Marc (eds.) [2009], Musical Gestures: Sound, Movement and Meaning, New York: Routledge.

Goldin-Meadow, Susan & Brentari, Diane [2017], Gesture, sign, and language: The coming of age of sign language and gesture studies, Behavioral and brain sciences, 40, e46.

Grote, Thomas, Genin, Konstantin, et al. [2024], Reliability in machine learning, Philosophy Compass, 19(5), e12 974, doi: 10.1111/phc3.12974.

Hannon, Michael [2021], Recent work in the epistemology of understanding, American Philosophical Quarterly, 58(3), 269–290, doi: 10.2307/48616060.

Jourdan, Théo & Caramiaux, Baptiste [2023], Machine learning for musical expression: A systematic literature review, in: Proceedings of the International Conference on New Interfaces for Musical Expression, edited by M. Ortiz & A. Marquez-Borbon, Mexico, 319–331, doi: 10.5281/zenodo.11189198.

Kendon, Adam [2004], Gesture: Visible action as utterance, Cambridge: Cambridge University Press, doi: 10.1017/cbo9780511807572.

Khalifa, Kareem [2017], Understanding, Explanation, and Scientific Knowledge, Cambridge: Cambridge University Press, doi: 10.1017/9781108164276.

Magalhaes, Michelle Agnes, Matuszewski, Benjamin, et al. [2020], STARTS Residency Public Report Constella(c)tions, Research Report, STMS – Sciences et Technologies de la Musique et du Son UMR 9912 IRCAM-CNRS-Sorbonne Université.

Matuszewski, Benjamin [2020], A Web-based framework for distributed music system research and creation, Journal of the Audio Engineering Society, 68(10), 717–726.

McNeill, David [2005], Gesture and Thought, Chicago: University of Chicago Press, doi: 10.7208/chicago/9780226514642.001.0001.

Nabi, Sarah, Esling, Philippe, et al. [2024], Embodied exploration of deep latent spaces in interactive dance-music performance, in: Proceedings of the 9th International Conference on Movement and Computing, 1–9, doi: 10.1145/3658852.3659072.

Peyre, Iseline [2022], Sonification du mouvement pour la rééducation après une lésion cérébrale acquise: conception et évaluations de dispositifs, Ph.D. thesis, Sorbonne Université, http://www.theses.fr/2022SORUS414.

Rombach, Robin, Blattmann, Andreas, et al. [2022], High-resolution image synthesis with latent diffusion models, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10 684–10 695, doi: 10.1109/cvpr52688.2022.01042.

Sanchez, Téo, Caramiaux, Baptiste, et al. [2021], How do people train a machine? Strategies and (mis)understandings, Proc. ACM Hum.-Comput. Interact., 5(CSCW1), art. 162, 1–26, doi: 10.1145/3449236.

Strevens, Michael [2016], How idealizations provide understanding, in: Explaining Understanding: New Perspectives From Epistemology and Philosophy of Science, edited by S. R. Grimm, C. Baumberger, & S. Ammon, London: Routledge, 37–49.

Sullivan, Emily [2022], Understanding from machine learning models, British Journal for the Philosophy of Science, 73(1), 109–133, doi: 10.1093/bjps/axz035.

Sullivan, Emily [2024], Do machine learning models represent their targets?, Philosophy of Science, 91(5), 1445–1455, doi: 10.1017/psa.2023.151.

Voillot, Marion, Matuszewski, Benjamin, et al. [2024], CoMo education: Collective and embodied storytelling in kindergarten using a gesture-sound interactive system, in: Proceedings of the 9th International Conference on Movement and Computing, 1–11, doi: 10.1145/3658852.3659070.

Haut de page

Notes

1 Following common usage [Chollet 2021, 7], we denote by “Shallow Learning models” the complement of the class of DL models within the class of ML models. By “DL model”, we denote Artificial Neural Networks (ANNs) with one or more hidden layers. Some SL models are not ANNs at all (e.g., Hidden Markov Models, like in the case of CoMo), others are ANNs with no hidden layer.

2 To be precise, the accelerometer device reports a complex raw signal that depends on two components: the so-called “linear acceleration” corresponding to the acceleration of the device and a “gravity component” corresponding, for each axis (x, y, or z) of the device’s local coordinate frame, to the projection of the global gravity vector onto that specific axis. The raw data, including both components, is sometimes called “acceleration with gravity”.

3 Concrete examples can be found in the following articles and videos: [Voillot, Matuszewski et al. 2024, (fig. 3)], the CoMo-Element tutorial, www.youtube.com/watch?v=XbCb_TAMbDA (accessed October 25th, 2025).

4 The workshops held by the Movement Analysis Network “movA” https://mova.one/ (accessed February 11th, 2026) gathered participants including dancers, musicians, Human-Computer Interaction researchers. See also the workshops described in https://gesturedesign.ircam.fr/ (accessed February 11th, 2026) [Dubos, Bevilacqua et al. 2017].

5 We do not mean to take a stance on the question whether understanding in general ultimately reduces to knowledge of an explanation, as some philosophers claim [e.g., Khalifa 2017]. What we are committed to denying is that interactive understanding, if it reduces to the knowledge of an explanation, reduces to something very much like the knowledge provided by X-AI systems, when they are successful.

6 Embodiment is essential in the case of CoMo because CoMo deals with gestures, but it does not follow from the account that interactive understanding always needs to be embodied. The account only entails that for users who do not master ML concepts the understanding of explanatory connections obtained by interaction may take the form of a know-how, rather than a know-that. This being said, gestures and more generally embodied activities (e.g., drawing by hand) are particularly favorable to interactive understanding because they give rise to rich forms of know-how.

7 We are grateful to an anonymous reviewer for raising this potential counterexample.

8 We are grateful to the same anonymous reviewer for raising this potential counterexample.

9 For a sense of this diversity in applications related to sound and music, including sound-gesture mappings, see [Jourdan & Caramiaux 2023].

10 Other techniques involving Model Agnostic Meta Learning (MAML) have been used recently to achieve few-shot learning in DL systems for real-time gesture classification [Dai, Zhou et al. 2025]. Other recent work on embodied latent exploration is also promising for making DL systems more understandable by users through embodied interaction [Nabi, Esling et al. 2024].

11 We thank four anonymous reviewers for extremely valuable comments that led to significant improvements in the paper.

Haut de page

Table des illustrations

Titre Figure 1: This figure presents the overall data flow within CoMo, from production of gesture data, to audio feedback through gesture recognition by the ML system. It also highlights (with dotted arrows) the feedback loop arising from the user’s trial-and-error interaction with the ML model.
URL http://journals.openedition.org/philosophiascientiae/docannexe/image/5306/img-1.png
Fichier image/png, 201k
Titre Figure 2: The four layers of movement and gesture representations defined in CoMo.
URL http://journals.openedition.org/philosophiascientiae/docannexe/image/5306/img-2.png
Fichier image/png, 369k
Haut de page

Pour citer cet article

Référence papier

Pierre Saint-Germier, Benjamin Matuszewski et Frédéric Bevilacqua, « Machine Learning, Understanding, and Interaction »Philosophia Scientiæ, 30-2 | 2026, 17-39.

Référence électronique

Pierre Saint-Germier, Benjamin Matuszewski et Frédéric Bevilacqua, « Machine Learning, Understanding, and Interaction »Philosophia Scientiæ [En ligne], 30-2 | 2026, mis en ligne le 01 mai 2026, consulté le 11 juin 2026. URL : http://journals.openedition.org/philosophiascientiae/5306 ; DOI : https://doi.org/10.4000/16953

Haut de page

Droits d’auteur

CC-BY-NC-ND-4.0

Le texte seul est utilisable sous licence CC BY-NC-ND 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont susceptibles d’être soumis à des autorisations d’usage spécifiques.

Haut de page
Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search