Navigation – Plan du site

A Multimodal corpus to check on pragmatic competence for Mild Cognitive Impaired aging people

Approche sur corpus des compétences pragmatiques et multimodales des personnes âgées présentant un trouble cognitif léger
Guillaume Duboisdindien, Cyril Grandin, Dominique Boutet et Anne Lacheret-Dujour

Résumés

Cet article présente un corpus vidéo d’analyse multimodale dont l’objectif principal est de modéliser et prédire les effets du vieillissement en situation de trouble cognitive léger sur les compétences pragmatiques et communicationnelles. Nous prenons comme observable les marqueurs pragmatiques verbaux et non-verbaux. Cette démarche, à l’interface des sciences du langage, des sciences cognitives et de la médecine réadaptative (l’orthophonie) s’inscrit dans un processus de recherche longitudinale en situation écologique (entretiens menés par des intimes des personnes âgées).
Nous présenterons en première partie de cet article les caractéristiques langagières, cognitives et sociales du vieillissement dans son continuum jusqu’aux troubles cognitifs léger et pathologiques. En seconde partie nous développerons l’intérêt d’une approche multimodale sur corpus notamment pour renseigner l’accompagnement non-médicamenteux et enrichir les connaissances orthophoniques. Enfin nous présenterons le corpus depuis sa conception expérimentale à ses résultats préliminaires qui concernent deux locutrices de l’étude âgées de plus 75 ans et qui présentent un trouble cognitif léger.
Les conclusions générales indiquent qu’avec l’avancée en âge, les marqueurs pragmatiques verbaux revêtent préférentiellement une fonction interactive permettant ainsi aux personnes avec TCL de maintenir les relations intersubjectives avec l’interlocuteur. Par ailleurs, au niveau non-verbal, les manifestations gestuelles sont de plus en plus mobilisées dans le temps avec une préférence pour les marqueurs pragmatiques non-verbaux à fonction référentielle et à fonction interactive. L’une permettant de compenser les manques du mot, difficultés de planification, accrocs discursifs ; l’autre optimisant et maintenant l’interaction avec l’interlocuteur.
Les cliniciens ont le devoir de développer leur pratique professionnelle par l’approche clinique basée sur des données probantes dont l’objectif majeur est de concilier la pratique clinique et les meilleures preuves issues de la recherche (Dollaghan 2007). Pour le cas de l’orthophonie, les cliniciens s'estiment très limités quant à cette approche (Lof 2011 ; McCurtin 2011) en particulier pour les patients avec TCL (Mungas
et al. 2010 ; Hopper 2013 ; Morello 2017) et plus spécifiquement lorsqu’il s’agit d’évaluer ou soutenir les fonctions langagières (Cummings 2014).
L’approche en TCL nécessite des corpus longitudinaux pour comprendre i) les évolutions naturellement en œuvre chez les sujets, ii) renseigner l’implication de la réserve cognitive chez chaque individu et iii) tirer avantage de ces paramètres comme bases de données attestées pour la recherche et la rééducation précoce. Nous désirons montrer quels sont les avantages des méthodes d’investigation scientifiques linguistiques et interactionnelles à travers le vieillissement fragilisé, pour les professionnels de la santé et les aidants au quotidien.

Haut de page

Texte intégral

The authors would like to thank:
- The European project CorpAGEst - People Marie Curie Actions (PIEF-GA-2012-328282) and Dr Catherine T. Bolly for her precious support and these evenings of rich discussions about aging, ethics and pragmatics.
- The France Médéric Alzheimer National Association and Paris Lumière (PLUM) for their confidence in our researches.
- MoDyCo lab UMR 7114 and Paris Nanterre University for their support.
- We warmly thank: Dr Pierre-François Roux, post-doctoral fellow at the
Nuclear Organization and Oncogenesis unit - Pasteur Institute (Paris) for assisting us with our statistical analyses and his brilliant sense for pedagogy.
- Special thanks to Dr Marion Blondel, current CNRS research fellow and Coralie Vincent, engineer and head of the experimental platform, members at the SFL lab, Pouchet, UMR 7023 (Paris), to believe in us.
- We would like to express our special thanks to the Pr L. Degand’s team from Louvain la Neuve University and Dr L. Meurant from Namur University, for your sense of reception, your help and your kindness.

Introduction

1The monitoring of elderly patients with cognitive and physiological multifactorial disorders at home or in healthcare institutions, increasingly encourages caregivers to consider the language resources of these patients, while aiming at maintaining communication and promoting their autonomy in a benevolent and medication-free environment. Current assessment tools focus mainly on losses, particularly in the language system, with limited reference to language use and to the resources preserved by the elderly. Those gaps hamper the effectiveness of speech-language interventions which rely on levels of evidence still limited in scientific resources (Dollaghan 2007). Our study offers an overview of a multimodal analysis focusing on Mild Cognitive Impaired People by mining longitudinal corpus built on recorded intergenerational exchanges in ecological situation, with an emphasize on the identification of pragmatic markers. By definition, pragmatic markers (PM) are rooted in the discussion situation and aim to maintain discourse consistency (Brinton 2010). We propose an approach to language aging in a continuum ranging from normal to pathological aging, in which we identify a profile of people at the frontier of potential dementia.

2We hypothesize that the identification of multimodal pragmatic markers, produced by people who are a priori at risk of developing dementia or of anchoring themselves in pathological aging, can help us to characterize inter-individual variations and significant compensatory communication skills in the aging process. We postulate that verbal and gestural pragmatic markers are relevant indicators for studying the subject's entrenchment in discourse (Davis & MacLagan 2016; Duboisdindien & Lacheret-Dujour 2017; Duboisdindien, Bolly & Lacheret-Dujour 2017; Hamilton 1999). We also consider the influence of interactional context on the use of these markers and what they can reveal about the emotional states and language skills of the elderly participant with respect to the proposed task: autobiographical recollection of recent or past events. The aim of our study is not to determine a quantitative threshold of multimodal pragmatic markers for the diagnosis of dementia, but rather to give insights in what is at stake for elderly people with the use or misuse of these markers and their pragmatic functions all along the cognitive alteration.

3We implemented the data processing following Kennedy’s (1998) recommendations on the balance to be found between an ecological study (i.e. non-invasive and spontaneous), and technical constraints. In doing so, we obtained a sufficient representativeness of the studied population, comparability between the sub-components of the corpus and the proposed tasks, as well as interoperability between the tools in order to systematize the analysis. The data processing consists of 6 steps: (i) development of an interview protocols inspired both by those written in CorpAGEst (Bolly & Boutet 2017) and our clinical experience, (ii) selection of participants, collection of field data for 14 months, sampling phase, digitization of video and audio data and their editing on the ORTOLANG scientific platform, (iii) transcription and alignment of audio data, (iv) annotation of audio and video data, (v) a unimodal and multimodal analyses of data, and (vi) systematic storage of original sources and annotated files.

4All the tasks proposed during the semi-structured interviews as well as the collection protocol correspond to the framework imposed by the ethics committee of the Psychological Sciences Research Institute of Louvain-la-Neuve University in Belgium.

5Our data include 20 hours of video recording corresponding to 36 interviews with nine speakers (mean age: 83 years; average score at MoCA-Test: 20/30). After subsampling, we ended with a total of 6 hours (30 minutes per speaker) for this specific study. From the results of our first analyses on the multimodal communicative features that characterize language evolution in aging, we expect that verbal deficits are accompanied/compensated by an increase in non-verbal acts with a specialization of both verbal and non-verbal pragmatic functions.

6The use of corpus in natural interaction informs us on different levels. The more complex the verbal content becomes, the more the elderly person uses referential and deictic gestures and intersubjective solicitations addressed to the interlocutor (signs of co-agreement, shared knowledge, interactive gaze) in order to keep communicating. This corpus delivers a different point of view compared to other investigations published so far on aging psychology and mostly based on data generated in laboratory conditions. Our approach allows us to grasp authentic interactions produced in natural exchange situations with their intrinsic complexity and enables a fine-grained functional analysis of productions in various modality. The project is in line with the work of C.T. Bolly (2013-2015), which focuses on healthy aging. The trends observed in this scope commit us to thinking of language as a resource for detecting dementia signs and understanding compensation strategies. Therefore, this work appears as an outstanding opportunity for clinical practitioners such as speech pathologists, physiotherapists, and psychomotor therapists to develop care protocols relying on verbal and non-verbal communication in aging. The significance of pragmatic marker functions in elderly speech, as well as approaches induced by linguistics and specifically pragmatics, contributes to broaden the urgent request to develop non-medicinal psycho-sociable methods, and evaluation tools for vulnerable old people to ensure their well-being.

1. Language and psychosocial aging characteristics

1.1. Language and Aging

7Language spans many levels of cognitive, functional and social implications. The effects of aging on language do not reach consensus in many studies. In cognitive psychology and neurogeriatrics, some conclusions focus mainly on the maintenance of language functions throughout adult development (Nef & Hupet 1992), while others have shown the decline of some of these functions with age (Shake, Noh & Stine-Morrow 2009). The pattern of the data is not homogeneous, according to the language activities considered (Mathey & Postal 2008). The language system would gradually disorganize in a singular and chaotic way in the heart of a dysfunctional system because of the advancing age.

8The action of executive functions is regularly designated as the factor responsible for these disorders. Executive functions refer to a heterogeneous set of high-level cognitive processes. At the interactive level, they make it possible to vary the processing and behavior of the information in real time. Many studies in psychology and neuroscience attest to a deficit of executive functions in the context of advancing age (Taconat & Lemaire 2014). The work of Colette et al. (2014) concludes that these executive deficits are related to disturbances of the inhibition process and the simultaneous maintenance of information processing in memory. This causes an increase in the length of conversations produced by the elderly. According to psychologists Spieler and Griffin (2006), there is a general slowing-down in conversational processes among seniors at all levels of the spoken chain: lexical representations, semantic outputs, phonological representations, and articulatory motor inductions.

9Verbal fluency is a skill that is blunted with old age, such that our max capacity to produce words in a limited time according to a precise criterion decrease. Troyer et al. (1997) show that older participants produce fewer words than younger participants in both semantic and phonological fluency tasks. More strikingly, older people would be more prone to verbal repetitions and to drifting from the target conversational theme, while facing issues when facing increasing grammatical complexity. Studies in clinical linguistics in natural, experimental, written or oral production highlight overall the same conclusions. As a consequence, elderly people tend to simplify their syntax and their discursive content is less dense (Kemper 2001).

10The pragmatic dimension, which refers to the use of language in a social context –pragmatic skills representing a part of social skills (Bates 1976; Cummings 2005 2009 2014)– is also studied in older adults’ speech. Authors have observed a strong tendency towards digression in normal aging, i.e. a propensity to move away from the topic of discussion, which considerably reduces overall coherence referred to as verbosity. According to Berrewaerts et al. (2003), the coherence of the discourse relies on the maintenance of the theme addressed by the speakers. People over the age of 60 tend to move away from the subject in 20% of cases, particularly in the case of autobiographical speech (Arbuckle & Gold 1995). This trend would again be linked to a lack of inhibition when processing irrelevant information.

11Speech management remains unchanged, but latency in response to an exchange is longer than in younger subjects (Ryan et al. 1995), which is reflected by a wider use of tools related to discursive cohesion such as references, substitutions, ellipses, and conjunctions.

12Clinical linguist Hamilton (1994, 1996), who developed the question of constructing the identity of older speakers through autobiographical content for nearly 20 years, also concluded that speech markers, speech breaks or even fillers as well as their atypical development within discourse became weakened with aging. According to the author, some markers fit into a new functional role: some connectors such as because, in fact, and then –mostly belonging to the structuring functional domain– would take an expressive functional dimension in the speech.

13In discourse analysis of the elderly subject, linguist Wray (2000, 2002) has described their tendency to make extensive use of formulaic language, designating repetitions, recurrences in formulations and anecdotes. This phenomenon would increase with age and would be preponderant in pathological subjects. Most of the studies conducted in cognitive psychology on this behavior describe a stereotyped behavior related to a deficit in both mental flexibility and in working memory. Tamir (1979) notes that the elderly sometimes have difficulties in either reopening the exchange, asking questions appropriately or strategically, or synthesizing information. When older people express their feelings, attitudes and verbalizations would be more dogmatic and assertive.

1.2. Social perception of cognitive impairment in aging and supportive approach

14With the aging of the population, a whole field of discussion has emerged to properly define characteristics of aging people, a task that is all the more difficult as it concerns a heterogeneous population with a wide variety of profiles (Mungas et al. 2010). This objective requires both intra- and inter-individual descriptive analyses. Indeed, intra-individual differences reveals differential aging according to domains, tasks, comparison of population and experimental conditions, while inter-individual comparisons highlight the diversity in aging revealing individual-specific trajectories influenced by living environment.

15Among the various profiles encountered in aging, there is an in-between profile referred to as an atypical profile characterizing subjects meeting both criteria characterizing their healthy peers, and criteria stamping problematic aging. These individuals, also referred to as cognitively fragile, are the subject of our study. We have oriented our approach starting at the age of 75 years old, which is considered as the entry into physical dependence following multiple degradations. Aging at this point in time is effectively recognized as a period of natural vulnerability where intra-individual differences are accompanied by inter-individual differences (Lemaire 2015).

16Characterizing this in-between profile is a particularly complex task since it is in fact part of a continuum of aging, as Loones et al. (2008) point out. According to these authors, MCI profile should not be confined to a state or stage of life, but rather should depict a slow development of individual’s vulnerabilities itself. It is therefore necessary to be able to identify the characteristics of fragility and their interrelationships as they evolve over time, along with social, psychosocial and physiological factors.

17It is well known that among the subjects initially considered as “cognitively normal”, while the majority does not decline, only a rather small subgroup shows a rapid decline. As for atypical, among the so-called cognitively fragile subjects, some improve, others remain stable, and others decline rapidly (Mungas et al. 2010).

18For these reasons, it is becoming all the more difficult to defend a categorical approach to aging while on the other hand, it is necessary to increase research on aging, particularly in order to uncover predictors, especially language predictors, in order to identify subjects at risk of rapid decline.

19Co-constructing dialogue to maintain the social thread is inherent in human interactions. This ability is maintained in people with cognitive impairment. However, in order to ensure this faculty, it is necessary to understand when atypical language situations arise and when markers of intersubjective maintenance are used to solicit the interlocutor. According to linguists and speech-language pathologists Guendouzi & Müller (2006), the majority of frail people are aware of their deficits and work hard to find ways to accommodate them so as not to suffer social consequences.

20Implicit ageism seems to affect communication with the elderly. The concept of ageism was first developed by Butler (1969) while describing discrimination towards elderly people perceived as atypical compared to a supposed ideal speaker –mastering each aspect of the communication and positioning itself as an authority to validate the discourse– or as a superspeaker (Rabatel 2004), establishing a hierarchy between the one who is informative, constant, efficient in his exchanges as opposed the one who stumbles, repeats itself, and parasites the discursive flow: the elderly speaker. The interactive environment immediately loses neutrality as well as any interpersonal dimension in which a form of equity, or even reciprocity at the heart of language acts, might have existed. Most of the time, elderly speak is favored when speaking to seniors who are trying to assert their independence. Likewise, such speech is used with people who are particularly weakened physically or otherwise. Thus, with these singular and maladaptative behaviors, we deprive the subjects of the faculty of meaning co-construction and therefore become an actor of their progressive isolation. As opposed to this stigmatizing posture, there is a constructive one in which –when an older person’s interlocutor perceives singularities and atypical behaviors which deviate from the norm he/she has integrated and which are cognitively expensive to treat– he must exploit the signs of communicative desire, whatever their modality (verbal or gestural), produced by the elderly and provide verbal and non-verbal support. The ideal speaker thus becomes a cooperative and sufficiently effective speaker to adapt to emerging language impairments.

2. Pragmatic competence and multimodal communication in language aging

21Bates (1976) broadened the pragmatic dimension to the use of language in the social context, in its relationship between the context of exchange on the one hand and its meaning on the other; influenced by the social relations between speakers and their mental states (Bernicot 1992). The interlocutors co-construct exchanges to maintain the social thread. This ability is deeply rooted in human interactions (Stivers 2008) and is maintained in people with cognitive impairment. Within the framework of studies on atypical communication, the multimodal approach (2.1) offers the possibility of linking linguistic information produced in different forms of communication, by analyzing their respective contributions to the development and perception of the message communicated by older people. In addition, a cluster of linguistic and extra-linguistic elements makes it possible to give weight/credit to pragmatic functions in the discourse of older interlocutors. This is particularly the case in discourse analysis, where pragmatic markers (2.2) have the virtue of being observable both verbally and non-verbally (2.3). To do so, it is necessary to take advantage of both the scientific literature in linguistic and the descriptions made in the field to propose a functional model of pragmatic skills integrating both verbal and non-verbal modalities (2.4).

2.1. Why must multimodal communication be considered by speech therapists?

22Considering language in its multimodal dimension offers a relevant grounding from the time we build interactive corpora, since speech in action is mostly accompanied by communicative gestures (McNeill 1992; Goldin-Meadow & Alibali 2013).

23The so-called multimodal communication is divided into two major modalities: (i) the verbal modality, which could be split in phonemic, lexical, syntactic-semantic and discursive levels (ii) and the visual modality, which encompasses gesturing and facial expressions. Multimodal analysis consists of linking language behaviors produced in different communication modalities, each of which contributes to the development and perception of the communicated message (Ferré 2011). From the psycholinguistic point of view, descriptions of non-verbal manifestations provide information on the evolution of communication through all ages of life from an integrated developmental perspective. As for the clinical field, it is increasingly interested in the question of multimodality with the aim of improving professional practices in healthcare situations (e.g. patient and therapist exchanges). Researchers and medical staff consider that gestuality facilitates interaction with the environment, improves the patient's quality of life and compensates for their disorders (Goldin-Meadow & Wagner-Alibali 2013). However, clinical research on this topic is still marginal, especially regarding people suffering from dementia, while practitioners subjectively indicate (without any analytical grid or standardized tests) their observations on the pragmatic, interactive and particularly gestural skills of the patients they meet. Nunes da Cruz Morello et al. (2017) conducted a systematic review of non-pharmacologic therapeutic interventions to maintain, assess or rehabilitate the language and communication skills of patients with ATD, and reveal this gap that significantly hinders guidance in evidence-based speech therapy and non-drug therapies. Most of the studies included in this systematic review show beneficial effects of speech therapy on the communication skills of MCI and ATD patients. Nunes da Cruz Morello et al. clearly show that studies involving several language skills (including pragmatics and multimodal) must be considered prior to any clinical intervention and with greater precision. The use of standardized tasks and accurate discourse measures will allow for greater data refinement and the generalization of ecological approaches. To this end, the authors indicate that it is fundamental to establish clearly defined discursive markers in analytical grids and to study them in depth. In this study, we propose to perform an integrative analysis of both verbal and non-verbal pragmatic markers.

2.2. Verbal pragmatic markers

24Verbal pragmatic markers (VPM) have been the subject of a multitude of linguistic studies without being the subject of a unified and consensual description today. Dostie (2004) & Crible (2017) explain that this problem stems from the changing nature of language in general, and from the variability in the discourse produced between speakers and contexts. However, linguists agree on two points. On the one hand, VPM are syntactically optional elements, with variable distribution, which fall into various categories: adjectives, adverbs, verbal forms, locutions, etc. On the other hand, from a functional point of view, VPM encode the internal structure of the discourse and its context, the relations between the enunciator and its discourse, and finally the intersubjective relations (Traugott 2010).

25According to Beeching (2008), pragmatic markers contribute to: 1) discourse progression, which concerns structuring and planning problems that the speaker may encounter during the production of a speech; 2) ecological interactions are suitable for the use of VPM.

26In the follow-up of Aijmer & Simon-Vandenbergen (2011) work, we consider VPM as elements that contribute to the interpretation of the context by other means than semantic decoding (Crible 2017) such as modal particles, greetings, interjections, discursive particles, response signals, speech markers and connectors, while always taking into account the discursive context.

27Literature indicates that no VPM has a single pragmatic function (Aijmer & Simon-Vanderbergen 2011; Bolly & Crible 2015; Crible 2016). They should therefore be considered as multifunctional. What finally characterizes them is the central pragmatic function they have, and their strong social and interpersonal function that allows communication between individuals. Furthermore, the importance of the environment in which a VPM is produced must be highlighted: when speaking, the enunciator preferably uses VPM in the presence of an interlocutor who influences the co-construction of the exchange. For Beeching (2008), (i) situations of verbal interaction would increase the use of VPMs, (ii) VPMs reflect the planning problems that the speaker may encounter during speech production and are involved in discourse progression.

28These observations helped build the model of identification and annotation of pragmatic (non)verbal functions, published by Bolly & Crible (2015), which allows a detailed description of the functions of pragmatic markers from a multimodal perspective. This model that we use in our study has its source in the taxonomy of pragmatic verbal markers developed by Crible (2014) and which contributed to the enrichment of the annotation protocol developed within the MDMA project: Model for Discourse Marker Annotation (Bolly, Crible, Degand & Uygur-Dixteshe 2015). The main ambition of this protocol is to propose an empirical method for the identification and annotation of VPM in oral French.

The method first aims at describing DMs (i.e. Discourse Markers) in clusters of variables and then, from a combinatorial point of view, in specific patterns. (Bolly et al. 2015: 3)

29A list of verbal markers and a functional taxonomy of these markers have been compiled. This taxonomy is divided into 4 main functional domains: referential, structuring, expressive and interactive function, following the approach of Halliday & Hasan (1976) and Bolly & Crible (2015). For each function and subfunction, we provide a functional mnemonic, examples as well as references supporting their validity.

2.3. Gestural pragmatic markers

30Kendon (1997) considers gesture as the expression of a thought or emotion, in movement. For the author, a pragmatic gesture is a voluntary action; it is necessarily visible and meaningful. He considers as gestures: those of sign language, conventional gestures (e.g. head refusals, hand signals to say goodbye, military salutes) and co-verbal gestures that he defines as speech-accompanying gestures. These gestures therefore have a communicative value and participate in the co-construction of exchanges (Goldin-Meadow 2003: 500).

31However, another category of gestures with strong pragmatic anchoring and underestimated in the studies also comes into play, particularly in studies on aging: they are adapters that tend to increase with age, especially in the context of fragile aging.

32Mol et al. (2012) and Bolly (2014) consider the manipulation of objects without a practical and instrumental purpose (scribbling doodles on a notebook or aligning objects on a desk) as adapters that also promote the cognitive comfort of the speaker in an exchange or listening situation. Alignment/adjustment or mirroring processes consist of more or less involuntarily imitating the other person during the exchange (elbowing the same way at the table while a person is speaking, standing up to a child to discuss with and support joint attention). Thus, gestures are not only interactive but also adaptive in order to facilitate collaboration and to enter into an intersubjective and shared dimension.

33With the objective of developing a multimodal analysis model in our study, Colletta’s (2009) perspective offers an interesting insight into the pragmatic functions of gestures within discourse, and a rigorous methodological framework for their functional annotation. The functional classification that was developed is particularly valid for several articulators, namely: manual gestures, head movements, facial expressions, changes in posture, body movements, and addressed gaze. Finally, such a classification values the whole intersubjective dimension intrinsic to gestures in corpus analysis (Bolly 2014).

34The functions initially identified from these works are the following four: referential, expressive, structuring and interactive, integrating the analysis model of the functional levels of language described by Halliday (1970). A theoretical combination has been established and published by Bolly & Boutet (2017) in their transversal multimodal study of healthy elderly communication, which shares the same foundations as those described earlier by Allwood (2010 - MUMIN Project).

35McNeill (1992) also greatly inspires our approach by distinguishing between propositional and non-propositional gestures. While the former facilitates the understanding of the discursive content and mark the speaker’s attitude; the others facilitate the flow of speech (fluidity, cohesion: McNeill 1992; McClave 1994), support interaction (Bavelas et al. 1995), or stand as emotional markers. Thus, propositional gestures including deictic gestures and representational gestures described above have a referential function and are linked to Halliday’s (1976) ideational level. Non-propositional gestures have structuring functions (textual level in Halliday), and are also involved in both interaction and expression (interpersonal level in Halliday).

36Gesture can convey meanings that do not appear –or only mildly– in the discourse. It stresses the importance of considering non-verbal behaviors in the communication of and with the person suffering from cognitive impairment (Hoffman et al. 1988). Furthermore, it illustrates how gesture can contribute in maintaining interpersonal communication by favoring understanding by the partner (Cassell et al. 1999; Schiaratura 2013). This is consistent with the work of Beattie & Shovelton (1999), which shows that the listener relies on gestures to obtain information that is not present in the discourse, such as the size, shape, and movement of the evoked objects.

37The model of annotation of pragmatic gestural and verbal functions published by Bolly & Crible (2015) was also inspired by the taxonomies of co-verbal gestures described in the dedicated literature (e.g. Bavelas et al. 1992; Colletta et al. 2009). The resulting multimodal annotation protocol comprises 44 functions grouped in 4 language domains mentioned above and inspired by Halliday (1970).

2.4. Modelling and developing a taxonomy of pragmatic functions in a multimodal perspective to help clinical practitioners in their approach

38In this context, the identification and assignment of functional domains to pragmatic markers (verbal and gestural) engages researchers to synthesize data that lead to consensus and to achieve an efficient and functional taxonomy. The purpose of these functions will be to help clinicians in assessing and identifying patients’ needs if significant changes occur over time or during the initial check-up.

39We know that language has several functions; using a more or less refined classification, linguists agree on a social or interpersonal function that allows communication between individuals. What about the analysis of the statements? This complex issue refers to the problem of how language functions relate to each other. Few authors have attempted to address this challenge. Halliday (1967, 1968, 1970) led the most thorough attempt, following the sketches of Hockett (1963) and Danes (1964). Criticizing the interest of distinguishing between competence and performance, i.e. between idealized knowledge and use in context, individualized use of language, Halliday studies language in relation to the types of situations in which it is used and the functions it has.

40This position leads to options of which at least one must be underlined: the basic unit of language is not the word or the utterance, as in previous studies, but the “text”, i.e. the set of coherent statements inserted within a discourse relevant to a situation. When the segment is subjected to analysis in our identification and annotation workflow, it will be considered as an embedded element of the discourse and not as a self-sufficient unit.

41Halliday distinguishes three functions within discourse: ideational, interpersonal and textual. The rhetorical function is one more function described by a more recent works of Bolly & Crible (2015); Bolly & Boutet (2017).

42The referential function (with a meaning close to ideational and cognitive functions) reflects the fact that language is used by the subject to represent processes, i.e. actions, events, states and relationships in which people, objects or abstractions are involved. From this point of view the sentence consists of the trial itself (usually expressed verbally) that fulfils specific functions: participating functions, including the roles of actor, patient and beneficiary, and circumstantial functions of time, place, manner, etc.

43At the gestural level, gestures with a referential function include gestures representative of the referent's reality. These gestures describe actions and spatial relations. This pragmatic function also includes the so-called deictic gestures, which consist in pointing with an articulator (finger, hand, head movement) towards the designated source.

Figure 1 a. Example of a referential gesture: light the fire (in ageSC3_r1_S1- 03:51) – Figure 1 b. Example of a deictic gesture: pointing the bunch of lavender (in ageSC3_r3_S5-00:59)

Figure 1 a. Example of a referential gesture: light the fire (in ageSC3_r1_S1- 03:51) – Figure 1 b. Example of a deictic gesture: pointing the bunch of lavender (in ageSC3_r3_S5-00:59)

44The interpersonal/expressive function takes into account the exchanges between interlocutors and their mode of involvement in the discussion: declarative, interrogative, imperative, etc. The components of the sentence are located in relation to the status that the speaker gives them. Halliday (1976) notes that it is within this interpersonal function that the predicative structure of a sentence can be located. The subject-predicate distinction has no self-meaning, since it relies on the mode of the sentence; consequently, the subject of the predicate is identical to the modal subject, the other elements of the sentence expressing the predicate.

45VPM with an expressive function are markers with an interpersonal and/or interactive meaning that call on the interlocutor to ensure their participation, their contextual anchoring, their emotions and shared knowledge. As an example, when the speaker uses PMs as you know, you see, do you understand, he/she expresses the intention to “ensure that certain pragmatic conditions necessary for the establishment of a dialogue are met” (Fernandez 1994: 83). Within this function, PMs play the role of speech regulators. Lakoff (1987: 122-124) defines the notion of hedge as “words whose function is to make things more obscure or clearer”. PMs as well, apparently, maybe are good candidates to illustrate this uncertain aspect in the speaker's speech.

46The expressive function is conveyed by gestures that express social attitudes, mental states and emotions, which facilitate the success of language acts, and comment on the discourse of both the speaker and the interlocutor (Colletta et al. 2009: 62).

Figure 2. Example of an expressive gesture “Surtout ne touche à rien – Don’t touch anything!” (in ageSC3_r1_S1-1:42)

Figure 2. Example of an expressive gesture “Surtout ne touche à rien – Don’t touch anything!” (in ageSC3_r1_S1-1:42)

47The sequential or structuring (textual) function relates to the organization of sentences within speech which creates the dynamic of communication. Two axes must be distinguished: 1) the thematic structure of the sentence; it is composed of two parts, theme and rheme. The theme is defined as what is spoken of, the psychological subject of the sentence, the medium on which the message is attached, whereas the theme is the body of the message (Halliday: 161); in the English language, the theme is the initial part of the sentence, the rheme the subsequent part. 2) the informative unity: discourse consists of a series of informative units whose limits are marked on the surface by intonation. Thus, each tonic group represents what the speaker decides to consider as an informative unit. This has its own internal structure: it is composed of elements that, compared to those above, express either new information or given information.

48Sequential or structuring pragmatic markers are generally opening, concluding or punctuating and support for discourse. These PMs are intended to help the speaker dividing the statement into different units of information and at the same time, helping the speaker decode these units. Consequently, they ensure a good discursive progression (ex: first of all, secondly, then).

49At the gestural level, the structuring function combines with the intonative structure of the utterance. These gestures have a prosodic salience function. They highlight the speech and they give the rhythmic organization of discourse.

Figure 3. Example of a structuring gesture “Parce que , je ne voulais pas manger -Because [structuring gesture] I didn’t want to eat!” (in ageSC3_r1_S1 – 02:05)

Figure 3. Example of a structuring gesture “Parce que , je ne voulais pas manger -Because [structuring gesture] I didn’t want to eat!” (in ageSC3_r1_S1 – 02:05)

50The rhetorical or interactive function is not present in Halliday’s functional taxonomy; it is –to some extent– a more contextualized shading of the ideational function with which it has discursive similarities, particularly on the relationship between segments. It was proposed by Degand (1998), Degand & Zufferey (2003), Gonzalez (2005) and Haselow (2011) in their studies on discursive markers. They translate what the speaker restrains in his metadiscourse: implicit assumptions and acts of language in the discursive construction. Interactive gestures synchronize the speaker’s and the interlocutor’s behaviors during social interaction and are often accompanied by gaze directed at the other person. Another type of gestures is considered in our study and within this function: adapters. These correspond to gestures directed towards oneself (rubbing one’s arm), others (stroking one’s hair while speaking), or an object (turning one’s wedding ring around one’s finger).

Figure 4. a. Example of an interactive gesture to Jules (her son) “Oui ! c’est ça ! ça laisse une bonne odeur sur le linge de maison hum - yeah! That’s it! It leaves a good smell in the hum household linen hum” (in ageSC3_r3_S5-03:56) b. Example of self-contact gesture (adapters): thumbs twiddling (in ageSC3_r1_S1-02:36)

Figure 4. a. Example of an interactive gesture to Jules (her son) “Oui ! c’est ça ! ça laisse une bonne odeur sur le linge de maison hum - yeah! That’s it! It leaves a good smell in the hum household linen hum” (in ageSC3_r3_S5-03:56) b. Example of self-contact gesture (adapters): thumbs twiddling (in ageSC3_r1_S1-02:36)

3. The corpus

51The main objective of this corpus is to offer a resource for 1) studying the pragmatic competence of elderly people (over 75 years of age) with MCI, 2) measuring the impact of MCI on their interactions, from a multimodal perspective and 3) characterizing sources of linguistic singularities discriminating subjects in order to focus on pragmatic resources. We hypothesize that the identification of pragmatic multimodal markers used by people a priori at risk of developing dementia or anchoring themselves in unhealthy aging, can help us characterize inter-individual variations and compensatory communication skills in the aging process. The underlying hypothesis that motivates us is that pragmatic verbal and gestural markers are relevant indicators for studying the anchoring of the subject in discourse. We also consider the influence of the interactive context on the use of these markers and on what they may reveal about the emotional states and language skills of the elderly subject in relation to the proposed task: autobiographical narration of recent or past events. We propose here a case study based on the production of two participants, Constance (speaker code: ageSC3) and Tristane (speaker code: ageIT1).

3.1. Methods and participants

52For the implementation of the workflow, we followed the recommendations of Kennedy (1998) suggesting to find a balance between a study with an ecological dimension (e.g. non-invasive and spontaneous), inherent technical constraints that guarantee both sufficient representativeness of the population studied and comparability between the sub-corpora and tasks proposed, as well as interoperability between the tools in order to systematize the analysis. This workflow is based on six steps: (i) the elaboration of interview protocols, inspired both by those written in CorpAGEst but also by our clinical experience; (ii) the selection of subjects, the recording of data in the field during 14 months, the sampling phase, the digitization of video and audio data and their editing on the ORTOLANG scientific platform; (iii) the transcription and alignment of audio data; (iv) the annotation of audio and video records; (v) the uni- and multimodal data analyses; and (vi) the systematic storage and conservation of raw data and annotated files.

53The interviews were all conducted with native French speakers. The selection criteria for female participants were as follows: (i) the participants had to be 75 years of age or older; (ii) the participants had neither neurological and/or psychological background (e.g. stroke, head injury, psychological disorders, alcoholism, etc.) nor advanced clinical signs of dementia. None of these people could be taking any symptomatic drug treatments for the disorders at the start of the study. They had to (iii) be cognitively fragile and have a mild and significant cognitive deficit, between 26 and 22 points/30 on the MoCA-Test cognitive assessment (Nasreddine et al. 2005), and respond to the self-evaluation of empathic skills (F-IRI, Gilet et al., 2013) without external influence. Finally, they had to (iv) be sufficiently autonomous. Autonomy was assed based on a modified version of the French national grid of autonomy and evaluation of dependence AGGIR [legifrance.fr].

3.2. Annotation of pragmatic markers

54For the manual annotation of the various sub-corpora, the text was first transcribed, segmented and aligned under PRAAT with the Easyalign application (Goldman 2011) and the gestures were annotated with ELAN (Wittenburg, Brugman, Russel, Klassmann, & Sloetjes 2006). The VPM identification stage was based on the annotation protocol developed for French within the MDMA project: Model for Discourse Marker Annotation (Bolly, Crible, Degand & Uygur-Dixteshe 2015), and applied to other languages: English, Spanish (Crible 2017), sign language (Gabarró-López 2017). A list of VPMs was edited in MDMA and other VPMs were collected during our readings on this domain (Dostie 2004) and during the processing of verbal data. A final list of 459 VPMs (DM_List) served as the basis for identification. In addition, the labelling of pragmatic functions of these VPMs and NVPMs followed the functional taxonomy (see Table 1) developed by Bolly and Crible (2015).

55The VPM treatment phase included a 3-layer annotation (tiers): i) the automatic extraction of potential VPMs by launching an automatic search on the basis of the final list of 459 VPMs; ii) a manual cleaning and disambiguation phase of VPMs in audio context (inter-annotator agreement); iii) the annotation phase of VPM functions directly in subdomains, the functional domains being essentially linked to one or another subdomain.

56In the third phase, the functional assignment of VPMs was done by labelling the sub-functions in a tier under the identified and disambiguated VPMs.

Figure 5. Example of functional labelling of VPMs (ageSC3_r2_S3)

Figure 5. Example of functional labelling of VPMs (ageSC3_r2_S3)

57In the above example, pragmatic verbal markers were identified (DM tier) and functionally annotated. In the context of this exchange, for example: the VPM alors that is associated with the main functional domain [interactive] (interpersonal level in orange) and the sub-domain planning [PLAN].

58Given the heterogeneous and polyfunctional nature of VPMs, we limited this labelling to a maximum of two functions in the case where one VPM had more than one function. These two labels are placed side by side in square brackets [ ] and with the + sign to indicate their association, the dominant sub-function first.

Figure 6. Example of polyfunctional labelling of a VPM (ageIT1_r1_S1)

Figure 6. Example of polyfunctional labelling of a VPM (ageIT1_r1_S1)

59In the example above, we observe that some pragmatic markers are associated with two sub-functions and sometimes two distinct domains. Hein belongs to the interactive domain (interpersonal level) but combines both the co-agreement [COGR] sub-function used to ensure that information has been shared, and the agreement [AGR] sub-function because it also expresses agreement on what was said by the speaker: “tu devais être jeune alors?” (Then, you must be young). In the case of donc we observe that this VPM is associated with the ideational domain and the sub-function consequence [CONS] because it indicates that the situation previously cited (her young age) had an effect on her emotional state (the trauma of war). In this context, donc also belongs to the expressive domain with a motivational sub-function [MOTIV] in an epistemic logic close to “je dis cela parce que” (I say that because).

60In this preliminary study, our objective is to observe the distribution of general functions among each of our participants from a longitudinal perspective. We will therefore only consider the main functional domain of VPMs, rather than their sub-function, which more specifically details the assigned domain. The non-verbal data annotation procedure based on the form of gestures (Müller et al. 2013, form-based procedure) follows the instructions suggested in CorpAGEst. This methodology was extended and applied in Bolly’s project to facial expressions, gaze, manual gestures and body gestures (a generic term denoting all of the following articulators: head, shoulders, torso, legs and feet). The gestural annotation we used is largely inspired by the Swedish multimodal analysis project MUMIN (Allwood et al. 2007), following a step-by-step list of physiological parameters and labels to segment and annotate the gestures of each articulator in ELAN. The choice of such a classification in relation to other existing annotation models lies in its exhaustiveness (all the articulators are represented) and its effective operationalization which makes it possible to compare the various articulators between them, for example, in order to observe a synchronization gesture-gesture or gesture-spoken (prosody included). The ELAN “templates” are the basic partitions containing both the articulator tiers and their controlled vocabulary implemented in the software. They are transposable from one annotator to another and follow an organization by group of articulators: facial displays and gaze, hand gestures, upper-body gestures; and finally, lower-body gestures.

61In this study, we will focus on the treatment of hand gestures. Bolly's model of annotation for manual gestures includes 21 lines of annotation. It contains the description of the manual motions previously segmented into phases: the orientation of the gestures themselves (configuration, position, motion, and orientation in space) (Stokoe 1960) as well as the identification of the contacts that accompany the adapters directed towards oneself (e.g. scratching the arm or temple etc.) and adapters pointing at an object (e.g. handling an object like a wedding ring or smoothing out sweater). Usually, hands are analyzed one after the other, tier after tier. However, the other challenge taken up by the protocol is to have integrated the symmetrical gestures of the hands in order not to miscount.

62The segmentation of gestures is broken down into phases. This stage serves as a founding basis for the functional annotation because all potentially significant gestural units of the patient studied during the exchange are noted. A gesture must be visually recognizable thanks to characteristics that we can break down here thanks to Kendon (1980) who distinguishes three major stages to describe the temporal course of the co-verbal gesture into phases: (i) Preparation [PREP] (optional phase): the hand leaves its previous rest position and enters movement. This phase may be followed by a pre-stroke hold before the actual gesture is performed. (ii) The stroke; it is the part that carries the meaning and expression of the gesture. At this point, the affiliated verbal expression and gesture synchronize. This phase is mandatory during a sign language sentence. This synchronization is called “growth point” by McNeill (2005). (iii) The return (optional): the hand then returns to a rest position and can be preceded by a post-stroke hold.

63During phase segmentation, the type of manual movement is coded within the controlled vocabulary. Our protocol counts 5 phases: Preparation [Prepa]; Stroke [Stroke]; Hold, Rest [Hold]; Return [Return]; Partial Return [Return-P]; Chain, Transition [Chain]. Finally, we have decided to identify only strokes since they are the most potentially significant on a pragmatic level.

Figure 7. Illustration of segmentation (left hand) in ELAN software

Figure 7. Illustration of segmentation (left hand) in ELAN software

64For the annotation of the pragmatic functions of gestural and verbal markers, in addition to the approaches of Crible (2014), Crible & Zufferey (2015), Bolly and Crible (2015), we took inspiration from the taxonomies of co-verbal gestures presented in the dedicated literature (Bavelas et al. 1992; Colletta et al. 2009). These pragmatic gesture functions are annotated in ELAN with the same methodology as described for the verbal modality: each manual action is potentially significant in context. Thus, each gestural phase –including strokes and peripheral phases (except holds, which are essentially static)– is considered by the annotator and manually labelled, first with the video context only, and then refined with the audio context.

3.3. Results on two case studies

65Constance is an 86-year-old woman; she lives alone in her own home. After a few years as a cleaner, she stayed at home to raise her children. Her sons and neighbors come to visit her daily. She does not require medical care and benefits from physiotherapeutic care. MCI appeared about three years ago, according to her relatives. Her regular lapses of memory led the family to consult a neurologist in April 2015 who did not detect any dementia but concluded that the cognitive fragility was moderately worsening over time. Constance's predominant communication complaint is lexical deficits (word finding problems). The following graph shows the set of scores for longitudinal cognitive assessments conducted every 4 months.

66As for Tristane, she is an 81-year-old autonomous woman. She does not require any special medical care. However, her family has alerted the general practitioner regarding memory issues affecting both discussions and daily life (forgetting recent discussions and family events: birth, death, marriage). The scores for the longitudinal cognitive assessments performed every 4 months are presented in the following graph. The medical diagnosis for Alzheimer's disease was given this year after two and a half years of follow-up (approximately every four months) with memory evaluation

67There has been a gradual decline in the scores of the screening MoCA-Test up to a problematic threshold. Complete neurological examinations were performed in April 2015 without revealing any AD or related syndrome. Constance is located in the clinical category of MCI patients.

Graphic 1. score results on longitudinal Moca assessment ageSC3 & ageIT1 during 15 months

Graphic 1. score results on longitudinal Moca assessment ageSC3 & ageIT1 during 15 months

68After exporting the VPMs data of the two speakers out from Elan, we first evaluated the quality of the corpus and validated it by carefully checking the presence and if necessary the well-foundedness of missing data.

69The data set that can finally be used for analysis includes 3931 VPM distributed as follows:

70In order to smooth the “speaker” effect and to take into account a possible idiosyncratic differential of the verbal content (prolific subject vs. taciturn subject), we chose to work in frequency. Still with this concern of normalization and in order to minimize the effects of extrinsic co-factors (different emotional state between two interviews), we decided to proceed to a normalization in frequency by task.

71Thus, for each speaker, the data will be expressed in frequency of use of each functional domain among all the VPMs expressed during a task.

72General characteristics of the VPM device: Use of VM (in frequencies per task and speaker)

73The figure below gives an overview of the data thus generated:

Figure 8. General characteristics of the VPM device: Use of VM (in frequencies per task and speaker)

Figure 8. General characteristics of the VPM device: Use of VM (in frequencies per task and speaker)

74According to this first figure, we can observe:

  • An extensive use of Interactive VPMs, whatever the speaker and the task considered (between 40 and 50% of VPMs used during a task) despite the progressive entry into the pathology.

  • A pragmatic profile of VPM use generally similar between speaker and proposed task (in general, in order of frequency of use: Interactive > Expressive > Structuring > Ideational).

75The interactive aspect prevails. Constance and Tristane use interpersonal VPMs to solicit the interlocutor and their shared knowledge in order to ensure interaction is maintained and remains anchored in discourse (in particular by inserting co-agreement VPMs). Thus, this situation of interaction disturbed by language fragilities (naming deficits, disfluencies, doubts) is dealt with by the speaker but can also be dealt with by relatives if they are attentive to these indicators of cognitive discomfort. Once these elements have been identified, they could be valuable resource tools for clinicians in order to intervene at times when tiredness occurs or if the theme addressed requires a major effort in the exchange. These markers would alleviate the feeling of powerlessness of older speakers who could overcome the situation by transmitting a positive and committed image of themselves in speech. The clinician must be sufficiently sensitive to these interaction marks.

Figure 9. Hierarchical clustering: Speakers vs. Use of VPM (in frequencies) per task

Figure 9. Hierarchical clustering: Speakers vs. Use of VPM (in frequencies) per task

76In order to characterize the system upstream, and to reveal any underlying structures, we then used a hierarchical clustering strategy (based on Pearson’s correlation and Ward’s D2 aggregation criterion), the idea being to graphically visualize the similarities and di-similarities between the different classes of VPM according to the way they are used by the different speakers. The heatmap below presents the results of this analysis. For a given cell, the warmer the colour is, the more important the use of a class of VPM during a given interview is for the speaker in question (and vice versa when it is coloured in cold blue).

77We see here (Figure 9) that:

  • The interactive VPMs are generally more mobilized, whoever the speaker and whatever the exercise, when compared with the other classes of VPM, corroborating the previous observations.

  • The way the interactive and ideational VPM types are used is relatively homogeneous across tasks.

  • The structure is more unclear concerning the VPM of structuring and expressive types.

  • AgeSC3 makes greater overall use of interactive VPMs.

  • AgeIT1 makes greater use of expressive and structuring VPMs.

78In addition to this analysis, we also characterized non-verbal manifestations in Constance’s speech. To do this, we counted the NVPMs (of hands) in r2 and r4, which allowed us to note an increase of 22% of NVPMs produced by Constance between these two interviews.

79

Graphic 2. Frequency of VPM versus NVPM between interview 2 and interview 4 with ageSC3

Graphic 2. Frequency of VPM versus NVPM between interview 2 and interview 4 with ageSC3

80Given these preliminary results, it remains to be seen whether NVPMs would increase over time in seniors’ discourse. In comparison, these NVPMs are more numerous than the VPMs produced within these two corpora. This finding may indicate a tendency for Constance to produce more gestures during interaction while verbal information content decreases.

81In the third and fourth interviews, referential gestures tend to take a deictic orientation by taking precedence over representational gestures.

82In an exchange about odor recollections, Constance is not able to name the stimulus, which is the smell of lavender. She explains to her son that she recognizes the smell but the word does not cross her mind. She ends up looking into her direct environment and pointing her finger at the bouquet of lavender placed in her kitchen while at the same time verbalizing her discomfort:

Constance: “euh…comment… euh… [pointing at the bouquet]”
Jules : “de la lavande oui !”
Constance : “oui de la lavande c’est ça … ah !”
* Constance: “hum…How can i…hum” [pointing at the bouquet]
Jules: “a bunch of lavander, yeah !”
Constance: “yeah that’s it, a bunch of lavander … ah !”
(see illustration 1.b. in this article : Example of a deictic gesture: pointing the bunch of lavender (in ageSC3_r3_S5-00:59))

83In the light of recent studies (Schiaratura et al. 2015; Carlomagno et al. 2005) we assume that representational gestures are also becoming too cognitively expensive for Constance. Indeed, it is a complex activity that involves both semantic memory (e.g. encyclopedic knowledge, language, and concepts) and executive functions that govern the cognitive mechanisms required to perform gestures and fine motor skills. Moreover, the overall situation is stressful for Constance experiencing naming and frustration for several months. Yet willingness to communicate is still there but manifests itself in a simpler form.

84Another interesting orientation concerns the developmental approach of communication throughout life. We can hypothesize that this duality between deficit phenomena and compensatory phenomena is to be analyzed under a retroactive light of human communication. During their development, young children gradually experiment with gesture and communication skills that will enhance their language acquisition. The development of referential gestures is gradually achieved through the deictics scale first (finger/hand pointing, joint attention and gaze designation), and then through ideational gestures that require greater cognitive skill and greater refinement on the semantic level as the young child develops his knowledge. During the aging process, and especially during the fragile and problematic aging, these gestural manifestations take the opposite path but always sign this need to communicate and share emotions. The cognitive stock could be involved in this process because it would contain all the neuronal capacities to cope with cognitive difficulties. Throughout their life experiences, the elderly have internalized a set of verbal and non-verbal behaviors that would allow them to compensate and maintain interactions in the event of difficulties in an individual-specific process. These attempts would be all the more valued and effective if the interlocutor, either close, clinician or all-coming, proves to be supportive and empathetic.

Conclusion

85This study sheds light on a point of view differing from the classical investigation methods proposed so far in aging psychology (for the most part based on data produced in laboratory conditions). Our approach give room for the expression of authentic interactions in natural exchange situations that can be observed as closely as possible. Our study develops several innovative aspects. First, the pragmatic competence of elderly people with mild cognitive deficits is still a marginal topic in linguistic. Secondly, it proposes a methodology based on the reasoned and explicit annotation of multimodal data in natural spontaneous exchange situations, which can therefore be used for other research perspectives. Third, the multidisciplinary dimension of this study, at the crossroads of pragmatics, discourse analysis, psychology of aging and multimodality, offers diversified avenues for the scientific community and –we hope– may encourage clinical research to develop corpus-based approaches. Indeed, the community is engaged in soliciting the humanities and social sciences for a better understanding of the language continuum and in developing clinical models favoring diagnosis and therapeutic support in an evidence-based approach.

86We propose a continuous approach to language aging and identify a singular profile for MCI people within this frame. The trends we highlight commit us to thinking of language as an interesting resource for detecting early markers of dementia. Without talking about pathological aging, it is necessary to observe the profiles of the participants in our study and to analyze the distinctions that exist between the common and heterogeneous pathways that they take over time. This has always been done after the diagnosis has been made. However, it seems crucial to carefully consider these pragmatic markers before the establishment of any obvious clinical signs in order to enrich research on pathological aging.

87Finally, it is worth developing a multimodal approach to account for the non-verbal compensatory elements within the deficit elements of these MCI people.

88At the end of this discussion on the communicational traits that would characterize this in-between profile in language aging, we postulate that verbal deficits are accompanied by an increase in non-verbal acts with a specialization of these non-verbal manifestations as the deficits increase. The more verbal content becomes difficult to represent manually, the more the elderly person relies on deictic gestures and intersubjective solicitations addressed to the interlocutor (signs of co-agreements, shared, interactive knowledge) in order to maintain the communication.

89In a long-term perspective, considering language in its plural dimension offers a relevant anchorage as soon as we build interactive corpuses. We have seen that speech in action is usually accompanied by communicative gestures. Follow-up researches on this topic would be beneficial in order to point the compensatory or facilitating nature of the gesture within the disturbed language, and to identify possible predictors of dementia like Alzheimer’s disease (AD). AD is characterized by a progressive deterioration of intellectual abilities, memory loss, attention difficulties and language disorders (particularly at the semantic level). The person's deep identity is altered, accompanied by behavioural and mood disturbances. These changes alter communication skills and disrupt not only the patient's life, but also social relationships.

90Currently, the few studies addressing these questions (Glosser & Barnoskir 1998; Taler & Philipps 2008; Schiaratura 2008; Taler, Baum, Saumier & Chertkow 2008; Davis & MacLagan 2016) engage research to develop models considering the communication of the elderly with MCI or DA: i) in its multimodal dimension, ii) interactive, iii) and in approaches toured on its manifestations at the level of discourse and more generally at the level of the pragmatic and social dimension. We emphasize the importance of considering non-verbal communication in its interactive dimension in people with MCI. If non-verbal and adaptive cues are not perceived by the interlocutor within repeated daily activities –care, meals, friendly exchange, activity– then the person is less and less likely to interact in this modality, at the risk of increasing the symptoms a little more by the effect of social and emotional comorbidity.

Haut de page

Bibliographie

Aijmer K. & Simon-Vandenbergen A. M. (2011). “Pragmatic markers”, Discursive pragmatics 8: 223-247.

Allwood J., Cerrato L., Jokinen K., Navarretta C. & Paggio P. (2007). “The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena”, Language Resources and Evaluation 41(3): 273-287.

Andrén M. (2010). Children’s Gestures from 18 to 30 Months (Vol. 50). Centre for Languages and Literature, Lund University.

Arbuckle T. Y. & Gold D. P. (1993). “Axging, inhibition, and verbosity”, Journal of Gerontology 48(5): 225-232.

Baines S., Saxby P. & Ehlert K. (1987). “Reality orientation and reminiscence therapy. A controlled cross-over study of elderly confused people”, The British Journal of Psychiatry 151(2): 222231.

Baltes P. B. (1997). “On the incomplete architecture of human ontogeny: Selection, optimization, and compensation as foundation of developmental theory”, American psychologist 52(4): 366.

Bates E. (1976). Language and context: The acquisition of pragmatics. Academic Press.

Baude O., Blanche-Benveniste C., Calas M.-F., Cappeau P., Cordereix P., Goury L. & Mondada L. (2006). Corpus oraux, guide des bonnes pratiques 2006. CNRS Editions, Presses Universitaires Orléans.

Bavelas J. B., Chovil N., Lawrie D. A. & Wade A. (1992). “Interactive gestures”, Discourse processes 15(4): 469-489.

Beattie G. & Shovelton H. (1999). “Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation”, Semiotica 123(1-2): 1-30.

Berrewaerts J., Hupet M. & Feyereisen P. (2003). “Langage et démence: examen des capacités pragmatiques dans la maladie d’Alzheimer”, Revue de Neuropsychologie 13(2): 165-207.

Boersma P. & Weenink D. (2009). Praat: doing phonetics by computer (Version 5.1. 05) [Computer program]. Retrieved May 1, 2009.

Bolly C. T. & Crible L. (2015). “From context to functions and back again: Disambiguating pragmatic uses of discourse markers”, Paper presented at the Panel Session Anchoring utterances in co(n)text, argumentation, common ground (Org.: K. Fischer, M. Alm), part of the 14th International Pragmatics Conference (IPra), 26-31 July 2015, Antwerp, Belgium.

Bolly C. T. & Boutet D. (2017). “The multimodal CorpAGEst corpus: Keeping an eye on pragmatic competence in later life”, Corpora 13(2).

Bolly C.T., Crible L., Degand L. & Uygur-Distexhe D. (2015). MDMA. “Un modèle pour l’identification et l’annotation des marqueurs discursifs ‘potentiels’ en contexte. Discours”, Revue de linguistique, psycholinguistique et informatique. A journal of linguistics, psycholinguistics and computational linguistics (16).

Bolly C. T. (2010). “Pragmaticalisation du marqueur discursif tu vois. De la perception à l’évidence et de l’évidence au discours”, in 2e Congrès Mondial de Linguistique Française, 45. EDP Sciences.

Bolly C. T. & Duboisdindien G. (2017). “and…and…you see, sweetheart?: Verbal and gestural pragmatic markers to compensate for the effects of cognitive frailty in aging”, Paper presented at the Panel session Pragmatics in real world par of The 15th International Pragmatic (IPra) Conference – 16-21th July 2017. Belfast, Northern Ireland.

Bordería S. P. (2008). “Do discourse markers exist? On the treatment of discourse markers in Relevance Theory”, Journal of Pragmatics 40(8): 1411-1434.

Bressem J. (2013). “A linguistic perspective on the notation of form features in gestures”, in C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill and S. Teßendorf (eds.), Body – Language – Communication (Vol. 1) (Handbooks of Linguistics and Communication Science 38/1), 1079-1098. Berlin, New York: Mouton de Gruyter.

Brinton L. J. (2010). 10. “Discourse Markers”, Historical pragmatics 8: 285.

Butler R. N. (1979). “Geriatrics and internal medicine”, Ann Intern Med 91: 903-908.

Colletta J. M. (2009). “Comparative analysis of children’s narratives at different ages: A multimodal approach”, Gesture 9(1): 61-96.

Collette F. & Salmon E. (2014). “Les modifications du fonctionnement exécutif dans le vieillissement normal”, Psychologie française 59(1): 41-58.

Crible L., Zufferey S. (2015). “Using a unified taxonomy to annotate discourse markers in speech and writing”, in Harry Bunt, Proceedings of the 11th Joint ACL - ISO Workshop on Interoperable Semantic Annotation (isa-11), 14-22.

Crible L. (2017). Discourse Markers and (Dis)fluency across Registers. A contrastive Usage-Based Study in English and French. U de Louvain. Thèse non publiée.

Crystal D. (2002). “Clinical Linguistics and Phonetics’ first 15 years: an introductory comment”, Clinical linguistics & phonetics 16(7): 487-489.

Cummings L. (2007). “Pragmatics and adult language disorders: Past achievements and future directions”, Seminars in Speech and Language Vol. 28, No. 2: 96-110.

Cummings L. (2009). Clinical pragmatics. Cambridge University Press.

Cummings L. (2014). Pragmatic disorders. Perspectives in Pragmatics, Philosophy & Psychology. Vol. 3. Springer Science & Business Media.

Davis B. H., MacLagan M. & Cook J. (2013). “Aw, so, how’s your day going?’: ways that persons with dementia keep their conversational partner involved”, Pragmatics in dementia discourse, 83-116.

Davis B.H. & MacLagan M. (2016). “Sociolinguistics, Language and Aging”, in Wright H. H. (ed.) Cognition, language and aging. John Benjamins Publishing Company, Chapter 9, 221-245.

Dollaghan C. A. (2007). The handbook for evidence-based practice in communication disorders. Paul H. Brookes Pub.

Dostie G. (2004). Pragmaticalisation et marqueurs discursifs – Analyse sémantique et traitement lexicographique. De Boeck, Duculot.

Duboisdindien G. & Lacheret-Dujour A. (2017). “Des aînés hors normes? Interroger le continuum du vieillissement langagier à travers une perspective interactionnelle”, Atypies langagières de l’enfant à l’âge adulte, apports de la psycholinguistique et des neurosciences cognitives. DeBoeck, 215-243.

Feyereisen P. La communication non verbale chez les sujets déments de type Alzheimer. Questions de Logopédie 1993; 27: 67-8.

Ferré G. (2011). “Analyse multimodale de la parole”, Rééducation orthophonique 246, 7385.

Gabarró-López S. (sous presse). “Les marqueurs du discours en langue des signes de Belgique francophone (LSFB) et en langue des signes catalane (LSC): les ‘balises-liste’ et les ‘palm-ups’”, in Ó. Loureda, G. Parodi, M. Rudka, S. Salameh (eds.) Marcadores del discurso y lingüística contrastiva en las lenguas románicas. Iberoamericana Vervuert.

Gilet A.L., Mella N., Studer J., Grühn D. & Labouvie-Vief G. (2013). “Assessing dispositional empathy in adults: A French validation of the Interpersonal Reactivity Index (IRI)”, Canadian Journal of Behavioural Science/Revue canadienne des sciences du comportement 45(1): 42.

Glosser G., Wiley M. J. & Barnoskir E. J. (1998). “Gestural communication in Alzheimer’s disease”, Journal of Clinical and Experimental Neuropsychology 20(1): 1-13.

Goldin-Meadow S. & Alibali M. W. (2013). “Gesture's role in speaking, learning, and creating language”, Annual review of psychology 64: 257-283.

Goldman J.-P. (2011). “EasyAlign: an automatic phonetic alignment tool under Praat”, Proceedings of InterSpeech, http://latlcui.unige.ch/phonetique/easyalign.php.

Gonzalez L. (2004). “Conséquences de la désafférentation sensorielle chez les personnes âgées”, La Revue de gériatrie 29(1): 61-66.

Guendouzi J. A. & Muller N. (2006). “Approaches to discourse in dementia”, Psychology Press.

Hamilton H. E. (1996). “Intratextuality, intertextuality, and the construction of identity as patient in Alzheimer’s disease”, Text-Interdisciplinary Journal for the Study of Discourse 16(1): 61-90.

Hamilton H. E. (2008). “Narrative as snapshot: Glimpses into the past in Alzheimer’s discourse”, Narrative Inquiry 18(1): 53-82.

Hamilton H. E. (2013). Language and communication in old age: Multidisciplinary perspectives. Routledge.

Hamilton H. & Chou W. Y. S. (eds.). (2014). The Routledge handbook of language and health communication. Routledge.

Hamilton H. E. & Hamaguchi T. (2015). “Discourse and aging”, The handbook of discourse analysis, 705-727.

Hansen M.B. M. (2006). “A dynamic polysemy approach to the lexical semantics of discourse markers (with an exemplary analysis of French toujours)”, Approaches to discourse particles, 2141.

Haselow A. (2011). “Discourse marker and modal particle: The functions of utterance-final then in spoken English”, Journal of Pragmatics 43(14): 3603-3623.

Hopper T., Bourgeois M., Pimentel J., Qualls C. D., Hickey E., Frymark T. & Schooling T. (2013). “An evidence-based systematic review on cognitive interventions for individuals with dementia”, American Journal of Speech-Language Pathology 22(1): 126-145.

Kemper S., Greiner L. H., Marquis J. G., Prenovost K. & Mitzner T. L. (2001). “Language decline across the life span: findings from the Nun Study”, Psychology and aging 16(2): 227.

Kendon A. (1997). “Gesture”, Annual review of anthropology 26(1): 109128.

Kendon A. (2004). Gesture: Visible action as utterance. Cambridge University Press.

Kennedy G. (1998). An introduction to Corpus Linguistics. London and New York, Longman.

Lemaire P. (2015). Vieillissement cognitif et adaptations stratégiques. De Boeck Superieur.

Lenoir R. (1979). “L’invention du ‘troisième âge’”. Actes de la recherche en sciences sociales 26(1): 57-82.

Lof G.L. (2011). “Science-based practice and the speech-language pathologist”, International Journal of Speech-Language Pathology 13(3): 189-196.

Loones A., David-Alberola E. & Jauneau P. (2008). “La fragilité des personnes âgées: perceptions et mesures”, Cahier de recherches, (256).

Mathey S. & Postal V. (2008). “Le langage”, Neuropsychologie du vieillissement normal et pathologique 16: 79-102.

McCurtin A. & Healy C. (2017). “Why do clinicians choose the therapies and techniques they do? Exploring clinical decision-making via treatment selections in dysphagia practice”, International journal of speech-language pathology 19(1), 69-76.

Mol L., Krahmer E., Maes A. & Swerts M. (2012). “Adaptation in gesture: Converging hands or converging minds?”, Journal of Memory and Language 66(1): 249264.

Morello A. N. D. C., Lima T. M. & Brandão L. (2017). Language and communication non-pharmacological interventions in patients with Alzheimer’s disease: a systematic review. Communication intervention in Alzheimer. Dementia & Neuropsychologia 11(3): 227-241.

Morgenstern A. & Beaupoil P. (2015). Multimodal approaches to language acquisition through the lens of negation. Vestnik of Moscow State University. Linguistics and literary studies. Issue 6 (717). Special issue on discourse as social practice, 435-451.

Morgenstern A., Leroy M. & Mathiot E. (2008). “Le pointage chez l’enfant: du gestuel au linguistique”, in Congrès Mondial de Linguistique Française. EDP Sciences, 164.

Müller N. & Schrauf R. (2014). Conversation as cognition: reframing cognition in dementia.

Mungas D., Beckett L., Harvey D., Tomaszewski Farias S., Reed B., Carmichael O., … DeCarli C. (2010). Heterogeneity of cognitive trajectories in diverse older persons. Psychology and aging 25(3): 606.

Nasreddine Z. S., Phillips N. A., Bédirian V., Charbonneau S., Whitehead V., Collin I., … Chertkow H. (2005). The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society 53(4): 695699.

Nef F. & Hupet M. (1992). “Les manifestations du vieillissement normal dans le langage spontané oral et écrit”, L’année psychologique 92(3): 393419.

Nussbaum J. F., Pecchioni L. L., Robinson J. D. & Thompson T. L. (2000). Communication and aging. Routledge.

Rabatel A. (2004). Interactions orales en contexte didactique. Presses universitaires de Lyon.

Rousseau T. (1995) Communication et maladie d’Alzheimer. Isbergues: Ortho-Edition.

Ryan E. B., Hummert M. L. & Boich L. H. (1995). “Communication predicaments of aging: Patronizing behavior toward older adults”, Journal of Language and Social Psychology 14(1-2): 144-166.

Salthouse T. A. (2010). “Selective review of cognitive aging”, Journal of the International neuropsychological Society 16(5): 754760.

Schaie K.W. (2005). “What can we learn from longitudinal studies of adult development?”, Research in human development 2(3): 133158.

Shake M. C., Noh S. R. & StineMorrow E. A. (2009). “Age differences in learning from text: Evidence for functionally distinct text processing systems”, Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition 23(4): 561-578.

Schiaratura L. T., Di Pastena A., Askevis-Leherpeux F. & Clément S. (2015). “Verbal and gestural communication in interpersonal interaction with Alzheimer's disease patients”, Gériatrie et psychologie neuropsychiatrie du vieillissement 13(1): 97-105.

Schiaratura L. T. (2008). “La communication non verbale dans la maladie d’Alzheimer”, Psychologie & NeuroPsychiatrie du vieillissement 6(3): 183-188.

Soubelet A., & Salthouse T. A. (2011). “Influence of social desirability on age differences in selfreports of mood and personality”, Journal of personality 79(4): 741-762.

Spieler D. H. & Griffin Z. M. (2006). “The influence of age on the time course of word preparation in multiword utterances”, Language and Cognitive Processes 21(1-3): 291-321.

Stivers T. (2008). “Stance, alignment, and affiliation during storytelling: When nodding is a token of affiliation”, Research on language and social interaction 41(1): 31-57.

Stokoe W. C. (1960). “Sign language structure (Studies in Linguistics”, Occasional paper 8.

Taconnat L. & Lemaire P. (2014). “Fonctions exécutives, vieillissement cognitif et variations stratégiques”, Psychologie française 59(1): 89-100.

Taler V. & Phillips N. A. (2008). “Language performance in Alzheimer's disease and mild cognitive impairment: a comparative review”, Journal of clinical and experimental neuropsychology 30(5): 501-556.

Tamir L.M. (1979). Communication and the aging process: Interaction throughout the life cycle. New York: Pengamon.

Taler V., Baum S. R., Chertkow H. & Saumier D. (2008). “Comprehension of grammatical and emotional prosody is impaired in Alzheimer's disease”, Neuropsychology 22(2): 188.

Traugott E. C. (2010). “(Inter) subjectivity and (inter) subjectification: A reassessment”, Subjectification, intersubjectification and grammaticalization, 2971.

Troyer A. K., Leach L. & Strauss E. (2006). “Aging and response inhibition: Normative data for the Victoria Stroop Test. Aging”, Neuropsychology, and Cognition 13(1): 20-35.

Van der Linden M. & Van der Linden A. C. J. (2014). Penser autrement le vieillissement: Des pistes concrètes pour une approche humaniste du vieillissement cérébral. Primento.

Wittenburg P., Brugman H., Russel A., Klassmann A. & Sloetjes H. (2006). “ELAN: a professional framework for multimodality research”, in 5th International Conference on Language Resources and Evaluation (LREC 2006), 1556-1559.

Wray A. & Perkins M. R. (2000). “The functions of formulaic language: An integrated model”, Language & Communication 20(1): 1-28.

Wray A. (2002). “Formulaic language in computer-supported communication: theory meets reality”, Language Awareness 11(2): 114-131.

Haut de page

Table des illustrations

Titre Figure 1 a. Example of a referential gesture: light the fire (in ageSC3_r1_S1- 03:51) – Figure 1 b. Example of a deictic gesture: pointing the bunch of lavender (in ageSC3_r3_S5-00:59)
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-1.png
Fichier image/png, 1,3M
Titre Figure 2. Example of an expressive gesture “Surtout ne touche à rien – Don’t touch anything!” (in ageSC3_r1_S1-1:42)
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-2.png
Fichier image/png, 604k
Titre Figure 3. Example of a structuring gesture “Parce que , je ne voulais pas manger -Because [structuring gesture] I didn’t want to eat!” (in ageSC3_r1_S1 – 02:05)
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-3.png
Fichier image/png, 555k
Titre Figure 4. a. Example of an interactive gesture to Jules (her son) “Oui ! c’est ça ! ça laisse une bonne odeur sur le linge de maison hum - yeah! That’s it! It leaves a good smell in the hum household linen hum” (in ageSC3_r3_S5-03:56) b. Example of self-contact gesture (adapters): thumbs twiddling (in ageSC3_r1_S1-02:36)
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-4.png
Fichier image/png, 1,2M
Titre Figure 5. Example of functional labelling of VPMs (ageSC3_r2_S3)
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-5.png
Fichier image/png, 93k
Titre Figure 6. Example of polyfunctional labelling of a VPM (ageIT1_r1_S1)
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-6.png
Fichier image/png, 86k
Titre Figure 7. Illustration of segmentation (left hand) in ELAN software
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-7.png
Fichier image/png, 304k
Titre Graphic 1. score results on longitudinal Moca assessment ageSC3 & ageIT1 during 15 months
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-8.png
Fichier image/png, 38k
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-9.png
Fichier image/png, 51k
Titre Figure 8. General characteristics of the VPM device: Use of VM (in frequencies per task and speaker)
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-10.png
Fichier image/png, 71k
Titre Figure 9. Hierarchical clustering: Speakers vs. Use of VPM (in frequencies) per task
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-11.png
Fichier image/png, 61k
Titre Graphic 2. Frequency of VPM versus NVPM between interview 2 and interview 4 with ageSC3
URL http://journals.openedition.org/corpus/docannexe/image/4295/img-12.png
Fichier image/png, 80k
Haut de page

Pour citer cet article

Référence électronique

Guillaume Duboisdindien, Cyril Grandin, Dominique Boutet et Anne Lacheret-Dujour, « A Multimodal corpus to check on pragmatic competence for Mild Cognitive Impaired aging people », Corpus [En ligne], 19 | 2019, mis en ligne le 01 janvier 2019, consulté le 15 octobre 2019. URL : http://journals.openedition.org/corpus/4295

Haut de page

Auteurs

Guillaume Duboisdindien

Université Paris Nanterre & MoDyCo UMR 7114

Cyril Grandin

Université de Lille & STL UMR 8163

Dominique Boutet

Université de Rouen & SFL UMR 7023

Anne Lacheret-Dujour

Université Paris Nanterre & MoDyCo UMR 7114

Haut de page

Droits d’auteur

© Tous droits réservés

Haut de page
  • Logo Revues électroniques de l’université de Nice
  • OpenEdition Journals