Navigation – Plan du site

AccueilNumérosHS1Partie I : L’audio-visuel comme d...From video game to ethnographic v...

Partie I : L’audio-visuel comme dispositif méthodologique pour capter et analyser les interactions multisensorielles

From video game to ethnographic video game

Jacques Ibanez Bueno et Lilyana V. Petrova

Résumés

L’évolution des méthodes visuelles transcende le film documentaire traditionnel, en embrassant divers formats numériques parallèlement au cinéma. Les jeux vidéo, les jeux sérieux, les applications téléphoniques ou la réalité virtuelle sont désormais utilisés pour effectuer des recherches, collecter des données et présenter les résultats à la communauté scientifique ainsi qu’au grand public. Dans cet article, nous étudions la relation entre le corps humain et la technologie du « jeu ». Grâce à une étude empirique de FIFA16 (le premier jeu FIFA avec des joueuses féminines), nous observons, filmons, décrivons et analysons les gestes et les discours des joueurs.euses de FIFA manipulant la manette. Nous nous concentrons sur les gestes mineurs (Manning, 2016) entourant l’outil lui-même (la manette), illustrant les processus d’immersion physique et narrative. Les méthodes choisies suivent les méthodes visuelles décrites ci-dessus : observation immersive, filmage, manipulation et représentation interactives. Concrètement, comme résultat logique de la requête épistémologique et comme l’un des principaux résultats de la recherche, nous proposons la création d’un « jeu ethnographique ».

Haut de page

Texte intégral

This research has been conducted with precious help from our students from Université Savoie Mont-Blanc, Master “Hypermedia and Communication” 2015–2016.

Introduction

1The visual methods which use moving images are no longer limited to the filmic format in its tradition of documentary film narration. The development of the digital image and its domination over the analogue one has given rise to many other formats since the end of the 20th century. There is no question of dismissing linear, digital or film-based documentary research, just as it would be inappropriate to consider that film in all distribution circuits (from the cinema to the digital platform) is doomed to disappear in relation to the latest digital formats. Based on research produced in interactive media format, in this paper we would aim to highlight some of the characteristics of the various formats added to the filmic format resulting from the history of a century and a quarter of cinematography. For this, it is, of course, necessary to rely on cinema and previously on photography in their historical and epistemological basis, within the diversity of the methodologies using images within the human and social sciences. For this, taking into account the contributions of pioneers such as Margaret Mead or Jean Rouch remains essential. In the continuation of the 1956 “Free Cinema Manifesto” promoting the plurality of forms and screen sizes in cinema, we adhere to the “Édito-Manifeste” of the Revue Française des Méthodes Visuelles, to “[e] xplore new experiences related to the digital by fully exploiting the opportunities offered by online publishing” (Bouldoires and Reix, 2017–2018).

2Without this list being exhaustive, we add video games, serious games, phone applications or virtual reality (VR) installations with headsets to existing productions such as documentaries in a contemporary art process, augmented documentary, transmedia documentary and web documentary (Sucari, 2012). Today, a sufficiently important corpus exists with very diversified productions allowing questioning the modalities of interactivity between the researcher (filming the research), the actors (subjects of the research project) in relation to concepts such as reality, corporeality and publicity.

  • 1 Navigation in the ethnographic game available here.

3Consequently, in this article we are investigating the relationship between the human playing body and the player’s controller. We are focusing on the minor gestures (Manning, 2016) surrounding the tool itself (the controller), picturing processes of both physical and narrative immersion. The decided upon methods are following the described above visual methods: immersive observation, filming and interactive manipulation and representation. Concretely, as a logical result of the epistemological query and one of the main research results, we are proposing the creation of an “ethnographic game”.1 The concluding remarks comfort us with the intention to apply innovative interactive hypermedia solutions to both interrogate the research subject and to represent the subsequent results.

1. Research on an online video game

4Among our field research practices, resulting in interactive visual productions, the one conducted on the online video game FIFA marketed by Electronic Arts seems particularly emblematic to us.

5Prior to the results, an empirical study using visual methods (Bouldoires and Reix, 2017–2018) was conducted. This study was based on an immersion within the immediate environment of five players, with visual and sound capture of the uses. In their natural environment, at home, the players participated according to the concept of participation (Ruby, 1996) by an active and contributive acceptance to the fact of being progressively filmed, in parallel with the capture of the images visible on the screen. This recording work was completed by semi-directive interviews before and after the recordings in order to extract a verbatim complementary to the original visual and sound data. Using the collected data, a merging process of camera synchronization was carried. This process allowed a sound and visual data analysis with a privileged focus on the body movements (from the most extensive to the smallest), in simultaneous connection with the visual and sound contents diffused by the playful interface.

6The soccer simulation video game FIFA16 was distributed in France from September 24, 2015. Since its release, players could use this game with Microsoft Windows, PlayStation or Xbox systems. In this release, the level of realism has been further perfected. Additionally, in its ability to play online, this release offered improved features (for example the draft mode) as well as the first appearance of female football players. The international success of the FIFA series was thus further confirmed on this occasion.

7The research undertaken here focuses on the physical dimensions of the uses of FIFA16. Does the physical experience of the body prevail in the process of appropriation of this online game? Is playingFIFA16 comparable to away from screen games such as table soccer? Or is the immersion not very sensorimotor and essentially produced by the visual elements and the interactive narration? The conceptual opportunities offered by the phenomenology of perception (Merleau Ponty, 1945) and their application to digital communication are here extended to video games to answer these questions. One of the aims of the research is to bring knowledge on the modalities of FIFA16 adoption by its users.

8Initially, the research questions were focused on the bodily dimension of the behaviour. What are the users’ perceptions and personal projections in the case of FIFA16? Is controller manipulation a continuation of the body (bodily techniques) in the tradition of McLuhan’s theories (McLuhan, 1964)? Is the physical experience predominant or is the immersion strictly visual?

9In the context of interactive corporeality, the proposals of two researchers are tested in relation to the concept of “projective acting body” (Amato and Weissberg, 2003, p. 244). Does FIFA16 offer the possibility of creating a “kind of dilation of our body” that will meet that of the avatar? The results produced move away from the hypothesis of a kind of reproduction of soccer movement (Hunter and Csikszentmihalyi, 2000), making FIFA16 a “reality” game. FIFA16 would be closer to a board game than to table soccer with a limited bodily involvement.

2. An evolving methodology

10An empirical study was set up using visual methods: a team of investigators observed and filmed users during their practice, using principles specific to hypermedia anthropology (Da Silva Ribeiro and Bairon, 2007) and sensory ethnography (Pink, 2009). The researchers then extracted visual data to co-design an interactive ludic hypermedia website.

11Thus, a comparative analysis was conducted between FIFA16 and other games such as Wii, board games, and table soccer. In the mentioned cases, the game does not depend on the use of the feet (unlike soccer), but on the use of the hands with a more or less developed handling technique. The game of soccer is delegated to the hands. With the Wii, for example, the player holds a controller that detects movements and allows physical participation in the game, developing the possibility of implication (Calleja, 2011), and bodily immersion. For board games, the hand is the executor of a premeditated operation (except for speed games like Jungle Speed), the player remains in a perspective of distance, in the logic of the projection, of representation. Finally, the table soccer (which retained our attention) seems to propose a situation in which the player reacts on the events of the game, without being completely immersed in a physical way. It can be considered as a transitional game because the game of soccer is delegated to the hands that animate the 11 figures (the same number as a traditional soccer team) and this in order to replay the soccer logic on the field delimited by the game table.

12It is in this sense that the table soccer is close to the FIFA type video games in that it positions the player at a distance from the game by the means of representation. At the same time, the table soccer is close to the real game of soccer in the sense that it is a mechanical device, and that the movements of the ball obey the physical movements of the whole body. The hands are used more than on the game controller. The speed, strength and technique of the players can directly influence the rhythm of the ball (by applying the desired spin for example). The hands can also move more freely as well as the body.

13Following these first comparisons, we proceed to the field study in order to observe the postures specific to the game of FIFA16 in its elastic similarity to that of table soccer. We will present them in two categories, in the first one we will situate these behaviours that bring the practice of FIFA16 closer to the body immersion. In the second category, we will talk about the examples that show a narrative immersion through visual, sound and interactive representation.

3. Body Immersion

14Although the importance of bodily activity in FIFA16 as in table soccer remains far from the physical training needs of the classic soccer field, there is still a strong bodily involvement that we have observed on several occasions.

15

Video 1 – “Physical reaction to events in the game” by Students Master “Hypermedia and Communication” 2015–2016.
Crédits : © J. Ibanez Bueno, L. Petrova

16Evidently, a player reacts physically to the game events. In the image appears a movement of the feet corresponding to the moment of the shot in the game. A spontaneous bodily reaction that indicates the existence of a relationship between what happens in the screen and related outcomes outside of it. According to Nadia Bianchi-Berthouz (2013), such behaviour can be considered as a sign of presence in the virtual (visual) environment, as the player behaves as if he or she were on the soccer field. Furthermore, the whole body reacts to events on the screen. Here we can see that, both players have a physical reaction every time the game stops.

17

Video 2 – “Loosening during breaks” by Students Master “Hypermedia and Communication” 2015–2016.
Crédits : © J. Ibanez Bueno, L. Petrova

18The rhythm of the game visibly influences the gestures and body postures. While both players appear “still” during the game, they seem more relaxed during the stop phase. The rhythm between their behaviour and the game is coordinated. With the same pair of players, we also observed the reactions when the game was won and noticed that some players reacted like fans of a soccer team by raising their hands to the sky. This is a “sensitive” experience that creates “sensations” in the sense given to this term by Maurice Merleau Ponty (1961). This experience of celebration is nevertheless equivocal since the player does not behave as if he were the actor of his success. He reacts as if it were his favourite team that had scored and not him. He does not seem to be aware of his own action. He is surprised and waits to see the rehearsal before he can confirm his excitement.

19Still on the side of the relation between the body and the playful device, by changing the play mode, from assisted mode to manual mode, we can observe the appearance of different physical reactions.

20

Video 3 – “Constraint” by Students Master “Hypermedia and Communication” 2015–2016.
Crédits : © J. Ibanez Bueno, L. Petrova

21In manual mode, the game becomes more difficult, more technical and this constraint influences the players’ reactions. They are more concerned and try to be very precise. Above all, it seems that their pleasure and disappointment are accentuated. According to the philosopher Hubert Dreyfus, “involvement is the ideal equipment by which he (the beginner) can go further” (Dreyfus, 2010).

22By focusing on several players’ hands and interactions (Ingold, 2013), differences in their ways of holding the controller become visible. In their user experience (Nacke et al., 2018), players who consider themselves less experienced move the controller as if to gain additional control over the game. The strong bodily dimension is evident: it ranges from the development of a manual technique to the sensations and moments of bodily immersion in connection with the practice (feet or hands).

4. Narrative Immersion

23In the consideration of the narrative immersion, it is a question here of observing the attachment between the player and the 11 avatars of the chosen team. There is a certain closeness between the players and their played characters. The latter correspond to real people like Christiano Ronaldo, Neymar or Cavani. The players will quote their names and then comment on their actions showing sympathy and closeness, and sometimes a phenomenological projection on the screen. In this case, they narrate their own story.

24

Video 4 – “Proximity” by Students Master “Hypermedia and Communication” 2015–2016.
Crédits : © J. Ibanez Bueno, L. Petrova

25This proximity does not always apply. A detachment is noticed in the games played with the female teams. By detachment, we understand here a distance of the players towards the female characters. Through our sample (essentially male), we notice that when playing with characters from the female teams, the players did not adapt their physical positions (contrary to when they play with male characters). They are not engaged in the game: they stay lying on the couch, without getting involved with sexist comments.

26

Video 5 – “Detachment” by Students Master “Hypermedia and Communication” 2015–2016.
Crédits : © J. Ibanez Bueno, L. Petrova

27Thus, despite the desire of the company Electronic Arts sports to include a female team in this new opus, these players are considered by the observed players not to have enough skills to generate real interest in the game (too slow and not powerful enough in the shots). However, and only at the end of the game, when the players feel threatened to loose the game, they re-enter playing mode, lean forward and engage in the competition. While playing, some players talk. They comment on the actions and events that are taking place.

28

Video 6 – “Commenting” by Students Master “Hypermedia and Communication” 2015–2016.
Crédits : © J. Ibanez Bueno, L. Petrova

29By commenting, the player changes his role in the game: he is no longer just the player, but also appears as a sports commentator. He takes an outside perspective and behaves strategically as if he were playing a board game. He calculates and evaluates his activities while playing. Players who describe themselves as experienced show less body and emotional expressions or are less excited than those who feel less experienced. They are also more focused and less expressive than beginners, creating a situation comparable to strategy games. The “focused” posture can be seen as a way to conduct a strategy. The experienced player knows how to react intuitively and reacts less emotionally to events in the game. As Hurbert Dreyfus (2010) states, “the beginner becomes a connoisseur who intuitively sees what to do without resorting to rules or remembering the case.”

30The players’ faces also provide insight into their engagement with the game. The majority of players observed have their eyes wide open and fixed on the screen, some also play with their mouths open biting their lips. One study (Rozin and Cohen, 2003) identified these expressions as signals of activity-oriented concentration. Various external events take place while the observed subjects are playing. A player is asked about these game events while playing. The question has to be repeated several times before he decides to answer it. In addition, he does not take his eyes off the screen. The player concentrates on the game as if refusing to be interrupted, similar to a strategic game posture.

31

Video 7 – “Facial expressions” by Students Master “Hypermedia and Communication” 2015–2016.
Crédits : © J. Ibanez Bueno, L. Petrova

Conclusion

32Based on the conducted observations, we could conclude that in the context of interactive corporeality, FIFA16 has the possibility to create this “dilation of our body” that Étienne Armand Amato and Jean-Louis Weissberg (2003) describe. The “projective acting body” is put in a situation of meeting that of the avatar. The produced results move away from the hypothesis of a kind of reproduction of the soccer movement (Hunter, 2000), making FIFA16 a “reality” game. FIFA16 suggests limited bodily involvement and a high level of concentration.

33However, it is the strategic type of games that rely primarily on focus, anticipation and the acquisition of effective knowledge of the game as well as self-awareness. This is a case similar to classic board games such as chess. This is why these postures and behaviours correspond to our observations of FIFA16 players. In this sense we propose a continuation of this research on the strategic dimension. The new hypothesis would become: Is FIFA16 strategy game?

34By its final visual and interactive format, the results of the research are not only presented in a traditional written form, presented in scientific conferences (Ibanez Bueno and Petrova, 2017), but also in an interactive form. In the first version, a website was created. Following this first version, a more complex production was developed in the form of an ethnographic video game. Although modest in its interactive dimension compared to commercial video games, the ethnographic game allows not only access to written results of the research and visual results in photographic or video form (which an academic website allows as well) but also a video game manipulation. The access to the informational elements is conditioned to an obligatory playful moment, asking an implication to the user by an interactive corporality. Curious to access knowledge about online video games, the user must himself practice and become a “projective acting body”. He must minimally feel a certain number of physical sensations linked to the manipulation of the device such as the vibrations related to the touch of the interface. This bodily experience is linked to the results of this research on the bodily dimension of the FIFA game. The superposition of the semiotic modalities and the modalities of use seems to us highly fertile in terms of transmission and assimilation of the content of the research. Faced with the youth of the medium and its mass success, this superposition facilitates the taking into account the unspeakable, the sensory, the affective and the “in the making”.

Haut de page

Bibliographie

AMATO Étienne Armand, WEISSBERG Jean-Louis, (2003), “Body meets interactivity: interface, narrativity, gesture”, in AKTYPI Madeleine, LOTZ Susanna, QUINZ Emanuele (ed.), Interfaces, Orléans, Éditions HYX (Anomalie digital_arts, 3).

BOULDOIRES Alain, REIX Fabien (2017–2018), “Édito-Manifeste”, Revue française des Méthodes visuelles, 1–2, [online] https://rfmv.u-bordeaux-montaigne.fr/numeros/1/manifeste/

BIANCHI-BERTHOUZE Nadia (2013), “Understanding the Role of Body Movement in Player Engagement”, Human–Computer Interaction, 28 (1), p. 40–75, [online] https://doi.org/10.1080/07370024.2012.688468

CALLEJA Gordon (2011), In-game: from immersion to incorporation, Cambridge, MIT Press.

DA SILVA RIBEIRO Jose, BAIRON Sérgio (2007),Antropologia Visual e Hipermedia, Porto, Ediçoes Afrontamento.

DREYFUS Hubert Lederer (2010), “Une phénoménologie de l’Acquisition de l’habileté comme base d’une anti-représentionisme Merlau-Pontien en sciences cognitives”, in ANDRIEU Bernard (ed.), La philosophie du corps. Expériences, interactions et écologie corporelle, Paris, Librairie Philosophique J. Vrin, p. 136–158.

HUNTER Jeremy, CSIKSZENTMIHALYI Mihaly (2000), “The Phenomenology of Body-Mind: The Contrasting Cases of Flow in Sports and Contemplation”, Anthropology of Consciousness, 11 (3–4), p. 5–24, [online] https://doi.org/10.1525/ac.2000.11.3-4.5

IBANEZ BUENO Jacques, PETROVA Lilyana Valentinova (2017), « FIFA 16 : un babyfoot en ligne ? De la pratique footballistique au jeu vidéo » in Sport et jeu vidéo, 2ème semaine internationale du corps Université Paris Descartes, 28 juin 2017.

INGOLD Tim (2013),Making. Anthropology, archaeology, art and architecture, Oxon-New York, Routledge.

MANNING Erin (2016), The Minor Gesture, Durham-London, Duke University Press.

MERLEAU-PONTY Maurice (1945),Phénoménologie de La Perception, Paris, Gallimard.

MERLEAU-PONTY Maurice (1961),L’œil et l’esprit, Paris, Gallimard.

MCLUHAN Marshall (2003),Understanding Media. The Extensions of Man (1964), Corte Madera/CA, Gingko Press Inc.

NACKE Lennart, DRACHEN Anders, MIRZA-BABAEI Pejman (2018), Games User Research, Oxford, Oxford University Press.

PINK Sarah (2009), Doing Sensory Ethnography, London, Sage.

ROZIN Paul, COHEN Adam B. (2003), “High Frequency of Facial Expressions Corresponding to Confusion, Concentration, and Worry in an Analysis of Naturally Occurring Facial Expressions of Americans”, Emotion, 3 (1), p. 68–75, [online] https://doi.org/10.1037/1528-3542.3.1.68

RUBY Jay (1996), “Visual Anthropology”, inLEVINSON David, EMBER Melvin (ed.), Encyclopedia of Cultural Anthropolog y, 4, p. 1345–1351, [online] https://carmenguarini.files.wordpress.com/2007/11/ruby-visual-anthropology-in-ency.pdf

SUCARI Jacobo (2012), El documental expandido: pantalla y espacio , Barcelona, Universitat oberta de catalunya.

Haut de page

Notes

1 Navigation in the ethnographic game available here.

Haut de page

Pour citer cet article

Référence électronique

Jacques Ibanez Bueno et Lilyana V. Petrova, « From video game to ethnographic video game »Revue française des méthodes visuelles [En ligne], HS1 | 2024, mis en ligne le 30 septembre 2025, consulté le 06 novembre 2025. URL : http://journals.openedition.org/rfmv/953 ; DOI : https://doi.org/10.4000/12mq7

Haut de page

Auteurs

Jacques Ibanez Bueno

LLSETI, Université Savoie Mont-Blanc

Lilyana V. Petrova

ETIS (CNRS/ENSEA/CY Univ.)

Haut de page

Droits d’auteur

CC-BY-NC-SA-4.0

Le texte seul est utilisable sous licence CC BY-NC-SA 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont susceptibles d’être soumis à des autorisations d’usage spécifiques.

Haut de page
Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search