Skip to navigation – Site map

HomeNuméros9DossierBetween the Photograph and the Fr...

Dossier

Between the Photograph and the Frame. The Fate of the Single Image in the Algorithmic Era

Entre la photographie et le photogramme. Le destin de l’image unique à l’ère des algorithmes
Barbara Grespi
Translation(s):
Entre la photographie et le photogramme. Le destin de l’image unique à l’ère des algorithmes [fr]

Abstracts

The use of deep-learning algorithms in the production of photographs has challenged a key distinction within media images: that between the photograph and the frame. Theory has often emphasized the non-coincidence of these two modes of the photographic, one incomplete and lacking its own visibility, the other autonomous, unique, and finite. But today, the intervention of algorithms at the stage of image capture weakens this distinction. Even the single photographic snapshot becomes a “vertical” concentration of several images, while an apparently fixed photograph can conceal a “horizontal” series of images that also allow the picture to be viewed as a clip. In the face of these “dense” snapshots, which no longer correspond to human perception and imply a very different dialectic between trace and visualization, singular and plural, snapshot and series, does it still make sense to juxtapose the photographic object and the photogrammatic material used within numerous types of imaging? Or should the photographic be redefined as a genetically serial process?

Top of page

Full text

  • 1 Roland Barthes, Camera Lucida: Reflections on Photography, trans. Richard Howard (New York: Hill & (...)
  • 2 Marta Braun, Picturing Time: The Works of Étienne-Jules Marey (Chicago: The University of Chicago P (...)
  • 3 This is precisely how the history of art has narrated photography, and this narrative has been acco (...)
  • 4 Bruno Di Marino, Pose in movimento: Fotografia e cinema (Turin: Bollati Boringhieri, 2009), 34.

1In the most enduring and influential of its interpretations, the photographic gesture corresponds to the fixation of a visual act. As the crystallization of a fleeting and punctuating perception from a stream of visual stimuli (Barthes), a “grammar and . . . an ethics of seeing” (Sontag), or the selection of a sight from “an infinity of other possible sights” (Berger), representational photography has been the technical image that has most taken on the task of materializing the human gaze in the twentieth century.1 The possibility of registering the gaze in its absolute instantaneousness was also a technological goal and an aesthetic principle of modernity; late nineteenth-century research into increasing the speed of the shutter, chasing the smallest photographic fraction of a second,2 and the birth, a few decades later, of photojournalism in the myth of the “decisive moment” consolidated the idea of photography as a unique and whole image that unites observer and observed at a precise point in time, by crystallizing an unrepeatable configuration of reality, magically frozen and deprived of duration.3 However, the full achievement of this aesthetic aim through the maximum chronophotographic punctual temporality coincided with the erasure of photography within the new language and art of the moving image. Photography in film, or cut loose and rendered fluid in the medium of film, becomes qualitatively different: an alignment of frames closely anchored to what precedes and follows them, unlike photographs, which remain unique and autonomous even when they are part of a series, or even a sequence.4

  • 5 Gilles Deleuze, Cinema 1: The Movement Image, trans. Hugh Tomlinson and Barbara Habberjam (London: (...)
  • 6 Jacques Aumont, L’Oeil interminable: Cinéma et peinture (Paris: Séguier, 1989).
  • 7 Garrett Stewart, “Photogravure: Death, Photography and Film Narrative,” Wide Angle 9, no. 6 (1987): (...)

2This distinction has been the subject of intense debate since the 1980s and has played a role in key contributions to film theory: at the beginning of the decade, it contributed to the Deleuzian idea of cinema as “movement-image,” a “mobile section” (coupe) to be contrasted with photography’s “immobile section.”5 Jacques Aumont derives from this a rigid opposition between instantaneousness and duration, filmic time that flows even in an apparently “still and flat” frame and photographic time that freezes the instant.6 In the same years, Garrett Stewart introduced the concept of the “photogrammatic” by reworking a central idea of so-called apparatus theory, which, in the 1970s, interpreted the illusion of movement as an ideological effect of the cinematic machine.7 His analyses of film, centered on the redemption of the photographic matter that forms the basis of cinema, have found a counterpart in that branch of contemporary art which, in the 1990s, deconstructed the moving image through various interventions that boycotted the strip of film in order to reveal the concreteness of the hidden frames and redeem the materiality buried beneath the imaginary (from Douglas Gordon to Steve McQueen, from Yervant Gianikian and Angela Ricci Lucchi to Martin Arnold).

  • 8 Raymond Bellour, “L’interruption, l’instant,” in L’Entre-Images: Cinema, Photo, Cinéma, Vidéo (Pari (...)
  • 9 Roland Barthes, “The Third Meaning,” Image Music Text, trans. Stephen Heath (London: Fontana Press, (...)
  • 10 Christa Blümlinger, “The Figure of Visual Standstill in R. W. Fassbinder’s Films,” in Between Still (...)
  • 11 Bellour, “Concerning ‘the Photographic,’” 261.

3A specific theory of the intricate exchange between the filmic and the photographic, intensified by the digital, was developed by Raymond Bellour, who has been reflecting, since the mid-1980s, on what he terms the entre-image.8 For Bellour, the “photographic” is a state of “in-betweenness,” a dimension that can only emerge in an interrupted movement or in a utopian stillness constantly haunted by a surrounding movement. Bellour draws on Barthes’s insights into the “third sense” and, in particular, his idea of the photogram not as raw photographic material devoid of aesthetic interest but rather as a paradigmatic image that participates in the film in a mysterious way, by subtraction and on the wave of chance. “The still,” he wrote, “is not a sample (an idea that supposes a sort of homogeneous, statistical nature of the film elements) but a quotation. . . . It is not a specimen chemically extracted from the substance of the film, but rather the trace of a superior distribution of [its] traits.”9 As a “random card . . . taken from the deck in search of an additional significance,”10 the frame condenses the spirit of the film precisely because it frees it from operative time, which creates the effect of movement, to make the mysterious interior of its images accessible. Bellour goes further, developing an aesthetics of confusion—i.e., pursuing the multiple states that the image takes on between film, video, and photography, against the specificity of the media that the digital has completely overtaken (and from which discourses on art have never particularly benefited).11

  • 12 Laura Mulvey, Death 24× a Second: Stillness and the Moving Image (London: Reaktion Books, 2006).

4The digital revolution at the end of the millennium, however, also reopens the question in another sense. Laura Mulvey’s Death 24× a Second: Stillness and the Moving Image (2006) rethinks the frame in the age of the VCR and DVD, celebrating its new visibility.12 For viewers of DVDs and MPEG files, film and photography merge in the practice of screen capture and the perfect freeze-frame; with the introduction of the technological tools of time manipulation, the secret particles of a film have become easily extractable and observable: images to be grasped, which the possessive viewer can appropriate. The frame becomes the part of the film that can be handled, touched, grabbed, as opposed to an intangible element in the optical flow.

  • 13 Mario Carpo, The Second Digital Turn: Design Beyond Intelligence (Cambridge, MA: The MIT Press, 201 (...)

5But shifting our attention from film to photography and, in particular, to the photography of the second digitization,13 which is heavily impacted by artificial intelligence, we are faced with a new stage in this dialectic. In a sense, algorithmic photographs are genetically also photograms. They come in series (linear, volumetric, or “vertical”), but conceal their composite plurality.

  • 14 Juan Fontcuberta, La furia de las imágenes: Notas sobre la postfotografía (Barcelona: Galaxia Guten (...)

6Even the simplest snapshot we take with our smartphones is a “thick” photograph, containing subsequent data captures, that is to say, several “frames.” It no longer corresponds to the recording of the photographer’s gaze, made possible by his readiness and ability to react to a sudden configuration of reality. It no longer preserves the perceptive act in the form of a trace of limitations that are technological but also physiological, such as the imperfect focusing of certain elements and the imprecise interruption of movement. This is already the case with the use of “simple” algorithms (i.e., not enhanced by AI), and to a greater extent with deep-learning algorithms that completely remove the signs of human limits even in the common practice of smartphone photography, by its very nature modeled on the immediacy of a now-naturalized gesture.14

Frames in Photography

  • 15 Bogdan Ionescu, Wilma A. Bainbridge, and Naila Murray, eds., Human Perception of Visual Information (...)

7The correction of the traces of human agency is already taking place as the index finger is pressing the shutter release: instead of recording the qualities and limits of an act of vision, the digital click delivers the shutter to the algorithms that optimize our way of looking by controlling the dynamic range. Dynamic range (DR) in photography is the ratio between the saturation point (the highest brightness a camera can capture) and noise (the lowest brightness it can capture before the signal disappears). This ratio is measured in f-stops, a unit that describes the difference between the lightest and darkest parts of a scene in powers of two. This means that in an image with a dynamic range of 4 f-stops, white is 24 (i.e., 16) times brighter than black. The human gaze naturally wanders, and the pupil constantly adjusts its aperture, adapting to the brightness of the different areas of the field of view; therefore, the brain mentally synthesizes all gazes to produce an image with a dynamic range that is potentially very high. However, when the eye dwells on a subject it reaches a DR of about 10 f-stops,15 very close to that of a good sensor, but not enough to focus on every part of a visible scene, which can have a range of up to 27 f-stops. Taking a photograph that is perfectly sharp and in focus therefore means dismissing the model of the human eye, overcoming its capabilities.

  • 16 Marco Fodde, “Mascherare o bruciare?,” Fotografia Reflex (November 2002): 63.
  • 17 See “Images composites,” Transbordeur: Photographie, histoire, société 7 (2023).

8Photochemical photography challenged this threshold of perception by resorting to retouching during positive printing—i.e., by making various exposure corrections that were often entrusted to the photographer’s hands, used as a mask to calibrate the different parts of the image.16 But the best-known and most anticipatory experiment is certainly the ciel rapporté of painter-photographer Gustave Le Gray (La Grande Vague, 1857), the result of the composition of two wet collodion negatives taken at different exposures, one to capture the brightness of the moving sea and the other to capture the translucence of the sky. More generally, recent studies have reconsidered the history of photography, recovering a significant line in which composition is primary to the recording of the signal, from the first photomontage to the retouching of the digital age.17

9In fact, with Photoshop, compositing different “negatives” in the same image became a common postproduction practice, but only with the algorithmic breakthrough did this “montage” become integrated with image capture. MEF (Multi-Exposure Image Fusion) algorithms are computational structures that compare several images by “stacking” them on top of each other in order to handle large differences in brightness and make the details of darker and lighter parts perfectly sharp.

  • 18 Fang Xu et al., “Multi-Exposure Image Fusion Techniques: A Comprehensive Review,” Remote Sensing 14 (...)

10MEF algorithms intervene in the HDR (High Dynamic Range) system activated by default on smartphones and presented as the basic mode of today’s photography. In HDR, each shot corresponds to at least three successive exposures, and this applies to every snapshot that appears on our screen. The snapshot acquires “depth,” becomes a virtual image, which does not correspond to the recording of a real look but shows how reality might appear to us if we could focus on its parts in three different ways, with three different simultaneous eye movements. The HDR system is the first of the most common algorithmic processes for serializing photography that are today enhanced by artificial intelligence (i.e., more powerful algorithms capable of deep learning). Regardless of the type of images they fuse—visible, infrared, multi-focus, and multi-exposure images—MEF algorithms are differentiated into three types: those operating in the spatial domain, in the transform domain, and through deep-learning methods.18

11Algorithms of the first type, which operate in the spatial domain, fuse the source inputs directly into the image plane after generating a map of weights for each input image. The working unit is the pixel: each pixel is assigned a weight (i.e., a luminance value) and the fused image is calculated as a weighted sum of the luminance values of each pixel. In other cases, the basic unit is the patch, a region of the image with a given step (i.e., distance between bytes). Patches at the same position in each image in the series are compared, and the patch containing the most significant information is selected to form the final fused image. Finally, if they do not work on pixels or patches, spatial domain algorithms proceed by optimization: the fusion becomes a probabilistic estimate of the global optimum solution. On the other hand, algorithms of the second type, based on the transform domain method, are a form of computational analysis of images that allows the coefficients of their formal characteristics to be obtained, compared, and merged to create a new image with all the best coefficients. This analysis is based on the transformation of the initial images by means of mathematical functions (such as the Pyramid transform), just as the synthesis of the final image employs other mathematical functions that go from the calculated coefficients back to the whole representing them.

  • 19 Jinhua Wang, Xuewei Li, and Hongzhe Liu, “Exposure Fusion Using a Relative Generative Adversarial N (...)
  • 20 Antonio Somaini, “Film, Media and Visual Culture Studies and the Challenge of Machine Learning,” NE (...)

12Finally, the MEF was optimized using deep-learning algorithms,19 variously based on vision machine techniques, including convolutional neural networks (CNNs) and generative adversarial networks (GANs), on whose description and interpretation there is already a significant body of reference literature.20

13In the case of deep-learning algorithms for MEF, image enhancement is therefore “reasoned” according to what the networks “see” (recognize) in them. With GAN-MEF, the fused image is obtained by a generating network from a pair of short and long exposure images, which is then evaluated by the discriminator. With CNN-MEF, the neural network is trained with indirect systems to extract a feature map from the images; local visibility maps are created for each of them, from which the fusion weight map is then determined.

14These systems have been in use on smartphones for a few years. On iPhones, the Deep Fusion system replaces HDR algorithms with neural networks, using a process that not only enhances each shot from a micro-stream of video but also optimizes it according to recognition systems. With Deep Fusion, the shot captures three different images: the first is a series of four frames at full shutter speed, necessary to perfectly freeze motion; the second is a series of four standard frames taken at normal speed; and the third is a longer exposure shot used to capture even the smallest details. The standard shot (the normal-speed series) is then merged with the long-exposure shot to create an image called a “synthetic long”; then the short-exposure frame with the most detail is selected and merged with the synthetic long. Neural networks assess how to combine these three differently “thickened” images, pixel by pixel, by identifying the main elements that compose them, each of which receives a specific treatment.

  • 21 Eivind Røssaak, “Algorithmic Culture: Beyond the Photo/Film Divide,” in Røssaak, Between Stillness (...)

15What with the basic algorithms was still a kind of calculated superimposition becomes, with the neural networks, a deep and transversal blending: a crumbling and recompositing of the frames of a linear micro-series into the characteristics of a static object. The photographic object thus comes to possess a multiple temporality, different from the instant; the photographic instant is “created” by compressing a duration, however minimal, and even photography becomes “a new (non)ground for unprecedented spatio-temporal explorations.”21

16Conversely, other deep-learning algorithms make it possible to create a short-duration video from a single photographic instant. Mulvey’s possessive cinephile, who interrupted the visualization of video data in order to appropriate a frame of the film, has their counterpart in today’s amateur photographer who, thanks to artificial intelligence, is able to bring their own snapshots to life in order to transform them into short videos.

  • 22 Somaini, “Algorithmic Images,” 99–100.

17In some experimental software, it is already possible to derive from a single image a linear series created through a verbal description of the desired movement. The production of images from text, a frontier of AI intervention on photography, recently discussed by Antonio Somaini, is based on latent diffusion models (LDM), which work in two stages: in the first, the initial images are transformed into neutral pixel surfaces (without recognizable shapes) through the progressive addition of noise—i.e., by redistributing pixels from high to low areas (forward diffusion). In the second, the network learns to calculate the amount of noise added in order to remove it and return from this state of neutrality to the original figure (reverse diffusion).22

18Although still at an experimental stage, a smartphone application called Pix2Gif based on the latent diffusion model has been available in the Apple App Store for some time now.23 Pix2Gif requires three inputs: a photograph, a textual description (of the desired motion), and a value for the magnitude of the motion (to be generated through a graphics-processing program). The training dataset is public:24 Tumblr GIF (TGIF) is an archive for research purposes that collects 100,000 GIFs (taken from Tumblr and from posts found on the Internet) and 120,000 sentences describing their visual content. The software works on this material basically by extracting the individual frames of a GIF and calculating the optical flow of the movement that characterizes it. It is then the optical flow data that is associated with the verbal description contained in the database, and from this association it becomes possible to derive other correspondences, such as that between the prompt description and the movement to be created.

19Pix2Gif appears like a neural version of the LIVE mode set by default on most smartphones. But LIVE photography is actually a mode (i.e., a mere display option), although the fact—one that is anything but trivial—is that through the photographic gesture (the single click) one actually captures a three-second video. LIVE photographs are both .mov and .jpeg files, transformed into each other by a compression algorithm. The correspondence between clip and still image is determined by a calculation of the “best” frame, which is often the one that lasts the longest, the most static, and/or the sharpest.

  • 25 Ken Timby, “‘Cinema in a Single Photo’: The Animated Screen Portrait of the 1910s,” in Between Stil (...)

20The LIVE mode is in turn the digital version of a series of early twentieth-century practices situated between photography and film well represented by the “animated portrait,” typical of the photography of the 1910s.25 The animated portrait was a manual and analogue version of the GIF image: what was (apparently) a photograph of a face was transformed into a movie of facial expressions by handling the support, bending it or pressing it; in this way, the vertical stripes into which the three overlapping frames were divided to form the portrait were combined differently, creating an effect of motion.

21However, the idea of the photograph as an image genetically tied to a single point in time is not really challenged by early twentieth-century filmed portraits and contemporary LIVE photographs, which simply show how our hands in one case, the device in the other, can act as an optical machine, running a series of frames, unfolding their overlap, or simply playing the video stream. The LDM techniques of video generation from the single image, on the other hand, represent a real challenge to the temporality of photography: they are able to infer moments after the one captured by the snapshot, transforming the photograph into a frame and inaugurating a different genesis—statistical and predictive—of the cinematic from the photographic, of the frame or series of frames from the photograph.

Towards the 3D Snapshot: Frames and Photogrammetry

22The temporality of photography is complicated, finally, by the emergence of the three-dimensional snapshot, the ultimate horizon of photo-telephony. The three-dimensional snapshot is in itself a temporal paradox: it implies the capture of several simultaneous images from different points of view by a single camera. The aim is not to extend the moment in order to capture more than one image in the time of a click but to simulate the simultaneity of what are actually successive shots.

  • 26 Paul Zammit, Guillem Carles, and Andrew R. Harvey, “Three-Dimensional Imaging and Ranging in a Snap (...)

23For some years, iPhones have been equipped with an extra back camera strategically placed at a distance from the others, which can be used for producing 3D shots. While 3D photography has never taken off as an artistic form or as an amateur practice, either in stereoscopic form or in the digital form allowed by Photoshop, in the new media scenario of extended realities, characterized by photorealistic environments to be inhabited and traversed with the body, the three-dimensional snapshot has once again become a strategic objective.26

24Many software packages already allow 2D shots to be transformed into 3D. At Apple, users mainly exploit PopPic, which uses the dual camera placed on the most recent iPhone to capture the image of the desired object from two viewpoints that are slightly offset from one another. By activating the cameras through this software, all the data necessary to model the scene captured by the two photographs in 3D can be extracted; the “drawn” model is then coated with the photographic “skin” from which it was derived and displayed: simply tilt the screen to get the impression of walking around the photograph (and to glimpse the boundaries of this surface).

  • 27 Barbara Grespi, “Archaeology of the Postphotographic Image: Photogrammetry, Photometry, and the Pos (...)
  • 28 Alexander R. Galloway, “Polygraphic Photography and the Origins of 3-D Animation,” in Animating Fil (...)

25The underlying principle of PopPic is that of photogrammetry, a technique originating in the nineteenth century that consists in recovering the volumetric dimension of spaces and objects from two-dimensional photographs, the perspective construction of which can be inverted in a certain sense by going back to the calculations and proportions applied to obtain it. The method expresses an idea of the photographic as a technique for measuring, quantifying, and ultimately datafying the world that while remaining subordinate in the twentieth century to the representative conception of the medium, confined within the perimeter of science (topography, astronomy, architectural surveying),27 has become in the new century one of the cornerstones of artistic and non-artistic image making. Harun Farocki rediscovered the photogrammetry devised by the Prussian engineer Albrecht Meydenbauer as the archaeology of the digital in his “essay film” Images of the World and the Inscription of War (1988), in which the use of photography as an instrument for taking possession of, colonizing, preserving, and simultaneously destroying the world is central and derives directly from its ability to translate itself into data, even in the analog era. A different archaeological root of photogrammetry was brought to light by Alexander Galloway, who identified an ancestor of current 3D printing practices in the late nineteenth-century photosculpture of François Willème, based on a series of photographs.28 In order to obtain a volumetric model of his chosen subject, Willème embedded twenty-four cameras in the circular walls of his domed studio and aimed them at the seated model in the center, producing his archaeo-photogrammetric series with two synchronous shots each taken from twelve different viewpoints. Although not guided by the imperialist spirit that Farocki recognizes in Meydenbauer, Willème's photogrammetry did not understand photography as the recording of a sensorial likeness to be imitated by art but rather as a method for creating an archive of exploitable measurements that would allow the exact modeling of the object.

  • 29 Chris Chesher, “Between Image and Information: The iPhone Camera in the History of Photography”, in (...)

26The advent of digital technology has drastically simplified the complex technique of analogue photogrammetry, making this system of massive serialization of the photographic image one of the main sites for rethinking the photographic, and its temporal dimension, in particular, further complicated, as we shall see, by the combination of photogrammetry and neural networks. Photogrammetry software makes it possible to process exponentially more images, and to automatize the calculations required to join the parallaxes and merge the frames. The process is predominantly desktop, although there is no shortage of mobile applications. Reality Capture and Metashape are the most widely used desktop software solutions in these fields, while Reality Scan is a smartphone version: it works on any iPhone or iPad device that supports at least iOS 16.0, increasing the many software products that, since 2008 and the creation of the App Store, have transformed photography into a technique that goes far beyond optics and light conversion to become a vast and articulated image-making practice.29

  • 30 Lev Manovich, “Media after Software,” Journal of Visual Culture 12, no. 1 (2013): 30–37.

27Compared to PopPic, which aims to automate the stereoscopy in the shot, drastically reducing rendering times (and the quality of the effect), Reality Scan remains a postproduction procedure that reinforces the “software-ized” character of post-photography.30 It requires a very precise shooting technique, reminiscent of the burst mode in photojournalism: the operator has to click rapidly and continuously while rotating around the chosen subject and holding the mobile phone at a stable height and distance, in order to obtain homogeneous frames that can be welded together. If the subject is not a living being, this is enough to simulate the simultaneity of the shots; conversely, if the subject is a living being, each of its small movements reveals the passage of time and exposes the false simultaneity of the photographic series, resulting in small figurative deformations. These deformations mainly concern the photographic “skin” of the 3D image—i.e., its coating—which represents the final stage of the photogrammetric process. Like PopPic, Reality Capture processes the proto-simultaneous series of photographs to derive a set of spatial coordinates, agglomerated in the so-called Point Cloud. The number and precision of the starting photographs determine the density and legibility of this cloud, whose points are then thickened, thanks to the fusion work of the algorithms, until they form a model of the photographed subject. The obtained model is, so to speak, “naked,” and in the subsequent meshing phase it must be coated with a texture: at that point, the source photographs become useful a second time thanks to their ability to mimic the visible and can be used to achieve a photorealistic effect. But until then, the photographs have only represented a repository of geometric and positional data (the metadata), dimensional measurements, and, above all, the distances of the elements from each other and from the viewpoint from which they were captured. But even in cases where a non-photographic meshing is chosen and the object created appears as a mere computer graphics simulation, photography remains, deep down, the structuring procedure that provides the indices to anchor the representation.

28In order to understand this specific role of the photographic in contemporary media objects, it is useful to analyze the successful experimentation with photogrammetry represented by Cilia (2021), a VR work by media artist Sara Tirelli based almost entirely on the creation of an immersive environment in 6DOF (six degrees of freedom, made functional by moving through space) using photogrammetry software. Tirelli interprets immersive media not so much as the culmination of a media history of optical illusion but as the dawn of new possibilities for mediating space with all the senses, transforming the new devices into techniques for capturing environmental vibrations. Cilia is a form of virtual installation, conceived as site specific: when it is “moved,” it requires a new rendering—i.e., the photogrammetric reconstruction of the new destination space. For the presentation at the Palazzo del Novecento in Milan, Sala Fontana was recreated with Reality Capture (figs. 1–6), using 100 photographs, then covered with a texture inspired by organic matter (the cilia, thin outgrowths of certain animal or vegetable cells, usually capable of beating rhythmically and producing locomotive currents). From inside the helmet, the user recognizes the room and orientates themself in it, while experiencing it in an unprecedented way, seeing it beyond the surface and perceiving it as a living body that senses our presence and responds to our gestures. In this sense, this work is an anti-photographic experience, within which, however, the chain of frames plays a central role: their hidden operation makes VR a kind of X-ray machine capable of visualizing not so much the skeleton of the world as something even deeper, which lies within its cells.

1. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021

1. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021

Screenshot of the RealityCapture photogrammetry process.

© courtesy of the artist, Sara Tirelli

2-3. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021

2-3. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021

Screenshot of the RealityCapture photogrammetry process.

© courtesy of the artist, Sara Tirelli

4-5. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021

4-5. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021

Screenshot of the Metashape photogrammetry process.

© courtesy of the artist, Sara Tirelli

6-7. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021

6-7. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021

Screenshot.

© courtesy of the artist, Sara Tirelli

  • 31 Fabio Remondino et al., “A Critical Analysis of NeRF-Based 3D Reconstruction,” Remote Sensing 15, n (...)

29The “radiographic” aptitude of photogrammetry is accentuated by the use of artificial intelligence, which also represents a significant boundary in this field, determined by a further complexification of the temporality of the image. If the basic Reality Scan algorithms distill the object from the photographs, discarding the surroundings, the NeRF (Neural Radiance Field) algorithms employed in other 3D photorealistic modeling software treat the photographed scene as a continuous volumetric field, from which they derive a set of 5D coordinates, consisting of three spatial positions (x, y, z) and two viewing directions.31 From these inputs the neural network is able to estimate other gaze perspectives to generate new visual information that enriches the views of the object. This means that the starting photographic series is implemented with other photographs, or rather other “photographic” data, that correspond to potential shots but not ones that have actually been captured. In some cases, a single photograph may be sufficient for a “neural representation,” as the 3D space that is generated from it is also called: by retrieving photographs corresponding to what the neural networks recognize around the object from the algorithms’ training dataset, the NeRFs expand the representation beyond what the source images recorded. For example, if the leaves of a tree are barely visible, the algorithms will be able to complement them from the tree they recognized on the basis of these leaves by searching the database to find it as a single specimen (whose image might exist thanks to geographic imaging) or as a class (using appropriately recalculated images of any tree). This is not exactly falsifying the photographic: the rendered object is really there, but what appears is not its presence there at that time but its presence there at other times or an iconic representation of its very plausible presence. In the series that feed the neural representations, “ground truth” frames are placed side by side with virtual ones, which have never possessed the status of images; this shifts the terms of our starting question from the dialectic between singularity and plurality to the distinction between different modes of the plural—e.g., “primary” series and mixed series. Mixed series allow the software to work with much less direct input, and in this way the data processing becomes much faster. This is why artificial intelligence plays a decisive role in 3D snapshot research.

  • 32 “NVIDIA Instant NeRF: NVIDIA Research Turns 2D Photos into 3D Scenes in the Blink of an AI,” posted (...)

30Speed and simplicity in the process is the goal of the NVIDIA Instant NeRF prototype launched in March 2022 and presented as a lightweight software that makes 3D photography possible from a handful of snapshots. The product demo quotes a famous photograph of Andy Warhol with his beloved Polaroid in his hands: in the video, a woman who looks like him and is also holding a vintage instant camera is photographed from four different angles; the resulting photos are merged and a perfect 3D rendering of the photographer in the studio is generated on the screen. “75 years ago, Polaroid made it possible to see a photograph in minutes,” the captions read, “now AI transforms 2D photos into 3D scenes in seconds” (fig. 7).32 The advertising strategy presents neural representation as an innovation in continuity with the history of photography, particularly its amateur practices, from Polaroid to smartphone photography. But the real goal is to ferry photography to extended realities and prepare the ground for its replacement by virtual images so that it sounds like a natural development of the technology.

8. Sara Tirelli, Cilia, virtual reality installation presented at the Espronceda Institute of Art and Culture, Barcelona, 2021

8. Sara Tirelli, Cilia, virtual reality installation presented at the Espronceda Institute of Art and Culture, Barcelona, 2021

Screenshot.

© courtesy of the artist, Sara Tirelli

The Photographic Illusion

31The comparison between photographic procedures characterized by the use of basic algorithms, which optimize calculation functions and measure and compare large quantities of shots, and procedures based on deep-learning algorithms, which, by recognizing and evaluating the characteristics of images, derive new ones, has sought to clarify a specific dimension of the contemporary photographic image: its nature as a serial object. Today, even the simplest photographic gesture does not capture a single set of data (one digital photograph) but captures a plurality of them, and their fusion, completion, and transformation, used in different types of visualization, constitute one of the fronts on which artificial intelligence acts and experiments.

32Algorithmic photography, characterized by composite sharpness, the ability to animate and expand in 3D, is the result of an invisible welding between chains of images, linear or simultaneous, stitched together by the transparent thread of calculation. Photography, then, as a synthesis of frames—if we extend the term to include all photographic images made in series and for a series—and a practice of their concealment within an aesthetic that reverses course with respect both to the electronic experimentation of the end of the last century, based on the emergence of the photogram from the flow (the Bellourian entre-image), and to the era of possessive screen capture triggered by the digital (in the cinefetishism described by Mulvey).

33Also to be considered is the hybrid nature of the chain of frames behind the dazzling surface of an algorithmic photograph, often composed of inputs both recorded by the camera and generated by neural processes (e.g., LDM or NeRF), which are able to deduce and create the preceding, successive, or geometrically contiguous frame of a series. These virtual frames—not only invisible but not directly derived from the visible—are mixed with captured frames to produce more extensive forms of photographic visualization that simulate a perceptual continuity which does not actually exist.

34In conclusion, the constitutive plurality of algorithmic photography transforms the experience of the photographic, giving it an illusory character similar to that which has always characterized cinema: the apparent continuity of movement produced by the flow of film frames, hidden in the stream, now finds a counterpart in the apparent continuity of the photographic form, also based on the concealment of the multiple shots that compose it.

  • 33 Joanna Zylinska, AI Art: Machine Vision and Warped Dreams (London: Open Humanities, 2020), 106.

35While representing an analogous logic of construction of the visible through the invisible, of the whole through the concealment of the parts, of the continuous through the fusion of the discontinuous, the photographic illusion expresses a different idea of the image: if modern optical toys celebrated the enchantment of the gaze and the “frenzy of the visible,” according to Linda Williams, algorithmic toys celebrate the contingency of image states and their decoupling from the act of seeing. As Joanna Zylinska writes, “Photographs cannot be treated as discrete entities because they are part of the larger technological network of production and perception: they are both objects to be looked at and vision-shaping technologies.”33 Photography, now a genetically plural image, becomes a transitional state and a point of access to every other iconic realm, between and beyond media.

Top of page

Notes

1 Roland Barthes, Camera Lucida: Reflections on Photography, trans. Richard Howard (New York: Hill & Wang, 1981; Susan Sontag, On Photography (New York: Picador, 1977), 3; John Berger, Ways of Seeing (London: British Broadcasting Corporation and Penguin Books, 1972), 10.

2 Marta Braun, Picturing Time: The Works of Étienne-Jules Marey (Chicago: The University of Chicago Press, 1992); Phillip Prodger, Time Stands Still: Muybridge and the Instantaneous Photography Movement (Oxford: Oxford University Press, 2003), 25.

3 This is precisely how the history of art has narrated photography, and this narrative has been accompanied by various philosophical discourses, including Vilém Flusser, Towards a Philosophy of Photography, trans. Anthony Mathews (London: Reaktion Books, 2000). However, we know that the photographic practices were indeed different in photojournalism too, as in the case of the Magnum photographers who used to produce not one perfect single shot but several contiguous images from which to choose. I thank Estelle Blaschke for reminding me of this.

4 Bruno Di Marino, Pose in movimento: Fotografia e cinema (Turin: Bollati Boringhieri, 2009), 34.

5 Gilles Deleuze, Cinema 1: The Movement Image, trans. Hugh Tomlinson and Barbara Habberjam (London: Continuum, 1986), 24.

6 Jacques Aumont, L’Oeil interminable: Cinéma et peinture (Paris: Séguier, 1989).

7 Garrett Stewart, “Photogravure: Death, Photography and Film Narrative,” Wide Angle 9, no. 6 (1987): 11–31; see also Garrett Stewart, Between Film and Screen: Modernism’s Photosynthesis (Chicago: University of Chicago Press, 1999), 27–73.

8 Raymond Bellour, “L’interruption, l’instant,” in L’Entre-Images: Cinema, Photo, Cinéma, Vidéo (Paris: Édition de la Différence 1990), 75–80; updated in Raymond Bellour, “Concerning ‘the Photographic,’” trans. Chris Darke, in Stillmoving: Between Cinema and Photography, ed. Karen Beckman and Jean Ma (Durham, NC: Duke University Press, 2008), 253–76.

9 Roland Barthes, “The Third Meaning,” Image Music Text, trans. Stephen Heath (London: Fontana Press, 1977), 67.

10 Christa Blümlinger, “The Figure of Visual Standstill in R. W. Fassbinder’s Films,” in Between Stillness and Motion: Film, Photography, Algorithms, ed. Eivind Røssaak (Amsterdam: Amsterdam University Press, 2011), 76.

11 Bellour, “Concerning ‘the Photographic,’” 261.

12 Laura Mulvey, Death 24× a Second: Stillness and the Moving Image (London: Reaktion Books, 2006).

13 Mario Carpo, The Second Digital Turn: Design Beyond Intelligence (Cambridge, MA: The MIT Press, 2017), esp. 70–71.

14 Juan Fontcuberta, La furia de las imágenes: Notas sobre la postfotografía (Barcelona: Galaxia Gutenberg, 2016).

15 Bogdan Ionescu, Wilma A. Bainbridge, and Naila Murray, eds., Human Perception of Visual Information: Psychological and Computational Perspectives (Cham, CH: Springer, 2022).

16 Marco Fodde, “Mascherare o bruciare?,” Fotografia Reflex (November 2002): 63.

17 See “Images composites,” Transbordeur: Photographie, histoire, société 7 (2023).

18 Fang Xu et al., “Multi-Exposure Image Fusion Techniques: A Comprehensive Review,” Remote Sensing 14, no. 3 (2022): 771.

19 Jinhua Wang, Xuewei Li, and Hongzhe Liu, “Exposure Fusion Using a Relative Generative Adversarial Network”, IEICE Transactions on Information and Systems E104D, no. 7 (2021): 1017–27.

20 Antonio Somaini, “Film, Media and Visual Culture Studies and the Challenge of Machine Learning,” NECSUS: European Journal of Media Studies 10, no. 2 (2021): 49–57; Ruggero Eugeni, Capitale algoritmico: Cinque dispositivi postmediali (più uno) (Brescia: Scholé, 2021), 89–90; Antonio Somaini, “Algorithmic Images: Artificial Intelligence and Visual Culture,” Grey Room 93 (Fall 2023): 75–115.

21 Eivind Røssaak, “Algorithmic Culture: Beyond the Photo/Film Divide,” in Røssaak, Between Stillness and Motion, 194.

22 Somaini, “Algorithmic Images,” 99–100.

23 Hitesh Kandala, Jianfeng Gao, and Jianwei Yang, “Pix2Gif: Motion-Guided Diffusion for GIF Generation,” arXiv, March 8, 2024, arXiv:2403.04634v2.

24 Tumblr GIF Description Dataset, https://raingo.github.io/TGIF-Release/.

25 Ken Timby, “‘Cinema in a Single Photo’: The Animated Screen Portrait of the 1910s,” in Between Still and Moving Images, ed. Laurent Guido and Olivier Lugon (New Barnet, UK: John Libbey, 2012), 97–111.

26 Paul Zammit, Guillem Carles, and Andrew R. Harvey, “Three-Dimensional Imaging and Ranging in a Snapshot with an Extended Depth-of-Field,” Imaging and Applied Optics 2016, OSA Technical Digest (online) (2016): CW2D.1.

27 Barbara Grespi, “Archaeology of the Postphotographic Image: Photogrammetry, Photometry, and the Post-Optical Regime (in Nineteenth Century Astronomy),” La Valle dell’Eden 41–42 (2023): 119–41.

28 Alexander R. Galloway, “Polygraphic Photography and the Origins of 3-D Animation,” in Animating Film Theory, ed. Karen Beckman (Durham, NC: Duke University Press, 2014). See also Brooke Belisle, Depth Effects: Dimensionality from Camera to Computation (Los Angeles: University of California Press, 2023).

29 Chris Chesher, “Between Image and Information: The iPhone Camera in the History of Photography”, in Studying Mobile Media. Cultural Technologies, Mobile Communication, and the iPhone, ed. Larissa Hjorth, Jean Burgess, and Ingrid Richardson (New York: Routledge, 2012), 98–117.

30 Lev Manovich, “Media after Software,” Journal of Visual Culture 12, no. 1 (2013): 30–37.

31 Fabio Remondino et al., “A Critical Analysis of NeRF-Based 3D Reconstruction,” Remote Sensing 15, no. 14 (2023): 3585.

32 “NVIDIA Instant NeRF: NVIDIA Research Turns 2D Photos into 3D Scenes in the Blink of an AI,” posted March 25, 2022, by NVIDIA Developer, https://www.youtube.com/watch?v=DJ2hcC1orc4&t=18s.

33 Joanna Zylinska, AI Art: Machine Vision and Warped Dreams (London: Open Humanities, 2020), 106.

Top of page

List of illustrations

Title 1. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021
Caption Screenshot of the RealityCapture photogrammetry process.
Credits © courtesy of the artist, Sara Tirelli
URL http://journals.openedition.org/transbordeur/docannexe/image/2603/img-1.jpg
File image/jpeg, 177k
Title 2-3. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021
Caption Screenshot of the RealityCapture photogrammetry process.
Credits © courtesy of the artist, Sara Tirelli
URL http://journals.openedition.org/transbordeur/docannexe/image/2603/img-2.jpg
File image/jpeg, 984k
Title 4-5. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021
Caption Screenshot of the Metashape photogrammetry process.
Credits © courtesy of the artist, Sara Tirelli
URL http://journals.openedition.org/transbordeur/docannexe/image/2603/img-3.jpg
File image/jpeg, 850k
Title 6-7. Sara Tirelli, Cilia, virtual reality installation presented at the Museo del Novecento, Milan, 2021
Caption Screenshot.
Credits © courtesy of the artist, Sara Tirelli
URL http://journals.openedition.org/transbordeur/docannexe/image/2603/img-4.jpg
File image/jpeg, 704k
Title 8. Sara Tirelli, Cilia, virtual reality installation presented at the Espronceda Institute of Art and Culture, Barcelona, 2021
Caption Screenshot.
Credits © courtesy of the artist, Sara Tirelli
URL http://journals.openedition.org/transbordeur/docannexe/image/2603/img-5.jpg
File image/jpeg, 303k
Top of page

References

Electronic reference

Barbara Grespi, Between the Photograph and the Frame. The Fate of the Single Image in the Algorithmic EraTransbordeur [Online], 9 | 2025, Online since 26 February 2025, connection on 13 January 2026. URL: http://journals.openedition.org/transbordeur/2603; DOI: https://doi.org/10.4000/13dx2

Top of page

About the author

Barbara Grespi

Barbara Grespi is a full professor in the University of Milan’s Department of Philosophy “Piero Martinetti,” where she teaches theories of the image in motion and media archaeology. Her research has explored the non-ocularcentric and gestural dimensions of optical media, such as photography, cinema, and more recently, extended realities. Her publications include Figure del corpo (2019), Harun Farocki (coeditor, 2017), Bodies of Stone in the Media, Visual Culture and the Arts (coeditor, 2020), Mediarcheologia (coeditor, 2023), Il postfotografico (coeditor, 2024).
Barbara Grespi est professeur titulaire au département de philosophie « Piero Martinetti » de l’université de Milan, où elle enseigne les théories de l’image en mouvement et l’archéologie des médias. Ses recherches explorent les dimensions non ocularo-centriques et gestuelles des médias optiques, tels que la photographie, le cinéma et, plus récemment, les réalités étendues. Elle a notamment publié Figure del corpo (Meltemi, 2019), Harun Farocki (coéditrice, Mimesis, 2017), Bodies of Stone in the Media, Visual Culture and the Arts (coéditrice, Amsterdam University Press, 2020), Mediarcheologia (coéditrice, Raffaello Cortina Editore, 2023), Il postfotografico (coéditrice, Einaudi, 2024).

Top of page

Copyright

CC-BY-NC-ND-4.0

The text only may be used under licence CC BY-NC-ND 4.0. All other elements (illustrations, imported files) may be subject to specific use terms.

Top of page
Search OpenEdition Search

You will be redirected to OpenEdition Search