Image & Narrative, Vol 10, No 3 (2009) 73
Jan Baetens
How Many is Multi? On the Example of Photo Narrative, High and Low.
Abstract
This article proposes a critical rethinking of the fashionable notion of multimodality as an extreme form of intermediality. Rather than a priori discarding the dead end of monomodality, it asks whether it is possible to reconsider it in the light of a “less is more” variant of multimodality. The example of the photonovella, at first sight a typical multimodal practice, demonstrates that monomediality has often been an ideal, both in the popular and in the high-brow version of the genre.
Résumé:
Cet article se propose de repenser critiquement le concept à la mode de multimodalité, forme extrême du phénomène d‟intermédialité. Au lieu d‟écarter d‟emblée la soi-disant impasse du monomédial, il se pose la question s‟il n‟est pas possible de l‟analyser sur de nouvelles bases comme une variante singulière du multimodal lorsque ce dernier obéit à la logique « moins c‟est plus ». l‟exemple du roman-photo, à première vue une pratique typiquement multimodale, démontre que le monomédial a souvent été un véritable idéal, aussi bien dans les formes populaires du genres que dans ses formes plus sophistiquées.
Keywords: photo narrative
The paradox of multimodality
Multimodality is a hype, obviously. It is rapidly taking the place of the remediation theory, which from Marshall McLuhan (1994) to Jay David Bolter and Richard Grusin (1999) has accompanied the rapidly changing media world as its first and ultimate theory, spanning more than five centuries of Western media technology, from the traumatic alpha of the “autonomous” typeface (the beginning of all evil, if one follows McLuhan) to the alpha of electric light (the final return to Creation‟s unity, always following the same author) or, two generations later, of virtual reality (the almost final, albeit unstable and therefore inaccessible step beyond mediatization, in Bolter and Grusin‟s theory). In both systems, multimodality is
Image & Narrative, Vol 10, No 3 (2009) 74 at the heart of media evolution: as McLuhan argues, each new medium absorbs the form of the previous one as its content (each new medium is therefore by definition more multimodal than the vanishing one); Bolter and Grusin rightly insist on the importance of multichanneling for the production of mimetic or realistic effects, which justify the shift towards always newer media (a computer enabling the easy mix of word and image fits the mimetic ideal better than the word processor, for instance). In this perspective, the ideal medium combines all five senses: it delivers messages that can be seen (words and images), listened to (sounds), felt (many media machines produce strong haptic sensations), tasted, and smelled (the last two aspects are not yet there in the commercial applications, but it can‟t be long before we will plug into the complete sensorium, which would not have come to a surprise to all the moviegoers of the fifties, and later of the I-Max theatres, willingly or unwillingly exposed to „shock‟ experiments addressing smell and taste: for a fictional reconstruction of this period‟s movie subculture, see Joe Dante‟s nostalgic –and regrettably underestimated– 1993 Matinee).
The gradual opening of the work of art to multimodality has demonstrated in retrospect that the avant-garde‟s inclination towards hybridization was right, and that (high) modernism‟s focus on medium specificity and on the necessity of freeing all media of what did not belong to their essence was debatable, if not simply wrong (teleology never works in one way: it does not only forecast the future, it also always rewrites the past). Hence the valorization of „mixed media‟ such as comics, photonovellas, cartoons, collages, and so on, which in their times may have been considered marginal or non-canonical, but which have shifted dramatically towards the very centre of what we define as art today.
Yet at the same time there may be some confusion here. For the increased diversity of media modes, i.e. signs and practices addressing a great variety of senses, is also clearly the result of a media concentration. As a matter of fact, multimodality appears always as a process that comes across many technical and cultural difficulties: first it is just a dream, or a desire; then it relies upon the combination of channels or technologies that cannot be seamlessly integrated; and only after other attempts it becomes possible to use no more than one medium to register, create, and display various types of signs. Let me mention three short, yet decisive examples of this process. The photographs that originally appeared in books had to be printed separately, before being glued on the blank pages of the already printed and bound book. It was only the discovery of the duo- or halftone technique that permitted the simultaneous production at print stage of the verbal and the visual. Cinema, secondly, which as we know
Image & Narrative, Vol 10, No 3 (2009) 75 has never been silent (it was nothing else but “mute”, to quote Michel Chion 1994), did not enter the sound era once it was possible to merge the sound track and the film stills on the same celluloid strip. And, finally, what we experience today on our desktop goes in the same direction: the merging and blurring of signs never comes naturally, it always takes time before one succeeds in combining seamlessly classes of signs (at a different level, one might add that this technical feature is continued by the institutional tendency towards media concentration: see TimeWarner/AOL). For we all know that in digital culture, we are still far from what we want the computer to do, since even today the integration of images in texts is less easy than some software producers want us to believe: "The notion that digital texts and images are infinitely fluid and malleable is an aesthetic conceit divorced from technical practice, a consensual hallucination in the same way that William Gibson's neuromantic „lines of light‟ delineate an imaginative ideal rather than any actual cyberspaces." (Kirschenbaum 2003: 154).
However, the more multimodal our messages become, the more we notice the gap between surface diversity and uniformization underneath (if our computers let us make or use multimodal texts so easily, it is in the very first place because their own “language” is a questions of 0‟s and 1‟s). This paradoxical relationship between means and ends (for one has to reduce the multiplicity of means in order to enable a multiplication of ends) may shed a new light on the efforts of high modernist practices to achieve medium specificity. Instead of immediately condemning it, as the current doxa claims, as a reactionary impetus towards an impossible (and ideologically suspect) „purity‟, one might ask indeed if the militant reductionism accomplished or at least pursued by medium specific approaches is totally opposed to issues of multimodality or if, on the contrary, it can be linked in its own way to the 20th Century poetics of hybridization. In other words: rather than a priori discarding the tendencies toward the dead end of monomodality, one should ask whether it is possible to reconsider this type of reductionism in the light of a “less is more” variant of multimodality.
Multimodality’s monomedialization
One of the strangest (and therefore most challenging) aspects of multimodal practices is their tendency towards modal simplification. I am thinking here in the very first place of the photonovella (but the graphic novel might be a case too, although one I will not be able to discuss in this article), which demonstrates a certain “resistance” towards what ought to be
Image & Narrative, Vol 10, No 3 (2009) 76 seen as its most characteristic feature: the merging of the photographically depicted and the verbal information delivered by captions and speech balloons.
In the field of the high-art photonovel (Baetens 1993, Ribière 1995) there has always been a strong rejection of the verbal elements, accused of a triple evil: first, claiming all the spectator‟s attention (just like in subtitled films, it is not easy to follow both the words and the images, and in our still iconoclastic culture, words are considered more valuable than images); second, disturbing the pictures‟ composition (if one has to „add‟ textual information to an already existing photograph, the composition of the images has a strong liability of losing its internal balance and structure); third, refusing the images to be in charge of the narrative (the sequential and causal relationships between the various pictures are ruled by the interaction of captions and speech balloons, not, as one would like it to be in the high-art sphere, by the intrinsic qualities of the images themselves). For all these reasons, one understands the fascination exerted by mute, i.e. apparently monomodal stories, in the field of photonarrative. The most famous example, together with the short stories by Duane Michals, is of course the 100 pages photographic novel by Marie-Françoise Plissart, Right of Inspection (1998). Yet as the very “Lecture” by Jacques Derrida which follows (and completes?) Plissart‟s pictures clearly demonstrates, the muteness of the images does not imply a vanishing of the words, on the contrary. It is to the very extent, Derrida argues, that the images are mute that they are able to generate (verbal) stories, and therefore, perhaps, to create new forms of multimodality.
A similar point can be made when one looks at the extreme opposite of the high-art photo narrative: the commercial photonovella, a much despised subfield of the romance genre (Lecoeuvre & Takodjérad 1991, Giet 1997). Born in the aftermath of World War II, the photonovella had strong ties with the world of popular cinema and was a typical example of „escapist‟ literature. Yet here too, although the material structures of this genre seem a good illustration of the merging of word and image imposed by the dominant paradigm of the cinema, a closer look reveals that the multimodal collage of pictures, captions and speech balloons, that has become so familiar, may prevent us from seeing the role and place of monomodal features. As Giovanni Fiorentiono (1995) has convincingly argued in a study of the photonovellistic adaptation of Dino Risi‟s masterpiece of „rose‟ (i.e. comical) Neo-Realism, Pane, Amore, e… (1955), the photonovella‟s possibilities of multimodal telling were manifestly underused, not just by lack of sophistication (this is the usual argument uttered by photo critics and historians who borrow their categories from Art), but in order to make room
Image & Narrative, Vol 10, No 3 (2009) 77 for what really mattered: the face of the actors (Sophia Loren and Vittoria De Sica, both bigger than life, each in his or her way), the emphasis put on their looks (there is not the slightest attempt to achieve a coincidence between the expression of the faces, often showed in profile, as kings and emperors on antique coins, and the content of what they are supposed to be saying), and the contrast between the leading actors‟ mythological universe and the daily (but extremely scenic) setting of Sorrento. Contrary to the film, whose story it „reproduces‟, the photonovella is, in a certain sense, in a pursuit of monomediality. The genre, which seems to be a transposition to the world of the book (with fixed images and texts) of the world of the cinema (with moving images and sound track), rapidly appears to be a continuation of a completely different genre: the (mythical) actor‟s portrait.
Monomediality in the photonovella
That the photonovella, which seems to be a number one candidate for any basic list of multimodal genres, displays a certain tendency towards monomediality may seem a paradox. Yet the very history of the photonovella easily demonstrates that the reluctance toward the mixing and blurring of modes and forms is one of its distinctive features, both in the popular forms of the genre as in its more elite or avant-garde occurrences. A closer look at some precise examples can make this clearer.
In the beginning of the photonovella, which appeared in the aftermath of World War II as an original combination of two existing genres, the illustrated film script on the one hand and the
graphic novel on the other hand (the best visual survey of this material can be found in
Lecoeuvre & Takodjérad 1991), one of the most dramatic problems the genre had to face was the integration of words and images, which led to systematic difficulties and communicational failures at various levels: technical, aesthetic, and narrative. In the first place, the encounter of textual and visual material within the same frame proved to be a technical nightmare: texts had to be added in handwritten form, and it appeared very complicated to obtain a sufficient chromatic contrast between letters and background: often white letters did not stand out enough against a too pale or too grey background or black letters were not contrasted sharply with the dark photographic spaces around them. Moreover, the print quality of the cheap magazines of these years was so poor, the rough paper absorbed so much ink that one had to guess the form and the meaning of the words more than one could read them (obviously, the first experiments with colour hardly increased the confidence of the public in the new genre). In the second place, the technical problems appeared also to have a strong aesthetic
Image & Narrative, Vol 10, No 3 (2009) 78 dimension. Given the fact that the texts were added to already existing photographs, either in the form of speech balloons or in the form of narrative captions, this insertion destroyed in many cases the internal composition of the picture, for there were only few pictures that displayed enough “empty”, i.e. semantically poor spaces, to be able to host more than just a few words. Furthermore, the very presence of textual material was accused of preventing from looking at the pictures themselves (a reproach often addressed at subtitles in film and television). In the third place, photonovellas were also insecure and vacillating in the synchronicity of their montage of word and images: many images were spoiled because the text came too early, or vice versa, and it was far from easy to “invent” new and smooth forms of reading rhythms in which the verbal and the iconic elements did complete each other without breaking the page lay-out.
These problems delayed the breakthrough of the genre as a new cultural form. During the first years of its existence, the photonovella had to compete with other cultural forms such as comic strips, and in this competition the photonovella was not always victorious. Photonovellas, short stories, illustrated instalments, cartoons, comic strips, etc. shared the same place in the specialized feminine press in which the photonovella first appeared and it took almost a decade before the domination of the photonovella became evident and this shift is clearly linked by the fact that the genre had overcome the difficulties created by its own multimodality. In a certain sense, one might of course argue that these technical, aesthetic and narrative difficulties were not specific to the genre of the photonovella and that a master as Hergé, who replaced in the comics field the European system of captions placed outside the visual frame by the American systems of visually integrated speech balloons, had needed also many years before he succeeded in avoiding the clumsiness of his first attempts. Yet what makes the photonovella so fascinating, for example in comparison with the comics and the graphic novel, is that in its search for solutions to its initial problems the genre has always looked for answers that were not insensitive to the idea of monomediality. Much more than the comics, photonovellas have always tried to keep words and images as neatly separated as possible, for instance by isolating the textual information in special frames (with no further visual information) or by presenting the speech balloons and narrative captions in ways that purposively refused to interact with the image: it is very hard to find experiments in “creative” lay-out, either at the level of the letters themselves (the typefaces that were used aimed mainly at remaining as invisible as possible) or at that of the composition of the frame or the page (the speech balloons and the captions have always made great efforts to “hide” from the
Image & Narrative, Vol 10, No 3 (2009) 79 reader‟s view). Therefore, it is not absurd to suggest that the photonovella, although it is obliged to combine words and images, at least in its popular forms, is less an example of multimodality than of “alternate monomediality”, creating a string of communicative units that produced an alternation of two categories of monomedial elements (words on the one hand and photographs on the other hand).
The fate of the more ambitious, high-art photonovella, which was heavily inspired by the experiments in conceptual narrative photography during the 1960s (the example of Duane Michals comes here immediately to mind), displays these monomedial tendencies with utmost strength and clarity. First, the works by artists such as Marie-Françoise Plissart (Plissart 1998) or Raymond Depardon (mainly in some of his early works like Correspondance new-yorkaise (Depardon and Bergala1981)) introduce a sharp material frontier between words and images: textual elements are only tolerated to the extent that they remain in the margins of the photograph, no material coincidence of the verbal and the iconic is accepted within the same frame (which is always that of the image). Second, they have a strong preference for textless and wordless stories (the ancient and of course very debatable idea of the “decadence” of the visual syntax of silent cinema at the emergence of the talkies is mentioned by several artists working in this field). A good visual story is a story that is told with no other means than purely visual ones, monomediality being considered the ideal form of all serious experiments in the field of sequential photography. Third, if text has to be present, it must be transformed in such a way that all traces of multimodality are whipped out: verbal information is thus no longer added to the image, it is photographed itself as part of the pictured universe, in order to become itself a visual element (sometimes at the price of its very readability). High-art photonovellas may differ from popular photonovellas in all possible respects (and the absolute split of their readership is the proof that their differences are considerable), yet at the level of their desire for monomediality their distinction tends to disappear.
The multi of mono
However, to insist on the monomodal tendencies within multimodality, at the two cultural extremes of the photo narrative field, is just half of the story. For, as the example of Plissart has already clearly shown, the rejection of any caption or balloon does not imply the impossibility of storytelling. Nor is the actor‟s mythology as imposed by the cinema portrait industry „mute‟, if only for that other „genre‟ well-known in „vernacular‟ literature: fan
Image & Narrative, Vol 10, No 3 (2009) 80 mail… (by the way, in the forties and the fifties most magazines specialized in the photonovella genre have always encouraged their readers to keep up a correspondence with the stars). For this reason, one may ask whether the relationship between mono- and multimodality should not be redefined in other, i.e. broader terms than the strictly technical terms that define the research agenda most of the times. In other words: rather than to circumscribe monomodality or multimodality in relationship to the type of signs materialized on the page (or on another host medium), one may also ask what is the relationship between means and ends, between technical devices used by the artist and the results accomplished at „reading‟ level.
The first aspect, that of the means and the devices, has to do with the issue of syncretism: some arts and practices are more or less multimodal than others, they use just one type of signs or they combine two or more types. The second aspect, that of the ends and the results, has to do with, broadly (metaphorically?) speaking, synaesthesia: some arts and practices succeed in delivering messages that are „processed‟ in ways that either reduce or enlarge the modal sensorium of the signs of the work itself. From that point of view, it is clear that both „mute‟ and „commercial‟ photonovellas are doing very well, although they are less multimodal, in the technical sense of the word, than the „sophisticated‟ word and images dialogue fostered in photography‟s play with words and texts (it is in no way a coincidence that almost every book on „photography & literature‟ is simply blind to the world of the photonovella). However, if one shifts from the level of syncretism to the level of synaesthesia (and I repeat that I am taking these terms as metaphors for „means‟ and „ends‟), the situation changes dramatically, for the resistance against words in the case of the high-art photonovella (which tends to delete them) as well as in the case of the commercial photonovella (which tends to have them ignored by actors and readers) generates, in a provocative and paradoxical way, an explosion of words and stories.
It may even be possible to draw more general conclusions from these examples. If one agrees that in spite of the monomodal tendencies in their technique, some photonarratives prove able to increase multimodality in audience‟s response, one should also ask whether there is not a
causal relationship at work between means and ends. In other words: would it not be possible
to suppose that the refusal of multimodality at technical level is the condition of the
achievement of multimodality at other levels? Given the importance of anti-multimodal
Image & Narrative, Vol 10, No 3 (2009) 81 be a sound „yes‟. Or, as the title of Roland Barthes‟s famous 1976 essay on Cy Twombly stated: “Non multa sed multum” (not many things, but one big thing).
And to generalize even further, one should ask also whether the conclusion reached in the case of the photonovella applies as well to cases of cooperation between arts. Our contemporary doxa, which promotes multimodality within each art‟s technique, is making also strong pleas in favour of second-degree multimodality, i.e. of the combination of hybridized arts and techniques. As a matter of fact, this is a mythological and ideological stance, which may not correspond with what artists are really doing. Of course, I am not able to answer these questions, but I would like to end with a provocative statement by Lev Manovich, the new media theoretician who, contrary to most of his colleagues in the field, has never concealed his belief in the stakes of medium specificity: “If we really look at the reality of the „integration of media‟ it turns out that it simply is not true. In the 20th
century or the 21st century, there are indeed hundreds of thousands of people who, in a mediocre way, work one day with interactive installations, the next day with sound, and after that with fiction stories. But among artist who are really great and define new directions, there is not a single person I can think of in the history of contemporary art who could work in more than one medium.” (Manovich 2006: 30).
Works quoted
Jan BAETENS (1993). Du roman-photo. Paris-Bruxelles: Les Impressions Nouvelles.
Jay David BOLTER and Richard GRUSIN (1999). Remediation. Understanding New Media. Cambridge, Mass.: MIT.
Michel CHION (1994). Audio-Vision. New York: Columbia University Press.
Raymond Depardon/Alain Bergala (1981). Correspondance new-yorkaise/Les absences du
photographe. Paris: Libération/Editions de l‟Etoile.
Giovanni FIORENTINO (1995). “Seduzione ottiche. L‟icona, il fotoromanzo, l‟istantanea”, in Id. (ed.), Luci del Sud. Sorrento un set per Sofia. Castellammare di Stabia: Eidos, 37-48. Sylvette GIET (1997). "Nous Deux 1947-1977: apprendre la langue du coeur". Louvain/Paris :
Image & Narrative, Vol 10, No 3 (2009) 82 Mathew KIRSCHENBAUM (2003). "The word as image in an age of digital reproduction", in
Mary E. HOCKS and Michelle R. KENDRICK (eds), Eloquent Images. Word and Image in the
Age of New Media, Cambridge, Mass. MIT, 137-156.
Fabien LECOEUVRE & Bruno TAKODJÉRAD (1991).Les années roman-photos. Paris: Editions
Veyrier.
LevMANOVICH (2006). “Media in a Software Age”, in Makhuzine, No 1, 23-30.
Marshall MCLUHAN (1994). Understanding Media. The Extensions of Man. With a new
introduction by Lewis H. Lapham. Cambridge, Mass.: MIT. (1st edition: 1964).
Marie-Françoise PLISSART (1998). Right of Inspection: Photographs by Marie-Francoise
Plissart, with a Lecture by. Jacques Derrida. Trans. David Wills. New York: Monacelli Press
(1st French edition: 1985).
Mireille RIBIÈRE (ed.) (1995). “Photo Narrative”, special issue of History of Photography,
vol.19-4.
Jan Baetens is chief editor of Image (&) narrative. Email: jan.baetens@arts.kuleuven.be
