Hypertext and Cultural Autobiography:
Talk with Your Hands Like an Ellis Island Mutt
Steven Wingate
South Dakota State University English Department, Box 2218
Brookings, SD 57007 1-720-256-8435
Steven.Wingate@sdstate.edu
ABSTRACT
Talk with Your Hands Like an Ellis Island Mutt is an interactive video, built using Korsakow cinema database software, made of interlocking fragments. It consists of 157 short clips (most 8-12 seconds in length) including altered images of my own hand gestures, archived records and objects of my immigrant ancestors, and archival footage of newcomers arriving at Ellis Island—
through which the majority of European immigrants passed as America’s population exploded between 1890 and 1920. It offers multiple unique playthroughs in an order determined both by interactor choice and the computational operation of its database.
CCS Concepts
• Applied computing Arts and Humanities Media Arts.
Keywords
Interactive cinema; interactive video; database cinema; Korsakow;
hypertext; visual hypertext; cultural autobiography.
1.1 Description of the piece
Talk with Your Hands Like an Ellis Island Mutt is an interactive video project that uses new media tools to explore cultural identity in a way that analog tools would not permit, suggesting that identity is the result of multiple lifelong collisions between elements of personality that we have inherited with or without knowing it.
Designed for performative screening or interactive exhibition and built in the Korsakow cinema database system, HandMutt embodies my experience as a second-generation American whose ancestors came through Ellis Island from Europe (Poland, Hungary, and Scotland between 1907 and 1914), and whose identity is formed by a multi-generational assimilation process that often feels perpetual. With three of my four grandparents being born outside the US in non-English speaking environments (and the fourth born here not long after his own parents arrived), I have been aware of my own otherness all my life, particularly in regard to my working class and Eastern European heritage.
Part of the expression of that heritage is talking with one’s hands—a habit attributed in America almost exclusively to
Figure 1. Various hand gestures from Talk with Your Hands Like an Ellis Island Mutt.
working class immigrants and their descendants. (One expected result of successful assimilation is the loss of this habit, which I did not achieve.) The idea for HandMutt came during the editing process for my digital lyric memoir daddylabyrinth, which premiered at the International Conference on Interactive Digital Storytelling in 2014 at the ArtScience Museum of Singapore. In its video footage I found many hand gestures that led me to reflect on my own heritage and on the multi-generational assimilation process of American immigrants. This resulted in the birth of HandMutt as a separate project.
The technology-enabled fragmentation of story, image, and consciousness in HandMutt reflects the fragmentation of cultural identity itself, which is often so dispersed in our lives that we can scarcely identify it. The work generates a nuanced, intimate understanding of the immigration and assimilation experience that would not be possible with traditional monolinear narrative and non-interactive media.
The video clips in HandMutt come from four distinct sources, and each is paired with short voice-over commentary by me. The largest group is a set of sixty-four of my hand gestures harvested from the video selfies in my digital work daddylabyrinth. Each of these gestures is isolated from its communicative context and broken down into micromotions—often only a few frames in length—that are repeated, reversed, and subjected to manipulations in frame rate using Final Cut Pro X editing software. This visual idea is an homage to my film mentor Ken Jacobs, whose performative Nervous System films of the 1980s and 90s created a similar “stutter” effect (one that we now see all around us, in altered form, thanks to GIF culture).
The second group of clips I worked with was harvested from public domain archival footage of immigrants arriving at Ellis Island in 1903 and 1906, which are subject to the same Ken Jacobs-esque manipulations. I left these untouched in terms of
Figure 2. Still with archival Ellis Island footage.
color to keep the historical record of immigration as cleanly represented as possible. These images, which were shot with a primitive stationary camera, are blown up to capture the micro- dramas embedded within the larger frame—a squabbling couple, a frightened child, an old woman uncertain of the life ahead of her.
HandMutt also includes clips of tangible family relics and photographs from my maternal and paternal families. I have few possessions from either side, and many have achieved near- iconographic status for me no matter how simple or utilitarian they are. The final group is simple text animations of keywords that are used in the Korsakow database that resonated with themes which emerged from the other three categories of clips. These represent the emotions and psycho-social forces that have shaped my relationship with my ancestry. The overall result of these multiple strands in conversation with one another is that of many stories weaving together to form a narrative experience without the benefit of Aristotelian coherence—a mosaic portrait of American assimilation and its multi-generational challenges.
Interactors can experience HandMutt via installation or web browser. They navigate through a series of brief clips, choosing the next one from a set of thumbnail images that appear at the end of each—all representing other nodes in the video database. The short length of the clips forces interactors into an active viewing mode in which they must make frequent choices about which clip to watch next without knowing how that choice will alter their movement through the material. Choosing a given thumbnail affects the next set of choices, but does not set the interactor on a
“path,” as there are no predetermined navigations in the work.
Instead, the next set of clips is determined by the keywords used to describe the one that has just played; while the number of variations is by no means infinite, the computational operation of the Korsakow database makes it extremely difficult to replicate the same navigation twice.
1.2 Artistic statement
I came to the electronic arts in mid-career as an evolution of my training and work in literature and film—particularly the experimental traditions of both. My first artistic education was in experimental cinema; I was a student of Ken Jacobs and later worked in a department with Stan Brakhage. I was also mentored by two experimental fiction writers, Ronald Sukenick and Steve Katz, who were at the forefront of the metafiction community in the 1970s that later spawned hypertext fiction. Because of this twin background I have always been invested in hybrid genre work and invented forms. It was not until this decade that I turned toward digital expression, to which I was drawn because it offers vastly more powerful tools than print and film alone in terms of
Figure 3. A view of the Korsakow database.
managing multiple perspectives and combinations of narrative elements. The most important aspect of digital arts to me is the ability to develop stories that are not pre-arranged into traditional Aristotelian act structures. I am interested in creating what Judy Malloy calls “narrabases” and Mark Amerika calls “narrative environments”—bodies of related emotional material that can be encountered in various configurations, each resulting in a unique aesthetic experience that relies on interactors’ choices and calls on their innate ability to co-create stories from discrete elements.
To achieve this I have sought out software systems designed to offer multiple possible relationships between the nodes in my narratives. Thus far I have worked with Scalar for daddylabyrinth:
a digital lyric memoir and Korsakow for HandMutt. In the future I am particularly interested in working with software that allows for unconscious interactor choice (using biometric input, for instance), which would enable me to develop ambient or proto- narrative experiences built up from thousands or even tens of thousands of narrative stimuli.
1.3 Hardware requirements, length, format,
HandMutt can be run on any computer via local files on a web browser. It has typically run on a Mac in Google Chrome. No special software is required to be installed. I can set up the project on my own machine (MacBook Pro) in the exhibition space and deliver a ten-minute presentation on the work. Alternately, I can provide an external hard drive with the video. An interactive setup of HandMutt is simple and flexible in terms of space; it requires only a computer, a monitor, a VGA or HDMI cord, a mouse or touchpad, and headphones (if desired), and can be spatially configured in a variety of ways based on exhibition needs. The project is designed for interactors to traverse it in half an hour.
1.4 Exhibition history
HandMutt was first exhibited in a partial screening at the Electronic Literature Organization’s 2015 conference in Bergen, Norway. It will premiere in full at the 22nd International Symposium on Electronic Art in Hong Kong in May 2016.
1.5 Links
A web version is at http://ellisislandmutt.korsakow.tv (login
<stevenwingate>, password <swsubmits>). Please note that after it premieres in full at ISEA, the login will no longer be required.
A two-minute video describing the project is at https://vimeo.com/134786308.
