Computational expressionism : a study of drawing with computation

(1)

computational

expressionism

A STUDY OF DRAWING WITH COMPUTATION.

by Joanna Maria Berzowska

BA Pure Mathematics, McGill University, Canada, June 1995 BFA Design Art, Concordia University, Canada, June 1995

Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, in partial fulfillment of the requirements for the degree of

Master of Science

Program in Media Arts and Sciences October 27, 1998

Certified by Walter Bender

-Senior Research Scientist, Program in Media Arts and Sciences Thesis Supervisor

Accepted by Stephen A. Benton

Chair, Departmental Committee on Graduate Students

/Program in Media Arts and Sciences

MASSACHUSETTS INSTITUTE OF TECHNOLOGY

MAR

19 1999

LIBRARIES

ROTC"

r vl ' I*v vvvU,

(2)

COMPUTATIONAL EXPRESSIONISM:

A STUDY OF DRAWING WITH COMPUTATION. by Joanna Maria Berzowska

Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning,

on October 27, 1998

in partial fulfillment of the requirements for the degree of Master of Science

abstract

This thesis presents computational expressionism, an exploration of drawing using a computer that redefines the concepts of line and composition for the digital medium. It examines the artistic process involved in computational drawing, addressing the issues of skill, algorithmic style, authorship, re-appropriation, interactivity, dynamism, and the creative/evaluative process.

The computational line augments the traditional concept of line making as a direct deposit or a scratching on a surface. Digital representation is based on computation; appearance is procedurally determined. The computational line embodies not only an algorithmic construction, but also dynamic and interactive behavior. A computer allows us to construct drawing instruments that take advantage of the dynamism, interactivity, behavioral elements and other features of a programming environment.

Drawing becomes a two-fold process, at two distinct levels of

interaction with the computer. The artist has to program the appearance and behavior of lines and subsequently draw with these lines by dragging a mouse or gesturing with some other input device. The compositions incorporate the beauty of computation with the creative impetus of the hand, whose apparent mistakes, hesitations and inspirations form a complex and critical component of visual expression.

Thesis Supervisor Walter Bender

Senior Research Scientist, Program in Media Arts and Sciences

(3)

computational

expressionism

Joanna Maria Berzowska Thesis Reader John Maeda

Assistant Professor of Design and Computation VIT Media Laboratory

Thesis Reader Scott Snibbe Member Research Staff Interval Research Corporation

(4)

acknowledgments

Heartfelt thanks to Anna and Tomasz for holding my hand and pulling and pushing me through the world these past 25 years. They have taught me about language, culture, desire, despair, courage and determination. They have taught me most of what I know and feel, and

I will be eternally grateful for their love. I also want to thank my little

sister Justyna for being a great roaming companion.

A tearful thank you to the late Gert-Jan Zwart, who taught me great

new things about life and, so very sadly, about death.

Special thanks to my advisor Walter Bender, whose insight, wit and enthusiasm shape the Media Lab in good ways. Additional special thanks to my thesis committee, John Maeda and Scott Snibbe, whose art and thought have and will continue to inspire my work.

Particular thanks to my unofficial thesis readers: Mike Best, Jack Driscoll, Nelson Minar, Arjan Schutte and Paul Yarin.

The Media Laboratory has provided the first social environment in my life where I have felt at ease. The people I interact with on a daily basis betray a lovely mixture of intellect and insanity. Among all these lovely people, there are a few whose friendship was indispensable and

invaluable in the task at hand: Mike Best, Sawad Brooks, Daniel Dreilinger, Nelson Minar, Arjan Schutte, Laura Teodosio, John Underkoffler, Alex Westner and Paul Yarin. I thank them all from the very bottom of my thumping heart.

Tremendous thanks to Jimmy Wondrasek for his sincere love and belief in me. He is unquestionably one of the pivotal people in my life. Equally tremendous thanks to Gaia Marsden for being such a close and

loving presence for almost ten years now. I also want to thank David Roselli for the inspiration he brought into my life for many years, Raz Schionning for showing me how to use a computer and for giving me direction and Meg Wilson, one of the most supportive and generous

(5)

introiduction

Computers have brought about many changes to facilitate and encourage a breadth of personal expression. Text editing, image editing, layout and publishing software allow the production of documents and other artifacts more quickly, and with an increased degree of control over the process. The ability to save multiple copies,

at various levels of completion, permits a more thorough and more efficient execution. The increasingly universal access to information and to the dissemination of information through personal printers and the World Wide Web have greatly simplified the process of receiving, as well as expressing and publicizing, opinions and feelings.

Computers and computer culture continue their

assiduous, penetrating and pervasive infiltration

of our homes and lives. They are quickly

becoming much more than tools for processing

4.A

r data. They have already become, in fact, for more and more of us, a primary mode of engaging the world. [SM196] They have become a primary outlet for expression, for the

Sketch made in the margin of a building of _{identity and for personal, emotive}

notebook (not the margin of the Word

document) by Joanna Berzowska interaction with others.

Computer developers are working hard to spawn products that purport to be conduits for creativity. New forms of computer art such as hyper-linked fiction, digital image manipulation, interactive graphics and interactive environments are interesting and significant artistic developments. It is more difficult, however, to find a viable computational outlet for drawing, painting, graffiti, for a true individuality of gestural, expressive style.

The very qualities that define computers as great tools for publishing and communication uses can be an impediment when we consider various implementations of drawing and painting programs. When we wish to sketch, doodle or gesture, we usually turn to different tools: pens, crayons and brushes. This is not because computational

environments do not engage the user in expressive ways, do not convey the richness of emotion and aesthetic awareness that some traditional media offer. Simulating the traditional media with computational analogues is not the answer. The answer lies in understanding and developing the innate expressive potential of computation.

This thesis is a step in that direction. It investigates and develops some aspects of the natural expressive language of computers, a language that allows individual style, computational sketching and doodling, and a feeling of having achieved a level of skill and intimate understanding of the medium. The work presented moves away from pre-programmed art tools (which allow a level of expression akin to collage) and allows the artist to program a multitude of personalized solutions. In the process, it reconsiders and reevaluates the skills of composition, line

making and dynamic, interactive forms, and approaches them at a rudimentary level within this new medium.

F .1

(8)

COMPUTATIONAL EXPRESSIONISM: Introduction

MOTIVATION

The motivation for this thesis is a desire to increase and improve the extent of possibilities for two-dimensional visual expression, in

particular for gestural expression such as drawing. It is a desire to define new outlets, new forms and methods for creativity using a computer. The thesis strives to better understand computation as a raw art material and to develop a vocabulary that will allow artists to

cultivate a closer relationship between expressive appetite and artistic product in the computational sphere.

The current state of visual digital art is greatly determined by the tools available for its production. Digital drawing tools are still modeled, to a great extent, on previous media and technologies. They are often inadequate in fulfilling the need for expression that traditional media can fulfill, and yet they hope to replace the use of paint, pastels and ink as our medium and material of choice.

Most commercially available software for visual arts' is based on one of two models: image manipulation techniques, and the replication of the visual characteristics of preceding art forms and styles. The former is an interesting beginning, but is limited by the fact that image filters provided by software packages result in a trite look, easily achieved and consistent between users. The latter revolves around direct metaphors of Van Gogh and Monet, pencils and erasers, paint buckets and spray brushes. This approach is limited by the fact that the

computer cannot hope to compete with the real thing, the metaphors are restricted, and they restrict the kind of expression we can produce. In addition, current computer interactions and interfaces are stereotyped as impersonal and lacking a certain intuitive, physical or familiar quality. The tradition or myth of engineers who design software while disconnected from anthropological concerns has spawned the extensive field of computer-human interaction design as separate from computer or software engineering.

Artists and programmers both need to better understand computation as an art medium so as to produce art pieces and tools that take full advantage of some of the characteristics of computation. Interactivity, dynamism and emergent behavior resulting from interaction among computational objects are only some of the fascinating aspects of this medium. A better understanding will give rise to a generation of digital art pieces, tools and materials that bring the same levels of emotional involvement and creative intensity in the computational realm that a dripping paintbrush, a thick chunk of charcoal or a textured piece of wood bring in the physical.

The focus should move away from duplicating the methods and materials we know from traditional media, and develop a different perspective on visual thinking. The concept of drawing with computation essentially means to program what sorts of shapes the linear motion of the hand will produce. For an artist to draw computationally means that the artist defines the patterns and colors that the brush produces, defines the behavior of the brush, provides its computational attributes, using algorithms and design principles. John Maeda2 has said that the designer must be an engineer and the engineer

'See Appendix B for a list of currently available software for visual arts.

(9)

must be a designer. As more people become comfortable with programming, these distinctions will disappear, and artists will have less need of an engineer at their side in order to bring their ideas to life. Intimate knowledge and understanding of the materials and methods are an inherent part of any category of art making. The degree of understanding and command of the medium is directly proportional to the degree of skill and expressive breadth.

To achieve an individual algorithmic style, the artist must customize the software used to create visual work. The artist must write code and eventually will create an aesthetic that is unique. As an artist's

algorithmic art matures it will achieve aesthetic qualities that are proper to the algorithmic style. This unique aesthetic or style does not mean that the algorithm will mimic work traced by the artist's hand in traditional media. Although it is just as individual, it is a separate form of thinking about visual representation.

FOCUS

The focus of this thesis is to introduce the concept of computational drawing, a method by which visual artists can use computation as an interactive and algorithmic tool to generate two-dimensional

compositions of line, shape and color. Computation is the art medium, the iterative process of drawing is the artform, the still drawings, printed on a piece of paper or displayed on a monitor are the end product.

Visual artists historically embrace new media and technologies, and strive to experiment and express themselves within the new criteria that these impose. As a result, they have been compelled to examine and adapt existing definitions of art. Computation has opened a large world of expressive potential, which is only starting to be explored. Among other things, computation has made us realize that the idea of drawing can be expanded to signify much more than the traditional tracing of static lines on a surface.

Algorithmic appearance, dynamic change and behavior are integrated into the traditional definition of a line to define a computational line. Algorithmic appearance implies a mapping of variable complexity between the gesture that is made through the input device and the visual representation on the display. Dynamic change alludes to marks that undergo some transformation over time, whether a shift in color, position, shape, or other physical attribute. Behavior means that the lines respond to each other when present on the same canvas. They can sense the presence, proximity, intersection (topology) of other lines, which in turn affect their dynamic behavior.

The objective of the work is to draw static drawings with computational lines. The act of drawing is a dynamic, interactive process of evaluating and reassessing choices, however the focus is not on interaction but on the creation of the drawing. The work centers on the concept of line making. Drawing, doodling, graffiti, sketching, the shaping of linear forms with movements of the hands are the central point of interest. The interaction is fascinating in itself, but only insofar as it embodies the process through which a drawing is generated. The process is really one of drawing, not of interactive performance.

(10)

PRODUCT

In addition to a discussion of computational drawing, the thesis includes a series of Java applets, artifacts that represent the concepts discussed. Each applet is a drawing application that presents one or more of the computational lines created by the author. Some illustrate the progression of one algorithm; others offer an array of various computational lines with widely ranging appearance and behavior. Each one allows a viewer to examine some aspect of computational drawing and computational lines, through a process of altering them and drawing with them. The applets are visualized as still images throughout the document, illustrating stages in the process of creating a drawing. Some still images represent finished drawings by the author, the end product of the computational drawing process. The

development of the work is described in the discussion section. The Java classes used to create the work are available in an annotated package. The code is freely distributed. Other artists are invited to experiment with available lines and to create their own by changing the present algorithms. There is no custom programming tool included with the code. Maeda, in Design by Numbers, provides a structured

environment where code is compiled and results are shown alongside each other. The editor is shown to the right of the display canvas. This allows designers to see the results of their code immediately. [MAE98] Since the present work is more concerned with drawing, it is beneficial to keep an element of distance between the code and the drawing. It is frustrating, clearly, for drawing to be such a bipartite process. One advantage is that this requires the artist to better understand the algorithms, instead of using a trial and error method of designing the computational lines.

All the code is available from http://www.media.mit.edu/-joey/demo/

or by contacting the author.

STRUCTURE

The Background section provides a motivational framework for Computational Expressionism. It describes the context of the work, introduces the history and significance of the term expressionism, the relevance of line and drawing, as well as the inspiration of John Maeda and his ideas on computational design.

The main section presents the body of work called Computational Expressionism, through an overview of projects created, and a discussion of Drawing with Computation, Algorithm, Dynamism, Artistic Process, Interface and Emergence as pivotal issues and questions. It also situates the present work in the context of interactive art, kinetic art, algorithmic art, and evolving (emergent/genetic) art. The section entitled Drawing with Computation serves as an introduction. It provides an outline of the author's definitions for the concept of computational lines and computational drawing. The third subsection discusses the creative process involved in this work, and introduces some examples produced by the author.

Three classification schemes permeate the organization of the text.

1. The qualities of computation that inform the present definition of

(11)

emergence. The first two make up two sections of this document, which also introduce the first two attributes of a computational line,

appearance and dynamism. Interactivity is discussed in the Artistic Process section. Emergence is introduced in its own section.

2. The work produced by the author for this thesis is grouped according to the three defined attributes of the computational line: appearance, dynamism (change over time) and behavior. The three attributes are discussed and illustrated as best as possible with still images.

Appearance is discussed in the Algorithm section. Another discussion of appearance is described in Manipulating the Canvas, where mappings of hand movement to graphical representation are presented in increasing levels of abstraction. Dynamic lines are discussed in the Dynamism section, and behavior, which is given a more rudimentary treatment than the other two, appears in the Emergence section.

3. Three groups of background work are used to illustrate the three

attributes. The subsections on algorithmic art, kinetic art, and evolving (emergent/genetic) art can be found successively under What is Skill Now? and Algorithmic Artists in the Algorithm section, Background in the Dynamism section and Definition in the Emergence section. The Discussion section presents some questions relating to the

evolution of the work, a description of evaluation, future developments and a conclusion. The Future Work subsection explains future

extensions and interests in genetic algorithms, interfaces that use drawing as a method for human-computer interaction, and developing

special programming languages for artists.

Appendix A presents a selection of work created and gives some of the technical details and Java code. Appendix B is an overview of available

(12)

COMPUTATIONAL EXPRESSIONISM: Background

background

CONTEXT

This work was produced within the Electronic Publishing Group, in the News in the Future (NiF) consortium at the MIT Media Laboratory, under the direction of Walter Bender, the principal investigator of NiF. Bender's varied interests include a desire to "build upon the interactive styles associated with existing media and extend them into domains where a computer is incorporated into the interaction." [BEND97] He professes a need for adaptive, dynamic, context-sensitive

representations of media objects, illustrated by applications such as customizable news browsers that are aware of changing environments. He helped spawn the present work by providing the inspiration of the Incredible Machine'. This is a game that hinges on the construction of virtual faux-physics environments out of computational elements that have specific form and behavior and interact in the workspace to produce working systems. The interactive parts provide an explicit

motion, shape and functionality and act/react with one another to produce emergent worlds whose behavior is both complex and interesting. The pieces are used to experiment and to solve puzzles. The game causes players to think of physics in a non-physical context. The physics model is stilted, simplified and embellished with elements that only the magic of computation allows. The elements each have a function associated with them and can be composed in ways that produce an engaging result. Bender proposed that a similar model would be interesting for drawing tools.

EXPRESSIONISM

The word expressionism has been applied to various art movements throughout history, all sharing a common preoccupation with the emotive, passionate qualities of material, subject and method, a desire to emphasize expression over realistic representation. [LYN97]

The original movement, exemplified

by artists such as Oskar Kokoschka,

Georges Rouault and Egon Schiele, flourished in Europe in the late 19th and early 20'h century. It is

characterized by a desire to depict more than objective reality, and focus on subjective emotions and responses that subjects evoke. [BR198] This is accomplished through distortion, exaggeration, primitivism and fantasy, as well as a vivid, violent, or dynamic

Drawing by Egon Schiele application of formal elements.

(13)

The German Expressionists developed a style notable for its harshness, boldness, and visual intensity. They used rough, distorted lines; crude brushwork and shrill colors to depict urban street scenes and other contemporary subjects in crowded, agitated compositions notable for their instability and their emotionally charged atmosphere. Many of their works express frustration, anxiety, disgust, discontent, violence, and generally a sort of frenetic intensity of feeling in response to the ugliness, the crude banality, and the possibilities and contradictions that they discerned in modem life. [BR198] [LYN97]

Abstract Expressionism refers to a body of American (centered on New York) painting executed in the 1940s and 50s. Work from this period, exemplified by artists such as Jackson Pollock, Willem de Kooning and Mark Rothko, was

non-representational and emphasized spontaneous, personal expression. This period of art history was also characterized by a great amount of experimentation with materials and

technique. [HAR97] [BR198] Painting by Mark Rothko

Artists were fascinated with the intrinsic expressive qualities of the materials, leaving behind traditional elements of composition and using the material naturally and intuitively to evoke sensual, violent, dynamic, mysterious, and lyrical element. The paint and tools were manipulated to express the force of the "creative

unconscious" in art, akin to surrealist

Painting by Willem de Kooning automatism. [BR198]

The preoccupation with and merging of material and subject is best illustrated by the work of Pollock, who used paint drippings as both style and content. Arthur Danto uses the term "the end-of-art condition" to describe "the fact that style becomes subject matter, and hence is something shown rather than used". [DAN97] The objective of Computational Expressionism sustains the spirit of these artists, by seeking out the natural expressive language of computers, to spawn an eloquent freedom, a vernacular of individual style and a level of visceral understanding of the medium.

LINE AND DRAWING

The line is crucial in this work, because computational expressionism is primarily concerned with the gestures that are made, that are reflected in the on-screen representation. Traditionally, the process of drawing involves hand gestures, dragging a soft or hard tool over a surface, to leave a deposit or an abrasion, a tracing. A drawing is the end product of a successive effort applied directly to the medium with a tool, producing linear additive or subtractive forms and silhouettes that follow and represent the hand's motion. Drawing is the art or technique of producing, with the hand, images on a surface, by making marks

(14)

with ink, graphite, chalk, charcoal, or crayon. It is a two-dimensional composition, relying aesthetically on gesture and line, an abstraction, lesser or greater, of spatial objects in the world to lines drawn on a plane. Drawing is distinguished from painting by an emphasis on form and shape, rather than mass and color. It is contrasted with other graphic arts in that a direct relationship exists between production and result, between the hand and the eye.

The line is an essential ingredient of drawing, although it can just as well be used to represent areas of tone or color. The line is an essentially abstract figure, absent in nature and appearing only as a border setting of bodies, colors, or planes. In art, it has been the vehicle

of a representational, more or less illusionist, rendition of objects. Only in very recent times has the line been conceived of as an autonomous element of form, independent of an object to be represented. [BR198]

Top left photo by Sawad Brooks. The drawings are made with ink and chalk, by Joanna Berzowska

Lines, depending on the medium and tool that shapes them, can have a large range of physical and dynamic qualities. They can be thick, regular, immutable, erased. The makeup of the materials used and the temperament of the gesture (speed, direction, and pressure) determine the visual nature of the lines. A combination of line shapes, even without reference to the medium used, provides the artist with a profusion of subjective form for the expression both of general stylistic traits and of personal temperament.

JOHN MAEDA AND THE ACG

John Maeda has been concerned with issues of computational aesthetics for many years, most recently as director of the Aesthetics &

Computation Group (ACG) at the MIT Media Laboratory. With his students, he is performing experiments in information navigation, models of digital performance, tangible interfaces, concrete

programming, and digital expression. He leads the study of "expressive aspects of computer-human interface from the viewpoint of traditional visual communication design"'. He is working towards developing a

(15)

new understanding of the field of computational aesthetics by forming a new breed of worker who simultaneously performs the roles of artist and of digital engineer. [MAE97]

Maeda is striving to define and realize the distinctive potential of computation as a powerful aesthetic instrument. He claims this potential is often misused in popular work, due to the ubiquity of simplistic and unimaginative digital design tools. Current tools

commodify the process of creating visual forms and attract underskilled designers. The issues associated with questions of originality as well as the pervasiveness of the medium compromise the quality of work.

DESIGN BY COMPUTATION

This thesis is inspired and informed by a course Maeda taught at the Media Laboratory in the spring of 1997. The course encouraged the exploration of computation as an expressive medium and emphasized the unique expressive properties of the medium in order to develop a set of extensions to the designer's creative vocabulary. Problem sets for the class resulted in a body of Java applets that illustrated elements of traditional design in an interactive or reactive computational form. Graphical elements, point, line, balance, rhythm, were programmed and could respond to user input (clicking and dragging the mouse across the canvas or drawing on a tablet) in ways that manipulated these elements of design and exposed aspects of composition to the viewer.

Stills from the Thales6 applet by Joanna Berzowska

Stills from the Rumba applet by Joanna Berzowska

Thales6 was created to explore and illustrate the element of repetition in composition. The Java program draws a series of concentric circles on the canvas. The image is interactive: the center of the circles follows the movement of the cursor. The constraints of the design are defined, but also invite the viewer to alter parameters of the work in order to give both visual depth and formal complexity to the composition. As designer, the programmer can experiment with shape, distance and behavior of the composition.

In Rumba, the illusion of dimensionality is heightened by decreasing the distance between adjacent shapes, and by constraining the smallest elements within the largest ones. Such an interactive study of an aspect

(16)

of design essentially alters the process of sketching. Instead of making several drawings by hand to explore repetition, the process demands a more thorough stage of planning. Sketching is paralleled in the repetitive and iterative process of programming and allows more thorough experimentation with form once the program is completed. Another element of design is shape, and the following stills are taken

from an applet that explored space, in particular form and contour. The problem asked students to investigate union and intersection of squares. When two objects are placed on a canvas, how do they interact, how do we view them, how do we represent them as a single shape? How do their silhouettes merge, intuitively, mathematically, and visually?

Four stills from the Sqr2 applet by Joanna Berzowska

To think about shape, within the realm of computation, one must first define intersection and union. The function for determining the area that is the union of both squares, given that the squares can be moved and scaled at will, is quite complicated, much more complicated, in fact, than determining the same information by hand. Concepts of mathematics can be used, but also can be twisted and reinterpreted. The union of squares, in the example above, is shown in an

architectural aesthetic, and encompasses more that the sum of the areas covered by each square. With computation, we can represent union as an extension in space, remaining within the parameters of geometrical union and intersection, but extending the concepts, and the lines, to achieve a visual balance.

The sum of questions posed by Maeda during the course inspired the present inquiry into the nature of computational drawing.

LL

(17)

COMPUTATIONAL EXPRESSIONISM

computational

Sherry Turkle has pointed out that the computer differs from other media in that it is both a constructive and a projective medium.

[TUR84] In the physical world, you cannot change the laws of nature to fit, for example, naive theories of motion. Computation allows

abstractions, exaggerations and playful representations that illustrate intuitive conceptions of physics. It also allows representations that completely disregard the laws of physics, and allows the construction of fantasy worlds. Computation invites us to represent, but also to interpret the world around us, through experimentation and abstraction. In this example, the line traces the movement of the cursor, which follows the gesture of the hand, holding a mouse or a pen. The tracing is displayed as a digital shadow on the monitor. The gesture is perfectly reflected in the pixels, until, several seconds later, the traced line begins to disappear from the monitor like a woven thread unraveling'.

Visually, the black bar acts as a magnet or a spring coil that pulls and erases the drawing.

Stills from the Linel applet by Joanna Berzowska

This illustrates the significance, but also the opportunities, provided by the task of drawing with a computer. In this realm, a line can be programmed to have behaviors that are unexpected, lyrical, metaphoric and even (though this serves no obvious purpose) unrelated to the motion made with the hand. The laws of physical motion, the laws of immutability of matter and other physical laws that we have learned from traditional drawing are similarly rendered invalid. We are free to pursue any interpretation of our ideas about drawing, and our ideas of the meaning of gestures.

Computational lines differ from those found in the physical world, whether on a paper surface or stretched out in space, projected onto a wall or implied, the boundary between two planes. They differ insofar as their appearance and behavior does not need to follow any laws of physics. They are not bound by the ballast of physicality, and thus can change form more easily than their physical counterparts. The very elements of composition and design become open to inquiry.

1 A straight line segment is drawn joining the point (x,,f(x,)), the oldest-drawn point, and

point (xfn.3),0), in the black bar at the top of the canvas, with latency. The pixel is

(18)

COMPUTATIONAL EXPRESSIONISM

This set of images is one solution to a problem posed by Maeda, to think of a representation of the transformation of three objects into two. This sort of magic is possible with computers, and a simple click of a mouse can launch a method that will transform a set of pixels into a form completely unlike their original state. In this case, the magic is simply that of thinking of space in terms of line and shape.

Three clicks of the mouse originally determine three points that describe the placement of three lines. Each point is represented as the vertex of a two-segment line. The y-coordinate determines the vertical position, the x-coordinate determines the angle at which the two segments meet. The subsequent click fills it the two areas delineated by the lines, as positive space polygons. The illusion of three objects becoming two is clear and simple.

Stills from the THREE applet by Joanna Berzowska

Computation allows a great fluidity of the formal elements of composition that were formerly thought to be static. It forces us to rethink very basic elements of composition and design, instead of

(19)

COMPUTATIONAL EXPRESSIONISM: Drawing with Computation 19

DRAWING WITH COMPUTATION

COMPUTATIONAL LINE

The constrained and stylized character of current two-dimensional digital art is directly related to the fact that most artists work with already existing tools. Programmed drawing tools are much more powerful aesthetically than traditional drawing media. A single

algorithm can generate whole images or patterns whereas a piece of chalk, no matter how dark, can only trace very elementary shapes. The stylistic nature of computational tools can influence the visual nature of the art to a greater extent. Algorithms are so controlling that we must carefully choose, as well as author, the tools and the methods with which we decide to create and manipulate digital media.

Calligraphy 1 is an early example of a computational line, created to

suggest the sort of difficult drawing that can be done with a fountain pen on a napkin. The fibers in the napkin interfere with the smooth motion of the pen. Blotches of ink can appear if the motion is paused. The thickness of the drawn line is determined by the speed with which

it is drawn. The direction of the drawing also affects the texture of the line. The line becomes thicker, and generates blotches when the speed decreases considerably. To create shapes, or filled areas of ink, one must slow down the motion of the hand.

Four stills from the Calligraphy I applet by Joanna Berzowska

This example shows how a single line tool (from now on referred to as a "computational line") allows one to draw lines of different thickness and to create shapes of various sizes. The size of the shape is a function of the time the line is paused (with the finger on the mouse button), and the shape orientation is determined by the direction of drawing.

(20)

COMPUTATIONAL EXPRESSIONISM: Drawing with Computation

The Calligraphy 1 line is very basic, the representation of the hand

gesture is closely mirrored by the algorithm, with minimal embellishments. It is something that will be referred to as a "direct line". It is also a static computational line: it follows the movement of the input device and leaves static remains on the display. As will be discussed in later sections, the computational line can also be portrayed as a pattern of shapes that holds a more abstract relationship with the gesture.

COMPUTATIONAL DRAWING

We "draw" on the computer monitor by dragging a mouse over the table. A computational line produces a visual outcome augmented by the representation of position, speed and direction of drawing. The hand gesture itself is encoded as a vector of point object. Each point object is a Cartesian coordinate pair and a time parameter, which records the exact time of creation. The points the line traverses and their temporal ordering are factors in determining the line's appearance. The following stills show some simple gestures drawn with the

WHEAT line, so named because of its appearance of wheat swaying in the wind. The first drawings illustrate single gestures, so as to highlight the versatility of this computational line. The left panel represents a movement from the bottom right corner to the top left. The right panel is a visualization of the inverse movement. The following two panels illustrate slower, more detailed movements. The spacing between consecutive strokes varies with the speed of drawing. The tilting direction preserves the direction of the gesture, the curved endpoints directly imply the direction of the hand's motion. It is an intuitive, albeit more abstract drawing tool. Complex shapes can be created by a very simple gesture.

on t l

The gestures above generated the WHEAT line compositions on the lett.

(21)

The next two panels illustrate a different use of the WHEAT line. Instead of creating linear shapes, it is used to create texture and areas of color. The gestures that generated these compositions are repeated movements back and forth, akin to shading with a pencil. The character of the line is evident from the shapes that are achieved.

\N

The shading gestures above generated the WHEAT line composition on the right.

The shaded gestures above generated the WHEAT line composition on the right.

Stills from the WHEAT applet by Joanna Berzowska

(22)

-COMPUTATIONAL EXPRESSIONISM: Drawing with Computation

CREATIVE (EVALUATIVE) PROCESS

The following study is an exploration of the WHEAT line. Starting with the computational line illustrated above, which offers a wealth of versatile expressive possibilities, variations in the algorithm are explored to achieve a variety of line tools. Varying the length of the stroke, the amount of latency, the smoothness of the curve, direction, and other characteristics produce a wide range of computational lines. The process of computational sketching, alluded to in the previous section, consists of making changes in the code, compiling and running the drawing program and making some sketches on the canvas. The drawings executed with the computational line instruct the next iteration of the algorithms. Changes can be saved to catalogue the progression of the computational line. The code evolves, and the algorithms are refined and varied. This iterative bipartite process of programming and drawing produces an individualized algorithmic style

for this particular piece.

The gestures above generated the compositions on the left

Stills from the WHEAT applet by Joanna Berzowksa ... .... ... .... .. ...

(23)

&0

The gestures above generated the compositions on the right.

The most successful algorithms are saved and incorporated into a formal applet. A color palette is selected by the artist for this particular computational line, and the resulting drawing tool is used to create more complex compositions, as illustrated in the last two small panels in this series. The final panel shows the simple interface for this particular example, which will be explained in the Interface section.

rrn,

Interface of the WHEAT applet by Joanna Berzowska

We will see in later sections that a computational line (representation of a gesture drawn with computational tools) has three attributes.

* The computational line has physical appearance, which can be a set

of points joining two endpoints; it can be a shape, a pattern, a representation of a mathematical algorithm, a color.

* The computational line has individual behavior: dynamic properties

such as a change of color over time, or movement across the canvas.

* Finally, the computational line has behavior in its interaction with the

other lines on the canvas. It can push them away with pseudo-magnetic forces, change their color, or affect their shape.

(24)

COMPUTATIONAL EXPRESSIONISM: Algorithm

ALGORITHM

Mathematics is the majestic structure of man to grant him comprehension of the universe. It holds both the absolute and the infinite, the understandable and the forever elusive. It has walls before which one may pace up and down without result; sometimes there is a door: one opens it-enters-one is in another realm, the realm of gods, the room which holds the key to the great systems. These doors are the doors of the miracles. Having gone through one, man is no longer the operative force, but rather it is his contact with the universe. In front of him unfolds and spreads out the fabulous fabric of numbers without end. He is in the country of numbers. He may be a modest man and yet have entered just the same. Let him remain, entranced by so much dazzling, all-pervading light. [LEC80]

DEFINITION

An algorithm is a "process, or set of rules, usually one expressed in algebraic notation, now used esp. in computing, machine translation and linguistics." [OXF98] It is a systematic, finite set of steps,

processes or operations that produce some result, such as the answer to a question or the solution of an equation.

Procedural models such as fractals allow the artist to create a high degree of complexity with relatively simple input information. Similarly, using combinations of functions as fundamental as sine and cosine can easily produce engaging compositions of the plane. On the right are pictured some early experiments. The x and y coordinate values of each pixel in the plane are used to generate the color values that are then painted on that pixel. The values obtained from the position of the cursor, either through clicking, or tracing the input device over the virtual canvas, add another parameter to the methods that determine color. Through experimentation, complex and interesting compositions are created from simple primitives of algebra and calculus.

MATHART

by Joanna Berzowska

Algorithms and computer programs are invariably procedural

constructs. A common reaction is that a mathematical formula and the graphs of functions cannot be considered art, as they are the products of a formal and directed process. It is true that a drawing program does not simply make art, but artists do not simply make art either, they follow a method. [VER94] What is often called talent or intuition is a complex form of reasoning, so complex, in fact, that we do not know how to describe it in words. Just as a traditional artist follows a learned and well-defined procedure in the course of creating, the algorithmic artist also uses a more rigid procedure of iterative reformulation of algorithms. The one essential element to the process, a developed

(25)

artistic procedure, is necessarily unique for each artist and for each work of art. The final composition, particular to each work, embraces more than computation, insofar as it is the product of creative approaches, reference, metaphor, memory, representation and

improvisation. [VER94] It is the product of the artist's experience and level of comfort with the medium, and is inspired by a history of sketching in computation and other media.

FIRST ATTRIBUTE: APPEARANCE

Every computational line has a component that is merely its physical appearance. That is the visual mark it leaves on the canvas. If the line is dynamic, its appearance can be thought of as its initial state as it is drawn, or simply its visual representation at any given moment in time. The rudimentary building block of a computational line is the vector of coordinate points that describe the drawn gesture. The appearance that we see represented on the drawing canvas is necessarily mathematically determined and incorporates elements of computation that extend the vector. We can think of the vector of points as a spine, and the computational line representation as ribs, or tentacles that the spine controls. It is important that what we see on the canvas can be directly related to gestures that are made. No matter how complex and evolved it is, the appearance must be governed, must be drawn by the hand.

These compositions are drawn with a traditional pencil, repeating the same motion several times to achieve a pattern of lines. A program can generate all the ribs, given the spine, and reflect the speed and direction of

Hand sketch by Joanna Berzowska drawing.

This particular sketch inspired the GRIDS applet. GRIDS started out as a desire to abstract the line itself into a series of endpoints of secondary ribs, lines emanating from a common point. In this case, the line drawn

in fact defines the border, the boundary of one or several shapes.

The compositions on the right were generated by the above gestures.

Stills from the WHEAT applet by Joanna Berzowska

The point of convergence of the ribs is determined either during the programming stage, or during drawing, where an initial click of the mouse marks the point of convergence, and the subsequent gesture

(26)

delineates the shape. Complexity can be added by defining more than one point of convergence, and moving the points onto, or far off the canvas. Examples are presented in the Development subsection.

RANDOMNESS AND THE HAND

These three panels were created with the same computational line consisting of a parametric curve algorithm. The quality of the drawing is immensely affected by the gestures that are made with the line.

The first panel illustrates a slow line drawn from the top left corner to the bottom right. The middle panel depicts a more irregular gesture that traces over the canvas several times. The right panel shows an erratic line that deviates greatly from the diagonal, and

The compositions below were generated by moves quickly in opposing directions. It the above gestures _{still retains some of the quality of}_{the shape}

but is expressive of the gesture.

Stills from the POPPY applet by Joanna Berzowska

The visual form exhibits a tension between pattern repetition and variation. The expressive, agitated quality of the drawing is attributed

to two factors.

First of all, the speed of the machine introduces a level of randomness into the drawing. Some aspects of the drawing are impossible to control because of the inability of the processor to keep up with the hand. Some algorithms are impossible to program because of speed issues,

and thus certain design decisions are based on the abilities of the technology to do what we want it to do. If an algorithm is too slow, we must change it, or eliminate it from our repertoire. This is not

necessarily different from other media, which also have their limitations. The difference becomes evident, however, in six months' time, when the things that were too slow suddenly become adequately

fast. Computation is a unique medium insofar as cutting age

technology, increasing speeds or storage capabilities often motivate art pieces. It is not necessary to be cutting edge or take full advantage of the technology to make meaningful computational art, although it does allow the artist to make things that are considered innovative.

Secondly, computational drawing integrates algorithm and hand gestures (drawn lines). We are able to create visual artifacts that display mathematical uniformity together with a more "human" irregularity. The unexpected variations, the playful imperfection of the human hand are a quality of expression that simple computer programs cannot presently replicate.

(27)

In one panel in his class, Maeda addresses the question of

indeterminacy as computation. He asks his students to question the nature of randomness and its aesthetic merits. He suggests that randomness is crass and overused, but can be used constructively to express surprise or the general unpredictability that is a common daily experience for most of us. Because in computation there is no such thing as true randomness, any number called random is still generated

by an algorithm. If we understand the nature of this algorithm, we can

use unpredictable behavior while still retaining knowledge and control of its actions. It is very important, for Maeda, to have complete control over the medium. [MAE98]

In the case of computational drawing, there are two levels of control. The first occurs in devising the algorithms, and the second consists of the hand-based gesture interaction that occurs while manipulating the algorithms. The control we exercise with the hand can be regarded as random, or certainly less deterministic than the control we exercise by asking the computer to render an equation. The use of irregular strokes can be compared to the introduction of randomness in image dithering. Because we cannot predict precisely where the stroke will be drawn, we are striving more for cloudy forms and textures.

The compositions above were generated

by these gestures.

Stills from the TUNNEL applet by Joanna Berzowska

The TUNNEL line can produce all of the above compositions of line and shape. As a single stroke, it creates series of fragmented circles of increasing diameter. The diameter increases as a function of position in the plane and direction of movement. Using the single computational line in different ways creates distinct algorithmic compositions. A single gesture is represented as a pattern, determined by the movement of the gesture. The rapid hand movement over a single area, shading areas of the canvas, creates areas of tone and shapes that emerge from these areas of tone. The shading is analogous to shading accomplished with a piece of chalk or a pencil on a piece of paper, but an algorithmic line determines a more complex pattern. The pattern is a function of position and direction of movement.

(28)

WHAT IS SKILL NOW?

The art of drawing has historically been the domain of skilled and talented artists who devoted much time and effort to the creation of images. Skills combined with creativity were prized. Artists spent considerable time in the studio, working through interminable sketches, to master these skills. Computers now provide a rapid and simpler process for creating many graphic representations that earlier

demanded a high level of skill. Computers offer easy access to precise detail and high resolution, to photo-realistic representations, stylized illustrations and technical sketches.

Sutherland's Sketchpad was the first program that demonstrated interactive drawing with a lightpen, directly on the CRT. It could create, manipulate, duplicate, and store engineering drawings on a computer. [SUT63] Considerable work has also been done for creating black-and-white illustrations, generally for engineering or graphical design work. Many programs are available to create design, layout and engineering drawings. The use of computation for line drawing involves creating specific sketching tools and beautifiers, which either make the drawing more geometrical, or more irregular, depending on whether the aim is to be ordered and precise or to emulate a hand-drawn aesthetic. High-resolution vector graphics also offer fun options for manipulating curves.

The automatic beautifier for drawings cleans up schematic drawings by removing hand-drawn irregularities. The system allows quick polygon-oriented sketches to be transformed into precise illustrations where lines are straight, precisely parallel and commensurate, sides of polygons are collinear, and vertices are aligned. [PAV85] [BOLZ93] Another type of application, Squiggle, [DEN95] [PRE93] adds the irregularity of the hand-drawn to static, geometric CAD output to make the results appear as if they were drawn without a computer. It

introduces a looser, more energetic look to computer-generated presentations by randomly tweaking the data file that specifies straight lines and curves. Pen-and-ink is an extremely limited medium, allowing only individual monochromatic strokes of the pen, yet skilled artists can create beautiful pen-and-ink illustrations incorporating a wealth of textures, tones, and styles. Salisbury's [SAL97] interactive system for creating pen-and-ink-style line drawings from grayscale images features a method where strokes of the rendered illustration follow the features of the original image.

A skeletal stroke [HSU931 [HSU941

[CRE98] is a drawing tool and image transformation instrument that changes the shape of pictures as if by bending or twisting, while conserving the aspect ratio of selected features on the picture. It is a vector graphics realization of a brush metaphor that uses arbitrary pictures as ink. The strokes are hierarchically structured so that they use a single skeleton to control an entire stroke and its features.

(29)

These methods are very powerful for illustration and technical drawing. Computer technology for producing more abstract and expressive work is less advanced. One option is to use one of the many available paint programs, which in their attempts to emulate traditional art tools often produce a sterile pastiche, a simulacrum, whose easily categorized style and generic aesthetic do not satisfy. Another option is to use image manipulation techniques, filters and other de-constructive methods to alter an existing image. Most of the published work on digital painting is concerned with the problem of emulating the process of traditional artists' tools, and creating both filters and interactive tools that produce the results that look similar to those produced by their predecessors.

One such method creates an image with a hand-painted appearance with a series of spline brush strokes chosen to match colors in the source photograph. Visual emphasis in the painting corresponds roughly to the spatial energy present in the source image. [HER98] This approach is noteworthy in terms of several innovations, long, curved brush strokes, a varying brush size, and changeable rendering styles.

Haeberli [HAE90] shows how scanned or rendered image information can be used as a starting point for painting by numbers. Many

companies sell products that combine the stylistic expressiveness of traditional artist tools with the speed, flexibility and resolution

independence of vector-based drawing. They are listed in Appendix B. Traditionally, artists are trained in hand-to-eye skills, not the more obviously rational skills necessary to conceptualize the mathematics of drawing a line from the point (90,30) to the point (10,140). Therefore, the current tools that invite digital image expression tend to shroud the tools' computational aspects, and provide a user interface that inserts a layer of artifice between the user and the tools. This layer of artifice introduces stylistic constraints, both by pre-defining aesthetic choices, and limiting the set of possibilities. Artists carry over skills from their previous work, and continue to use the new medium in the old way.

Needless to say, this is inadequate. The skills necessary to develop an expressive proficiency in the medium are those of thinking

mathematically, of programming, of translating artistic vision into a concise, mathematical algorithm, and approaching composition in an explicitly procedural way.

Composition created with the JoeyGraphics Java class

The importance of skill, of the mastery of a medium is partially lost in using commercial image software. The act of programming, however, provides an environment that must be learned and conquered, the long road that must be traveled and explored so as to develop a closer relationship, a conspiracy, between the artist and the medium. This sort of complex understanding is necessary to move beyond a preoccupation with the medium, and allow a greater comfort that will cause the artist to concentrate on content and meaning instead of tools.

(30)

ALGORITHMIC ARTISTS

This sort of thinking is not new, and many artists have been working with algorithms to create visual art. Algorithms are after all simply a set of steps or processes necessary to execute a task. It can be argued that the act of drawing in a more traditional medium is also an algorithmic process, [VER98] but a far more complex one. Algorithms are not a phenomenon unique to programming, but the advent of computers has given visual artists have the technology for composing art works algorithmically in ways than were unavailable before. [VER98] This subsection is an algorithmic art gallery, with sample images and quotations from a set of algorithmic artists.

Touching is a very broad concept. In these images, lines are playing a game of touching, of near-touching, of avoiding, of seeking, of crossing and intermingling: a manifestation of the purity of the line and an invitation to meditate.

Algorithmically generated drawings, drawn on a pen plotter, constitute a very small segment within the area of computer artwork. It is this small segment, however, which I find most fascinating. This has to do with the

archaic notion of a mechanical extension

to the drawing hand, unlocking a universe

of machine-generated drawings utterly

different from hand drawings. [DEH98] b97.9.3 by Hans E Dehlinger

My interest in computer imaging is

producing images that have never been imagined. That means I am not reproducing the images in my brain but

accepting or selecting the images generated by math based processing by a preset algorithm. Computer is my collaborator and my role in the creation process is setting parameters, changing surface attributes and waiting for the

Yoshiyuki Abe _{results processed by the computer.}

[ABE97]

The computer became a physical and intellectual extension in the process of creating my art. I write computer

algorithms i.e. rules that calculate and then generate the work which could not be realized in any other way. My artistic goal is reached when a finished work can dissociate itself from its logical content and stand convincingly as an independent abstract entity. [MOH98a] My art is not a

mathematical art, but an expression of my artistic experience. [MOH98b]

(31)

Roman Verostko

Most of my work for the past 40 years has been with pure visual form ranging from controlled constructions with highly studied color behavior to spontaneous brush strokes and inventive non-representational drawing. Such art has been labeled variously as "concrete", "abstract", objective", and

"non-representational". In its purest form such art holds no reference to other reality. Rather one contemplates the object for its own inherent form similar to the way one might contemplate a flower or a seashell. Procedures for creating such art have been evolving from the work of its pioneers in the first decade of this

Roman Verostko century. [VER98]

The world of forms available for artists who create such art is vast, an uncharted frontier of "unseen" worlds waiting to be "discovered" and concretized. With the advent of computers, I began composing detailed procedures for generating forms that are accessible only through extensive computing. My on-going work concentrates on developing this program of procedures for investigating and creating such forms.

By joining these procedures with fine arts practice, I create objects to

be contemplated much as we contemplate the forms of nature. [VER98]

I started this so long ago that many of the

wonderful tools which are available now to artists did not exist. At that point I was forced to make my own and I had to program them. This is the origin of my work, which is generally personal in its goal and forms. Everything I do is based on mathematics and geometry and line arts. I chose to use a plotter because it was the most affordable piece of hardware I could put my hands on at

the time. ... It was a very dynamic process for

me and this process is going to end because I am using a technology which is becoming

(32)

Interruptions, 1968 Vera Molnar Variations Sainte-Victoire, 1996 Vera Molnar

The image obtained by a painter using a computer stops being an accumulation of unknown badly defined forms and colors. It becomes instead a pattern of thousands of distinct, intermittent, and quantified points. The position in space, the colorimetric values of these thousands of points, are perfectly defined and numerically accountable. In this way, the painter controls each one of these points. At any moment, the artist is able to modify the value of one or several points, or even the total number of them. As a result, innumerable successive approaches (many sketches, to use the accepted history-of-art term) can be shown on the screen. Proceeding by small steps, the painter is in a position to delicately pinpoint the image of dreams. Without the aid of a computer, it would not possible to materialize quite so faithfully an image that previously existed only in the artist's mind. This may sound paradoxical, but the machine, which is thought to be cold and inhuman, can help to realize what is most subjective, unattainable, and profound in a human being. [MOL98]

With technology it is possible to manifest mathematical ideas as images, sounds, sculpture and even poetry. Artists in all media have found mathematical processes of value in their creative enterprise. These processes are often described using algorithms. In describing mathematical processes with algorithms, beauty and meaning can be discovered. Numbers are mapped into light and/or sound, and perceived through the senses as objects. It is the mathematical source of these works that has aesthetic worth. Algorithms, implemented on computers, make it possible for us

to see and hear the beauty of mathematical

processes. [EVA98]

I had clear ideas about abstract form in

painting and intuitively recognized the potential for expanding this new twentieth century visual language

through the computer. ... I seek to expand a radical new language in art. I do not feel impressed by ray tracing or

by scanning photographs. I can draw

well enough and render in traditional

academic ways. ... It is not the

computer's ability to imitate optics that I find fascinating. The computer, as a digital media, has allowed me to deepen visual ideas of form and illusion not

(33)

In my early work, I created a sense of presence of invisible forces in nature. For me, these forces in nature are metaphors for the interpersonal dynamics between people. I created algorithmic images, using mathematical descriptions of phenomena such as light reflecting off of

irregular surfaces, that embodied these dynamic forces. In these drawings, environmental phenomena that we sense, like the wind, were visualized and given a physical presence. Algorithmic patterns were also created on fabric using heat-transfer xerography. This mapping of environmental behaviors onto cloth propelled this algorithmic representation back into the natural

worl. [TU98]Construction E5, 1975 Joan Truokenbrod

world. [TRU98]

My drawings and paintings are made in a

two part process. First, images are generated on the screen of a microcomputer using a variety computer programs. Some of these programs employ simple random

procedures, others utilize permutations of graphic elements such as cellular

automations. Next, a rectangular section of the image is plotted, pixel by pixel The pixels can be drawn as circles, filled boxes,

i

IIcrosses,

and so forth. They can be large or

small, and can be mapped onto various geometric surfaces, such as planes,

cylinders, and cones, Finally, these surfaces

Mark Wilson are projected into perspective space. I have written all the software. [WIL98]

18 G 90 Mark Wilson

These working procedures offer much greater versatility and freedom from the traditionally small photographically-based computer graphic formats. The overall picture-making process is reminiscent of collage. Elements are selected and placed on the surface. Compositional decisions are made step by step. Thus, the final appearance of the image is not predetermined by the machinery, but by artistic judgment. It would be impossible to realize my works using any other medium. I have attempted to directly use the digital nature of this medium. Rather than trying to disguise pixels, they have become the central element of my artmaking. Technology underlies all of my working procedures, but the ultimate goal is a simple one: to delight and intrigue the eye of the viewer with images that can be seen only with the aid of a computer. [WIL98]

Computational expressionism : a study of drawing with computation

computational

expressionism

MAR

19 1999

LIBRARIES

ROTC"

abstract

computational

expressionism

acknowledgments

contents

introiduction

assiduous, penetrating and pervasive infiltration

of our homes and lives. They are quickly

4.A

MOTIVATION

FOCUS

PRODUCT

STRUCTURE

background

CONTEXT

EXPRESSIONISM

LINE AND DRAWING

JOHN MAEDA AND THE ACG

DESIGN BY COMPUTATION

computational

DRAWING WITH COMPUTATION

COMPUTATIONAL LINE

COMPUTATIONAL DRAWING

CREATIVE (EVALUATIVE) PROCESS

&0

rrn,

ALGORITHM

DEFINITION

FIRST ATTRIBUTE: APPEARANCE

RANDOMNESS AND THE HAND

WHAT IS SKILL NOW?

ALGORITHMIC ARTISTS

i

IIcrosses,