VECTOR SPACES
The research described in this chapter arises from the author’s work in designing and building a wearable graphics production facility used to create a new kind of visual art over the past 15 or 20 years. This work bridges the gap between computer graphics, photographic imaging, and painting with powerful yet portable electronic flashlamps. Beyond being of historical significance (the invention of the wearable computer, mediated reality, etc.), this background can lead to broader and more useful applications.
The work described in this chapter follows on the work of Chapter 4, where it was argued that hidden within the flow of signals from a camera, through image processing, to display, is a homomorphic filter. While homomorphic filtering is often desirable, there are occasions when it is not. The cancellation of this implicit homomorphic filter, as introduced in Chapter 4, through the introduction of an antihomomorphic filter, will lead us, in this chapter, to the concept of antihomomorphic superposition and antihomomorphic vector spaces. This chapter follows roughly a 1992 unpublished report by the author, entitled “Lightspace and the Wyckoff Principle,” and describes a new genre of visual art that the author developed in the 1970s and early 1980s.
The theory of antihomomorphic vector spaces arose out of a desire to create a new kind of visual art combining elements of imaging, photography, and graphics, within the context of personal imaging.
Personal imaging is an attempt to:
1. resituate the camera in a new way — as a true extension of the mind and body rather than merely a tool we might carry with us; and
2. allow us to capture a personal account of reality, with a goal toward:
a. personal documentary; and
179
b. an expressive (artistic and creative) form of imaging arising from the ability to capture a rich multidimensional description of a scene, and then “render” an image from this description at a later time.
The last goal is not to alter the scene content, as is the goal of much in the way of digital photography [87] — through such programs as GIMP or its weaker work-alikes such as Adobe’s PhotoShop. Instead, a goal of personal imaging is to manipulate the tonal range and apparent scene illumination, with the goal of faithfully, but expressively, capturing an image of objects actually present in the scene.
In much the same way that Leonardo da Vinci’s or Jan Vermeer’s paintings portray realistic scenes, but with inexplicable light and shade (i.e., the shadows often appear to correspond to no single possible light source), a goal of personal imaging is to take a first step toward a new direction in imaging to attain a mastery over tonal range, light-and-shadow, and so on.
Accordingly, a general framework for understanding some simple but impor-tant properties of light, in the context of a personal imaging system, is put forth.
5.1 LIGHTSPACE
A mathematical framework that describes a model of the way that light interacts with a scene or object is put forth in this chapter. This framework is called
“lightspace.” It is first shown how any of a variety of typical light sources (including those found in the home, office, and photography studio) can be mathematically represented in terms of a collection of primitive elements called “spotflashes.” Due to the photoquantigraphic (linearity and superposition) properties of light, it is then shown that any lighting situation (combination of sunlight, fluorescent light, etc.) can be expressed as a collection of spotflashes.
Lightspace captures everything that can be known about how a scene will respond to each of all possible spotflashes and, by this decomposition, to any possible light source.
5.2 THE LIGHTSPACE ANALYSIS FUNCTION
We begin by asking what potentially can be learned from measurements of all the light rays present in a particular region of space. Adelson asks this question:
What information about the world is contained in the light filling a region of space?
Space is filled with a dense array of light rays of various intensities. The set of rays passing through any point in space is mathematically termed apencil. Leonardo da Vinci refers to this set of rays as a “radiant pyramid.” [88]
Leonardo expressed essentially the same idea, realizing the significance of this complete visual description:
The body of the air is full of an infinite number of radiant pyramids caused by the objects located in it.1These pyramids intersect and interweave without interfering with each other during their independent passage throughout the air in which they are infused. [89]
We can also ask how we might benefit from being able to capture, analyze, and resynthesize these light rays. In particular, black-and-white (grayscale) photography captures the pencil of light at a particular point in space time (x, y, z, t) integrated over all wavelengths (or integrated together with the spectral sensitivity curve of the film). Color photography captures three readings of this wavelength-integrated pencil of light each with a different spectral sensitivity (color). An earlier form of color photography, known as Lippman photography[90,91] decomposes the light into an infinite2 number of spectral bands, providing a record of the true spectral content of the light at each point on the film.
A long-exposure photograph captures a time-integrated pencil of light. Thus a black-and-white photograph captures the pencil of light at a specific spatial location(x, y, z), integrated over all (or a particular range of) time, and over all (or a particular range of) wavelengths. Thus the idealized (conceptual) analog camera is a means of making uncountably many measurements at the same time (i.e., measuring many of these light rays at once).
5.2.1 The Spot-Flash-Spectrometer
For the moment, let us suppose that we can measure (and record) the energy in a single one of these rays of light, at a particular wavelength, at a particular instant in time.3We select a point in space(x, y, z)and place a flashmeter at the end of a collimator (Fig. 5.1) at that location. We select the wavelength of interest by adjusting the prism4 which is part of the collimator. We select the time period of interest by activating the trigger input of the flashmeter. In practice, a flashmeter integrates the total quantity of light over a short time period, such as 1/500 of a second, but we can envision an apparatus where this time interval can be made arbitrarily short, while the instrument is made more and more sensitive.5 Note that the collimator and prism serve to restrict our measurement to light traveling in a particular direction, at a particular wavelength,λ.
1Perhaps more correctly, by the interaction of light with the objects located in it.
2While we might argue about infinities, in the context of quantum (i.e., discretization) effects of light, and the like, the term “infinite” is used in the same conceptual spirit as Leonardo used it, that is, without regard to practical implementation, or actual information content.
3Neglecting any uncertainty effects due to the wavelike nature of light, and any precision effects due to the particle-like nature of light.
4In practice, a blazed grating (diffraction grating built into a curved mirror) might be used, since it selects a particular wavelength of light more efficiently than a prism, though the familiar triangular icon is used to denote this splitting up of the white light into a rainbow of wavelengths.
5Neglecting the theoretical limitations of both sensor noise and the quantum (photon) nature of light.
Pencil
Figure 5.1 Every point in an illuminated 3-D scene radiates light. Conceptually, at least, we can characterize the scene, and the way it is illuminated, by measuring these rays in all directions of the surrounding space. At each point in space, we measure the amount of light traveling in every possible direction (direction being characterized by a unit vector that has two degrees of freedom). Since objects have various colors and, more generally, various spectral properties, so too will the rays of light reflected by them, so that wavelength is also a quantity that we wish to measure. (a) Measurement of one of these rays of light. (b) Detail of measuring apparatus comprising omnidirectional point sensor in collimating apparatus. We will call this apparatus a ‘‘spot-flash-spectrometer.’’
There are seven degrees of freedom in this measuring apparatus.6 These are denoted by θ, φ, λ, t, x, y, and z, where the first two degrees of freedom are derived from a unit vector that indicates the direction we are aiming the apparatus, and the last three denote the location of the apparatus in space (or the last four denote the location in 4-space, if one prefers to think that way). At each point in this seven-dimensional analysis space we obtain a reading that indicates the quantity of light at that point in the space. This quantity of light might be found, for example, by observing an integrating voltmeter connected to the light-sensing element at the end of the collimator tube. The entire apparatus, called a “spot-flash-spectrometer” or “spot-spectrometer,” is similar to the flash spotmeter that photographers use to measure light bouncing off a single spot in the image.
Typically this is over a narrow (one degree or so) beam spread and short (about 1/500) time interval.
Suppose that we obtain a complete set of these measurements of the uncountably7 many rays of light present in the space around the scene.
6Note that in a transparent medium one can move along a ray of light with no change. So measuring the lightspace along a plane will suffice, making the measurement of it throughout the entire volume redundant. In many ways, of course, the lightspace representation is conceptual rather than practical.
7Again, the term “uncountable” is used in a conceptual spirit. If the reader prefers to visualize the rationals — dense in the reals but countable — or prefers to visualize a countably infinite discrete lattice, or a sufficiently dense finite sampling lattice, this will still convey the general spirit of light theorized by Leonardo.