Book Chapter
Reference
Temporal Analysis of Contrast and Geometric Selectivity in the Early Human Visual System
BURGI, Pierre-Yves, PUN, Thierry
Abstract
A new approach in image analysis is presented, which makes explicit use of the notion of time. The purpose is to extract image features, such as edges, with the constraint that pertinent features should appear first. In particular, it is desired that strong contrasts are detected before weaker ones (contrast strength selectivity), or high curvatures are detected before straight lines (geometric selectivity). A simple electrical circuit is studied first to model contrast selectivity; a relationship between contrast strength and latency can be derived. The relation between stimulus geometry and latency is then explored by means of two types of receptive fields, namely lateral geniculate nucleus (LGN)-Iike and end-stopped cells. Concrete experimental results are presented that sustain our belief that time favors asynchronous processing in order to prevent overflow in the visual system by the visual information.
BURGI, Pierre-Yves, PUN, Thierry. Temporal Analysis of Contrast and Geometric Selectivity in the Early Human Visual System. In: Blum, B. Channels in the Visual Nervous System:
Neurophysiology, Psychophysics and Models. London : Freund, 1991. p. 273-288
Available at:
http://archive-ouverte.unige.ch/unige:17932
Disclaimer: layout of this document may differ from the published version.
1 / 1
···.···.·caAN~tsLs ...•. IN •.• TI-IE.·.YI6I.JAt..···
·.· .... ·.·.N ... ·
".. · .•...•.. ·.E
',,< ---",•.•... ··· ... R ...•..
,--,-¥ ...• · •.. · ... O .. · .•.. · .. ··.·· .... l.I.6.· •.. ·.· .. · .•..•... · ..•.. 6 .... · .. ·'I.· ... · .. ·.6 .. · •.. T. E .... ·.·.M ...•.. · ...• : .•
N~I.JR9P~~S.IOLOG~, ... :;~./
P6¥CI-IOPHY'SIC6 AND<"
MODE.L6
Editor: B. Blum
@J 1991: Freund Publishing House Ltd.
London and Tel Aviv
TEMPORAL ANALYSIS OF CONTRAST AND GEOMETRIC SELECTIVITY IN THE
EARLY HUMAN VISUAL SYSTEM
Pierre-Yves Burgi and Thierry Pun
Artificial Intelligence and VISion Group, Computing Science Center, University of Geneva, 12, me du Lac, CH-1207 Geneva, Switzerland
Abstract
273
A new approach in image analysis is presented, which makes ex- plicit use of the notion of time. The purpose is to extract image features, such as edges, with the constraint that pertinent features shonld appear first. In particular, it is desired that strong contrasts are detected before weaker ones (contrast strength selectivity), or high curvatures are detected before straight lines (geometric selectivity). A simple electrical circuit is studied first to model contrast selectivity; a relationship between contrast strength and latency can be derived.
The relation between stimulus geometry and latency is then explored by means of two types of receptive fields, namely lateral geniculate nucleus (LGN)-Iike and end-stopped cells. Concrete experimental results are presented that sustain our belief that time favors asynchronous processing in order to prevent overflow in the visual system by the visual information.
Key words: neuronal model; latency; temporal precedence; contrast and geometric selectivity; image analysis
1. Introduction
This paper deals with an important dimension used in human vi- sual perception: time. There are two aspects to this notion of time.
First, evaluating the direction and speed of a moving object implies a neural circuitry that is sensitive to time differences and able to process visual signals that are not generated at the same moment. Secondly, an often neglected aspect of time concerns what happens to visual sig- nals when the retinal image is completely stabilized (single image frame). Will visual signals generated by higher contrasts occur before those generated from lower contrasts? Could some geometric ar- rangements (such as corners) be perceived before others (such as edges)? The present article is concerned with this second aspect re- lated to time and aims at answering the above two questions. It shows that a temporal precedence amongst visual stimuli can be defined and, furthermore, that stimuli having temporal precedence are those seemingly the most pertinent.
Section 2 details the temporal precedence model and is divided into two parts. The first part discusses the separation between strong and weak contrasts. Despite numerous studies of threshold selection (e.g. recently, Haddon, 1988; Hertz and Schafer, 1988; Lacroix, 1988;
Sahoo, Soltani and Wong, 1988), the problem of retaining pertinent edges (called "prominent edges" in Haddon, 1988), and suppressing non-important edges (called "weak edges" in Lacroix, 1988; "false edges" in Hertz and Schafer, 1988) has not yet heen satisfactorily solved. It will be shown that an elegant way of avoiding this problem is to associate a neuronal latency to the edge strength. One problem that is not addressed herein is that edges perceived as "true" ones may have lower contrast than edges perceived as "false" ones. It should first be demonstrated that this well known and recurrent problem is not cognitive before attempting to incorporate it in the present low level analysis.
The second part of section 2 deals with geometric features and is linked to the notion of receptive field (RF). A RF is characterized by its response profile and shape which determine its selectivity to geo- metric patterns. Again, the use of temporal analysis indicates a corre- spondence between latencies and stimulus geometry.
Some experimental results are presented in section 3. They are followed in section 4 by a more general discussion regarding the con- cept of precedence. Mathematical details of the model are given in the Appendix.
P.-Y. Burgi and T. Pun 275
2. Temporal analysis
Both contrast strength selectivity and geometric features selectivity are included in our model; they are treated sequentially for the sake of clarity. The former is studied by means of a simplified electrical circuit and the latter using single cell models.
2.1 Contrast strength selectivity
In the human visual cortex, intensities of quantitative stimuli are usually transmitted by periodic or nearly periodic trains of pulses (exceptions are found in the retina). It is also known that neurons are similar to leaky integrator elements. We are interested in a model in- corporating both features, that is, a model combining periodic pulse trains with leaky elements (Burgi et al., 1989).
Figure 1 illustrates how an intensity is converted into a rate of fir- ing. Computing elements are neuron-like and are modeled by the leaky integrator electrical model shown in Figure 2. This model is pre- sented in more detail and described mathematically in the Appendix.
All--- - n n
A . J - I - -
n
Fig. 1: Intensity translated into frequency. ton Is the duration of the spike and is supposed to be constant. The frequency f = (t + t ,,),' is a monotonic function of the Intensity: f = u . A + {3 angn
yiefi:ls tOff = (u . A + {3r' -ton with u a positive constant. Note that even a zero intensity is converted into a train of pulses of period {3 + ton'
ton toJ!
I 11 IJ 11
potential outputiute gration energy decay
Fig,2: Two modules corresponding to two stages: (1) Integration during ton
= 1.0 ms with an excitatory conductance gexo = 0.28 p.S. an excitatory potential Eexc = 20.0 mV: (2) energy decay with gexo = O. In both modes E'eak = -71.0 mV. g, •• k =0.08 p.S. C = 2 nF. threshold = -45.0 mV. Conductances are measured in [siemens] noted S.
The electrical circuit (Figure Al for details) is composed of one passive conductance gleak' one time-dependent conductance g ,a membrane capacity C and two voltage sources E'eok and E "",.""'fhe time-dependent conductance is serially linked to the positive voltage source E . The current generated by this branch is injected into (i) the capacity and (ii) the leakage pathway. In this latter pathway, the passive conductance gleak is serially linked with the negative voltage source E
1eak,
In order to ease the analysis, a simplifying assumption is made: the time-dependent conductance has a constant positive value during ton and zero value during ton A more realistic model would require a time varying conductance based upon the alpha synaptic function (for details on neuronal modeling, see Wilson and Bower, 1989).
Simulations indicate however that both models yield identical qualitative results (Figure 3).
The membrane capacity of the cell integrates pulses through the conductance gexc during ton' while the stored energy leaks through conductance gl during ton Electrical parameters are inspired from those found in Wehmeier er al. (1989). A temporal analysis of this cir- cuit shows that succeeding pulses at the entry cause the output poten- tial to oscillate while rising steadily towards an asymptotic value. This value is a monotonic function of the spike frequency. A more inter-
P.-Y Burgi and T. PUll 277
esting problem is to determine the precise instant at which the asymp- totic potential reaches a fixed threshold. The time difference between this instant and the stimulus appearance is called the latency of the cell. Figure 3 shows computed latencies for trains of pulses with fre- quencies ranging from 1 to 800 Hz. Details are to be found in the Ap- pendix.
Higher frequencies yield shorter latencies; cells with short latencies are said to have temporal precedence over cells with longer latencies.
See also in Kurogi (1987) the so-called "maximum detection" function, detecting the cells whose potentials first reach the threshold value.
The curve in Figure 3 clearly indicates that latencies increase rapidly for low frequencies and that under a certain critical frequency the threshold will never be reached. Inversely, for the upper half range of frequencies latencies are short and decrease slowly. This result is in agreement with studies in psychophysics, in particular with those found in Greenlee and Breitmeyer (1989). Consequently, high fre- quencies corresponding to strong contrast (value A in Figure 1) will exert an action first. They have, therefore, temporal precedence over lower frequencies (weak contrasts) whose exponentially deferred ac- tion becomes negligible.
Fig. 3:
100 A B
'"'
!
50"
"
c::
\~
"
;;
...l
0 I
200 400 600 800
Frequency (Hz)
Latency of the electrical model for periodic trains of pulses (electrical parameters as in Figure 2). The curve labeled A has a time-varying conductance based on the alpha synaptic function; this curve was obtained by simulation (Wilson & Bower. 1989; para- meters are: T " 1.0 ms: 9 eak" 0.28I'S), The curve labeled B has the simplified time-varying cgnductance presented In the text, and was determined using equations A 1.4 and A 1 .5 from the Appendix. For the curve B, below 89 Hz, the potential does not reach the threshold and so latencies are undefined. The critical frequency (see text) is therefore 89 Hz with a latency of 97 ms.
2.2 Geometric selectivity
Classically in computer vision (e.g. Marr, 1982), images are pro- cessed through a hierarchy of analysis levels: matrices of points of an image enter the first processing level and the resulting output matrix is transmitted to the next level. This action is synchronous, in the sense that all pixels are transmitted at the same time. Herein we want to stress the unlikeliness of such a synchronism. This has already been demonstrated above for intensity (of contrast). A similar phenomenon will be shown in this paragraph for geometric features. Key points, that is specific geometric patterns, such as edges, chevrons (V-shapes or corners), junctions and curves, have inherent latencies. This obser- vation should however be qualified: latencies are related to the detec- tor specificity and a detailed analysis requires taking detector re- sponse into account.
Two geometric feature detectors are described and used. They cor- respond to visual receptive field (RF) whose shape and response (weight profile) mimic those found in the LGN and in the visual cor- tex. Concentric on-center off-surround LGN cells are modeled by the difference of two Gaussians (DOG); for examples of parameters see Wehmeier et al. (1989). These cells are known to enhance contrasts and to respond weakly to uniform surfaces. From the visual cortex, end-stopped cells were chosen for their ability to be excited either by line ends or by corners (von der Heydt, 1987) as well as for their selec- tivity to curvatures (Dobbins et aI., 1989). They are described by the difference of two elliptiC Gabor functions (for Gabor functions, see Marcelja, 1980; Daugman, 1983) at the same position with identical orientations but one with a small receptive field (small component) and the other with a large receptive field (large component). Parame- ters are adapted from those found in Dobbins et al. (1989).
To evaluate latencies to various stimuli the weighting functions of the RF are convolved against the image containing the stimulus. Re- sults of the convolution are then converted into rates of firing and la- tencies computed as in section 2.1.
The spatial linearity of the convolution operator is acceptable if the stimulus has a constant intensity. Without this assumption of constant value, non-linearity pervades and to convolve is consequently no longer correct. In such circumstances an extension of our electrical circuit would be necessary with an inhibitory conductance for negative weights (of the weighting profile). The model would furthermore con- tain as many inputs as there would be image elements in the spatial span of the RF and each channel (either excitatory or inhibitory)
P.-Y. Burgi and T. PUll 279 would have to integrate its train of pulses. Such a model is described in Wilson and Bower (1989).
Latencies obtained confirm that key points will appear first, whereas diffuse light and edges invoke longer latencies and will ap- pear later. This fundamental result is valid for both kinds of RFs.
Furthermore, results obtained with end-stopped cells show that variation in the radius of curvature of lines implies correspondingly varying latencies (Figure 4)_ Varying responses (in terms of rate of firing) to different radii of curvature are extensively discussed in Dob- bins et aL (1989). The supplementary step performed herein relates rates of firing to latencies. It is intended to show that the curvature whose value is adapted to the RF sizes has a short latency while those differing in this optimal value have steep increases in latencies. There- fore, the more bent a line is, tbe shorter the latency of the cell, assum- ing that the smallest radius of curvature is above a certain size related to the size of the components. Sharper curves are thus favored.
Similarly, LGN cells appear to have shorter latencies for chevrons with correspondingly smaller angles between the sides of the V-shape.
In this case, however, the function relating latencies to the varying an- gle seems to be non-monotonic and further studies are necessary.
~
5 '"
~
...
'"
~
=
j
100
50
....
...
o
20 40 60 80 100Radius of curvature
Fig. 4: Latency of end-stopped cells vs. various radii of curvature of line segments. Fi~er parameters (see equation A2.4J are: WR = 2.5, AR = 3, MR = 4.3; large component = 25 and small component = 11 (see equations in Appendix A.2J. Electrical parameters are as in Figure 2.
The stimulus was centered on the peak of the positive lobe of the RF.
3. Results
Figure 5 illustrates the temporal precedence of corners over simple lines when LGN-Iike cells are used. The whole image has a constant intensity: the lines have value 1, whereas the background is set at 0;
the convolution is therefore applicable. As expected, key points Gunctions and corners) appear first.
time: 0
\
lat: 5.28 ms lat: 8.0 ms lat : 10.44 ms
Fig. 5: Latencies of features using the receptive field of a LGN-like cell (see equations in Appendix A.2.0. Circles show the loci of the activated cells. Fmer parameters are: WR = 2.5. AR = 1.0, MR = 18.0 and size = 15.
In order to show responses of end-stopped cells to stimuli with varying radii of curvature we chose the profile of a mouse (Figure 6).
The whole image has a constant intensity: the lines have value 1 whereas the background is set at 0 and again the convolution is appli- cable. We have seen before that sharper curves should be favored.
Moreover, in order to increase the sensitivity of cells over lines, a di-
P.-Y. Burgi and T. Pun 281
rectional selectivity was added using different angles in equation AZ.3.
Angles were varied between O· and 170· by steps of 10· _ Results are shown in Figure 6. Ellipses indicate the loci of the activated end- stopped cells_ Simulation was stopped at time 8.52 ms when key points were effectively detected.
A
B
Fig_ 6: A: profile of a moUse. Various radii of curvature allow testing of latencies of the end-stopped cells. B: loci and orientations of the activated cells. Filter parameters are WR = 2.5. AR = 3. MR = 4.8; large component = 15 and small component = 7; electrical parameters are those used in Figure 2.
4. Discussion
On the basis of simple neurophysiological assumptions, a new mechanism of temporal precedence has been presented: we demon- strate that the more "important" a feature is, the quicker the informa- tion contained in it is transmitted. Instead of simultaneously detecting all image elements, they are distributed naturally through time ac- cording to their relative importance. For a subsequent system receiving this ordered information, it is easier either to enhance the competition or to benefit from the cooperation between cells. The problem of deciding between pertinent edges and weak edges has been greatly simplified: the former appear before the latter.
Furthermore it has been shown that sharp curves have short latencies and thus are "important". This is in accordance with the observation that strong curvatures are determinant for the perception of shapes (Attneave, 1954).
However, this definition of "importance" is circular. We have shown that some features, a priori deemed to be important, appear first. Alternatively, we could have used temporal precedence as a means of defining the importance of a feature. We are not attempting here to answer the question whether it is the perception of visual stimuli which has incited the development of cells responding better and faster to a particular feature.
The presented model has purposely been kept simple in order to keep track of what is going on. Its domain of applicability concerns rather local stimuli; strict comparisons with more global psychophysi- cal experiments are therefore premature (e.g. Krose and Julesz, 1989;
Treisman, 1985). In spite of the limitations of this model, its plausibil- ity can be emphasized by relating temporal precedence to two psy- chophysical studies, respectively showing that precedence exists in the first visual levels (oculomotor system) as well as at higher levels (inferior temporal cortex).
The first study relates the luminance and the oculomotor system.
When the visual system is confronted with sudden stimuli, an attentive mechanism is aroused with the effect of moving the eye. It has been shown that the latency of the resulting saccadic eye movements is de- pendent upon the luminance of the target stimulus. Furthermore Doma and Hallett (1989) have shown that latencies are affected by spectral components by virtue of differences in response characteris- tics between rods and cones. Thus early in the visual pathway a tem- poral competitive mechanism is functional and implies a specific se- lection.
P. -Y. Burgi and T. Pun 283
The second study deals with geometric patterns. Supposing laten- cies in the early visual pathway do exist, they must be present at every level along the hierarchy of the visual system. Consequently, the la- tency specificity must likewise increase when going higher up in the levels. Optican and Richmond (1987) have studied, in the primate in- ferior temporal cortex, the temporal modulation of spike trains; they indeed found latencies specific to different two-dimensional Walsh stimuli, corroborating our beliefs (see also Richmond et aI., 1987;
Richmond and Optican, 1987).
5. Conclusion
The fact that signals are transmitted asynchronously between the many levels of the human visual system is often overlooked. By using very simple models, this paper has shown that a temporal analysis ap- plied to one level is already sufficient to discriminate between image features. Reasoning in terms of temporal precedence is a new ap- proach which helps to alleviate the problem of fixing feature detection threshold. For these reasons we believe the time dimension to be es- sential in image analysis. Ongoing research now involves more com- plete models in order to emphasize other temporal mechanisms.
Acknowledgements. The support of the Swiss National Fund for Sci- entific Research (FNRS) (grants 20-26475.89 and 4023-027036) is gratefully acknowledged.
References
Attneave, F. (1954). Some informational aspects of visual perception. Psychological Review, 61,183-193.
Burgi, P.-Y., Pun, T. & Pellegrini, C. (1989). Temporal analysis and modeling of the visual circuitry. Perception, 18, SOL
Daugman, J.G. (1983). Six formal properties of two-dimensional anisotropic visual filters. IEEE Transactions all Systems, Man, and Cybernetics, 13,882-887.
Dobbins, A., Zucker, S. W. & Cynadel', M.S. (1989). Endstopping and curvature. Vi- sion Research, 29,1371-1387.
Doma, H. & Hallett, P.E. (1989). Variable contributions of rods and cones to saccadic eye-movement latency in a non-foveating task. Vision Research, 29, 563- 577.
Greenlee, M.W. & Breitmeyer, B.G. (1989). A choice reaction time analysis of spa- tial frequency discrimination. Vision Research, 29, 1575-1586.
Haddon, J.E (1988). Generalized threshold selection for edge detection. Pattern Recognition, 21, 195-203.
Hertz, L. & Schafer, RW. (1988). Multilevel thresholding using edge matching.
Computer Vision, Graphics, and Image Processing, 44, 279-295.
Krose, B.JA. & Julesz, B. (1989). The control and speed of shifts of attention. Vision Research,29,1607-1619.
Kurogi, S. (1987). A model afnenral network for spatiotemporal pattern recognition.
Biological Cybernetics, 57,103-114.
Lacroix, V. (1988). A three-module strategy for edge detection. IEEE Transactions on Pattern AnaJysis and Machine Intelligence, 10,803-810.
Marcelja, S. (1980). Mathematical description of the responses of simple cortical cells. Journal of the Optical Society America, 70,1297-1300.
Marr, D. (1982). Vision. San Francisco, CA: W.H. Freeman.
Optican, L.M. & Richmond, B.J. (1987). Temporal enCOding of two-dimensional pat- terns by single units in primate inferior temporal cortex. III. Information theo- retical analysis, Journal of Neurophysiology, 57,162-178.
Richmond, B.J., Optican, L.M., Podell, M. & Spitzer, H, (1987). Temporal encoding of twa-dimensional patterns by single units in primate inferior temporal cortex. 1.
Response characteristics. Journal of Neurophysiology, 57, 132-146.
Richmond, B.J. & Optican, L.M, (1987). Temporal encoding of two-dimensional patterns by single: units in primate inferior temporal cortex. II. Quantification of response waveform. Journal of Neurophysiology, 57, 147-161.
Sahoo, P.K., Soltani, S. & Wong, A.K.C. (1988). A survey of thresholding techniques.
Computer Vision, GraphiCS, and Image Processing, 41, 233-260.
Treisman, A. (1985). Preattentive proceSSing in vision. In Human and Machine Vi- sion II, ed. Rosenfeld, A., pp. 313-334. Academic Press.
von der Heydt, R. (1987). Approaches to visual cortical function. Reviews ofPhysiol- ogy, Biochemistry, and Pharmacology, 108,70-150.
Wehmeier, U., Dong, D., Koch, C. & van Essen, D. (1989). Modeling the mammalian visuaJ system. In Methods in Neuronal Modeling, ed. Koch, C. & Segev, I., pp.
335-359. Boston, MA: The MIT Press.
Wilson, M.A. & Bower, LM. (1989). The simulation of large-scale neural networks.
In: Methods in Neuronal Modeling, ed. Koch, C. & Segev, I., pp. 291-233. Boston, MA: 11,e MIT Press.
P.-Y. Burgi and T. Pun 285
Appendix
A.I Electrical equations
c
t
Fig, A 1: Electrical circuit of a neuron-like computing element.
Figure Al defines the capacity C, the excitatory conductance gexc' the excitatory vOltage source E exc' the leakage conductance gleak' an~
the leakage vOltage source Eleak, In the following equations, I
designates the i'h period, each period being defined as t + t 'if The most important electrical equations are therefore: an 0
(a) during ton:
( 'ooJ
' 0 0u·
l = E· r 1 - e" + U· 1 -l ' e "(AU)
(b) during t of/
where
E = Eexc·gexc+Eleak·gleak r gexc + gleak
~l = ----,'--C
gexc + gleak
~2 = C
gZeak
Expressing the potential after j period(s) or spikes yields: (Al.3) E . k· (1- (k'· k'" )i) + E . k'· k" . (1- (k'· k'" )i-l)
U _ r leak
i - I (k' . k"') +
where k' = e 1:1
k = 1-k'
Eleak· (k,)i. (k,,,)i - 1
_ tolf
klH = e
,
2k" = l-ktl!
We now want to express the integer number I of periods required for the potential to reach a threshold. Mathematically, this corresponds to determining I in the equation Ul
=
s, where s is thethreshold.
The desired number I of periods is:
1= ~l . ~2 (AlA)
1 r leak lit
(
s· (l - (k' . k"')) - (E . k) - (E . k' . k")
J
og Eleak. (1- (k' . k'''» - (Er· k· k"') - (E
leak. k") . k
The latency is therefore:
(Al.5)
P.-Y. Burgi and T. Pun 2Jl7
A.2 Receptive field equations
i) Circular DOG filter Gaussian function:
1
1 2 0-2
G (a,size) = - - - ; ; - - ' e
2· IT' aZ • size (A2.1) The parameter size designates the dimension of the window containing the Gaussian function; it corresponds to the spatial extent of the receptive field The spatial extent of the weighting profile is givenbya.
Difference of two Gaussians:
(A2.2) GDOG = csG(al'size) -cp(a2,size)
where c and c. modulate the Gaussian amplitude. They are related by equatio~ A2.4; values used are given in the text.
ii) End-stopped cell
Normalized Gabor function (with phase
=
0 and OJ=
I):g (ax' ay,
e,
size) = (A2.3)Small receptive fieJd:gs = g (a ,a
,e,
size!)Xl Yl
Large receptive field:g1 = g (a ,a
,e,
sizez) , with sizez > size!Xz Yz
with
WR = AR = =
0"
Y, 0"
x,
(A2A)
