• Aucun résultat trouvé

To demonstrate the importance of this temporal integration stage, a seemingly simple prob- lem is considered (Burgi and h n 1991b; Burgi 1992a). k t an image, formed by a lattice of points, be divided into a left and a right part of equal area but of differing luminance values referred to as the background for the part of lower luminance and as the foreground for the part of higher luminance (e.g. left in Figure 5. IOA). Output responses of the feedforward architec- ture to this image are estimations of the location of the boundary separating the background from the foreground (e.g. Figure 5.10B,C). For a given column, the higher the response, the higher the confidence in the boundary location.

While this problem is easily solvable for a noiseless image, for very low signal-to-noise ratios it becomes critical. To establish the role of the temporal integration stage, white noise is added on this image. Comparisons between asynchronous and synchronous approaches are given in Figure 5.10 and Figure 5.11 for, respectively, Gaussian white noise of SNR 0.1 and uniform white noise of SNR 0.12. Brightness amplitude range is [ O . . .255] corresponding, given a linear relation, to the latency range [22 ms

. .

.2 ms ]

.

For the asynchronous simulations, sampling times were 2 ms (Figure 5.10) and 1 ms (Fig- ure 5.1 I), corresponding to 11 and 22 iterations respectively. Conversely, for the synchronous simulations, sampling time were 22 ms, corresponding to only one iteration. Results in Figure 5.10C and Figure 5.1 1C clearly demonstrate that boundary location is correctly performed with the asynchronous approach, as shown by the peak in estimation of the location in the middle of the image, whereas the synchronous approach gives a wrong location (the maximum response is not situated in the middle of the image).

To illustrate the effect of varying the sampling along time, different values have been cho- sen to process asynchronously the image with Gaussian white noise. Results shown in Figure 5.12 make clear that an increase in sampling time yields a decrease in the performance of boundary location. Note that the limit case in this increase corresponds to the synchronous pro- cessing.

CHAPTER 5. Asynchronous Visual Processing I. Foundations

A:

SNR = 0.1

without noise with Gaussian noise

E3: synchronous processing asynchronous processing with vectorial coherence cooperation with vectorial coherence cooperation

Figure 5.10 : Application of the feedforward model to locate the boundary separating two regions of differing lu- minances pertmbed by Gaussian white noise. (A): Gaussian white noise of standard deviation 79 has been added on a step of height 25 (125-100), resulting in a SNR of 0.1; (B): Coherency detection in vertical image columns.

For the asynchronous processing, sampling time of 2 ms; (C): Slices of (B), with numbers on the right indicating the worst peak-to-peak ratio (the highest divided by the second highest). Images 128 x 128; isotropic filter 11 x 1 1; anisotropic filters 21 x 21, aspect ratio 0.5. Brighmess range is [O.. .255] corresponding to the latency range

[22 ms . . .2 rns ] . Note that because of the reproduction on paper, the edge in (A) might not be visible and, for that reason, a dashed line has been added.

Asynchronous model. I. Feedforward

A:

SNR = 0.12

B:

synchronous with vectorial asynchronous with vectorial coherence cooperation coherence cooperation

Figure 5.11 : Application of the feedforward model to locate the boundary separating two regions of differing lu- minances perturbed by uniform white noise. (A): Uniform white noise of range 50 has been added on a step of height 5, resulting in a SNR Of 0.12; (B): Coherency detection in vertical image columns. For the asynchronous processing, sampling time of 1 ms; (C): Slices of (B), with numbers on the right indicating the worst peak-to-peak ratio (the highest divided by the second highest). Image 128 x 128; isotropic frlter 1 I x 11; anisotropic fdters 21 x 21, aspect ratio 0.5. Brightness range is [O ... 2551 corresponding to the latency range [22 ms . . .2 ms] . Note that because of the reproduction on paper, the edge in (A) might not be visible and, for that reason, a dashed line has been added.

CHAPTER 5. Asynchronous Visual Processing I. Foundations

Figufe 5.12 : illustration of the effect of varying the sampling time on the performance. Noisy step as in Figure 5.10 (Gaussian white noise and SNR=O. 1). Sampling time values are shown above graphs. Performance is mea- sured as the peak-to-peak ratio (the highest divided by the second highest) with the values shown on the right of the graphs. Increasing sampling time diminishes the asynchrony in processing, the limit being the synchronous case. Filter parameters as in Figure 5.10.

In order to illustrate the notion of optimum in processing, which is hypothesized to result from the increase in transinformation, a 3-D plot of the temporal evolution of the cooperative stage output (before the temporal integration stage), for every column image, is shown in Fig- ure 5.13. Peaks in coherence are temporally situated in a period of time where dynamic transin- formation is higher than synchronous transinformation (for a contrast of 25, it has been esti- mated that the maximum in transinformation occurs at x, = -39.2, which, for the image shown in Figure 5.10, would correspond to t = 6.8 ms ).

Asynchronous model. 11. Recursive with diffusion

Coherency f

u '

Time

Figure 5.13 : Illustration of the dynamic evolution of the feedfoxward architecture outputs, before the temporal integration stage. h u g e and parameters as in Figure 5.10. Coherency is calculaed according to equation (5.12).

The indication "middle" corresponds to the edge position. On the time axis, the indication "Opt" refers to the time where there is an optimum in response (obtained at t z 7 ms ), and the indication "Syn" corresponds to the re- sponse obtained from the synchronous case ( t = 22 ms ). Note that the temporal integration stage summates, for a given position, all the coherence responses along time (from t = 0 to t = 22 ms ).

5.3 Asynchronous model. 11. Recursive with diffusion

5.3J

Introduction

Most of the processing stages of this architecture are identical to those used in the feedfor- ward version (section 5.2). Only the new disposition of the processing stages and the adjunction of a new diffusive stage confer to this architecture new properties. Thus, the isotropic and anisotropic filtering stages (including the rectifications), the cooperation stage (when used), and the temporal stage are identical to those already described, respectively, in

5

5.2.2, § 5.2.3,

9

5.2.4, and tj 5.2.5.

The general architecture is first described in

5

5.3.2. Then, and for the sake of clarity, the diffusive stage is described in four successive paragraphs. The first

(5

5.3.3) deals with the the- ory invoked by the diffusion equation. The second 5.3.4) addresses the question of distin- guishing an edge from a surface. The third

( 5

5.3.5) presents some practical aspects of the dif- fusion equation. The fourth

(5

5.3.6) explains how edges, needed for determining the diffusive coefficients, are obtained. Finally, in § 5.3.7 a method to fix some essential parameters of the model is suggested.

5.3.2

Global architecture

The global architecture is shown in Figure 5.14 where it can be seen that the diffusive stage receives an asynchronous data flow. As it will be explained in the next paragraphs, this stage requires coefficients to be controlled. Thus, these coefficients must be reevaluated continuous- ly. This state of affairs implies the existence of a recursive loop linking the diffusive stage with

110 CHAPTER 5. Asynchronous Visual Processing I. Foundations the gradient calculation stage. The resulting architecture, with its dynamic evolution, can be de- scribed as follows: (i) an image with differing luminances yields an asynchronous data flow where latencies are chosen to be linearly related to luminances; (ii) the data flow enters the first isotropic filtering stage; (iii) output of this stage is temporally integrated; (iv) output of this stage is diffused; (v) conductance coefficients are determined dynamically to control diffusion.

The role of the temporal integration has afready been discussed in § 5.2.5. It is situated at the output of the isotropic filtering in order that the diffusive stage benefits from the increase in transinformation, resulting in a better edge estimation.

Figure 5.14 : Recursive architecture with diffusion. Edge estimation comprises the anisotropic and competitive stages (and optionally the cooperation stage). Outputs of the temporal integration stage are Xij defrned in equa- tions (5.34): outputs of the edge estimation are

sfx. S k

defined in equations (5.37). Note the stage which converts luminance values into an asynchronous flow of data. This asynchronous data flow is indicated by hatched mows.

This asynchronous recursive architecture has two dynmic data flows. First, the data flow yielded by luminance differences. Second, the data flow formed by the loop linking the diffu- sive stage with the edge estimation stage. Conceptually, both data flows are indistinguishable.

Practically, and for reasons of optimization, these two data flows are considered independently.

Given a sampling time for the asynchronous data flow, the data flow formed by the loop is sub- divided into a finer sampling time.

53.3

Diffusion equation

The diffusion equation used in the present context describes the propagation of neuronal activity from one neighbor to another along time. Propagation velocity is controlled by time- varying anisotropic coefficients, Membrane potential is supposed to have a decay and nellronal activity is initiated by sustained inputs. The anisotropic diffusion equation corresponding to these assumptions is written as follows (adapted from Cohen and Grossberg 1984):

where b(x, y, t), M, c(x7 y, t), and X(x, y, t ) stand respectively for the membrane potential, the decay, the anisotropic conductance coefficient, and the sustained input. The operators div and

V

indicate, respectively, the divergence and the gradient with respect to the space variables.

Note that if c(x, y, t) is a constant, this equation reduces to the isotropic heat diffusion equation.

Convergence of equation (5.14) is demonstrated in Appendix 12 in accordance with properties already stated in Briffod and Burgi (1991). Its capacity to enhance object contours and smooth

surface inhomogeneities is now established according to a development inspired from Perona and Malik (1990).

First, let the anisotropic coefficients be a function of the gradient V b(x, y, t) :

Choice of the function g is made relatively to three requirements: (i) when there is no edge (g(O)), conductance is maximum; (ii) the sharper an edge, the lower the conductance; (iii) con- ductance is nonnegative. According to these three statements, function g has to be nonnegative monotonically decreasing.

Equation (5.14) is rewritten in a simpler but equivalent form, limited to one spatial dimen- sion for the sake of clarity:

a a

where b, = -b(x, t ) and b, = -b(x, t ) ; function (I is defined as

a t

ax

5.3.4

Edge sharpening and surface smoothing

An important property expected from equation (5.16) is that an edge considered as such (see below) must sharpen as time progresses (increase in edge slope), and surface inhomoge- neities, delimited by the edges, smoothen. Mathematically, these conditions require for an edge that:

and the inverse relation for surfaces. As stated in Perona and Malik (1990), the increase in slope cannot be caused by a scaling of the edge as time progresses, because this would violate the maximum (also demonstrated in Perona and Malik 1990). The origin of the notion of scaling comes from the Laplacian of Gaussian filter whose output indicates edge positions by the so-called zero-crossings (see

5

3.5.8). The scale of this filter, defining its spatial exten- sion, has been compared to time, rendering such a filtering analogous to an isotropic diffusion.

A similar demonstration of the maximum principle for the isotropic diffusion has thus been done to ensure that zero-crossings never occur as scale increases (or as time progresses). Such a demonstration can be found in Yuille and Poggio (1987). Nevertheless, if they do not occur, zero-crossings can move and disappear (by fusion of two zero-crossings for instance). With re- spect to the isotropic diffusion, the anisotropic diffusion "behaves" better. Indeed, if edges sharpen along time, zero-crossings do not move and do not disappear. This question of scale- space representation for anisotropic diffusion can be found in Saint-Marc et al. (1991).

To verify relation (5.18), the temporal variation of gradient b, is expressed:

1. The Inaxinurn principle states that all maxima of the solution of the equation in space and time belong to the initial condition, and to the boundaries of the domain of interest provided the conductance coeffi- cient is positive (from Perona and Malik 1990).

CHAP l'bK 5. Asynchronous Visual Processing I. Foundations

For this latter relation to be true, function b(x, t) must be of class C'

.

If at any position and at any time c(x, t) > 0 , then by virtue of definition (5.15) and of the three above enounced require- ments for function g , gradient b, exists. Furthermore, if inputs X are supposed to appear pro- gressively, then it is reasonable to consider the function b(x, t) smooth and thus of class C' .

Using equation (5.16) and equation (5.19), it becomes:

Considering a positive step edge (left part is darker than the right part; for the opposite situa- tion, signs in the development below must be inverted), then, at the edge position, bxx = 0 and b,, << 0. For edges to be sharpened, relation (5.18) must be verified. At, or near the edge, equa- tion (5.20) becomes:

and for verifying relation (5.18):

Generally, X, > Mb,; also tern b, is negative (see above). Thus, condition (5.22) is obtained if $' < 0. Alternatively, smoothing is guaranteed when

For verifying this relation, equation (5.21) can be rewritten as:

XX < Mb, - $'&Xx

Given b,,, > X,, condition (5.23) is obtained if @' > 0 .

There are several possible choices of function g, and thus @(b,). For instance,

Graphs of function @(b,), using definition (5.25) for g(b,), is shown in Figure 5.15 for three different values of a. For

a

> 0, there exists a certain threshold value K, related to E and a, below which $(b,) is monotonically increasing ($' > 0, smoothing), and beyond which $(b,) is monotonically decreasing

(9'

< 0, sharpening); the value of this threshold is given by:

Asynchronous model. 11. Recursive with diffusion 113

The definition of an edge depends on parameter E ( a is set to a fixed value). This statement brings anew the perpetual threshold dilemma: how to fix a threshold? (Burgi and Pun 1991a).

Generally, the answer implies heuristic methods. To give an example, in Perona and Malik (1990) 1 /& is set equal to the 90% of the integral of the histogram of the absolute values of the gradient throughout the image. Another example is given by Shah (1991) who uses a regular- ized threshold1. More generally, and despite numerous studies of threshold selection (e.g. Had- don 1988; Hertz and Schafer 1988; Lacroix 1988; Sahoo et al. 1988, recently Hancock and Kit- tler 1991), the problem of retaining pertinent edges (called "prominent edges" in Haddon 1988), and suppressing non-important edges which are to become smooth surfaces (called

"weak edges" in Lacroix 1988; "false edges" in Hertz and Schafer 1988) has not yet been sat- isfactorily solved. The way parameters are fixed in the present asynchronous architecture is dis- cussed in

5

5.3.6.

Figure 5.15 : Function $(b,) given for three a values: 0, 1, and 5. For a = 0, the function is only monotonically increasing; for a > 0, the function is monotonically increasing as long as b, < K and monotonically decreasing as long as a > K. Parameters are: 6 = 1, E = 0.5. K values are 2.0 and 1.53 for a = 1.0 and a = 5.0 respec- tively.

53.5 Practical aspects of the diffusion equation

Interestingly, Grossberg's feature contour system (FCS) utilizes definition (5.25) with a = 0. For such a value, function $(b,) is only monotonically increasing (Figure 5.15) and, thus, edges are not kept. This situation would be disastrous if conductances were iteratively up- dated, which is not the case in Grossberg's architecture. Once the boundaq contour system (BCS) has given an estimate of the edge positions, conductances are determined using defini- tion (5.25). Then, the stable solution of the diffusion equation is calculated2 (according to one

1. Regularization rnakes an ill-posed problem, such as edge detection, well posed by introducing suitable a priori knowledge (see Poggio et al. 1987).

2. Not excluding an iterative method. But the point is that conductances are determined only once.

114 CHAPTER 5. Asynchronous Visual Processing I. Foundations of the various existing methods discussed in Briffod and Burgi 1991).

For an asynchronous architecture, it is of primal importance to have a monotonically de- creasing function $(b,) if edges are to be sharpened. The solution adopted, oddly enough, makes use of Grossberg's architecture, but the edges are iteratively sharpened through a com- petitive stage. A similar approach has been adopted by Waxman et al. (1989) where local max- ima are kept track as diffusion progresses. It turns out that this solution ensures the existence of sharp edges without having to employ the power function required when a > 0 , a function not necessarily biologically plausible. In fact, function (5.25) with a = 0 represents the equi- librium solution of an accumulation-depletion equation used to model biological mechanisms (Grossberg 1984).

For the implementation of the anisotropic diffusion equation, equation (5.14) can be dis- cretized on a square lattice:

db,

- = - Mb,

+ C

(bpq - bij) cPq,

+

.Xv

d t

(P, 4)

'

Nrj

where Nij is the set of positions (p, q ) corresponding to the 4-nearest-neighbors:

Nij = { ( i - 1, j ) , ( i + 1, j ) , ( i , j - 1 ) , ( i , j + 1 ) ) . Conductance coefficients are defined as follows:

where Spq and S, are the 2-D gradients, at positions (p, q) and (i, j) , determined according to a formula described in the next paragraph.

Equation (5.27) can advantageously be solved by direct integration. It gains in stability and simplicity. Assuming a first-order form for this equation with constant coefficients A and B over the time interval d t for a particular position (i, j ) , equation (5.27) can be written as:

with

Then, y at time t

+

dt is given by:

The updating of the coefficients A and B corresponding to different spatial positions is made asynchronously: as soon as a new value y is calculated, it replaces the old value.

Equation (5.29) is of the type describing an RC electrical circuit (see Appendix 7). Also, the equivalent time constant of this diffusive equation can be expressed:

Obviously, such a time constant depends on the conductance coefficients but its range of val- ues, for a 4-nearest-neighbors, can be determined:

5.3.6 Edge estimation

A good estimation of the edge loci is required if the diffusion is to be well controlled. Also, using a = 0 in definition (5.25) implies that threshold K does not exist and only a competitive mechanism can guarantee the sharpening of edges. From the output of the anisotropic filtering (see Figure 5.14), including rectification, several equations are now described which govern the edge estimation.

Multiplicative stage

The output of the isotropic filtering stage represents gradient values of luminance which are temporally integrated. When the output of this temporal integration is multiplied with the output of the anisotropic filtering stage, a better edge estimation occurs (experimentally veri- fied). One explanation of this increase in the performance of edge estimation could be that this multiplication limits spatially the effect of the anisotropic filtering. Temporal integration and multiplication operations are given by the equations:

where Xy is the output of the isotropic filtering stage, described by equation (5.10), W q the output of the temporal integration stage, and the output of the anisotropic filtering stage of orientation k, described by equation (5.11). Note that t - 1 indicates the actual time minus the sampling time, In the forthcoming equations, specification of time is dropped.

Competitive stage

There is no binary decision to classify an edge as such or a surface as such but only gradient values (see

5

5.3.4). Competition keeps only the higher gradient value in a neighborhood which is defined with respect to a receptive field perpendicular to the gradient orientation. The recep- tive field equation, adapted from cooperative receptive field equations found in Grossberg and Mingolla (1 985b), is:

116 CHAPTER 5. Asynchronous Visual Processing I. Foundations where k is the orientation, P is the distance where the exponential has its maximum, and T is a constant defining the receptive field profile. Examples of such receptive field profiles are shown in Figure 5.16. For a set of images containing gradient of orientation k , competition is applied as follows:

i

0 x > y

where [x]

*

=

1 x s y

Vij defines a neighborhood around (i, j)

.

Note that k' = k

+

a/2, as illustrated in Figure 5.16.

Figure 5.16 : Receptive ular to edgeso which are (B): k = 90 and T =

field profiles for the competitive stage defined by equation (5.35). OrientatiOon perpendic- indicated by das$ed lines (not shown in C). Parameters are in (A): k = 90 and T = 1 ; 10; (C): k = 60 and T = 20. P is set to 20 for the three cases.

Multi-orientation projection

Anisotropic filtering is not limited in the number of orientations by opposition to conduc- tance coefficients which are defined with respect to only two orientations: horizontal and ver- tical. Thus, a projection of n orientations to 2 is required and is based on the sine and cosine of the angles:

Indices h and v in this equation stand for the horizontal and vertical orientations.

Cooperation

When only two orientations are used (horizontal and vertical), the cooperative stage de- fined in $5.2.4 can be incorporated in the edge estimation stage. Beyond two orientations, an-, other method has to be contrived. One possibility, already discussed in $ 5.2.4, could utilize oscillatory neuronal structures, but such a solution has not yet be investigated.

Conclusions

5.3.7

Parameters

The question of parameters is now addressed, In § 5.3.5, it has been formulated that if pa- rameter

a

is set to zero, the binary decision to classify a gradient value into an edge or into a surface is not required. In this case, however, the diffusive process does not sharpen the edge (already pointed out in $ 5.3.4). One solution to this problem consists in using a competitive stage. With this solution, the threshold problem is avoided, although the problem of fixing pa- rameters is not. According to the analogy with the RC electrical circuit, the diffusive time con- stant (equation (5.32)) depends on parameters 6 and e. ffiowledge of the velocity of the

The question of parameters is now addressed, In § 5.3.5, it has been formulated that if pa- rameter

a

is set to zero, the binary decision to classify a gradient value into an edge or into a surface is not required. In this case, however, the diffusive process does not sharpen the edge (already pointed out in $ 5.3.4). One solution to this problem consists in using a competitive stage. With this solution, the threshold problem is avoided, although the problem of fixing pa- rameters is not. According to the analogy with the RC electrical circuit, the diffusive time con- stant (equation (5.32)) depends on parameters 6 and e. ffiowledge of the velocity of the