On the other hand, each regional minimum of the gradient image is the attraction point of a catchment basin

(1)

CORINNE VACHIER

Centre de Math´ematiques et Leurs Applications, Ecole Normale Sup´erieure de Cachan 61, av. du Pt Wilson, 94235 Cachan, France

Corinne.Vachier@cmla.ens-cachan.fr

FERNAND MEYER

Centre de Morphologie Mathématique, Ecole Nationale Supérieure des Mines de Paris 35, rue Saint Honoré, 77305 Fontainebleau, France

Fernand.Meyer@cmm.ensmp.fr

Abstract. The watershed transform is the basic morphological tool for image segmentation. Watershed lines, also called divide lines, are a topographical concept: a drop of water falling on a topographical surface follows a steepest descent line until it stops when reaching a regional minimum. Falling on a divide line, the same drop of water may glide towards one or the other of both adjacent catchment basins. For segmenting an image, one takes as topographic surface the modulus of its gradient: the associated watershed lines will follow the contour lines in the initial image.

The trajectory of a drop of water is disturbed if the relief is not smooth: it is undefined for instance on plateaus. On the other hand, each regional minimum of the gradient image is the attraction point of a catchment basin. As gradient images generally present many minima, the result is a strong oversegmentation. For these reasons a more robust scheme is used for the construction of the watershed based on flooding: a set of sources are defined, pouring water in such a way that the altitude of the water increases with constant speed. As the flooding proceeds, the boundaries of the lakes propagate in the direction of the steepest descent line of the gradient. The set of points where lakes created by two distinct sources meet are the contours. As the sources are far less numerous than the minima, there is no more oversegmentation. And on the plateaus the flooding also is well defined and propagates from the boundary towards the inside of the plateau. Used in conjunction with markers, the watershed is a powerful, fast and robust segmentation method. Powerful: it has been used with success in a variety of applications. Robust: it is insensitive to the precise placement or shape of the markers. Fast: efficient algorithms are able to mimic the progression of the flood. In some cases however the resulting segmentation will be poor: the contours always belong to the watershed lines of the gradient and these lines are poorly defined when the initial image is blurred or extremely noisy. In such cases, an additional regularization has to take place. Denoising and filtering the image before constructing the gradient is a widely used method. It is however not always sufficient. In some cases, one desires smoothing the contour, despite the chaotic fluctuations of the watershed lines. For this two options are possible. The first consists in using a viscous fluid for the flooding: a viscous fluid will not be able to follow all irregularities of the relief and produce lakes with smooth boundaries. Simulating a viscous fluid is however computationally intensive. For this reason we propose an alternative solution, in which the topographic surface is modified in such a way that flooding it with a non viscous fluid will produce the same lakes as flooding the original relief with a viscous fluid. On this new relief, the standard watershed algorithm can be used, which has been optimized for various architectures. Two types of viscous fluids will be presented, yielding two distinct regularization methods. We will illustrate the method on various examples.

Keywords: image segmentation, watershed transform, viscosity

(2)

1. Introduction

A digital image is nothing but a collection of pixels with color or intensity attributes distributed on a grid.

Segmentation is the process of regrouping pixels into regions representing meaningful entities. Hence segmentation follows an ambitious goal: contouring and recognizing objects of interest. In many circumstances, this goal is too complex and segmentation will process in multiple stages: for instance construct regions which are homogeneous for some criterion, and then regroup regions according to some other criteria. Formally, each stage of segmentations amounts to create a partition of the space E into a set of regions (R1,R2, . . . ,RN) verifying E = ∪iRi and Ri ∩Rj = ∅; each regions being homogeneous for some criterion like color, tex- ture, motion. In some cases, one desires also to control the size and/or the number of regions (if a relatively simple description of the scene is required). Moreover the process should be robust and yield the same result if the image is degraded (presence of noise, blurring, poor lighting conditions. . . ).

Many segmentation paradigms have been proposed in the literature: statistical methods, fuzzy classifi- cations, Markov fields, neurons networks or PDE- based methods such as bayesian models, snakes or active contours models, Mumford and Shah’s functional, etc. . . Analogies with physical phenomena have inspired some particularly interesting models: various energy minimizations as in the balloon model [5], application of Fermat’s law of minimal action [6, 7], or adaptation of Coulomb’s law [8]. In the present paper we will flood a topographical relief and construct watershed lines.

Watershed-based segmentation has been invented by C. Lantu´ejoul and S. Beucher in 1979 [2]. The

Figure 1. A gray-scale image, its gradient norm and the representation of the gradient image as a topographic surface.

Figure 2. The relief flooding scenario: points where two lakes meet define the watershed line.

fundamental idea is the following: a gradient image is interpreted as a topographic relief where the contours correspond to crest lines of the relief. More precisely, the contours are the divide lines separating the various catchment basins: a drop of water falling on a divide line may glide on any side of it and reaches one of the two adjacent catchment basins, following a line of steepest slope. Inversely, a flooding scenario, in which the minima serve as sources, will also follow the lines of steepest descent from bottom to top ; the watershed line will then be the meeting line of two flooding fronts coming from two distinct minima (see Figs. 1 and 2).

Among all the segmentation methods, the watershed transform is certainly one of the most popular judging from the great diversity of applications in which the method has been successfully applied. The morphological segmentation paradigm is a two steps procedure:

during the first step, the image is analyzed and some germs or markers are introduced within each object of interest; the precise location of the marker inside the object has no importance. This crucial step may be done iteratively or interactively, permitting an elegant and easy decision procedure. During the second step the strongest and best contours separating the markers are detected. This last part of the segmentation is fully au- tomatic and is completely parameter free. Furthermore,

(3)

or interactive segmentation schemes. Since its inven- tion in 1979, many authors have contributed to make the watershed-based segmentation paradigm powerful by the way of important advances on theoretical, algorithmic or experimental levels. With steadily growing processing capacities , increasingly complex problems of segmentation may be tackled. The list of publica- tions on those topics is very long, but here are some synthesis articles: [18, 21, 24].

There is however a drawback with the watershed segmentation: the strength of the watershed due to the ab- sence of parameters is also its weakness. In some cases, when the images to be segmented are corrupted by blurring, noise, poor lighting conditions contours are poorly defined, the segmentation must result from a compromise between a complete adherence to the data (and possibly to the noise) and a certain amount of modelling

; poorly defined parts of the contour have sometimes to be interpolated by fitting a model to the better defined parts. Energy-based methods (like the snakes, the balloon model. . . ) go along this line: smoothness terms are incorporated in the model [4, 5, 11, 28, 35]. Follow- ing our flooding analogy, one may imagine introducing some degree of stiffness and a higher smoothness by flooding the topographic relief with a viscous fluid.

This option is that of Hieu, Worring and Boomgaard [10] who suggest to add a smoothness term in the watershed energy or that of Salembier [25], Marcotegui [13]

or Serra [27] who suggest to regularize the lakes along the flooding scenario. Alternatively, the relief itself may be modified in such a way that, by flooding this new relief with a viscous fluid or a non viscous one, one would produce exactly the same progression of the flooding, and hence the same placement of the watershed lines.

The advantage is that an ordinary watershed transform can be used on this new topographic surface, for which, as we already mentioned, extremely fast algorithms ex- ist. Furthermore, if the process has to be repeated, as in interactive segmentation where various sets of markers may be used until a satisfactory result is obtained, the topographic surface has to be smoothed only once.

This second promising alternative is chosen in this article.

regularization to the topography of the relief. This idea is discussed here and compared to another model based on another model in which the viscosity decreases with the temperature, as a function of the altitude of the relief.

This paper is organized as follows: in the next section, some key points of the watershed-based segmentation are recalled and discussed. Then, the concept of viscous watershed line is introduced. Two models of viscosity are proposed and the viscous transformation of a relief is defined. The last part includes an experimentation with the viscous watershed transform applied on various image types.

2. The Watershed-Based Segmentation 2.1. The Relief Flooding Scenario

and Others Definitions

In topography, the watershed line refers to a ridge that divides areas drained by different river systems while a catchment basin is the geographical area draining into a river or reservoir.

The fundamental idea leading to the watershed- based segmentation is built on an analogy. In standard image segmentation applications, contours correspond to high luminance transitions, i.e. points where the gradient norm takes high values. The analogy consists in regarding the gradient norm image (or any other contours image) as a topographic relief in which the crest lines represent the contours. In order to get closed contours, we only consider a subset of the crest lines, namely the watershed lines, or boundaries of the catchment basins. (see Fig. 2).

Concurrently to the original pragmatic definition of the watershed transform proposed by Lantu´ejoul and Beucher [2], a more formal definition exists in terms of skeleton by influence zone. The watershed line is the set of points at equal distance of the image minima, according to a certain distance: the topographic distance. Distance-based formulations of watershed transform are due to Meyer in the discrete case [17] and to Schmidt and Najman in the continuous case [23].

(4)

Much more usually, the watershed line is presented as the set of meeting points of water in a relief flood- ing scenario. The image being seen as a topographic relief, it is progressively flooded, the water entering by the regional minima. Gradually the level of water goes up causing the appearance of a number of lakes (one per minimum). All over the flooding process, each lake takes the exact shape of the valley. Watershed points are meeting points of lakes of different sources (see Fig. 2). This formulation leads to very efficient algorithmic solutions [12, 15, 24, 33, 34]. In this flooding scenario, the lakes correspond to the level sets of the functions, so the watershed points may be extracted level by level. Denoting by Xh( f ) the level set of f at level h,

Xh( f )= {x∈E, f (x)≥h}

the negative sets Sh =X^c_hare the level sets of the lakes.

As the level of the flooding increases, new lakes ap- pear and existing lakes become larger. At some pass points of the relief, two lakes separated at level (h−1) meet at level h: in order to prevent this meeting, the geodesic skeleton of influence of the lakes at level (h−1) is constructed within the lakes of level (h). This process is repeated for all successive levels, keeping the lakes separated along the watershed line. The lakes at level (h−1) play the role of markers and the level set (Sh( f )) the role of reference set (see Figs. 3–5). This formulation is due to Lantu´ejoul and Beucher [2].

We recall that the skeleton by influence zone as- sociates with a family of components C₁,C₂, . . .Cn

Figure 3. The relief and its level sets.

Figure 4. Flooding of the relief (white sets correspond to lakes). From left to right: a first lake appears at the flooding level 1; a second lake appears at level 2. The two lakes grow then meet at level 5 then 6. At each level h, the watershed points correspond to the skeleton by influence zone of the lakes at level (h−1) into the level set [Xh( f )]^c.

Figure 5. The relief and the dividing line calculating by the watershed transform.

called markers, the connected set of points at equal euclidian distance from two different components:

{p∈ E,d( p,Ci)=d( p,Cj),i= j}

In the geodesic case, the space E is replaced by a refer- ence set R and a condition is added: the components Ci

and Cjmust belong to the same connected component of R and of course the separating points p are selected in R.

The advantage of this formulation of the watershed construction is to stress the contribution of each level set. For smoothing the topographic surface, we will apply to each level set of the topographic surface a closing by a disk, whose radius depends on the altitude.

Now, how is the watershed transform used in prac- tice for segmenting an image f ? First the image edges are enhanced by computing their gradient magnitude

||∇f||. This can be for example approximated by the discrete morphological gradient δ( f )−ε( f ), where δ( f )= f ⊕B is the flat dilation of f by a small disk B andε( f ) = f B is the flat erosion of f by B.

After the edge enhancement, the segmentation process starts with creating flooding waves that emanate from a set of markers (feature points inside desired regions) and flood the topographic surface||∇f||. The simplest markers are the regional minima of the gradient image.

However, very often, the minima are extremely numerous, leading to an over-segmentation. For this reason, in many practical cases, the watershed will take as sources

(5)

Figure 6. Profile of the original function f and of the marker function g. The reconstructed function having as minima those of g is represented in bold.

of the flooding a smaller set of markers, which have been identified by a preliminary analysis step as inside germs of the desired segmentation.

In the case where the sources for the flooding are not all minima of the topographic surface, two solutions are possible. First, use the markers as sources: in this case, catchment basins without sources are flooded from already flooded neighboring regions. Such a flooding algorithm using hierarchical queues has been described in [3]. The second solution consists in modifying the topographic surface as slightly as possible, in such a way that the markers become its only regional minima.

This operation is called swamping. If m1,m2, . . .mk

are the binary markers, we construct a marker function g defined as follows: g = +∞ outside the markers and g = f inside the markers. On the other hand, the topographic surface f is modified by constructing the highest flooding of f below function g; in other terms, we perform a reconstruction closing of f using g as marker [19]. The process is illustrated by Fig. 6. Sev- eral operators have been invented for selecting the per- tinent markers: let us mention the r -h extrema [26], the dynamics [9], the waterfall [1] and the generalization of these concepts in terms of extinction functions [29, 31, 32]. The basic idea is to carry out a granulometry of the scene (i.e. a multi-scale filtering of the original scene), to study individually each object evolution throughout the simplification scales in order to evalu- ate their persistence although interpreted as a signifi- cance degree [29, 31, 32]. Filters compatible with such an analysis are the morphological levelings [20, 30].

2.2. Precision and Robustness of the Watershed Transform

By construction, the localization of the contours extracted by the watershed transform is entirely determined by the topographic surface: the lakes follow faithfully the borders of land and their contours can

Figure 7. Original image (image to be segmented).

be rather irregular and chaotic. This is the case in presence of noise, or poorly defined gradient images, as is the case when the original image is blurred.

For segmenting the central part of the flower presented in Fig. 7, two flooding sources have been manu- ally placed: one inside and one outside the heart of the flower. As usually, transitions of luminance correspond to a high value of the gradient norm, so the relief to be flooded is the gradient image. Here, the gradient corresponds to the morphological gradient (dilation minus erosion by a disk of size one).

Without pre-processing, the watershed line is poorly localized (see Fig. 8). This was foreseeable: the precise localization of the heart of the flower is hard to determine as the image is fuzzy. During the flooding procedure, water leaks between contour fragments and some lakes meet at wrong places. . . This phenomenon is quite frequent in the case of noisy data (and especially when gradient images are considered).

When contours are blurred or badly defined, the segmentation must result from a compromise between a complete adherence to the data and a certain amount of modelling. As illustrated in our example, in the watershed-based scenario, irregularly shaped lakes, when they meet naturally, create irregular watershed lines. Regularizing the watershed can only proceed from some modelization and smoothing of the water front. Either the water front itself is smoothed out, as if the fluid had some viscosity or alternatively the topographic surface itself has to be smoothed out, and some

(6)

Figure 8. From left to right: gradient image (relief to be flooded), flooding sources (one is placed in the center of the flower; the other is the edge of the field) and watershed line(superimposed to the original image).

Figure 9. From left to right: closed gradient (the relief to be flooded is closed by a disk of radius r0=30), flooding sources superimposed to the closed gradient and watershed line computed on the closed relief (superimposed to the original relief).

fjords have to be filled. In both cases the lakes created during flooding will present smooth contours.

In fact, the simplest modification of the relief pro- ducing smoother divide lines is the morphological closing (see Fig. 10). A large closing produces a much smoother image. However the contour is still poorly localized if one constructs the watershed associated to the sources, as illustrated in Fig. 9.

This example illustrates a well known duality: without pre-filtering, the segmentation is precise but noise sensitive; with pre-filtering, the segmentation is robust but less precise. Our purpose is to modulate the smoothing to the topography in order to obtain more sensible results than with a plain closing. We want the contour to stick to the data where the data are sure and to stick more to a model where the data are poorly defined.

Figure 10. Effect of a closing on a thin contour line: in black, the original contour; in gray, the closed contour. Each point outside the closed contour belongs to a disk of radius R.

3. The Viscous Watershed Transform

3.1. Oil and Mercury Flooding Scenarios

Our goal is to regularize the watershed line in order to gain in robustness while preserving a great precision of the localization in places where contours are well defined.

We will distinguish two types of scenarios.

The first scenario is well adapted for almost all gradient images for which bottoms of valleys are of gray level zero: as a consequence, the luminance of a point in a gradient image equals to its contrast. This is (almost) true even if the image is degraded by blurring. In such a situation, higher values of the gradient are the sign of a well defined contour in the original image: the watershed line has to follow the relief more faithfully in these areas (and this independently of the depth of the adjacent valleys). Inversely, low values of the gradient indicate the presence of blurring due to motion or poor focus; for such parts of the contour, a higher regularization has to take place.

This may be accomplished by using an oil type of fluid: such a fluid enters more and more deeply in a narrow isthmus when its temperature increases. Sup- pose now that the temperature of the fluid increases with the altitude of the relief; this means that the lakes will better stick to the relief as their altitude increases and the contours at higher altitude gain in precision.

(7)

Figure 11. The regularization (i.e. the closing activity) has to be locally adapted to the gradient values. In points of low gradient, an important regularization is needed (a closing by a large disk can be applied) while a weak regularization is sufficient at points of high gradient values (a closing by a small disk is applied).

In other situations, the valleys depth will be the fundamental parameter of the gradient for modulating the regularization and a second scenario is needed. In such a situation one desires a smoothing which decreases with the depth of the valleys (this behavior is illustrated in Fig. 11). A mercury type of fluid will do the job: such a fluid enters more and more deeply in a narrow isthmus or fjord when submitted to an increasing pressure;

the radius of curvature of its meniscus increases with the pressure. The pressure at a given point of a lake is equal to the height of the column of liquid above this point. This means that, as the level of the lake increases, the pressure at a given altitude increases and the fluid is able to enter more and more within the fjords, whereas at the surface of the lake, the smoothing is maximal.

We will call the first type “oil flooding” and the second type “mercury flooding” and present below how to implement them.

For instance, let us concentrate on the viscous wa- tershed line, i.e. the set of points separating the viscous lakes. If f denotes the original relief, the goal is to find a modified relief g such that the viscous watershed line of the original relief coincides with the standard watershed line of the modified relief:

Wsh(g)=ViscWsh( f )

As said in Section 2.1, the watershed transform of a function g can be expressed as skeletons by influence computed on the level sets of g: the watershed points are points of the level sets Xh(g)^clocated at equal geodesic distance of the lakes at level h −1 (see Fig. 4). The water lakes appearing by flooding of g are components of Xh(g)^c. So, the construction of g will be fully defined

as soon as we have defined how each level set of f is transformed during the viscous flooding simulation:

∀h, Xh(g)^c=Xh(ViscousLakes( f ))^c

3.2. Viscous Flooding Simulation

The concept of viscous flooding has been first proposed by Meyer [16]. It is inspired from a physical measure where the rock granulometry is measured by injecting mercury in the rock: the amount of mercury increases as the pressure allows reaching narrower holes in the rock. This process has inspired Matheron for introducing families of increasing openings [14] as the basis of granulometric measurements.

A smoothed version of a set may be obtained by taking the union of all disks of a given radius included in this set; the smoothing increases with the radius of the disks, as more and more details cannot be covered by such disks:

γr( A)=

x∈E

B_x^rB^r_x⊂ A

γr is the opening by the structuring element B^r. If B^r is the euclidian disk of radius r , the opened set may be interpreted as the space filled by a viscous fluid with a given viscosity [14]; the viscosity r being a function of the pressure for mercury types of fluids and function of the temperature for oil types of fluid. Hence openings by disks of decreasing radius will represent the space filled by a viscous fluid with decreasing viscosity (increasing pressure for mercury or temperature for oil) (see Fig. 12). When the fluid gets less viscous, the

(8)

Figure 12. Lakes formed by fluids of decreasing viscosity: they are represented via openings by disks of decreasing radius.

space filled by the fluid gets larger; it can be defined as the setγr(S) with r ≤r0. Of course, the viscosity of a fluid is determined by the relation linking r to pressure or temperature.

Recall that we do not wish to model the fluid, but to smooth the relief in such a way that a non viscous fluid will behave as a viscous one: opening the lakes byγrin order to make them viscous or closing the background of the lakes, i.e. the level set Xh( f ) by the closingϕr

will have the same effect:

∀h ≥0, Xh[ϕr( f )]=ϕr[Xh( f )]=[γr(Xh( f )^c)]^c

3.3. Oil Flooding

We illustrate on Figs. 13–15 the viscous flooding scenario with an oil type of fluid. Figure 13 represents a relief with two catchment basins and 6 gray levels.

There are two regional minima, one at level 1 on the left and one at level 2 on the right. The temperature is supposed to be indexed on the gray levels: at level 1, the temperature T₀is cold and the oil extremely viscous.

The minimum on the left is large enough and a viscous

Figure 13. Original relief.

Figure 14. Simulation of the viscous flooding. Because of the viscosity, the lakes apparition and meeting may be delayed (in comparison with the non viscous case). The watershed line is still a skeleton by influence zone but now associated with the set of viscous lakes.

Figure 15. The watershed line and the viscous watershed line.

lake appears. When the lake reaches level 2 its temperature increases and the fluid becomes less viscous. The valley on the right is too thin, so it is not flooded. At flooding level 3, the viscosity has again decreased; at this step, two lakes are formed. When the flooding level increases, the lakes temperature also increases and the level sets enlarge. At flooding level 5, the meeting of the water fronts is possible. As in the non viscous case, the watershed points correspond to points of a skeleton by influence zone but the sets to be considered are not the same: they correspond to the level sets of the viscous fluid lakes (see Fig. 15). In the present section, it is explained how these sets are expressed. As illustrated in Fig. 15, the watershed line derived from the viscous model differs from the standard watershed line in points of contour of low gray level.

It is now time to detail the viscous flooding of our relief. We consider a topographic surface f defined on a domain E and taking its values in [0,M]. We are interested in computing the modification to apply to each threshold Xh( f ), defining the maximum ex- tension Xh(T ( f )) of a viscous fluid at this altitude.

A non viscous fluid will then follow the same exten- sion. At flooding level h, the temperature of the fluid is

(9)

Figure 16. Oil flooding of a single cylinder: in the lowest levels, the temperature is low so the fluid is of high viscosity; in surface, the temperature increases and the fluid is less viscous.

uniform and equal to Th; the viscosity radius associated with the fluid at this temperature is r (h). The viscous lakes are defined by the opening of size r (h):

γr (h)

X^c_h

(see the Fig. 16).

We are now at the heart of the method: the con- struction of a modified relief (denoted T ( f )) having the same level sets as the viscous lakes formed in our relief. T ( f ) must satisfy:

∀h≥0, Xh(T ( f ))=

γr (h)(Xh( f )^c)c

=ϕr (h)(Xh( f )) When h increases, the level set Xh( f ) decreases.

But r (h) also decreases and thus the operator ϕr (h)

decreases. Hence the series Xh(T ( f )), produced by applying a decreasing operator to a decreasing series of sets are decreasing as h increases; they are thus really the level sets of a function T ( f ). An explicit formula for this transformation is the following:

T ( f )=

h≥0

ϕr (h)(h.χh( f ))=

h≥0

h.χh

ϕr (h)( f ) with χh( f )( p)=

1 ∀p∈ Xh( f ) 0 elsewhere

In this formulation, h.χh( f ) simply corresponds to the level set Xh( f ) represented at the altitude h.

This formulation may be directly implemented: for each level h, the level set of the input function f is extracted then closed using a disk of radius r (h). The output function results from the superposition of the closed level sets; the sets being repositioned at their initial altitude.

morphological closing: f ≤ T ( f ) ≤ ϕr0( f ).

Note the hierarchical action of the viscous closing:

low altitude level sets are severely closed whereas the highest level sets are nearly preserved. The computation of the viscous closing involves the computation of a number of binary closings (one per level h), represent- ing a non negligible cost. However, the viscous closings is generally computed on gradient images presenting a reduced number of gray levels. Furthermore in case of interactive segmentation for example where the number, position and shape of the markers are adjusted by hand, numerous computations of the watershed have to be performed; each of them flooding the same topographic surface with a different set of sources. In this case the unique modification of the topographic surface is advantageous compared to a method trying to model the flooding itself.

3.4. Mercury Flooding

Flooding with a mercury type of fluid has already been addressed in [22]. There is however a difference in the presentation which follows. In [22] we have considered an infinitely high column of fluid, which is not realistic.

In the present analysis, we limit the column of fluid to the grey tone range [0,M] of the image. During flooding, a mercury type of fluid enters more and more deeply in a narrow isthmus or fjord when submitted to an increasing pressure; the radius of curvature of its meniscus increases with the pressure. Let us consider the mercury flooding of a relief f at level h and detail how the geometries of the lakes are defined.

We consider a topographic surface f defined on a domain E and taking its values in [0,M]. We are interested in computing the modification to apply to each threshold Xh( f ), defining the maximum exten- sion Xh( ˜T ( f )) of a viscous fluid at this altitude. A non viscous fluid will then follow the same extension.

Let us consider a threshold Xt( f ). When the flooding with mercury reaches the level t, its maximal extension will be limited by the closingϕr (0)of Xt( f ). The radius r (0) is the maximal radius of curvature of mercury at atmospheric pressure; let us call this pressure P0. When

(10)

Figure 17. On the left: cylinder to be flooded; on the right: column of viscous fluid. In the bottom of the cylinder, the fluid is more compressed; it behaves as if it were less viscous.

the level of flooding increases and reaches the height h,there will be a pressure proportional to the height h−t of fluid above the level t. Let us call this pressure P(h−t). The maximal radius of curvature of mercury at pressure P(h−t) will be r (h−t)<r (0). Hence the fluid will have entered more deeply into the fjords of

Xt( f ),limited by the setϕr (h−t)[Xt( f )].

We may now compute the threshold Xh[ ˜T ( f )] at altitude h of the smoothed relief. It will be the contri- bution of all levels below h. Imagine that we consider an aerial view of the flooding at level h. The contri- bution of level k itself isϕ_{r (h−k)}Xk( f ) as illustrated on Fig. 17 in the case of a single cylinder. One verifies that the contribution of level h itself isϕr (0)Xh( f ). Consid- ering the aerial view, the flooding will be the union of all floodings at levels up to h. It is limited by a set which is the intersection of all limiting sets for all levels up to h:

0<k<hϕr (h−k)Xk( f ). Let us remark that we have two conflicting evolutions in the seriesϕr (h−k)Xk( f ):

when t increases, the closingϕr (h−k)increases and the threshold Xk( f ) decreases: see Figs. 17 and 18.

On the other hand Xk( f )=Xh( f +h−k),where f +h−k is the vertical translation of f by the value h−k loped to the maximal value M. Hence

Xh[ ˜T ( f )]=

0<k<h

ϕr (h−k)Xh( f +h−k).

But since the closing ϕr (h−k) is a flat operator, the threshold commutes with the closing and we have

Xh[ ˜T ( f )]=

0<k<h

Xhϕr (h−k)( f +h−k)

= Xh 0≤k≤h

ϕr (h−k)( f +h−k)

Figure 18. Profile of the relief and of the mercury flooding. The contribution of the cylinder fkof basis Skat level h≥k isγr(h− k)(Sk). Its contribution at level h<k is∅.

Defining t =h−k, we get

Xh[ ˜T ( f )]=Xh 0≤t≤h

ϕr (t)( f +t)

In this formula the height h appears not only in the threshold level but also in the range of the parame- ter t which varies from 0 to h. What happens when u takes a value l >h ? The ground level of the deepest lakes of the function f +l is greater than or equal to l. Hence the threshold Xh( f +l) represents the whole domain of definition E of the function f and the clos- ingϕr (h)of E is again E. Hence all levels of t above h in

0≤tϕr (t)( f +t) have no influence on its threshold at level h. Hence

Xh[ ˜T ( f )]=Xh 0≤t≤h

ϕr (t)( f +t)

=Xh 0≤t

ϕr (t)( f +t)

and since the height h is not present anymore in our parenthesis, we may identify the functions which are thresholded: ˜T ( f )] =

0≤tϕr (t)( f + t), which is the formula established in [22]; the only difference being the range of t varying between 0 and M: see Fig. 18.

Finally:

T ( f )˜ =

t≥0

ϕr (t)( f +t)

(11)

roughly closed (ϕr (0)( f )), then details are re-injected via softer closings (ϕr (t)( f +t) with r (t)≤ r (0)). As the flooding level h increases (h→ ∞), the fluid at the basement is more and more compressed (it seems like a non-viscous fluid (r (h) → 0), the data are less and less filtered (ϕr (h)( f +t)→ f +h).

3.5. Comparison of the Models

The effects of our viscous transforms are illustrated on Figs. 19 and 20. Both T and ˜T ( f ) are anti-extensive and more precise than the standard morphological closing ϕ( f ≤T ≤ϕand f ≤T˜ ≤ϕ).

At last, by definition of our viscous transforms, we have

∀h, Xh[T ( f )]=ϕr (h)[Xh( f )] and Xh[ ˜T ( f )]=

t≥0

ϕr (t)[Xh( f+t)]⊂ϕr (h)[Xh( f )]

Figure 19. From left to right: original relief (gradient of the flower image of Fig. 7), effect of the viscous transformation of size 30 (mercury model: ˜T ), effect of the viscous closing of size 30 (oil model: T ), and comparison with the standard closing of size 30. In order to improve the visibility of the images, the gray-levels dynamics have been normalized for each image.

Figure 20. From left to right: original relief (ultrasound image of the left ventricle of the heart), effect of the viscous transformation of size 20 (mercury model: ˜T ), effect of the viscous closing of size 20 (oil model: T ) and effect of a standard closing of size 20 . In order to improve the visibility of the images, the gray-levels dynamics have been normalized for each image. Without enhancement, we have: I d≤T˜ ≤T≤ϕ.

fluids of different viscosity invade the relief: the viscosity of the fluid in a lake depends on the depth of the lake. In a single cylinder having its basis at level k, all the sections being equal, the less viscous fluid im- poses its geometry: it is the fluid located at the bottom of the valley, i.e. at level k. The same result may be obtained with oil, considering that the viscosity radius of mercury at the pressure P(h−k) equals the viscosity radius of the oil at temperature T (h−k): see Fig. 21. In other words, the mercury and oil scenarios are equivalent if the temperature of the oil is not indexed on the flooding level but on the depth of the valleys: oil pene- trates inside the relief with the temperature T₀and then becomes hotter when the oil lakes depth increases.

Note that, in order to improve the visibility of the images presented in Figs. 19 and 20, gray-levels dynamics have been changed from one image to another. For this reason, the inequalities between the transformed images are not preserved on our illustrations.

In our original oil model, the temperature is a function of the flooding level and not of the valley’s depth.

Of course, the two models are equivalent if the relief

(12)

Figure 21. A same set of viscous lakes is obtained by mercury flooding (on the left) and oil flooding (on the right).

is a union of cylinders with all minima at level 0. This configuration is common in segmentation applications as we will see in Section 4. As an example, gradients of constant piece wise functions are of this type.

4. Application to Image Segmentation

4.1. Case of a Thin Contour Line

In order to better understand the behavior of the viscous watershed transform we have constructed a series of synthetic test images (Figs. 22–24). It consists in a thin and broken contour line dividing the space into two parts. In the first image, on the left of Fig. 22, the contour line is bright, having a high altitude. The vis-

Figure 22. Viscous closing of a synthetic image. The original contour is a one pixel thin line. The viscous closing gradually thicks the contour.

Figure 23. Viscous Watershed Transform computed on contours of increasing height (with a constant viscosity): the stronger the contour is, the softer the regularization is.

Figure 24. Viscous Watershed Line in the case of a contour of variable height.

cous closing of the relief produces a thick contour zone whose crest line remains close to the fine initial line;

smooth versions of the original contours are added at lower altitudes. Remark that in the particular case of this test example, the image consists in a cylinder and both oil and mercury flooding produce the same result.

In the present examples, the viscosity is set to 20 (more exactly, 20 is the radius of the structuring element used for a fluid at the temperature of reference). Then, the viscosity decreases as the altitude increases: if h de- notes the altitude of the source, the viscosity at level (h+1) is 19, 18 at level (h+2). The effect on the watershed construction is best illustrated by the series of Fig. 23, where the same contour line is represented with increasing heights. As foreseen, the strongest regularization of the construction of the watershed line occurs for the contours with a low altitude. As the altitude increases, the watershed line is less and less smoothed out. The last example in Fig. 24 summarizes all these effects: again the same contour, but with a varying altitude, low in the central part of the image and high on the top and bottom of the image. The contour which is produced follows our expectation: a strong smoothing in the central part of the image and gradually no smoothing towards the borders of the image.

4.2. Segmentation of Fuzzy or Blurred Images We consider again the example of the flower presented in Section 2.2. As noted previously, the image is fuzzy and contouring the heart of the flower is delicate. The

(13)

computation of the plain watershed transform directly of the gradient image leads to a incorrect result.

The Fig. 26 presents the segmentation obtained via a viscous watershed transform. Both oil and mercury flooding models are experimented. The results are simi- lar: in both cases, the viscous watershed line is correctly localized whereas the standard watershed produces incorrect contours (see Fig. 25).

Finally we test the robustness of the viscous watershed transform against noise. Gaussian noise has been added to the original gradient image: see Fig. 27. Non viscous and viscous watershed lines are computed on the degraded gradient. In all cases, adding noise hardly changes the watershed positioning. In particular, the viscous model gives as good results as without noise.

For both models, the quality of the segmentation is not really affected by blurring: see Fig. 28.

4.3. Generic Images Segmentation

In many image processing tasks, such as the processing of multimedia images, we encounter images of very different nature. We now present various examples of image segmentation tasks. In this section, the viscous watershed transform is applied on images of very different types. The segmentation algorithm proceeds in three steps:

1. a first segmentation of the image by the standard watershed transform computed on the original gradient image is performed (see Fig. 29). The sources have been chosen among the largest and most contrasted regional extrema of the original image. For this step, one can refer to [21, 31]. This first seg-

Figure 26. Viscous closings of the gradient image and associated viscous watershed lines (oil type and mercury type).

mentation is used to build a mosaic image, i.e. a piece-wise constant image where the segmented regions are filled with the local mean gray-values of the original image.

2. thin contour lines are extracted computing the morphological gradient of the mosaic image (see Fig. 29).

3. the viscous closing of the image of the thin contour lines is performed (Fig. 30) and the viscous watershed transform is computed (Fig. 31). It is recalled that in this case (where all the relief valleys are at the same level) oil and mercury models are equivalent.

Why do we work with thin contour lines? Remem- ber that our regularization is based on hierarchical closings. When contours are thick or close to one another, they may coalesce by closing (as the closing does not preserve the homotopy). This effect is limited if the contours are fine. In order to avoid this drawback, one could replace the morphological closing by a homotopic transformation (a thickening for example) in order to prevent this phenomenon. When thin regions have to be segmented, this point is of great importance (see for example the arm of the table tennis player on the Fig. 31: for the strongest closing, the contours of the arm coalesce into a central skeleton line).

The Figs. 31 and 32 present the results obtained by the previous algorithm for different viscosities of the fluid. Note that, instead of increasing the viscosity of the fluid, it is possible to lower the heights of the contour arcs: this allows to speed up the computation.

The proposed regularization adapts the closing size to the gradient norm, i.e. the height of the contours.

This process adapted when a high gradient value means

(14)

Figure 28. Viscous closings of the noisy gradient image and associated watershed transforms (oil type and mercury type).

Figure 29. From left to right: original image, morphological gradient, result of the Watershed Transform (in white), gradient of the mosaic image (each segmented region is filled with the local mean value of the original image; then the morphological gradient of this mosaic image is computed).

Figure 30. Viscous closing of the mosaic gradient for different viscosity radii: from left to right, the viscosity radius is 5, then 10 and 20.

Figure 31. Watershed line computed on the precedent filtered relief (from left to right: r0=5,10,20).

precise contours and low gradient values fuzzy contours. This is not always the case, a poorly contrasted object can have extremely sharp contours and a low gradient norm: on the other hand the gradient line would be narrow. Adapting the algorithm to such a situation is the object of our current work: the idea is to distinguish fuzzy contours, which have to be smoothed, and sharp contours which have to be tracked with preci-

sion. This means that in that case, the viscosity has to be indexed to the contour thickness rather than to its height.

4.4. Segmentation of the Heart Ultrasound Image We consider now another example, that of an ultrasound image representing the left ventricle of the heart:

(15)

Figure 32. From left to right: original image, Watershed Lines (on top) and Viscous Watershed Lines (on bottom) obtained from two different sets of flooding sources. In the second case, a more important viscosity is imposed.

see Fig. 33. In that case especially, the relief to be flooded is the original image itself, as its gradient does not improve the localization of the contours. The strat- egy involves the following steps:

1. the computation of the viscous transformation of the original image.

2. the imposition of markers inside and outside the ventricle.

3. the computation of the watershed transform on the gradient of the viscous closed image.

Two models have been tested: the oil and the mercury. Equivalent viscosities have been considered: the viscosity radius r0at reference temperature and at atmospheric pressure is the same in both cases. The mercury flooding scenario has been especially conceived for solving problems comparable to those introduced by the ultrasound example. Results are presented on Figs. 34 and 35. As previously, the result obtained by this method is very good.

We compare our segmentation to the contour produced via a geodesic snake [4]. The results are sim- ilar: see Fig. 36. This was foreseeable: the evolution

Figure 33. Ultrasound image of the left ventricule of the heart.

Figure 34. From left to right: relief closed by viscous closing (oil model), gradient of the viscous closed image and watershed transform.

of a viscous lake may be interpreted as an active variety, the viscosity playing the role of the rigidity. The difference between the two approaches lies in the mathematical formulation: in our solution, the geometrical

(16)

Figure 35. From left to right: relief closed by viscous closing (mercury model), gradient of the viscous closed image and watershed transform.

Figure 36. Segmentation based on a geodesic active contour.

constraints have been expressed as pre-processing.

Nevertheless, note that the viscous watershed-based algorithm presents a major advantage in comparison with the active contours models: it depends on a smaller number of parameters. In particular, the convergence of the snake type algorithms is always a delicate matter: the contour evolution stops when reaching a local minimum of the energy functional, which does not always yield the correct result. In our paradigm, the convergence is guaranteed on a global optimum: the process stops when the relief is entirely flooded.

5. Conclusion

Introducing some type of viscosity in the watershed transform is not a new topic in mathematical morphology. The originality of our work consists in expressing the viscosity as a pre-filtering of the contour image;

the original segmentation paradigm and its qualities are preserved. The proposed pre-processing, the so-called viscous closing, consists in closing the contours, with closing sizes adjusted to the contour heights. Two types of viscous floodings have been presented: one with an oil type of fluid, the second with a mercury type of fluid.

The oil type of fluid is more adapted to cases where low contours are supposed to be poorly localized and can be smoothed out, and higher contours are supposed to

be more precise and should be smoothed less. Mercury type on the contrary is adapted to cases where higher levels of the contours are prone to leakages of flooding and a higher regularization is required. We noticed a drawback of the method, when small details are present in the figure, whose width is comparable to the sizes of our closings. In such cases non desirable effects, as merging of contours may occur. These could be pre- vented by homotopic thickenings instead of plain closings. A second direction of improvement would be to create a smoothing model, which better recognizes the precision or fuzziness of the contours, independently of their height; like that, the level of smoothing of each piece of contours could be better adjusted to its quality.

Both directions of improvement will be studied in a forthcoming work.

References

1. S. Beucher, “Watershed, hierarchical segmentation and water- fall algorithm,” in ISMM’94: Mathematical Morphology and its Applications to Image Processing, Sept. 1994.

2. S. Beucher and C. Lantu´ejoul, “Use of watersheds in con- tour detection,” in Proc. Int. Workshop on Image Processing, Rennes(France), Sept. 1979, pp. 17–21.

3. S. Beucher and F. Meyer, “The morphological approach to segmentation: The watershed transformation,” in Mathematical Morphology in Image Processing, 1993, pp. 433–481.

4. V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic active con- tour,” International Journal of Computer Vision, Vol. 22, pp. 61–

79, 1997.

5. L. Cohen, “On active contour models and balloons,” Com- puter Vision Graphics Image Processing: Image Understanding, Vol. 53, pp. 211–218, 1991.

6. L. Cohen, “Minimal paths and deformable models for im- age analysis,” Actes des journées d’études SEE: le traitement d’image l’aube du XXIième siècle, Mars 2002.

7. E. Deléchelle and J. Lemoine, “La trajectoire déformable: un modèle optique des contours géodésiques fondé sur le principe de fermat,” VI’99, Trois-Rivières, Québec, Canada, Mai, 1999.

8. T. Grigorishin and Y. Yang, “Image segmentation: an electro- static field based approach,” Vision interface’98, Vancouver, 1998, pp. 279–286.

9. M. Grimaud, “New measure of contrast: Dynamics,” in Image Algebra and Morphological Processing III, Proc. SPIE, San Diego, CA, 1992.

10. H.M. Worring and R.V. den Boomgaard, “Watersnakes: En- ergy driven watershed segmentation,” Technical Report 12, In- telligent Sensory Information Systems Group, University of Amsterdam, Oct. 2000.

11. M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active con- tour models,” Int. J. Computer Vision, Vol. 1, No. 4, pp. 321–331, 1988.

12. J.-C. Klein, F. Lemonnier, M. Gauthier, and R. Peyrard, “Hard- ware implementation of the watershed zone algorithm based on a hierarchical queue structure,” in Proc. IEEE Workshop

(17)

8 congr`es RFIA, Lyon-Villeurbanne, 1991, pp. 847–857.

16. F. Meyer, “Inondation par des fluides visqueux,” Technical Re- port Note interne CMM, Ecole des Mines de Paris, 1993.

17. F. Meyer, “Topographic distance and watershed lines,” Signal Processing, 1994, pp. 113–125.

18. F. Meyer, “An overview of morphological segmentation,” Inter- national Journal of Pattern Recognition and Artificial Intelli- gence, Vol. 17, No. 7, pp. 1089–1118, 2001.

19. F. Meyer and S. Beucher, “Morphological segmentation,” Jour- nal of Visual Communication and Image Representation, Vol. 11, No. 1, pp. 21–46, 1990.

20. F. Meyer and P. Maragos, “Morphological scale-space represen- tation with levelings,” Scale-Space’99, LNCS, 1999, pp. 187–

198.

21. F. Meyer, A. Oliveras, P. Salembier, and C. Vachier, “Morpholog- ical tools for segmentations: Connected filters and watershed,”

Annals of Telecommunications, 1997.

22. F. Meyer and C. Vachier, “Image segmentation based on viscous flooding simulation,” in Proceedings of ISMM’02, H. Talbot and R. Bear (Eds.), CSIRO, Sydney, 2002, pp. 69–77.

23. L. Najman and M. Schmitt, “Watershed for a continuous func- tion,” Signal Processing, Juil 1994, pp. 99–112.

24. J. Roerdink and A. Meijster, “The watershed transform: Defi- nitions, algorithms and parallelization strategies,” Fundamenta Informaticae, Vol 41, pp. 187–228, 2001.

25. P. Salembier, “Morphological multiscale segmentation for image coding,” Signal Processing, Vol. 38, No. 3, pp. 359–386, 1994.

26. M. Schmitt and F. Prˆeteux, “Un nouvel algorithme en morpholo- gie math´ematique: Les r-h maxima et r-h minima,” in Proc.

2ieme Semaine Internationale de l’Image Electronique, Avr, 1986, pp. 469–475.

27. J. Serra, “Viscous lattices,” in Proc. of ISMM’02, Sydney, H.

Talbot and R. Bear (Eds), CSIRO, 2002, pp. 79–90.

28. J. Sethian, Level Set Methods (Evolving Interfaces in Geome- try, Fluid Mechanics, Computer Vision, and Materials Science), Cambridge University Press, 1996.

29. C. Vachier, Extraction de caract´eristiques, segmentation d’image et morphologie math´ematique, PhD thesis, Ecole des Mines de Paris, Dec 1995.

30. C. Vachier, “Extraction de caract´eristiques par analyse mor- phologique multi-´echelle,” in Proc. of GRETSI, Toulouse, Vol. 1, Sept. 2001.

31. C. Vachier and F. Meyer, “Extinction value: A new measurement of persistence,” in Proc. of 1995 IEEE Workshop on Nonlinear Signal and Image Processing, Vol. I, Juin 1995, pp. 254–257.

32. C. Vachier and L. Vincent, “Valuation of image extrema using alternating filters by reconstruction,” Image Algebra and Mor- phological Processing, Proc. SPIE, San Diego, CA, Juil 1995.

33. L. Vincent, Algorithmes morphologiques `a base de files d’attente

Corinne Vachier received an engineer degree from the Ecole Supérieure d’Electricité, Paris and a Ph.D. in Mathematical Morphology from the Ecole des Mines de Paris, respectively in 1991 and 1995. From 1992 to 1995, she was research engineer in General Electric Medical Systems, Buc, France and phd student in the Centre de Morphologie Mathématique (CMM) of the Ecole des Mines de Paris. She became in 1996 an associate professor at the Univer- sity Paris 12. She joined Jean-Michel Morel’s Team at the Centre de Mathématiques et Leurs Applications (CMLA) at the Ecole Na- tionale Supérieure de Cachan in 2001. Her research interests include mathematical morphology with emphasis on multiscale representa- tions. Current applicative interests are focused on medical imaging.

Fernand Meyer got an engineer degree from the Ecole des Mines de Paris in 1975. He works since 1975 at the Centre de Morpholo- gie Math´ematique (CMM) of the Ecole des Mines de Paris, where he is currently director. His first research area was ”Early and au- tomatic detection of cervical cancer on cytological smears,” subject of his PhD thesis, obtained in 1979. He participated actively to the development of mathematical morphology: particle reconstruction, top-hat transform, the morphological segmentation paradigm based on the watershed transform and markers, the theory of digital skeleton, the introduction of hierarchical queues for high speed watershed computations, morphological interpolations, the theory of levelings, multiscale segmentation.