Grey-level hit-or-miss transforms—Part I: Unified theory

www.elsevier.com/locate/patcog

Grey-level hit-or-miss transforms—Part I: Unified theory

Benoît Naegel

, Nicolas Passat

, Christian Ronse

aEIG-HES (École d’Ingénieurs de Genève), 4 rue de la Prairie, CH-1202 Genève, Switzerland

bLSIIT, UMR 7005 CNRS-ULP (Laboratoire des Sciences de l’Image, de l’Informatique et de la Télédétection), Bd S. Brant, BP 10413, F-67412 Illkirch Cedex, France

cInstitut Gaspard Monge, Laboratoire A2SI (Algorithmique et Architecture des Systèmes Informatiques), Groupe ESIEE, Cité Descartes, BP 99, F-93162 Noisy-le-Grand Cedex, France

Received 14 November 2005; received in revised form 27 April 2006; accepted 2 June 2006

Abstract

Thehit-or-miss transform(HMT) is a fundamental operation on binary images, widely used since 40 years. As it is not increasing, its extension to grey-level images is not straightforward, and very few authors have considered it. Moreover, despite its potential usefulness, very few applications of the grey-level HMT have been proposed until now. Part I of this paper, developed hereafter, is devoted to the description of a theory leading to a unification of the main definitions of the grey-level HMT, mainly proposed by Ronse and Soille, respectively (part II will deal with the applicative potential of the grey-level HMT, which will be illustrated by its use for vessel segmentation from 3D angiographic data). In this first part, we review the previous approaches to the grey-level HMT, especially thesupremalone of Ronse, and theintegralone of Soille; the latter was defined only for flat structuring elements (SEs), but it can be generalized to non-flat ones. We present a unified theory of the grey-level HMT, which is decomposed into two steps. First afittingassociates to each point the set of grey-levels for which the SEs can be fitted to the image; as in Soille’s approach, this fitting step can beconstrained. Next, avaluation associates a final grey-level value to each point; we propose three valuations:supremal(as in Ronse),integral(as in Soille) andbinary.

䉷

Keywords:Mathematical morphology; Hit-or-miss transform; Grey-level interval operator; Morphological probing

1. Introduction

Consider a Euclidean or digital spaceE(E=RⁿorZⁿ).

ForX ∈ P(E), writeX^c=E\X (the complement of X), Xˇ = {−x |x ∈X}(the symmetrical ofX), and forp∈E, X_p= {x+p |x ∈ X}(the translate ofXbyp). Then the Minkowski addition and subtraction are defined by setting forX, B∈P(E):

XB=

b∈B

Xb and XB=

b∈B

X₋b.

This leads to the operatorsB :X→XB(dilation byB) andε_B :X →XB (erosion byB); hereBis considered

∗Corresponding author. Tel.: +41 022 338 05 66.

E-mail address:benoit.naegel@hesge.ch(B. Naegel).

0031-3203/$30.00䉷2006 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

doi:10.1016/j.patcog.2006.06.004

as a structuring element (SE) that acts on the binary image X. (NB: Our terminology follows[1,2], in accordance with the algebraic theory of dilations and erosions; it is slightly different from that of Refs.[3,4], in the sense that for some operations, the SEBis replaced by its symmetricalB, seeˇ Refs.[2,5]for a more detailed discussion.)

The hit-or-miss transform (in brief, HMT) uses a pair (A, B) of SEs, and looks for all positions whereAcan be fitted within a figureX, andBwithin the backgroundX^c, in other words it is defined by

X(A, B)= {p∈E|Ap⊆X andBp⊆X^c}

=(XA)∩(X^cB). (1) One assumes that A∩B = ∅, otherwise we have always X(A, B)= ∅. One calls A andB, respectively, the fore- ground and background SE. In practice, one often uses bounded SEsAandB.

This operation was devised by Matheron and Serra in the mid-sixties [3,6], and has been widely used since. It represents the morphological expression of the notion of template matching.

The binary HMT is often applied in shape recognition, for example in document analysis[7–9]. Hardware implemen- tations with optical correlators have been studied in Refs.

[10–15]. These implementations seem interesting, since computational time is independent from the size of the SE used, which is obviously not the case with software ones.

A recurrent issue consists in determining the SEs in order to cope with the noise and the variability of the patterns to be recognized.

Zhao and Daut[16]propose a method to match imperfect shapes in an image. They start with a set of shapes to be recognized, then smooth each element of this set by some kind of opening. The boundaries of these smoothed sets are then used as SEs for the HMT.

Doh et al.[17]discuss the choice of SEs for the recogni- tion of a class of various objects. They start from two sets: a set of hit SEs (i.e., SEs that fit the objects to be recognized) and a set of miss SEs (SEs that fit the background). Their conclusion is to use a synthetic hit SE composed of the in- tersection of all hit SEs and a synthetic miss SE composed of the union of all miss SEs.

Bloomberg et al.[8,9]introduce a blur HMT which con- sists in dilating both setXand complementX^c. They also propose to subsample the SEs by imposing a regular grid.

This allows the HMT to be less sensitive to noise while pre- serving the global characteristics of the shape.

The operatorX→(X(A, B))Ahas been considered in Ref.[18](it was suggested to the author by Heijmans), and later in Ref.[4, p. 149], where it was calledhit-or-miss opening. It is idempotent and anti-extensive, like an opening, but not increasing.

Although the HMT is widely used in binary image pro- cessing, there are only a few authors who considered its possible extension to grey-level images (we review the main works in the next section). The main difficulty resides in the fact that this operator uses both the set X and its comple- mentX^c, and is thus neither increasing nor decreasing. Let us explain how to removeX^c from definition (1).

LetA, B∈P(E)such thatA⊆B. Consider theinterval [A, B] = {C∈P(E)|A⊆C⊆B}.

Then we define_[A,B], theinterval operator by[A, B], by setting for everyX∈P(E):

_[A,B](X)= {p∈E|X_−p∈ [A, B]}

= {p∈E|A_p ⊆X⊆B_p}. (2) Heijmans and Serra[19]were the first to consider such an operation, but they wrote itX (A, B)instead of_[A,B](X).

Clearly _[A,B](X)=X(A, B^c). Here the inclusion con- straintA⊆B(without which we always get_[A,B](X)= ∅) corresponds to the disjointness conditionA∩B^c= ∅of the

corresponding HMT X(A, B^c). In practice, one usually choosesAand the complementB^cofBto be bounded.

This variant formulation was fruitful. First it allowed to give a very short proof of the theorem of Banon and Bar- rera [20], namely that every translation-invariant operator is a union of HMTs. More precisely, given a translation- invariant operator : P(E) → P(E), Matheron’s kernel [6]is the set

V()= {A∈P(E)|0∈(A)}, (3) and indeed Matheron showed thatifis increasing, we have

(X)=

A∈V()

XA (4)

for everyX∈P(E), in other wordsis a union of erosions.

Consider now thebi-kernel[19]

W()= {(A, B)∈P(E)²|A⊆B,[A, B] ⊆V()}, (5) then an elegant proof in Ref.[19]shows that for everyX∈ P(E)we have

(X)=

(A,B)∈W()

_[A,B](X), (6)

in other words is a union of interval operators (equiva- lently, of HMTs).

However, the main advantage of considering an interval operator (2) instead of a HMT (1), is that it gave way to the first theory (by Ronse[18]) of interval operators on grey- level images and more generally on complete lattices, in particular the operatorsA_[A,B]are part of a very interesting family of idempotent and anti-extensive operators, called in Ref.[18]open-over-condensations.

A few years later, Soille[4,21]gave independently another definition of a HMT for grey-level images. His framework was restricted to the use of flat SEs and of discrete grey- levels. However, as we will see in the next section, it can easily be generalized to non-flat structuring functions and to images with arbitrary grey-levels (continuous or discrete).

Moreover, he introduced the possibility of constraining the HMT; as we will see later on, this constraining of the HMT can also be applied to Ronse’s version.

When it is extended to non-flat SEs, the unconstrained version of Soille’s HMT has some resemblance with Ronse’s interval operator[18], and is also very similar to an opera- tion introduced by Barat et al. [22–24] under the name of morphological probing.

The authors have successfully applied grey-level HMTs to the detection of blood vessels in 3D angiographic images [25–28]. In fact, we used both Ronses and Soille’s uncon- strained versions, but also some new variants. Therefore we have felt that it would be useful to make a review of the dif- ferent grey-level HMTs found in the literature, and to give a unified theory containing each one as a particular case.

The paper is organized as follows. In Section 2 we re- view the various approaches to the grey-level HMT found in the literature, mainly the ones of Ronse[18], Soille[4,21]

and Barat et al.[22–24]; we generalize Soille’s approach to arbitrary (not necessarily flat) SEs and arbitrary (not nec- essarily discrete) grey-levels. We will see that these HMTs can be better understood by expressing them as grey-level extensions of the interval operator_[A,B](2).

In Section 3 we give a unified theory of grey-level interval operators. Such an operator can be decomposed into two steps:

(i) afittingwhich extracts from a grey-level image and a pair of structuring functions, a set of pairs (p, t)(p a point,ta grey-level); we have two versions (following the approaches of Ronse and Soille), and each one can optionally beconstrainedas in Soille’s approach;

(ii) avaluationwhich constructs from this set of pairs(p, t) the resulting grey-level image; we have three versions:

a supremal one (following Ronse), an integral one (following Soille), and abinaryone (which produces a binary image).

This gives thus in theory a set of six unconstrained grey- level HMTs, and six constrained ones (however, there is some redundancy in this set).

The Conclusion summarizes our findings and gives some perspectives for further research. In particular, we have not extended our theory to the general framework of complete lattices, nor have we analysed the operators obtained by composition of the HMT and the dilation by the foreground SE (both things were done in Ref.[18] for one version of the HMT).

Part II of this paper[29]will provide a review of our work on the application of grey-level HMTs to the detection and enhancement of blood vessels in 3D angiographic images, but also algorithmic remarks about grey-level HMT, still valid for more general applications.

2. Existing approaches to the grey-level HMT

We will review the various forms given in the literature for the grey-level HMT. But let us beforehand recall the basics from grey-level morphology[1,30].

We consider a space E of points, which can in general be an arbitrary set. However, in order to define translation- invariant operators (like the dilation and erosion by a SE), we need to add and subtract points, so in this case we assume E to be the digital space Zⁿ or the Euclidean space Rⁿ, for which the addition and subtraction of vectors are well- defined.

We have a setT of grey-levels, which is part of the ex- tended real line R=R∪ {+∞,−∞}. We requireTto be closed under non-void infimum and supremum operations (equivalently,T is a topologically closed subset of R); for

example we can take T =R, T =Z=Z∪ {+∞,−∞}, T = [a, b] (a, b ∈ R,a < b) orT = [a . . . b] = [a, b] ∩Z (a, b∈Z,a < b). ThenTis acomplete lattice[1]w.r.t. the numerical order . Write and ⊥, respectively, for the greatest and least elements ofT.

Grey-level images are numerical functionsE→T, they are generally written by capital lettersF, G, H, . . .The set T^E of such functions is a complete lattice for the compo- nentwise ordering defined by

FG ⇐⇒ ∀p∈E, F (p)G(p),

with the componentwise supremum and infimum operations:

i∈I

F_i :E→V :p→sup

i∈I F_i(p) and

i∈I

F_i :E→V :p→inf

i∈I F_i(p).

Let us now introduce some notation. GivenF, G∈T^E, we writeG?F (or equivalently,F>G) if there is someh >0 such that for every p ∈E we haveG(p)F (p)+h. For F ∈T^E andp∈E, thetranslateofFbypis the function F_p : E→ T : x →F (x−p). Thesupportof a function Fis the set supp(F )of points ofEhaving grey-levelF (p) strictly above the least value⊥:

supp(F )= {p∈E|F (p) >⊥}, (7) and thedual supportofFis the set supp^∗(F )of points ofE having grey-levelF (p)strictly below the greatest value: supp^∗(F )= {p∈E|F (p) <}. (8) For everyt ∈T, writeC_t for the functionE→ T with constant value t: ∀p ∈ E,C_t(p)=t. We see in particular that the least and greatest elements of the lattice T^E of numerical functions are the constant functionsC_⊥andC, respectively. For anyB⊆Eandt ∈T, thecylinder of base B and level tis the functionC_B,t defined by

∀p∈E, C_B,t(p)=

t ifp∈B,

⊥ ifp /∈B. (9) Note in particular thatC_t=C_E,t. Also, forh∈Eandt ∈T, theimpulsei_h,t is the cylinderC_{h},t, thus

∀p∈E, i_h,t(p)=

t ifp=h,

⊥ ifp=h. (10) ForF ∈T^E, we havei_h,tF ifftF (h), and

F =

{ih,t |h∈E, t ∈T , tF (h)},

in other words every function is a supremum of the impulses below it.

The dual cylinder of base B and level t is the function C_B,t^∗ defined by

∀p∈E, C_B,t^∗ (p)=

t if p∈B,

if p /∈B. (11) ForV , W ∈T^EwithVW, we have theinterval[V , W]=

{F ∈T^E|VFW}.

Every increasing operator : P(E) → P(E) on sets extends to a flat operator ^T : T^E → T^E on grey-level images [31]. For everyF ∈ T^E andt ∈ T we define the threshold set[1]

X_t(F )= {p∈E|F (p)t}.

ClearlyXt(F )is decreasing with respect tot. Now ^T is defined by applyingto each threshold set and stacking the results. Formally:

^T(F )=

t∈T

C_{(Xt(F )),t}, (12)

so that for every pointp∈Ewe have ^T(F )(p)=

{t ∈T |p∈(X_t(F ))}. (13) In particular, whenE=RⁿorZⁿ, the dilationBand erosion ε_Bby a SEBextend as follows:

^T_B(F )=

b∈B

F_b and ε_B^T(F )=

b∈B

F_−b, (14) so that for every pointp∈Ewe have

^TB(F )(p)=

b∈B

F (p−b) and

ε^T_B(F )(p)=

b∈B

F (p+b). (15)

We will also writeFBandFBfor^T_B(F )andε^T_B(F ), respectively.

Let us now consider morphological operations with SEs that are functions instead of sets. Here grey-levels will be added and subtracted in formulas, thus in order to avoid grey- level overflow in computations,T must necessarily be un- bounded (so=+∞and⊥ =−∞), in fact we assume that T=RorZ(however,T=aZ= {az|z∈Z} ∪ {+∞,−∞}, wherea >0, is also possible). LetT=T\{+∞,−∞}, the set of finite grey-levels. We saw above that a functionFcan be translated by a pointp ∈ E, this is a horizontal trans- lation; now there is also a vertical translation, namely by a finite grey-levelt ∈T, transformingFintoF+t. Combin- ing both, we get the translation by(p, t), the translate ofF by(p, t)isF(p,t)=Fp+t :x →F (x−p)+t. We con- sider impulsesi_h,t only fort ∈T. Theumbraof a function F ∈T^E is the set

U(F )= {(h, t)|h∈E, t∈T, tF (h)}. (16)

Note that for an impulsei_h,t, we have i_h,tF iff(h, t) ∈ U(F ), and

F =

{i_h,t |(h, t)∈U(F )}. (17) For F, G ∈ T^E, we can define the Minkowski addition FGand subtractionFGas follows:

FG=

(h,t)∈U(G)

F_(h,t)

and

FG=

(h,t)∈U(G)

F(−h,−t). (18)

At every pointp∈Ewe have (FG)(p)= sup

h∈E(F (p−h)+G(h))

= sup

h∈supp(G)(F (p−h)+G(h)) and

(FG)(p)= inf

h∈E(F (p+h)−G(h))

= inf

h∈supp(G)(F (p+h)−G(h)).

SinceT=RorZ, the termsF (p−h),F (p+h), andG(h)can have an infinite value, so the expressionsF (p−h)+G(h) andF (p+h)−G(h)can take the form+∞−∞or−∞+∞, which are arithmetically undefined; then their evaluation is achieved by the following rules:

• In the formula for (FG)(p), we consider that +∞ =Tand−∞ =

∅, so+∞ − ∞ =

t∈T

t∈∅

(t+t)=

∅ = −∞, in other words expressions of the form+∞ − ∞or−∞ + ∞must be evaluated as−∞.

• Dually, in the formula for (FG), we consider that +∞ =

∅and−∞ =T, so expressions of the form +∞ − ∞or−∞ + ∞must be evaluated as+∞. We obtain thus the dilation and erosion byG, namelyG : F → FGandε_G : F →FG. These two operations form anadjunction[1]:

∀F1, F2∈T^E, F1GF2 ⇐⇒ F1F2G. (19) Consider thesymmetricalGˇ ofGdefined byG(x)ˇ =G(−x), and the grey-level inversion T → T : t → −t, which extends to functions by transforming F into −F : x →

−F (x). From Eq. (18) is easily seen that

−(FG)=(−F )Gˇ and

−(FG)=(−F )G,ˇ (20) in other words, erosion is thedual under grey-level inversion of the dilation with the symmetrical structuring function. Let us define thedualofGasG^∗= − ˇG:x → −G(−x).

Taking a setB∈P(E)as SE, the flat dilation and erosion by B seen in Eqs. (14) and (15) are a particular case of dilation and erosion by a grey-level function, since we have FB=FC_B,0 and FB=FC_B,0. (21) More generally, fort∈T, we have

FCB,t=(FB)+t and

FCB,t=(FB)−t. (22)

Structuring functions of the formC_B,0 are also called flat SEs.

Grey-level Minkowski operations do not always preserve the bounds of image grey-levels:

Lemma 1. LetF, G∈ T^E such thatF (p)∈ [a, b]for all p∈E,and letg=supp∈EG(p).Then for allp∈Ewe have (FG)(p)∈ [a+g, b+g]and(FG)(p)∈ [a−g, b−g].

Proof. From Eq. (18) we check easily that for any t ∈ T, CtG = Ct+g and CtG = Ct−g. The fact that

∀p ∈ E, F (p) ∈ [a, b], means that CaFCb. Hence we get Ca+g = CaGFGCbG = Cb+g and Ca−g=CaGFGCbG=Cb−g, that is∀p∈E, (FG)(p) ∈ [a +g, b+g] and(FG)(p) ∈ [a−g, b−g].

The next result is fundamental for our analysis:

Proposition 2. Let F, V , W ∈ T^E, p ∈ E and t ∈ T. Then:

(i) V_(p,t)F iff(FV )(p)t. (ii) V_(p,t)>F iff(FV )(p) > t.

(iii) FW_(p,t)iff(FW^∗)(p)t. (iv) F>W(p,t)iff(FW^∗)(p) < t.

Proof. (1) (FV )(p)t means i(p,t)FV, and by adjunction (19) this is equivalent to i(p,t)VF; but i_(p,t)V =V_p,t, so the result follows.

(2)V_(p,t)>F iff there is someh >0 withV_(p,t)F−h;

by item 1, this is equivalent to ((F −h)V )(p)t, in other words (FV )(p) −ht for some h >0, that is (FV )(p) > t.

(3) By grey-level inversion, FW_(p,t) iff −F − (W_(p,t))= (−W )_(p,−t). Applying item 1, this is equiv- alent to ((−F )(−W ))(p) − t. Inverting again, this means −((−F )(−W ))(p)t; by duality (20),

−((−F )(−W ))= −(−F )(−W )^∨=FW^∗, and the result follows.

(4)F>W(p,t)iff there is someh >0 withF+hW(p,t); by item 3, this is equivalent to((F+h)W^∗))(p)t, in other words (FW^∗)(p)+ht for some h >0, that is (FW^∗)(p) < t.

Let us apply this result to the case where (FV )(p)or (FW^∗)(p)has an infinite value:

Corollary 3. LetF, V , W ∈T^E andp∈E.Then:

(i) (FV )(p) = +∞ iff (∀t ∈ T, V_(p,t)F ) iff

t∈TV_(p,t)F.

(ii) (FV )(p)= −∞iff(∀t ∈T, V_(p,t)F ).

(iii) (FW^∗)(p)= +∞iff(∀t ∈T, FW_(p,t)).

(iv) (FW^∗)(p) = −∞ iff (∀t ∈ T, FW(p,t)) iff

F

t∈TW(p,t).

Proof. Items 1 and 2 follow from item 1 of Proposition 2, and the fact that +∞is the only value t for allt ∈ T, while−∞is the only onet for allt ∈T. Items 3 and 4 follow from item 3 of Proposition 2, and the fact that+∞

is the only valuet for allt ∈T, while −∞is the only one t for allt ∈T.

Note that ifFhas all its values in an interval[t₀, t₁] ⊂R, and sup_p∈EV (p)=v∈Rand infp∈EW (p)=w∈R, then by Lemma 1,FV andFW^∗will have all their values in the intervals[t₀−v, t₁−v]and[t₀−w, t₁−w], respectively, hence infinite values do not occur in such a case.

2.1. Ronse’s supremal interval operator

The basic ideas in Ronse’s approach[18]are to start from the interval operator (2) instead of the HMT, and to consider the fact that a grey-level image is a supremum of impulses (17) as the parallel of the fact that a set is a union of single- tons. We still assume that T =Zor R. We define thus for V , W ∈T^Esuch thatVWthesupremal interval operator ^S_{[V ,W]}by setting for everyF ∈T^E:

^S_{[V ,W]}(F )

=

{i_(p,t)|(p, t)∈E×T, F_(−p,−t)∈ [V , W]}

=

{i(p,t)|(p, t)∈E×T, V(p,t)FW(p,t)}. (23) Note that following Ref.[19], Ronse wroteF (V , W )for ^S_{[V ,W]}(F ). By Proposition 2, fort∈T,V_(p,t)FW_(p,t) iff(FW^∗)(p)t(FV )(p). Hence for everyp∈E, ^S_{[V ,W]}(F )(p)=sup{t∈T|V_(p,t)FW_(p,t)}.

Now forab, we haveb=sup{t ∈T |atb}, except ifa=b= +∞, in which case we get the empty supremum, that is−∞. We obtain thus:

^S_{[V ,W]}(F )(p)

=

(FV )(p) if(FV )(p)(FW^∗)(p)

= +∞,

−∞ otherwise. (24)

Note that if (FV )(p) = (FW^∗)(p) = +∞, by Corollary 3 we have FW_(p,t) for all t ∈ T, so that ^S_{[V ,W]}(F )(p)= −∞, and not+∞.

A B

V W

E T

a

b

B

Fig. 1. Top: The two structuring elementsA(in black) andB(in grey).

Bottom: the cylinderV =C_A,a (in black) has support A and the dual cylinderW=C_B,b^∗ (in grey) has dual supportB.

In practice, one usually choosesVwith bounded support, andW with bounded dual support (i.e., W^∗ has bounded support). For example, we can takeV=C_A,aandW=C_B,b^∗ , see Fig. 1; then W^∗ =C_B,ˇ−b and by Eq. (22), FV = (FA)−a andFW^∗=(FB)ˇ −b, so that Eq. (24) gives here:

^S_[V ,W](F )(p)

=

⎧⎨

⎩

(FA)(p)−a if(FA)(p)

(FB)(p)ˇ +a−b= + ∞,

−∞ otherwise.

ForA,Banda fixed,^S_{[V ,W]}(F )increases with b, as more and more points will get the value(FA)(p)−a instead of−∞. We illustrate this inFig. 2.

The operatorV^S_{[V ,W]}mapsF ∈T^E on {V_(p,t)|(p, t)∈E×T, V_(p,t)FW_(p,t)}.

It is idempotent and anti-extensive like an opening[18], but not increasing. It is part of a family of operators calledopen- over-condensations.

In Ref.[18]the theorem of Banon–Barrera, Refs. (5) and (6) was also extended to grey-level images (and more gener- ally, in a complete lattice where Minkowski operations are properly defined[2]): every translation-invariant operator is a supremum of supremal interval operators.

2.2. Soille’s integral HMT

Soille [4,21] assumes discrete grey-levels (an interval in Z) and flat SEs. If we return to formula (13) for the

0

−1 T

E A

B B

Fig. 2. Here E=Zand T =Z. On top we show the two structuring elementsAandB(the origin being the left pixel ofA), with the associated levels a=0 andb= −1 (thusV =C_A,0 andW=C_B,−1^∗ ). Below we show a functionF, and in grey we have^S_{[V ,W]}(F ), forming three peaks.

The left peak would disappear forb−2, and the right one forb−3.

construction of the flat operator^T from anincreasingset operator, the set of allt ∈T such thatp∈(Xt(F ))is a closed interval[⊥, b], wherebgives the value^T(F )(p) (NB: this holds because we have discrete grey-levels; oth- erwise we could have the half-open interval[⊥, b[). This is no longer valid ifis not increasing; in particular, ifis a HMT, we will see below that it is an interval, but generally not containing⊥. The idea in Refs.[4,21]is to take as value of the grey-level HMT the length of that interval.

LetA, B ∈P(E)be disjoint SEs, and consider the finite grey-level set Tˆ = [t0. . . t1] ⊂ Z. Soille’s (unconstrained) HMT on grey-level images, written UHMTA,B, is defined [4, Eq. (5.3), p. 143]by setting for everyF ∈ ˆT^Eandp∈E:

UHMTA,B(F )(p)

=card{t ∈T |p∈X_t(F )(A, B)}. (25) Note that the resulting grey-level values will be non- negative, in fact they belong to the interval[0, t1−t0]. We illustrate it inFig. 3(to be compared withFig. 2).

In order to analyse Soille’s operator, we embed the grey- level setTˆ intoZ:

Proposition 4. LetA, B∈P(E),T=ZandTˆ=[t₀. . . t₁] ⊂ Z. For every t ∈ T, F ∈ ˆT^E and p ∈ E,we have p ∈ X_t(F )(A, B)iff

(C_A,0)_(p,t)=C_Ap,tF>C_B^∗_p,t=(C_B,0^∗ )_(p,t), iff(FB)(p) < tˇ (FA)(p).

Proof. Recall that q ∈ Xt(F ) iffF (q)t. The condition p ∈ Xt(F )(A, B) means that Ap ⊆ Xt(F ) and Bp ⊆ Xt(F )^c. The first partAp⊆Xt(F )translates as: for every q ∈ Ap,F (q)t; on the other hand, for q /∈Ap, we have

A B T

E B

Fig. 3. Here E=Zand T = [0. . . t₁] ⊂Z. On top we show the two structuring elementsAandB(the origin being the left pixel ofA). Below we show a functionF(the same as inFig. 2); the dots indicate the pairs (p, t)withp∈Xt(F )(A, B), and in grey we haveUHMT_A,B(F ).

always F (q) − ∞; hence Ap ⊆ Xt(F ) ⇔ CAp,tF. The second partB_p ⊆X_t(F )^c translates as: for everyq ∈ B_p, F (q) < t, that is F (q)+1t; on the other hand, for q /∈B_p, we have always F (q)+1t₁+1+ ∞; hence B_p ⊆X_t(F )^c⇔F>C_B^∗_p,t. Thereforep∈X_t(F )(A, B) iffC_Ap,tF>C_B^∗_p,t. Now clearlyC_Ap,t=(C_A,0)_(p,t)and C_B^∗_p,t=(C_B,0^∗ )_(p,t). Applying Proposition 2, and the fact that (C_B,0^∗ )^∗=C_B,0ˇ , the condition(CA,0)_(p,t)F>(C_B,0^∗ )_(p,t) is equivalent to(FC_B,0ˇ )(p) < t(FCA,0)(p), in other words by (22),(FB)(p) < tˇ (FA)(p).

We get thus:

UHMTA,B(F )(p)

=max{(FA)(p)−(FB)(p),ˇ 0}, (26) in other words[4, Eq. (5.4), p. 143]it has value

(FA)(p)−(FB)(p)ˇ

if(FA)(p) > (FB)(p), and 0 otherwise.ˇ

From Proposition 4, we see that Soille’s grey-level HMT is not restricted to flat SEs; the two sets A and B corre- spond implicitly to the cylinderC_A,0and the dual cylinder C_B,0^∗ . Also, it does not require discrete grey-levels; we have simply to measure at each point p the half-open interval ](FB)(p), (Fˇ A)(p)]. Now the Lebesgue measure inR and the discrete measure (cardinal) inZ, when applied to a half-open interval]a, b], both give its lengthb−a.

Assume thusT =Z orR. Letmes be the measure used onT(Lebesgue’s forT=Rand discrete forT=Z). For V , W ∈T^Esuch thatVW, we define theintegral interval operator^I_{[V ,W]}by setting for everyF ∈T^E andp∈E:

^I_{[V ,W]}(F )(p)

=mes({t∈T|V_(p,t)F>W_(p,t)})

=mes({t∈T|(FW^∗)(p) < t(FV )(p)})

=max{(FV )(p)−(FW^∗)(p),0}. (27)

0 1 2 3 4 5 6 0 1 2 3 4 5 6 0 1 2 3 4 5 6

0 1 2 3 4 5 6

0 1 2 3 4 5 6

(b)

(c) (a)

(d)

(e)

Fig. 4. (a) The function F; we have F^V = F (with V =C_{0},0). (b) The function F=F^{W∗ (with W} =C_{−1},0^∗ ) is the translate of F by +1. (c) G=^I_{[V ,W]}^{(F )} is given by G(p)=max{F (p)−F(p),0}; we have G=V(G)=V^I_{[V ,W]}^{(F ),} and G=G^{V. (d) G}=G^W∗ is the translate of G by +1. (e) H =^I_{[V ,W]}^(G) is given byH (p)=max{G(p)−G(p),0}; we have H=V(H )=V^I_{[V ,W]}^(G)=(V^I_{[V ,W]}^{)2(F ).}

In the third line of the equation, an expression of the form +∞−∞or−∞+∞for(FV )(p)−(FW^∗)(p)must lead to the value 0. Indeed, if(FV )(p)=(FW^∗)(p)= +∞, Corollary 3 gives FW_(p,t) for all t ∈ T, while if (FV )(p)=(FW^∗)(p)= −∞, Corollary 3 gives V_(p,t)F for allt ∈T; in both cases the second line of the equation givesmes(∅)=0.

We can take, as above with Ronse’s operator,V =C_A,a andW=C_B,b^∗ . ForA,Bandafixed, increasingbincreases ^I_{[V ,W]}(F )by the same amount on all points having non- zero value. For flat SEs (V=C_A,0andW=C_B,0^∗ ), we obtain Soille’s original operatorUHMTA,B.

As can be seen withFigs. 2and3, the two interval opera- tors^S_{[V ,W]}and^I_{[V ,W]}can be used to detect in an image all locations pwhere the grey-level on Ap is higher than that on Bp by at least some height h: here we take V =CA,a

andW=C^∗_B,bwithh=a−b. While^S_{[V ,W]}behaves as the erosionε_V at such locations, ^I_{[V ,W]}measures the effective difference between the grey-levels inAp andBp.

Note that, contrarily toV^S_{[V ,W]}, the operatorV^I_{[V ,W]}is not necessarily idempotent. Take for exampleE=Z, the flat SEsA={0}andB={−1}(thusV=C_{0},0andW=C_{−^∗ _1},0).

ThenV=εV is the identity, whileW^∗is the translation by +1. We illustrate inFig. 4the construction ofV^I_{[V ,W]}(F ) and(V^I_{[V ,W]})²(F )forFgiven byF (z)=zforz=1. . .5 andF (z)=0 otherwise.

Soille introduced a constrained variantCMHTA,B of his HMT. Here we assume that one of the two SEs A and B contains the origin o. If o ∈ A, in Eq. (25) we re- quire that p ∈ Xt(F )(A, B) fort =F (p), which means that (FB)(p) < F (p)ˇ (FA)(p); if the requirement is not met, the result is 0. As o ∈ A, we always have

(FA)(p)F (p), hence we get CHMTA,B(F )(p)

=card{t ∈T |(FB)(p) < tˇ (FA)(p)=F (p)}, in other words it is equal to

⎧⎨

⎩

(FA)(p)−(FB)(p)ˇ ifF (p)=(FA)(p)

> (FB)(p),ˇ

0 otherwise.

Ifo ∈ B, in Eq. (25) we require thatp ∈ Xt(F )(A, B) fort=F (p)+1, which means that(FB)(p) < F (p)ˇ + 1(FA)(p) that is (FB)(p)ˇ F (p) < (FA)(p), and the result is 0 if this condition fails. As o ∈ B, we always have(FB)(p)ˇ F (p), so we get

CHMTA,B(F )(p)

=card{t ∈T |F (p)=(FB)(p) < tˇ (FA)(p)}, in other words it is equal to

⎧⎨

⎩

(FA)(p)−(FB)(p)ˇ if(FA)(p)

> (FB)(p)ˇ =F (p),

0 otherwise.

In order to generalize this to arbitrary structuring func- tions, we can forget the requirement thatAorBcontains the origin, but keep only the constraint thatF (p)=(FA)(p) or F (p)=(FB)(p). Thus we obtain, forˇ V , W ∈ T^E such thatVW, theconstrained integral interval operator ^C_{[V ,W]}, which gives for everyF ∈T^E andp∈E:

^C_{[V ,W]}(F )(p)

=

^I_{[V ,W]}(F )(p) ifF (p)=(FV )(p) orF (p)=(FW^∗)(p),

0 otherwise. (28)

2.3. Barat’s morphological probing

Barat et al.[22–24] introduced under the name ofmor- phological probingan operation which has some similarity to the integral grey-level interval operator^I_{[V ,W]}. We con- sider again two structuring functionsV , W ∈T^E; the idea is to measure at each pointp ∈ E two numerical values t_v and t_w defined as follows: t_v is the greatest t such that V_(p,t)F, whilet_w is the leasttsuch thatFW_(p,t); then one associates topthe valuet_w−t_v.

From Proposition 2 and Corollary 3, we have (FV )(p)=sup{t∈T|V_(p,t)F}

and

(FW^∗)(p)=inf{t ∈T|FW_(p,t)}. (29) Moreover:

• if(FV )(p)= ±∞,(FV )(p)is the greatestt ∈T such thatV_(p,t)F;

t_v t_w

p

p W

W

V V

Fig. 5. Left: In morphological probing, we look for the least interval [t_v, t_w]such thatV_(p,tv)^FandFW(p,tw). Right: In the integral in- terval operator, we look for the greatest interval{t|V_(p,t)^F>^W(p,t)}.

• if(FW^∗)(p)= ±∞,(FW^∗)(p)is the leastt ∈T such thatFW_(p,t).

Thus Barat’s morphological probing operator MP_{V ,W} is given by

MP_{V ,W}(F )(p)=(FW^∗)(p)−(FV )(p) (30) for every F ∈ T^E andp ∈ E. We have ^I_{[V ,W]}(F )(p)= max{−MP_{V ,W}(F )(p),0} by comparison to Eq. (27). We illustrate in Fig. 5 the difference between morphological probing and the integral grey-level interval operator.

Contrarily to the two interval operators seen above, here we do not require on the structuring functionsVandWthat VW, but rather that we always haveFW^∗FV. For example consider two functionsG_v, G_wdefined on a support S, such that−∞< G_w(p)G_v(p) <+∞for allp∈S, and letV , W be defined byV (p)=G_v(p)andW (p)=G_w(p) forp ∈S, whileV (p)= −∞andW (p)= +∞forp /∈S.

Here we will have (FW^∗)(p)= sup

h∈S(F (p+h)−G_w(h)) inf

h∈S(F (p+h)−Gv(h))

=(FV )(p).

In Refs. [22–24] the particular case where G_v=G_w was considered. For instance, ifG_v=G_w has constant value 0 onS, we getV =CS,0 andW=C_S,0^∗ , as in the left image inFig. 5.

2.4. Other works

Khosravi and Schafer[32]use a single structuring func- tionVand define a grey-level HMT onFas the arithmetical sum[FV]+[(−F )(−V )]; by duality (20), this is equal to[FV] − [FV^∗]. This is thus the same as^I_{[V ,V]}, ex- cept that negative values are not changed into 0.

Schaefer and Casasent[13]use two structuring functions V andW, and define a grey-level HMT on F as the meet [FV] ∧ [(−F )W] (however, they use a non-standard notation for expressing this).

Raducanu and Graña[33] compare the grey-level HMT (GHMT) defined by Khosravi and Schafer[32]with an op- erator called the level set HMT (LSHMT). This operator consists in applying a binary HMT to the successive thresh- olds of a function F and of a structuring function G, and keep the supremum of all results:

FG=sup{t∈T |p∈Xt(F )(Xt(G), Xt(G)^c)}.

3. Unified theory

From the two interval operators described in Sections 2.1 and 2.2, we see that both involve two steps: first a fitting which associates to an imageFa set of pairs(p, t)∈E×T, for which the translatesV(p,t)andW(p,t)have some relation to F; it can eventually be associated to the operation of constraining; second avaluationwhich derives from this set of(p, t)a new grey-level image.

Assume V , W ∈ T^E with VW. The fitting used in Ronse’s supremal interval operator will be writtenH_{V ,W}, it is defined by

H_{V ,W}(F )= {(p, t)∈E×T|V_(p,t)FW_(p,t)}. (31) Another one was used in Soille’s integral interval operator, we write itK_{V ,W}, it is defined by

K_{V ,W}(F )= {(p, t)∈E×T|V_(p,t)F>W_(p,t)}. (32) Next, the constraining is the operator C_{V ,W} : T^E → P(E×T), associating to a functionF :E→T the set C_{V ,W}(F )= {p∈E|F (p)=(FV )(p)

or F (p)=(FW^∗)(p)} ×T. (33) We get thus the two constrained fittingsH_{V ,W}^C and K_{V ,W}^C defined by

H_{V ,W}^C (F )=H_{V ,W}(F )∩C_{V ,W}(F ) and

K_{V ,W}^C (F )=K_{V ,W}(F )∩C_{V ,W}(F ). (34) The valuation must associate to any subset ofE×T a functionE→T. The one used in Ronse’s supremal interval operator is the upper envelope operatorS, associating to any Y ∈P(E×T)the function

S(Y ):E→T :p→sup{t∈T|(p, t)∈Y}. (35) Note thatSis a dilation in the algebraic sense[1], that is S

i∈I

Y_i

=

i∈I

S(Y_i); (36)

the adjoint erosion[1]is the map associating to a function Fits umbraU(F ), see Eq. (16).

Soille’s integral interval operator uses another one, written I, associating to anyY ∈P(E×T)the function

I (Y ):E→T :p→mes({t∈T|(p, t)∈Y}), (37) wheremesmeans the measure (Lebesgue’s forT=Rand discrete for T=Z). Following Ref. [19], for a sequence Xn of sets and a setX, we writeXn ↑X to mean that the sequenceXn (n∈N) is increasing (i.e.,Xn⊆Xn+1for all n ∈ N) and converges toX(i.e., X=

n∈NX_n); similarly for a numerical sequencer_n,r_n↑rmeans that the sequence is increasing and converges to r(i.e.,r_nr_n+1 for alln ∈ N, andr=sup_n∈Nr_n). A well-known property of measures is that for a sequence X_n of measurable sets, X_n ↑ X

⇒ mes(X_n) ↑ mes(X) (see Theorem 1.8(c) on p. 25 of Ref.[34]). We have thus for a sequenceY_n(n∈N)inE×T: Y_n↑Y ⇒I (Y_n)↑I (Y ), (38) which is weaker than being a dilation, as in Eq. (36).

We introduce a third valuation, the binary one B, which associates to anyY ∈P(E×T)the set

B(Y )= {p∈E| ∃t ∈T, (p, t)∈Y}. (39) We can represent it as a function with value+∞onB(Y ) and−∞elsewhere, this gives thus the binary mask valuation Massociating toYthe function

M(Y ):E→T: p→

+∞ if∃t ∈T, (p, t)∈Y,

−∞ otherwise. (40)

Note thatBandMare also dilations in the algebraic sense, that is

B

i∈I

Y_i

=

i∈I

B(Y_i)

and M

i∈I

Yi

=

i∈I

M(Yi); (41)

their adjoint erosions are, respectively; for B the map P(E)→ P(E×T):X →X×T, and for M the map {−∞,+∞}^E→P(E×T):F →supp(F )×T=U(F ).

Composing one ofHV ,W orKV ,W, optionally constrained by intersection with CV ,W, by one of S,I andM, we ob- tain an interval operator. We have thus six unconstrained operatorsSHV ,W,SKV ,W,I HV ,W,I KV ,W,MHV ,W and MKV ,W, as well as six constrained ones,SH^C_{V ,W},SK^C_{V ,W}, I H^C_{V ,W},I K^C_{V ,W}, MH^C_{V ,W} andMK^C_{V ,W}. We see then that Ronse’s supremal interval operator is SHV ,W, Soille’s un- constrained integral interval operator is I KV ,W, while the constrained one isI K^C_{V ,W}. In Refs.[27,28]we used a union ofBHV ,W for various choices of pairs(V , W ), as a form of