Image filtering using morphological amoebas

HAL Id: hal-01431825

https://hal-mines-paristech.archives-ouvertes.fr/hal-01431825

Submitted on 25 Jan 2017

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

Image filtering using morphological amoebas

Romain Lerallut„ Etienne Decencière, Fernand Meyer

To cite this version:

Romain Lerallut„ Etienne Decencière, Fernand Meyer. Image filtering using morphological amoebas.

Image and Vision Computing, Elsevier, 2007, 25 (4), pp.395-404. �10.1016/j.imavis.2006.04.018�.

�hal-01431825�

Image ltering using morphologi al amoebas

Romain Lerallut

∗

, Étienne De en ière, Fernand Meyer

Centre de Morphologie Mathématique, É ole des Mines deParis

35rue Saint-Honoré, 77305 Fontainebleau, Fran e

Abstra t

Thispaperpresentsmorphologi aloperatorswithnon-xedshapekernels,or

amoe-bas, whi h take into a ount the image ontour variations to adapt their shape.

Experimentsongrays aleand olorimagesdemonstrate thatthese novellters

out-perform lassi al morphologi aloperationswitha xed,spa e-invariant stru turing

elementfor noiseredu tionappli ations.Proof-of- on eptstestsarethenperformed

on 3Dimages to showthe highnoise-redu tion apa ity ofamoeba-based lters.

Key words: Anisotropi lters,noise redu tion, morphologi al lters, olorlters,

3Dimage pro essing

1 Introdu tion

Noiseis possibly the mostannoyingproblem inthe eld of image pro essing.

There are two ways to work aroundit: eitherdesign parti ularlyrobust

algo-rithms that an work in noisy environments, or try to eliminate the noise in

a rst step while losing as little relevant information as possible and

onse-quently use a normallyrobust algorithm.

Thereareof oursemanyalgorithmsthataimatredu ingthe amountofnoise

in images. Most are quite ee tive but also often remove thin elements su h

as anals orpeninsulas. Even worse, they an displa ethe ontours and thus

reate additionalproblems ina segmentation appli ation.

∗

CorrespondingAuthor.

image

Fig.1.Classi noiseltering(b)removesmu h ontourinformation.Re onstru tion

( )nds not only the ontours,but alsoall the noise onne ted to theobje t.

In mathemati al morphology we often ouple one of these noise-redu tion

lters toare onstru tion lterthat attempts tore onstru t onlyrelevant

in-formation,su h as ontours,andnot noise.However, afaithfulre onstru tion

an be problemati when the ontour itself is orrupted by noise. This an

ause great problems in some appli ations whi h rely heavily on lean

on-tour surfa es, su h as 3D visualization, so a novel approa h was proposed:

morphologi alamoebas.

An amoeba (here Amoeba proteus) is a

genus of protozoa that moves by

proje t-ing pseudopods and is a well-known

rep-resentative uni ellular organism.They are

foundinsluggishwaters alloverthe world,

both fresh and salt, as well as in soils and

asparasites. They now begin a new life in

the eld of imagepro essing.

2 Amoebas: dynami stru turing elements

Prin iple

Classi lter kernel

Formallyat least, lassi lters work on a xed-size sliding window, be they

morphologi aloperators(erosion, dilation) or onvolution lters, su h as the

isotropi Gaussian diusion smooths the ontours when its kernel steps over

astrong gradient area.

Fig.2.Closing ofanimage bya largestru turing element. Thestru turing element

doesnot adaptitsshape and mergestwo distin t obje ts.

Amoeba lter kernel

Having made this observation, Perona and Malik [1℄ (and others after them)

have developed anisotropi lters that inhibit diusion through strong

gra-dients. The rst work on non-xed shape stru turing elements was done by

Braga-Neto in [7℄ though it was restri ted to alternate sequential lters of

openingsand losings.Wewere inspired by theseexamples todene

morpho-logi allterswhosekernelsadapttothe ontent ofthe imageinorder tokeep

a ertainhomogeneousnessinsideea hstru turingelement(see gure3)while

at the same time keeping their size in he k. Tomasi and Mandu hi have

de-s ribed in [5℄. the idea of oupling performed a geometri distan e between

pixelsanda distan ebetween theirvalues, whi hoersremarkable properties

for our intended use.

Theinterestofthisapproa h, omparedtotheanalyti aloneisthatitdoesnot

depart greatly from what we use in mathemati al morphology,and therefore

most of our algorithms an be made to use amoebas with little additional

work. Most of the underlying theoreti al groundwork for the morphologi al

approa h has been des ribed by Jean Serra in his study [2℄ of stru turing

fun tions, although until now ithas seen littlepra ti aluse.

Fig.3. Closing of animage by anamoeba.The amoeba doesnot ross the ontour

and assu hpreserveseven thesmall anals.

Fig.5.Behavior ofan amoeba on various relief types. Stronggradients shouldslow

or even hamperthe growth oftheamoeba

entered. Figure 4 shows the shape of an amoeba depending on the position

of its enter. Note that in at areas su h as the enter of the dis , or the

ba kground,the amoeba is maximallystret hed, while itis relu tant to ross

ontour lines.

When an amoeba has been dened, most morphologi aloperators and many

othertypesoflters an beusedonit:median,mean, ranklters,erosion,

di-lation,opening, losing,even more omplexalgorithmssu hasre onstru tion

lters, levelings, oodings, et .

Constru tion

Amoeba distan e

In general, a ltering kernel of radius

r

is formally dened on a square (or a hexagon) of that radius, that is to say on the ball of radius

r

relative to the norm asso iated to the hosen onne tivity. We will keep this denition

hangingonlythe norm,using onethat takesintoa ount the gradientofthe

image,so that we get the behavior des ribed ingure 5.

Denition 1 Let

d

pixel

be adistan e denedbetween the values of theimage, for example a dieren e of gray-value, or a olor distan e.

Let

σ = (x = x

0

, x

1

, . . . , x

n

= y)

a path between points

x

and

y

. Let

λ

be a realpositive number. The length of the path

σ

is dened as

L(σ) =

n

X

i=0

[1 + λ.d

pixel

(Image(x

i

), Image(x

i+1

))]

The amoeba distan e with parameter

λ

is thus dened as:











d

λ

(x, x) = 0

d

λ

(x, y) = min

σ

L(σ),

minimum taken on all paths between x and y

Itit importanttorealizethat

d

pixel

has nogeometri alaspe t, itisadistan e omputed only on the values of the pixels of the image. Furthermore, if

n

is the number of pixelsof a path

σ

, then

L(σ) ≥ n

(sin e

λ ≥ 0

),whi hbounds the maximal extension ofthe amoeba.

This distan e also oersaninteresting in lusionproperty:

Property 1 At a given radius

r

the family of the balls

B

λ,r

relative to the distan e

d

λ

is de reasing (for the in lusion),

0 ≤ λ

1

≤ λ

2

⇒ ∀(x, y), d

λ

1

(x, y) ≤ d

λ

2

(x, y)

⇒ ∀r ∈ R

+

, B

λ

1

,r

⊃ B

λ

2

,r

Whi h may be useful when buildinghierar hies of lters, su h asa family of

alternate sequential lters with strong gradient-preserving properties.

The pilot image

We have found that the noisein the image an often distort the shape of the

amoeba.Forthisreason,weoften omputetheshapeoftheamoebaonanother

image. On e the shape is omputed, the values are sampled on the original

image and pro essed by the lter (mean, median, max, min, ...). Usually,

the other imageis the resultof a strongnoise removallteringof the original

imagethat dampens the noisewhile preserving asmu haspossiblethe larger

ontours.A large Gaussianworks fairlywell,and an beapplied very qui kly

with advan ed algorithms, however we will see below that iterating amoeba

Adjun tion

Erosionsand dilations an easily bedened onamoebas.However itis

ne es-sarytouse adjoint erosionsand dilationswhen usingthem todeneopenings

and losings:

δ(X) =

S

x∈X

B

λ,r

(x)

ǫ(X) = {x/B

λ,r

(x) ⊂ X}

These two operations are at the same time adjoint and relatively easy to

ompute, ontrary tothe symmetri alones that use the transposition,whi h

is not easy to ompute for amoebas. See [2℄ for a dis ussion of the various

formsof adjun tion and transposition of stru turingfun tions.

Algorithms

Thealgorithmsusedfortheerosionanddilationarequitesimilartothoseused

withregularstru turingelements,withtheex eptionofthestepof omputing

the shapeof the amoeba.

The openingusing thesealgorithms anbeseen asthe gray-levelextensionof

the lassi binary algorithm of rst taking the enters of the ir les that t

inside the shape (erosion), and then returning the union of all those ir les

(dilation).See [10℄ for a more detailed des ription of the algorithmsused for

adjointerosion and dilation.

Complexity

The theoreti al omplexity of a simpleamoeba-based lter (erosion,dilation,

mean, median) an be asymptoti allyapproximated by:

T (n, k, op) = O

h

n ∗



op(k

d

) + amoeba(k, d)

i

Where

n

is the number of pixels in the image,

d

is the dimensionality of the image(usually 2 or3),

k

isthe maximum radius ofthe amoeba,

op(k

d

₎

ost of the operation and

amoeba(k, d)

is the ost of omputing the shape of the amoeba for agiven pixel.

The shape of the amoebas is omputed by a ommon region-growing

imple-mentation using a priority queue. Depending on the priority queue used, the

omplexity of this operation is slightly more than

O(k

d

₎

(see [3℄ and [4℄ for

advan edqueueing data stru tures).

Therefore, forerosion,dilationormean asoperators,wehave a omplexityof

alittlemorethan

O(n ∗ k

d

₎

whi histhe omplexityofalteronaxed-shape

kernel. It has indeed been veried in pra ti e that, while being quite slower

than with xed-shape kernels (espe iallyoptimized ones), lters using

amoe-bas tend to follow rather well the predi ted omplexity, and do not explode

(tests have been performed on 3D images, size 512x512x100, with amoebas

with sizes up to 21x21x21).

4 Results

Alternate sequential lters

The images of gure 6 ompare the dieren es between alternate sequential

lters (ASF) built on lassi xed shape kernels and ASFs on amoebas in

the ltering of the image of a retina. The lter should be able to redu e the

amount of ba kground noise while preserving the shapeof the vessels.

Median and mean

In the ontext of image enhan ement, we have found that a simple mean or

median oupled with anamoeba formsa verypowerful noise-redu tion lter.

The imagesingure 7showthe median andthe mean omputed onamoebas

omparedtothose builtonregularsquarekernels.Thepilotimagethatdrives

the shape of the amoeba is the result of a standard Gaussian lter of size

3 on the original image, and the distan e

d

pixel

is the absolute dieren e of gray-levels.

Forthe ltersusingamoebas,the medianlterpreserveswellthe ontour,but

the meanlter gives amoreaestheti ally pleasing image. Ineither ase,the

results are learly superior to lterings by xed-shape kernels, as seen in the

pass ond pass

(d)AmoebaASF:rst

pass

(e) Amoeba ASF:

se -ond pass

(f) Amoeba ASF:

fourthpass

Fig. 6.Alternate sequential lters on lassi kernels and on amoebas. The amoeba

preserves extremely wellthebloodvesselswhile stronglyatteningthe otherareas.

Mean and median for olor images

In the ase of olor images, the mean is repla ed by the mean on ea h olor

omponentof the RGB olorspa e. Forthe median,the point losesttothe

bary enter is hosen. Other distan es or olorspa es an be used, depending

onthe appli ation,the type of noiseand the quality of the olor information.

Iteration

The quality of the ltering strongly depends on the image that determines

theshapeofthe amoeba.Thepreviousexampleshaveused theoriginalimage

ltered by a Gaussian, but this does not always yield good results (also see

dian median mean

Fig. 7. Results of a  lassi  median ltering and two amoeba-based lterings: a

median anda meanon Edouard Manet'spainting Lefre.

It isfrequent indeedthat asmalldetailof the imagebeex essively smoothed

in the pilot image, and thus disappears ompletely in the result image. On

the other hand, noisy pixels may be left untou hed if the pilot image does

not eliminate them. A possible solution isto improvethe quality of the pilot

image,sothatithelpsthe amoebainpreservingthesefeatures.Su hanimage

should be well-smoothed in at regions, while preserving as well as possible

the ontour information.One goodmethodto ompute su h animage would

be of ourseto use an amoeba-basedlter !

We willpro eed in two steps: the rst one follows the s heme des ribed

ear-lier,usingthe Gaussian-lteredoriginalimage asapilot, withlarge amoebas,

and outputs a well-smoothed image in at areas while preserving as mu h

as possible the most important ontours. The se ond step takes the original

imageasinput andthe lteredimageasapilot, withsmaller amoebas.These

amoebas don'tneed to beas large asthe rst ones,sin e their shapes willbe

omputed on a very smooth image, and therefore they will preserve well the

dian median mean

Fig. 8.Color images: results of a  lassi  median ltering, and two amoeba-based

lterings: a median and a mean. As a simple extension of the grays ale approa h,

ea h hannelof the pilot image hasbeen independently smoothedbya Gaussian of

size 3.

Althoughthis renement of the pilot image ouldbeiterated, we have found

inpra ti ethat on eisenoughtoredu ethenoisedramati ally(seegure9).

Thismethodisalsoveryusefulfor olorimages,sin ethe amoeba-basedpilot

image provides better olor oupling through the use of an appropriate olor

distan ethansimplymergingtheresultsofaGaussianlteringofea h hannel

independently.

5 Appli ation to 3D images

3D images and ontour noise

Whilenoise redu tionis ani e bonusfor 2D images,things are dierent with

sian lter amoeba mean lter

(d)Resultimage:amoebameanwith

Gaussian pilot

(e)Resultimage:amoebameanwith

amoeba pilot

Fig. 9. Comparison between two pilot images: a Gaussian one, and one based on

a strong amoeba-based ltering. With the amoeba pilot image the hand is better

preserved, and the eyebrows do not begin to merge with the eyes, ontrary to the

Gaussian-basedpilotimage.Havingbothlessnoiseandstronger ontoursinthepilot

image also enables the use of smaller values on the lambda parameter so that the

amoeba will stret h more in the atter zones, and thus have a stronger smoothing

Fig.10.A syntheti 3Dvolume presentingmanysimilaritieswithmedi al imagesof

the ardia regions. Left:original image. Right:image withaddition of noise.

informationforthe omputationofthe shadingof thevoxels andmany

appli- ations,espe iallyinthe medi al eld,use mostlythe ontour informationto

visualize the various obje ts and as su h are very vulnerable to noise on the

ontours (see gure 10).

When displaying dire tly 3D data (as opposed to studying a sta k of 2D

im-ages),itisessentialthat theuserbeabletoseethe obje tsthey areinterested

in. This is why most modern renderers in lude a omplex transparen y and

shading model that makes it possible to peek far inside the image to display

the interesting obje ts. To make a quantitative analysis easier, a syntheti

image was reated that presents many similarities with 3D s anner images,

espe ially images of the ardia region: strong textures and thin vessels to

preserve.

Figure 10 illustrates the problem due to strong noise in a 3D image and

g-ure 11 shows that levelings are inee tive as they suppress the noise inside

the obje ts but not on the ontours. However, as with most morphologi al

tools,amoebasare adimensionaland anbe usedwithoutmodi ation on3D

images.

Median ltering withamoebas

One important aspe t of the ltering of su h medi al images is that those

imagesare monospe tral: oloringand,ultimately,tissueidenti ationisdone

ing

Fig.11.Althoughthe medianltersmu hnoise away,there onstru tion(neededto

re over the position of the ontours) re onstru ts most of thenoise on the borders

oftheobje t, removing most ofthe transparen y.

dramati ee t on the visualization and onsequently on the interpretation

of the images. This is why we have hosen to rst test the median: with its

property of returning only values existing in the image, the median lessens

the risk of misinterpretation. However, traditional median ltering does not

preservewellthe ontours,andmayremovesmalldetailswhi hmaybe ru ial

toa physi ian's analysis, hen ethe use of amoebas.

For ea h pixel the pro essing is done in two steps: rst ompute the shape

of the amoeba entered on the pixel and then sample the values of the pixel

inside the amoeba, feed them to the median operator and write the result at

the enter of the amoeba in the output image.

Results for 3D images

Figure12shows theresultofamedianlter omputedonamoebas.Itisplain

toseethatmostofthetransparen yee thasbeenpreserved,whi hindi ates

that most of the noise has been ltered onthe ontours as well as inside the

obje ts.

Figure 13 is a zoom of gure 12 entered on the thin stru tures. On e again

wehavegoodresults inthepreservation ofthose elongatedstru tures.This is

absolutelyessentialsin e thistypeof featuresin lude riti albody partssu h

dian omputed onamoebas

Fig. 12. The median applied to the amoeba re overs most of the information

very well, in luding thin details. The leaner gradient on the ontours results in

anear-perfe t transparen yee t.

(a)Original image (zoomed) (b) Amoeba median lter

(zoomed)

Fig.13.The amoeba-based median lter re overs verywell the ontours, aswell as

thetransparen y.

It is extremely hard to quantify in a meaningful way the results of su h a

ltering, espe ially sin e the usual signal-to-noiseratio does not express well

the fa t that we may tolerate small variations in many pla es but not a few

strongvariationsin riti alareas.Amoremeaningfulmeasureforthisproblem

with that of the orresponding voxel in the ideal image. When a voxel in

the ltered image ontributesin the same amountto the visualization asthe

orresponding voxel in the ideal image,then itsasso iated quality measure is

equaltoone.Thegreaterthedieren ebetweenboth ontributions,the loser

tozero the measure willbe. Results are then averaged in the whole image as

well asseparatelyon ea h omponent.

Figure 14shows a omparison between an amoeba-based median lter and a

lassi alternate sequential lter (ASF). The results show learly that while

the ASF is as good as the amoeba on the larger stru tures (spheres B2, B3

and B5), it fails ompletely topreserve the thin stru tures su h as the rings.

Furthermore, there is a lear shift of the values, espe ially on the outermost

rings, whi h may ause diagnosti errors.

Simple optimizations

Animportantfa ttotake intoa ountis thatthe radius parameterislikean

amount of energy given to the amoeba. It an be used either to limbslopes

(with a penalty given by the

λ

parameter) or it an be used to expand in at areas. This amount of energy needs to be quitehigh so that the amoeba

an jumpover noisypixels(though not toohigh so that itdoes not ross too

mu h over strong gradient lines). However su h a high energy means that in

at areas the amoeba will grow to a very large size, whi h means that not

only will the shape be ostly to ompute but the resulting sample of pixel

values willbequitelargeandsothe lteroperatorwillbea ordinglylong. A

verysimpleyetdramati allyee tiveoptimizationisthustoimpose anupper

bound onthe size of the amoeba, the value of whi h depends on the type of

noise and the hara teristi size of the image elements. This an redu e the

ost of omputation by an order of magnitude without any dete table lossof

ee tiveness.

Another form of optimization is to ompute the shape of the amoeba on a

slightlylteredversionoftheoriginalimage,su hasaGaussianltering.This

willredu e smallnoise withoutmovingtoomu hthe ontoursand enablethe

use of smaller amoebas.

6 Con lusion and future work

Wehavepresented hereanew typeof stru turingelementthat anbeusedin

dian

0

0.2

0.4

0.6

0.8

1

Texture 50

B1

B2

B3

B4

B5

P1

P2

Rings

Global

Amoeba

ASF3

(d) Measureof the results

Fig. 14. Comparison of the results of an amoeba-based median lter and an

alter-natedsequential lterofsize upto3.Themeasureisperformedonea h omponent

ofthe image (spheres B1-B5, parallelepipedsP1 and P2,rings,and total average).

ltersbuiltuponthosestru turingelements anbemademorerobustonnoisy

images and in general behave in a more sensible way than those based on

xed-shapestru turingelements.Inaddition,morphologi alamoebasarevery

adaptable and anbeused on olor imagesaswell asmonospe tral onesand,

like most morphologi al tools,they an be used on images of any dimension

output smoother images that may be more pleasing to the eye but ould be

harder tosegment.

Itispossibletouseamoebasto reatere onstru tionltersandoodingsthat

take advantage of the ability to parametrize the shape of the amoebas based

onthe image ontent. However, the behaviorsof the amoebas are mu h more

di ulttotakeinto a ountwhen they are usedin su h omplex algorithms.

In parti ular, if onnexity is important (su h as in a re onstru tion lter),

then amoeba with amaximum radius of one pixel shouldbeused.

The results show that simple extensions of the s alar algorithmsto the RGB

spa ealreadyyieldex ellentresults,espe iallywheniterating.Theuseofmore

per eptual distan es(HLSorLAB) wouldprobablyprevent someunwanted

blending of features, although this is as yet onje tural and will be the basis

of further work.

The ltering of 3D images by morphologi al amoebas, though still in its

in-fan y,seemsverypromising.Expressingthe ouplingbetweenimagedataand

geometrythroughakernelmakesitpossibletoimplementamu hlargerrange

of lters to an image than was possible before. Another area where

improve-ment is to be expe ted is the omputation of the shape of the amoeba. Not

only should it be possible to use elaborate gradient estimation su h as

pro-posedin[6℄,butalsoprovidingmore omplexbehaviorsfortheamoebas,su h

as anin ompressible minimum element,to guarantee atleast some diusion,

or on the ontrary a minimum size requirement to prevent diusion through

smallholes.

Finally,itisimportanttonoti ethatthisamoebaframeworkisgeneralenough

toa ommodateothertypesofdistan es. Thedistan epresented here ouples

geometryandgreylevels(or olordistan es),butothersimilars hemes anbe

expressed in termsof amoebas.For instan es,the approa hes presented in[7℄

and [8℄, whi h oer very interesting results, an be implemented by amoebas

withthe appropriatedistan e,whi hthusinheritallthe possibilitiesavailable

totheonesdes ribedinthispaper.Thesevariousapproa hesshowtheviability

and thevitality ofthe amoebaframework,aswellasitsappli abilitytomany

elds of resear h.

Referen es

[1℄Perona, P. and Malik, J., S ale-spa e and edge dete tion using anisotropi

diusion,IEEETransa tionsonPatternAnalysisandMa hineIntelligen e,vol.

and mathemati al morphology - Volume 2: Theoreti al advan es, Chapter 2, pp

37-46, A ademi Press,1988

[3℄Cherkassky,B.V.andGoldberg,AV.,Heap-on-toppriorityqueues,TR96-042,

NEC Resear h Institute,Prin eton,NJ, 1996

[4℄Brodnik, A. et al., Worst ase onstant time priority queue, Symposium on

Dis rete Algorithms,2001,pp 523-528

[5℄Tomasi, C. and Mandu hi, R.,Bilateral Filtering for Gray and Color Images,

Pro eedings of IEEE International Conferen e on Computer Vision, Bombay,

India, 1998

[6℄Catté, F. et al., Image sele tive smoothing and edge dete tion by nonlinear

diusion,SIAMJ.Numeri alAnalysis, Vol.29 No.1,pp182-193,February1992

[7℄U.M. Braga-Neto, "Alternating Sequential Filters

byAdaptive-NeighborhoodStru turingFun tions.",Pro eedingsofInternational

Symposiumon Mathemati alMorphology ISMM'96,pp.139-146, 1996.

[8℄Debayle, J. and Pinoli, J.-C., Multis ale image ltering and segmentation by

meansofadaptiveneighborhoodmathemati almorphology,Pro eedingsofIEEE

International Conferen e onImage Pro essing, Genoa,Italy,2005

[9℄Boehm, M. Contribution à l'amélioration du rendu volumique de données

médi ales 3D, PhD Thesis, Centre de Morphologie Mathématique, É ole des

Mines de Paris, 2004

[10℄Lerallut, R. and De en ière, E. and Meyer, F., Image pro essing using

morphologi al amoebas, Pro eedings of the 5th International Symposium on