HAL Id: hal-00918290
https://hal.archives-ouvertes.fr/hal-00918290
Submitted on 13 Mar 2014
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Distributed under a Creative Commons Attribution - NonCommercial - NoDerivatives| 4.0
Statistical region-based active contours for segmentation: an overview
François Lecellier, Stéphanie Jehan-Besson, Jalal M. Fadili
To cite this version:
François Lecellier, Stéphanie Jehan-Besson, Jalal M. Fadili. Statistical region-based active contours
for segmentation: an overview. Innovation and Research in BioMedical engineering, Elsevier Masson,
2014, 35 (1), pp.3-10. �10.1016/j.irbm.2013.12.002�. �hal-00918290�
Statistical region-based active contours for segmentation: an overview
F. Lecellier
a, S. Jehan-Besson
b, J. Fadili
baXLIM-SIC, UMR CNRS 7252, Bd Marie et Pierre Curie BP 30179, 86962 Futuroscope Chasseneuil cedex, France
bGREYC CNRS-ENSICAEN-Universit´e de Caen, UMR 6072, Bd du Mar´echal Juin, 14050 Caen, France
Abstract
In this paper we propose a brief survey on geometric variational approaches and more precisely on statistical region-based active contours for medical image segmentation. In these approaches, image features are considered as random variables whose distribution may be either parametric, and belongs to the ex- ponential family, or non-parametric estimated with a kernel density method.
Statistical region-based terms are listed and reviewed showing that these terms can depict a wide spectrum of segmentation problems. A shape prior can also be incorporated to the previous statistical terms. A discussion of some optimization schemes available to solve the variational problem is also provided. Examples on real medical images are given to illustrate some of the given criteria.
1 Introduction
Due to the rapid evolution of medical imaging systems, image segmentation
faces various and increasingly difficult challenges. In order to cope with the
growing variety of data encountered in medical imaging, segmentation tools
must be able to handle different noise models and to provide a way to include
geometric and photometric priors. To this end, geometric variational approaches
that consider image segmentation as a continuous optimization problem are
particularly well adapted (see for example [1–3]). Indeed, the computation of a
given shape (i.e. segmentation) can be advantageously modeled as the optimum
of a wisely chosen continuous energy functional. Continuous energy functionals allow the use of photometric image properties such as texture [4–8] and noise [9–12], as well as geometric properties such as the prior shape of the object to be segmented [13–18], see also [19] and more recently [20] for some reviews on deformable models and active contours in medical image segmentation.
In this paper, we propose to focus on region-based terms that take benefit of the probability density function of a given image feature inside the region of interest [1, 9, 21]. We more particularly review the optimization of diver- gences between pdfs which represent a general setting for both segmentation and tracking in medical images. When considering a segmentation framework, we aim at maximizing the distance between the pdf of the inside region and the pdf of the outside region. When considering a tracking application, we aim at minimizing the distance between the pdf of the region of interest and a reference one. The pdf can be considered as parametric (e.g. Gaussian, Rayleigh ...) or non parametric (no assumption is made on the law). In the literature, region tracking using non parametric probability density functions has been first pro- posed in [22] for video and then developed for cardiac structures tracking in perfusion MRI (p-MRI) sequences in [23]. On the other hand, some authors [24] have also proposed to take benefit of the maximization of the Bhattacharya distance of non parametric pdfs for segmentation. Concerning parametric pdfs, we restrict our study to the exponential family as first proposed in [9]. The rationale behind using the exponential family is that it includes, among others, Gaussian, Rayleigh, Poisson and Bernoulli distributions that have proven to be useful to model the noise structure in many real image acquisition devices (e.g.
Poisson for photon counting devices such as X-ray or CCD cameras, Rayleigh
for ultrasound images, etc). To cope with the occlusion and missing data and
to alleviate initialization issues in medical image segmentation, a shape prior on
the structure to be isolated can also prove necessary. For space reasons, we re-
strict our review to the use of the Legendre moments in a variational approach
first proposed in [16] and applied to echocardiography in [25] and in cardiac
segmentation on non-contrast CT images in [18].
One of the difficult point concerning variational approaches remains the as- sociated resolution schemes. We propose here to review two main schemes. The first one is based on advanced and efficient optimization tools (e.g. originating from shape derivation tools [26] and the second one relies on efficient non-smooth convex optimization tools as proposed in [27]). Some segmentation examples of medical structures are taken (brain and cardiac MRI, echocardiography, MRI perfusion) to show the adaptability of such statistical segmentation methods.
2 Statistical region-based terms for segmentation
A region-based segmentation problem aims at finding a partition of the image domain Ω into n regions {Ω
1, .., Ω
n} of respective boundaries {∂Ω
1, .., ∂ Ω
n} that minimizes the following criterion:
E(Ω
1, .., Ω
n, Γ) =
n
X
i=1
E
ri(Ω
i) + λE
b(Γ) , (1) where E
riis the region-based term related to the domain Ω
i, and Γ = S
ni=1
∂Ω
i. The regularization term E
bis balanced with a positive parameter λ.
We focus here on statistical region-based term E
rithat takes advantage of the pdf of some image features y whose realizations take values in χ ⊂ R
pwithin each region Ω
i(e.g. when considering the intensity in grey level images y(x) = I(x) with x the pixel location, p = 1 and χ = [0, 255]). This framework is then also adapted to the use of vectorial image features such as the coefficients of the wavelet transform or optical flow vectors. At this stage, we consider two main classes of region descriptors detailed in the two sections below. We also provide a comparison for brain MRI segmentation in section 6.2.
2.1 Statistical descriptors based on regions integrals
For the first kind of region descriptors, we consider the minimization of region integrals of some well chosen functions of the pdf of the feature y within the region Ω
i, namely p(y(x), Ω
i). We then consider as a region-based term:
E
ri(Ω
i) = Z
Ωi
Φ(p(y(x), Ω
i)dx , (2)
with Φ at least C
1and Lebesgue integrable function. Let us note that when Φ(t) = − log(t) the function (2) is known as the log-likelihood score function, used to describe the homogeneity of a region. It has been introduced in [1] using Gaussian pdfs and in [9] using parametric pdfs from the exponential family. In [12], we consider a general setting replacing the − log by any proper function Φ within the exponential family. We also elucidate the impact of the estima- tion method of the hyper-parameters on the shape derivatives of the criterion.
Such a criterion has been also investigated using non parametric pdfs and some functions Φ related to mutual information and entropy as developed in [28, 29].
2.2 Statistical descriptors based on features integrals
Another class of statistical region descriptors may be interesting since it can depicts both supervised and unsupervised segmentation. In this class of statistical descriptors, we consider integrals over the feature domain χ and so we do not make any assumption on the independence of the random variable y(x) on the contrary to the region descriptors described using (2). Let us then introduce the following functional which represents the distance, or more generally the divergence, between the current pdf estimate p(·, Ω
i) and another one q(·) for some appropriate function Ψ comparing pdfs:
E
ri(Ω
i) = Z
χ
Ψ (p( y, Ω
i), q( y )) dy . (3) We can introduce for example the Hellinger distance or the commonly used Kullback-Leibler divergence. Some other distances or divergences can be intro- duced to improve the accuracy and robustness of the segmentation such as the Wasserstein distance [30] or alpha-divergences [31]. Segmentation of complex regions can also take benefit of local estimation of the pdfs on small patches as proposed in [32].
Such divergences represent a general setting for both segmentation and track-
ing in medical images. Indeed, we may cast the segmentation problem as the
maximization of the distance between the pdfs of the feature in the inside and
outside regions. In order to fix ideas, let us consider a partition of an image in
two regions Ω
1and Ω
2. The segmentation may be formulated as the maximiza- tion of the following criterion:
E
r(Ω
1, Ω
2) = Z
χ
Ψ (p(y, Ω
1), q(y, Ω
2)) dy . (4) In the tracking problem, one aims at finding a consistent region Ω
ithrough a series of images. We can assume statistical similarity between the pdfs of the region in two consecutive images. We then search for the domain that minimizes the functional (3) where q(·) is a reference pdf that has been learned from the region of interest:
E
ri(Ω
i) = Z
χ
Ψ (p(y, Ω
i), q
ref(y)) dy . (5) This framework may also be applied to supervised segmentation where a refer- ence pdf is learned on the region of interest.
3 Statistical features description
For all the above criteria, the pdf can be either parametric or non-parametric.
Parametric pdfs are useful for the segmentation and tracking of homogeneous regions. The pdf can then be chosen according to the noise model as detailed thereafter. However parametric models may be subject to bias when the pdf model is only an approximation of the true noise model. In this case non parametric pdfs may be useful as well as in the case of segmentation or tracking of non homogeneous regions.
3.1 Parametric pdfs within the exponential family
In this case, image features are considered as random variables whose distri-
bution belongs to some parametric family which is chosen according to the noise
model. The exponential family is a flexible class of distributions that includes,
among others, Gaussian, Rayleigh and Poisson distributions that have proven
useful to model the noise structures in medical imaging systems. More formally,
the current pdf estimate p(·, Ω
i) is now parametrized by a vector θ
i∈ Θ ⊂ R
κ(e.g. we have κ = 2 and θ
i= (µ
i, σ
i)
Twhere µ
iis the mean and σ
ithe variance for Gaussian law), or equivalently, by the natural parameter vector η
iin the terminology of the canonical exponential family [33]. The estimate of θ
i(or η
i) (performed using the maximum likelihood method for example) explicitly depends on the corresponding region, and this dependence must be taken into account when computing the derivatives of the functional. In [12], the expres- sions of the associated evolution equations were computed and detailed.
3.2 Non-parametric pdfs
Here we consider non-parametric density estimates using kernel-based meth- ods. Given a region Ω
i, we can estimate the pdf of the feature y through the use of the Parzen method [34]. Let K
h: χ → R
+be the Parzen window in general chosen as a Gaussian kernel of 0-mean and variance h (scale parameter):
ˆ
q(y, Ω
i) = 1
| Ω
i| Z
Ω
K
h(y(x) − y) dx,
where y(x) is the value of the feature of interest at the point x of Ω
i.
Such non parametric pdfs may be helpful for tracking non homogeneous regions (an example is given in section 6.3 for perfusion MRI) but also for the statistical description of more complex features such as the coefficients of linear texture transforms (e.g. wavelet, waveatoms). Some results are given in [8, 25].
4 Segmentation with a shape prior
Medical structures are often really complex and sometimes not very con- trasted. The addition of a shape prior in the energy functional can then be crucial for many applications. Different approaches were considered that can be categorized as follows : minimization of a boundary-based functional [15];
shape prior through a signed distance function [13]; scale and translation invari- ant shape moments, e.g. Legendre as in [16].
In [17], we propose to combine statistical data term with a shape prior com-
puted using its Legendre Moments. Indeed, moments [35] give a region-based
compact representation of shapes through the projection of their characteris- tic functions on an orthogonal basis such as Legendre polynomials. Scale and translation invariance can be advantageously added as in [16]. When combining statistical region-based terms and shape prior term, the criterion becomes:
E(Ω
in, Ω
out, Γ) = E
in( x, Ω
in) + E
out( x, Ω
out) + αD(Ω
in, Ω
ref) + βE
b(Γ) , (6) where D(Ω
in, Ω
ref) is defined as the Euclidian distance between the moments of the evolving region and the ones of the reference shape:
D(Ω
in, Ω
ref) = kλ(Ω
in) − λ(Ω
ref)k
22, (7) where λ(Ω
i) are the moments of the region Ω
i.
To drive this functional towards its minimum, the obtained evolution equa- tion is iteratively run without the shape prior, then the shape prior term is updated, and the active contour evolves again by running the PDE with the shape prior. This procedure is repeated until convergence. An example of the segmentation of the left ventricle in echocardiography is given in section 6.1.
5 Resolution schemes for active contour segmentation
The statistical criteria introduced above are generic and allow to well de- scribe numerous medical image segmentation problems. However they imply the estimation of the optimal domain using an appropriate resolution method.
This is the key part in variational approaches and we here propose to focus on two main efficient methods to solve the previous optimization problems.
5.1 Methods based on the evolution of a deformable model
These methods are based on the computation of an evolution equation that
will drive an active shape towards the boundary of the object to segment. Such
a deformation is directed by a geometric partial differential equation (PDE). Let
us denote by Γ(s, τ ) the active contour where s is a parameter of the contour
and τ is an evolution parameter. The contour evolves according to the following
well known PDE:
∂Γ(s,τ)∂τ= FN where N is the normal vector and the velocity F must be deduced from each considered criterion.
In order to perform the computation of this PDE, several methods have been introduced. The first one is the well-known approach proposed in [2], which consists in transforming the initial criterion by introducing the level set function U and computing the associated Euler-Lagrange equations. The second one has been proposed by [36] and takes advantage of shape derivative tools introduced in [26]. Let us suppose that the shape derivative of the criterion E(Ω) in the direction V may be written as follows:
< E
′(Ω), V >= − Z
∂Ω
speed(x, Ω)(V(x) · N(x))da(x) . (8) If we want to minimize E, we can choose F = speed(x, Ω). On the contrary, to maximize the criterion, we take the opposite sign for the velocity.
Some other approaches are also worth mentioning. For example, some au- thors propose to directly take benefit of B-splines in the level set formulation [37] allowing efficient resolution schemes, or to implement fast Newton-type minimization schemes based on second order derivatives in [38].
5.2 Convexification method
This approach was proposed [27] for the classical non-convex variational segmentation problem:
min
Ω1E = Z
Ω1
(I( x ) − c
1)
2dx + Z
Ω\Ω1
(I( x ) − c
2)
2+ λ Z
∂Ω1
g( x )da ( x ) , where c
iis the mean value of the intensity within the corresponding region, and g(·) is some scalar function on the boundary.
The key idea now is (i) to reformulate this functional by introducing the characteristic function u of Ω
1; (ii) to use the co-area formula to re-express the boundary integral in terms of the TV semi-norm of u; and (iii) to relax u to live in [0, 1]. Altogether, this yields the non-smooth but convex functional:
0≤u≤1
min E
GM= Z
Ω
u(x)(y(x)−c
1)
2+(1−u(x))(y(x)−c
2)
2+λg(x)|∇u(x)|
dx .
(a) (b) (c)
Figure 1: Segmentation of a cardiac MRI slice obtained using the convexification approach [27] (a) and active contours [2] (c) (using the initial contour given in (b)).
In [39], the authors show that, for fixed c
1and c
2, if u
∗is a global minimizer of E
GM, then for almost every α ∈ [0, 1], Ω
C(α) := { x ∈ Ω : u
∗( x ) > α} is a global minimizer of E. The energy E
GMcan be solved using efficient non- smooth convex optimization tools as proposed in [27].
In Fig. 1, we give a comparison between active contours and the convexifica- tion approach from [27] for a MRI slice of the 2009 MICCAI challenge database [40] . Using active contours, and given that the energy E is non-convex, the final contour Fig. 1.(c) depends on the choice of the initial contour of Fig. 1.(b), which is only a stationary point (not necessarily a minimizer even local). Using the convexification approach, the obtained u
∗is indeed a global minimizer, but must be thresholded using the parameter α. One of the advantage of convex optimization tools is the low computational cost of its resolution schemes. The introduction of more complex descriptors is also possible and an interesting ap- plication was proposed in [18] for cardiac segmentation with a prior shape based on Legendre moments.
6 Medical image segmentation examples
In this section we present some results on medical image segmentation using
the criteria introduced in section 2. We propose different examples using both
parametric and non parametric pdfs for different applications: echocardiogra-
phy, brain MRI and perfusion MRI.
6.1 Noise and shape prior for cardiac echography segmentation
We propose to illustrate the interest of using shape prior on real echocardio- graphic images. We use a parametric Rayleigh model for the noise distribution with an anti-log-likelihood homogeneity criterion, and a shape prior term with invariant Legendre moments and an alternating minimization scheme. In order to deal with the shape variability of the heart on images, we choose α = 0.7 and β = 0.4 on eq. (6). Those parameters give the best results on quantitative and qualitative evaluation of the segmentation as shown in [25].
We conduct our experiment on both parietal and apical views and on con- trast and classical echocardiography. The results on apical view for classical echography are depicted in Fig. 2. The importance of adding a shape prior in echocardiographic data is here demonstrated. In addition to this comparison, we compare also this method to another well-known segmentation method, the Active Appearance and Motion Models (AAMM) proposed by Bosch [41]. In Fig. 2 (d), we present the segmentation result of the echographic image using AAMM. We also compute the mean distance between AAMM and expert seg- mentation (cross points on Fig. 2 (e)), and the mean distance between active contours and expert segmentation (circle points on Fig. 2 (e)). As depicted on Fig. 2 (e), active contours are closer to expert segmentation than AAMM. In addition, we compute Dice coefficient for both methods, on AAMM it is 0.74 and on active contour it is 0.83. Further quantitative evaluation results may be found in [25].
6.2 Parametric pdfs for brain MRI segmentation
Here we give an example on simulated brain T1-weighted MRI images pro- vided by the Montreal Neurological Institute Brain Web URL. We perform the segmentation of the WM on a brain MRI image with a noise of 7% (Brain Web offers noise levels ranging from 1 to 9%). For such images the problem of segmentation is particularly critical for both diagnosis and treatment purposes.
It becomes necessary to obtain a robust segmentation of the different tissues
(WM (White Matter), GM (Gray Matter), or Cerebrospinal Fluid (CSF)) or
(a) (b) (c)
0 2 4 6 8 10 12 14
2 4 6 8 10 12 14 16 18
our approach AAMM
(d) (e)
Figure 2: Segmentation of the left ventricle in echocardiography using a Rayleigh model of the noise without any shape prior (a) and with a shape prior (b). The expert contour is given in (c) and the segmentation result of AAMM method [41] is given in (d). The graphic (e) gives the mean distance computed on 14 images between the AAMM and expert (cross points) and between the shape and noise method and expert (circle points).
structures. The noise model in MRI images is assumed to be represented by a Rician distribution. However, for large signal intensities the noise distribution can be considered as a Gaussian distribution (this is the case for the WM or the GM). For the CSF which has a low signal intensity, the noise model can be approximated by a Rayleigh noise.
In Fig. 3, we first compare the Chan-Vese method [2] to the (anti-)log- likelihood with Gaussian noise [9] and divergence maximization using Kullback Leibler (KL) for the WM segmentation. We can note that the KL criterion acts as an extractor of the most prominent Gaussian in the initial mixture of Gaussians. The two other criteria separate the mixture without extracting a single Gaussian. In order to extract single Gaussian pdfs using the Chan-Vese method, a multiphase procedure can be advantageously used (see [42]). From a quantitative point of view, our method gives a dice coefficient of 0.91, a False Positive Fraction (FPF) of 0.8% and a True Positive Fraction (TPF) of 84%
(computed on the whole volume). We can remark that our method gives a
(a) Chan & Vese (b) log likelihood (c) KL maximization
Figure 3: T1-weighted brain MRI segmentation results (extraction of the White Matter).
Column (a): final contour and pdfs for the Chan & Vese method [2], column (b): for the log-likelihood method [1], column (c): for the maximization of the KL divergence.
very small number of false positive voxels, at the price of a higher number of missing voxels. The last point is due to the regularization term which acts by minimizing the curve length. Some complementary results are depicted in [43].
6.3 Non parametric pdfs for perfusion MRI tracking
The perfusion MRI (p-MRI) has emerged as a primordial clinical investi- gation tool in the evaluation of cardiac diseases. Spatio-temporal tracking of myocardial dynamics during the first transit of the contrast bolus allows the identification of hypoperfused or ischemic regions. An automatic quantification tool relies on the accurate segmentation and tracking of the cardiac structures.
The main difficulty lies in the fact that the different regions (myocardium, left ventricle) are not homogeneous. We give here an example of tracking using non parametric pdfs. From an initial segmentation of the left ventricle, we track this structure along the sequence by minimizing the distance between the pdf of the intensity of the current region and the pdf of the intensity of the previous one.
In Figure 4, we show the evolution of the curve in one frame of the sequence
and the joint evolution of the pdf of the inside region. It is interesting to see
how the pdf converges towards the reference pdf as shown in Figure 4. This example shows that this tool can be used for segmentation tracking but some improvements are needed notably concerning the fact that the criterion needs both the inside reference pdf and an outside reference one. The outside refer- ence pdf can be for example computed on the narrowband as proposed in [44]
in order to be more representative.
(a) reference segmentation (b) initial contour (c) final contour
0 20 40 60 80 100 120 140 160 180
0 0.01 0.02 0.03 0.04 0.05 0.06
0 20 40 60 80 100 120 140 160 180
0 0.01 0.02 0.03 0.04 0.05 0.06
(d) initial pdfs (initial contour) (g) final pdfs (final contour)
Figure 4: Minimization of the distance between the current region pdf (blue or dashed line) and a reference one (red or solid line) which is computed from the segmentation of LV in the previous frame (the pdf is the pdf of the intensityIwithin the region).