Parallel Double Snakes. Application to the segmentation of retinal layers in 2D-OCT for pathological subjects

HAL Id: hal-02559220

https://hal.archives-ouvertes.fr/hal-02559220

Submitted on 30 Jun 2020

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

Parallel Double Snakes. Application to the segmentation

of retinal layers in 2D-OCT for pathological subjects

Florence Rossant, Isabelle Bloch, Itebeddine Ghorbel, Michel Paques

To cite this version:

Florence Rossant, Isabelle Bloch, Itebeddine Ghorbel, Michel Paques. Parallel Double Snakes.

Appli-cation to the segmentation of retinal layers in 2D-OCT for pathological subjects. Pattern Recognition,

Elsevier, 2015, pp.3857-3860. �10.1016/j.patcog.2015.06.009�. �hal-02559220�

Parallel Double Snakes. Application to the segmentation of retinal

layers in 2D-OCT for pathological subjects

Florence Rossant

a,n

, Isabelle Bloch

b

, Itebeddine Ghorbel

a,b,c,1

, Michel Paques

d

a_{Institut Supérieur d'Electronique de Paris (ISEP), Paris, France} b_{Institut Mines-Telecom, Telecom ParisTech, CNRS LTCI, Paris, France} c

FOVEA Pharmaceuticals, Paris, France

d

Clinical Investigation Center 1423, Centre Hospitalier National des Quinze-Vingts, Paris, France

a r t i c l e i n f o

Keywords:

Parametric active contours Double snakes

Approximate parallelism Optical coherence tomography Retinal layers

a b s t r a c t

In order to segment elongated structures, we propose a new approach for integrating an approximate parallelism constraint in deformable models. The proposed Parallel Double Snakes evolve simulta-neously two contours, in order to minimize an energy functional which attracts these contours towards high image gradients and enforces the approximate parallelism between them by controlling their distance to a centerline under regularity constraints of this line. The proposed approach is applied on retina images, for segmenting retinal layers in optical coherence tomography images of pathological subjects (and it applies to healthy subjects as well). Results are evaluated by comparing with manual segmentations for three retinal layers, and provide a similarity index above 0.87, sensitivity between 0.85 and 0.93, and specificity between 0.84 and 0.94. These results are within the range of intra and inter-expert variability. Moreover, quantitative studies demonstrate that, in our application, our Parallel Double Snake (PDS) model outperforms other parametric active contour algorithms integrating parallelism information.

1. Introduction

The use of a priori information is an interesting way to overcome usual difficulties inherent to the segmentation task and to increase both accuracy and robustness in segmentation methods, by restrict-ing the space of solutions. In the case of the segmentation of linear structures, a strong assumption can be made regarding the local parallelism of their boundaries. For example, such situations occur for the detection of roads or rivers in aerial images, or for the delineation of blood vessels in medical imaging. Therefore, several segmentation approaches integrating a parallelism constraint have been developed, such as statistical[1]or fuzzy[2,3]models, Kalmanfilters[4] and parametric active contours[5–8].

In this paper, we address the problem of the segmentation of retinal layers imaged with Optical Coherence Tomography (OCT), in the foveal area. All these structures are characterized by the approximate parallelism of their boundaries. We propose an original segmentation method, based on parametric active contour models, with a new energy term modeling this approximate parallelism, as an extension of a preliminary work in[9].

Active contours were originally introduced by Kass et al.[10]in order to detect the boundary of an object in an image. A curve V(s), parametrized by the curvilinear abscissa s, evolves dynamically, from its initial position towards the edges of the object to be extracted, under the action of two main types of forces: internal forces that control the regularity of the curve and image forces that express the attraction towards contours. The curve evolution derives from the minimization of an energy functional, which is, in the most general case, composed of three terms:

EðVÞ ¼ EIntðVÞþEImageðVÞþEExtðVÞ ð1Þ

The image energy (EImage) attracts the curve towards the features to be detected, such as edges or lines. The internal energy (EInt) imposes regularity properties. Eqs. (2) and (3) give common examples of such energy terms, as proposed in[10]:

EImageðVÞ ¼ Z 1 0 PImageðVðsÞÞ ds ¼  Z 1 0 ∇IðVðsÞÞ  2 ds; ð2Þ EIntðVÞ ¼ 1 2 Z

α

ðsÞ ∂V ðs; tÞ ∂s   2þ

β

ðsÞ ∂2Vðs; tÞ ∂s2   2ds; ð3Þ where∇I denotes the image gradient.

The external energy (EExt) can express additional contextual information. Usual limitations of this model are its sensitivity to the initialization and the difficulty to converge towards the correct Contents lists available atScienceDirect

journal homepage:www.elsevier.com/locate/pr

Pattern Recognition

n_{Corresponding author.}

E-mail address:Florence.Rossant@isep.fr(F. Rossant).

1

solution in the presence of high levels of noise. Thus, the addition of contextual or shape information can help to gain in robustness and accuracy. In the following review, we focus on the modeling and integration of a parallelism constraint.

Parallelism has been modeled in several works[5–8]. The Twin Snakes algorithm[5]is an extension of the traditional snake, designed for the detection of two parallel contours simultaneously, in high-resolution images. Traditional snakes cannot detect accurately close edges, and the curve often jumps from one to the other. To overcome this problem, a mutual attraction force (spring force) is introduced, which is expressed at point s as EextðVðsÞÞ ¼

δ

ðsÞðdðsÞd0Þ2, where

δ

ðsÞ

is a weighting factor, d(s) the actual distance between the point s and its twin partner and d0the desired distance. This latter parameter has to be initialized beforehand, which constitutes a strong limitation of this model. Moreover, this formulation assumes that the distance is constant all along the structure (strict parallelism), which is generally not the case in real images.

The Ribbon Snakes [6] are modeled by a centerline VðsÞ ¼ ðxðsÞ; yðsÞÞ associated with a half width w(s), defining the left and right borders of the ribbon, VL(s) and VR(s). It has been initially designed for automatic road extraction from aerial images. The energy functional follows the traditional formulation but applies to vðsÞ ¼ ðxðsÞ_{; vðsÞ; wðsÞÞ. Moreover, the authors proposed} to modify the image energy as:

EImageðVÞ ¼

Z1

0 ∇IðVLðsÞÞ∇IðVR

ðsÞÞ

ð Þ  nðsÞ ds; ð4Þ where n(s) is the normal to the centerline, oriented from VLto VR for bright structures. The internal energy is defined as in Eq.(3). No additional term is added with respect to the standard model, so, no specific control of the parallelism constraint is possible.

The Lateral Coupled Snakes [8] are designed for searching for worms in 2D images and segmenting them. This model is composed of two contours VLand VRwhose points are paired in a one to one correspondence. The line linking the two points of each pair is supposed to be normal to both curves VL and VR. To ensure the parallelism between VLand VR, a shear energy Esis introduced, and expressed as EsðVLðsÞÞ ¼n

c_{ nð}c_nÞn

J nc_{n J} and EsðVRðsÞÞ ¼ EsðVLðsÞÞ, where n

is the normal unit vector and nc_{the unit vector pointing from V} Lto VR. This model requires a strict parallelism between both curves. In case the two sides are not perfectly parallel, the authors propose to use the mean normal direction instead of the unit normal vector, but this suggestion has not been implemented in the experiments.

Finally, the Ribbon of Twins method [7]takes back the ideas developed in the Twin Snakes [5]and Sandwich Snake models

[11]. It consists of four curves which evolve simultaneously: two of them inside the contour, the two others outside. The two outside contours are connected by pull forces to the corresponding inside contours (similarly to the sandwich model), while the inside contours are connected by push forces to each other (twin snakes). This model is robust to high levels of noise and can locate boundaries under difficult conditions such as lack of gradient or occlusion. However, it requires a precise initialization, the prior knowledge of the ribbon width, and it is computationally more expensive than other models.

Some approaches have also been proposed within the frame-work of implicit deformable models. For instance in [12], the segmentation of the cortex in magnetic resonance images of the brain is addressed using two coupled level sets. The coupling is embedded in the evolution functions of the level sets, so as to keep the surfaces within the normal range of distances. In [13], two level sets are coupled to force the two contours to have a distance close to a given value d (the distance between the epicardium and the endocardium in this work, for the segmentation of the left ventricle). Another application to heart imaging was proposed in

[14], for the segmentation of the heart in real-time 3D

echocardiography, where again a level set approach is adopted. Region information is modeled using statistical distributions, and the coupling is implicit in the joint probability model. Additionally, an incompressibility constraint is added, which can be interpreted as another coupling, but over time, between successive frames of the sequence.

In the domain of the segmentation of OCT images of the retina, several algorithms including parallelism information have been also proposed. Yazdanpanah [15] presented an active contour model, derived from the Chan and Vese energy functional[16], where the interfaces are approximated by circles. The parallelism results from this shape constraint, which is neither suitable for the analysis of the foveal area of the retina nor for less constrained applications. Finally, extending the prior work published in[17,18], Chen et al.[19]proposed a method for the global and simulta-neous segmentation of surfaces, in 3D OCT images. A cost function includes surface terms, region terms, and interactions in the form of a penalty function depending on a threshold on a vertical distance, requiring many parameters to be set.

All these methods make a strong assumption about the usual distance (or usual volume) between the evolving surfaces, which is assumed to be constant or, at least, evolves within a range of admissible values. In this paper, we consider only parametric deform-able models, with the objective to avoid to have pre-fixed threshold values, while controlling the strength of the parallelism constraint.

In the context of clinical studies in ophthalmology, we need extracting thin, elongated structures that can be visualized in several types of retina images, such as retinal layers in OCT images. These structures are difficult to segment since they present varying thicknesses and are generally weakly contrasted. In addition, the parallelism between the main sides is more or less strict, requiring to model and parameterize explicitly this kind of information. Moreover, in some cases, diseases can affect the width or the grey levels of the searched structures, increasing dramatically their variability. The limitations of the existing models, as described above, make them difficult to use for solving these issues.

In this paper, we propose a new parametric active contour model, that allows the simultaneous evolution of two curves, under a parallelism constraint. The advantages of our method are as follows:

(i) The distance between both contours has not to be known accurately as prerequisite. On the contrary, it is adjusted during the evolution process and can vary along the boundaries. (ii) The parallelism constraint can be strengthened or relaxed

locally by changing a weighting parameter.

(iii) All these interesting features are obtained without increasing the computational cost of the algorithm. Consequently, this model is well adapted to a large range of applications, including the ones mentioned above.

The novel Parallel Double Snake (PDS) method is presented in

Section 2, and discussed in Section 3. Its application for the

segmentation of retinal layers is presented in Section 4, along with quantitative evaluation and comparison with other methods integrating a parallelism constraint.

2. Parallel Double Snake model (PDS) 2.1. The model

The proposed Parallel Double Snake model is dedicated to the automatic extraction of two almost approximate parallel curves. It consists of a centerline, VðsÞ ¼ ðxðsÞ; yðsÞÞ, parametrized by s and a local half-thickness b(s). It implicitly defines two approximate parallel

curves, V1and V2, representing the boundaries of the structure. Let us denote by nðsÞ ¼ ðnxðsÞ; nyðsÞÞt the normal vector to the curve V(s)

(Fig. 1), with nxðsÞ ¼  y 0_ðsÞ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi x0_ðsÞ2_{þ y}0_ðsÞ2 p and nyðsÞ ¼ x 0_ðsÞ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi x0_ðsÞ2_{þ y}0_ðsÞ2 p . Then, points of the curves V1and V2are given by

V1ðsÞ ¼ VðsÞþbðsÞnðsÞ; V2ðsÞ ¼ VðsÞbðsÞnðsÞ: ð5Þ

Our aim is tofind the centerline V(s) and the associated half-thickness b(s), so that the curves V1 and V2match the searched image contours, which are assumed to be approximately parallel. For this, we introduce a new energy functional, defined as EðV; V1; V2; bÞ ¼ EImageðV1ÞþEImageðV2ÞþEIntðVÞþRðV1; V2; bÞ ð6Þ

In this equation, EImage and EInt are defined as in the “classical” snakes (Eqs.(2) and (3)), in order to attract the curves V1and V2 towards the high image gradients while preserving the regularity of V. In Eq.(3), the weighting factors

α

ðsÞ and

_β

ðsÞ control the tension and rigidity of the centerline, respectively. Thus, EInt ensures the global regularity of the structure to be segmented while EImageallows for a precise delineation of its boundaries. The last term RðV1; V2; bÞ in Eq. (6) controls the smoothness of the

thickness variation and prevents it from sharp changes. It is thus de_{fined as a function of the derivative of b:}

RðV1; V2; bÞ ¼ 1 2 Z 1 0

φ

ðsÞðb0_ðsÞÞ2_{ds ð7Þ}

For strictly parallel curves, b(s) is constant and the related energy is zero. The more general formulation where b depends on s allows for thickness changes along the structures, which is required in several applications (seeSection 4).

φ

ðsÞ is a weighting parameter which may vary depending on s, to locally strengthen or relax the parallelism constraint.

Although the formalism used to define the curves is identical to the one proposed in[6], the definition of the energy functional is very different. First, the image energy terms, EImageðV1Þ and

EImageðV2Þ, are not explicitly related to the normal n(s) to the

centerline. On the contrary, the classical formulation is kept, which makes the model simpler and more general (especially, no assumption is made on the relative intensity of the structure to be segmented). Moreover, the coupled term RðV1; V2; bÞ allows

evolving and controlling the half-thickness independently, accord-ing to both image and regularization forces. Theses features make the Parallel Double Snakes model very general and adaptable, as discussed inSections 3 and 4.4.2.

In the following, we assume that the two weighting para-meters,

α

ðsÞ and

_β

ðsÞ, are set to constant values. Let us denote by PTot the potential so that EðV; V1; V2; bÞ ¼

R1

0PTotðs; V; V1; V2; bÞ ds.

The minima of EðV; V1; V2; bÞ are found by solving two coupled

Euler–Lagrange equations. The first one expresses the minimiza-tion of EðV; V1; V2; bÞ with respect to the centerline V(s): ∂PTot

∂V dsd∂P∂VTot0þd 2

ds2∂P_∂VTot″ ¼ 0, or



_α

V″ðsÞþ

_β

V⁗ðsÞþ∇PImageðV1ð ÞsÞþ∇PImageðV2ð ÞsÞ ¼ 0 ð8Þ

Let us denote by Fðx; yÞ ¼ ∇PImageðx; yÞ the image force at (x,y). Eq.

(8)is very similar to the traditional formulation. The external force applied on each point of the centerline is defined as the sum of the forces applied on the corresponding border points V1ðsÞ and V2ðsÞ.

Thus, the curve moves globally towards the searched position. The second Euler–Lagrange equation expresses the minimiza-tion of EðV; V1; V2; bÞ with respect to the half-thickness b(s): ∂PTot

∂b dsd∂P_∂bTot0 ¼ 0, which leads to

nðsÞ; FðV2ðsÞÞFðV1ðsÞÞ   

_φ

ðsÞb″ðsÞþ

_φ

0_ðsÞb0 ðsÞ h i ¼ 0 ð9Þ Consequently, the local distance between V1and V2is driven by the difference of the image forces applied respectively on V1and V2and projected on the normal to the centerline V. Let us take a simple example as illustration. Here we assume that

φ

ðsÞ is constant and the centerline is at the correct position, but that b(s) is locally too high or too low. The local forces FðV2ðsÞÞ and

FðV1ðsÞÞ point globally towards opposite directions (Fig. 2), and the

projection of their difference on the normal is negative or positive. Thus, b(s) increases or decreases so that V1 and V2move toward the correct position. The last term in Eq.(9), related to b0ðsÞ and b″ðsÞ, imposes the curves V1 and V2 to be regular. The chosen weighting parameter controls the strength of the parallelism constraint. A very high value leads to a strict parallelism with b constant along the curves, while a very small value allows high variations of b(s). Hence, our coupled energy term allows adjusting locally the half-thickness under forces derived from image infor-mation, while imposing a parallelism constraint whose strength can be explicitly modulated according to contextual information.

To solve these equations, we classically introduce the time variable t:

γ

∂Vðs; tÞ_∂t 

_α

V″ðs; tÞþ

β

V⁗ðs; tÞF Vð 1ðs; tÞÞF Vð 2ðs; tÞÞ ¼ 0 ð10Þ

γ

b∂bðs; tÞ_∂t þ nðsÞ; F Vð 2ðs; tÞÞF Vð 1ðs; tÞÞ   

_φ

0_ðsÞb0 ðs; tÞ

φ

ðsÞb″ðs; tÞ ¼ 0 ð11Þ where

γ

and

γ

bare viscosity parameters. Eqs.(10) and (11)are then discretized by using classical numerical approximations of deriva-tives: central difference in space and backward difference in time. The minimization is realized by updating sequentially V(s), V1ðsÞ

and V2ðsÞ, b(s), V1ðsÞ and V2ðsÞ, and repeating these four steps until

convergence. The initialization of the curve V and the parameter setting are specific for each type of application. They will be detailed inSection 4.

2.2. Derived models

This general model can be modified in order to handle specific applications where the central curve is fixed and known in advance. In the case where there are still two moving curves V1

Fig. 1. Parametric representation of the Parallel Double Snakes.

Fig. 2. Variations of the half-thickness b(s) given by the second Euler–Lagrange Eq.

(9): (a) the obtained forces on each contour (green and blue arrows) and (b) the variation of b(s). (For interpretation of the references to color in thisfigure caption, the reader is referred to the web version of this paper.)

and V2evolving with respect to afixed central curve V, the energy functional reduces to EðV1; V2; bÞ ¼ EImageðV1ÞþEImageðV2ÞþRðV1;

V2; bÞ, and this energy is minimized only with respect to b,

according to Eq.(9).

Another interesting model is the one where only one curve V1 evolves with respect to a fixed one V. This time, the energy functional is defined as EðV1; bÞ ¼ EImageðV1ÞþRðV; V1; bÞ, and the

evolution of b, and consequently V1, is given by nðsÞ ; FðV1ðsÞÞþ

φ

ðsÞb″ðsÞþ

_φ

0_ðsÞb0

ðsÞ

h i

¼ 0.

The double snake model could be easily extended to the simultaneous segmentation of any number of approximately parallel curves. In our application, instead of such a global approach, we prefer a sequential method, where retinal layers that are easier to segment are processedfirst, while more difficult ones are handled later in the sequence. This allows us to benefit from previous steps to have a good initialization for the difficult layers, and thus compensates for the noise, lack of contrast or low signal to noise ratio.

3. Discussion

In this section, we propose to compare the performances of our Parallel Double Snake (PDS) model with those of other parametric active contour models: the classical model of Kass and the main other approaches including a parallelism constraint, already intro-duced and discussed in Section 1: the Twin Snakes[5], the Ribbon of Twins (ROT) [7] and the Ribbon Snakes [6]. We have re-implemented the algorithms according to the information pro-vided by the authors, except the publicly available standard model. The segmentation of the Hyper Reflective Complex in OCT images of the retina (Fig. 9) is an interesting case study since it consists in the detection of two almost parallel curves, with a slowly varying distance between them (from 6 to 22 pixels). Moreover, although the interfaces are relatively well contrasted, high gradients in the choroid may disturb the movement of the curve towards the outer interface (Fig. 9), justifying the introduction of structural information.

Our evaluation is about the following criteria: the convergence and the accuracy of the segmentation results, the sensitivity to initialization, the number of parameters to set and the sensitivity to parametrization.

Table 1 summarizes the energy functionals and the specific

parameters of each approach. All weighting parameters, in our experiments, are fixed to a constant value over the entire curve. The ones that are common to all algorithms are the weighting parameters of the internal energy,

α

and

β

. The image forces are derived from the Gradient Vector Flow[20], calculated withfixed parameters (Gaussian filter with

σ

¼2, regularization parameter

μ

¼0.1, 5 iterations), except in the Ribbon Snakes since image forces derive from a specific image potential in this case. Note that the image forces are always weighted by 1. Also, we fixed the viscosity parameter

γ

¼10 in the evolution equations ([10, Eq.

(10)], which, in this study case, ensures a correct convergence

speed without any oscillations around the local minimum reached by the algorithm.

3.1. Comparison with the classical active contour model

Wefirst studied the behavior of the standard snake according to the parameter settings, given two different initializations, one mostly inside the structure and the other one outside (Fig. 3). We found no parametrization ð

α

;

β

Þ leading to a correct segmentation for both initializations. Moreover, the outer border (in blue) of the HRC is never correctly detected all along it, when the lower curve is

initialized out of the structure, inside the choroid (second column

in Fig. 3). On the opposite, the Parallel Double Snakes converge

perfectly towards the interfaces, for a large range of initializations and parametrizations, as it will be shown inSection 3.3.

In order to illustrate the contribution of the parallelism energy term, we applied the standard active contour and our method with the same initialization and the same common parameters (Fig. 3) (

α

¼10,

β

¼0,

γ

¼10). Here, a maximal number of iterations nbiter¼ 500 ensures convergence. The results inFig. 3show that

in the case of an initialization inside the searched area (a), the standard active contour algorithm is unable to find the right contour. This under-segmentation is clearly visible near the foveal area (indicated by a red line). Our method overcomes this problem thanks to the new energy term modeling the approximate paral-lelism. Furthermore, with an initialization outside the HRC area (b), our algorithm manages to avoid local extrema of the gradient and converges towards the right position. In the case of standard active contours, we obtain an over-segmentation of the HRC area (see in particular the periphery of the HRC).

These examples illustrate how the parallelism information embedded in the PDS model enables the contours to avoid being trapped in local maxima, leading to the robustness and the accuracy mentioned above.

3.2. Comparison with other parametric models with parallelism information

Keeping

α

¼10,

β

¼0 and

γ

¼10, we applied then the Twin Snakes

[5]with different values for d0, ranging from 6 pixels, the minimal thickness of the HRC at the periphery, to 22 pixels, the maximal thickness at the foveola. The initialization outside the HRC was used

(Fig. 3(b)). Three values were tested for the weighting of the

parallelism information. No correct segmentation was obtained, whatever the parameter settings. Fig. 4illustrates the behavior of the model. For

δ

¼0.5 (Fig. 4(a)), the parallelism constraint is not strong enough and the lower curve (in blue) is trapped by the local high gradients in the choroid. Increasing the parallelism weight, with

δ

¼1, is not a solution, as illustrated inFig. 4(c) and (d), since it is not possible to set a value for d0that is adequate all along the HRC: the segmentation is correct only on the part of the HRC whose thickness is close to d0. The value

δ

¼2 imposes a strict parallelism between both curves, which is obviously not suitable (Fig. 4(b)). These experiments show that the Twin Snakes model is valuable for the extraction of curves featured by a strong parallelism, and whose distance is accurately known beforehand. It is not adapted to the extraction of elongated features with variable thickness.

The Ribbon of Twins (ROT) model provides a solution to this limitation of the Twin Snakes, by adding two other curves so that each searched interface is sandwiched between two curves. The main drawbacks of this approach are its additional complexity (4 curves instead of 2) and the difficulty to automatically initialize the curves in a real application context. We started from the initialization illustrated in Fig. 5(a), with two curves inside the HRC and 2 curves outside, as required. We performed exhaustive parametrization tests with d0Af6; 7; …; 21; 22g and

δ

Af0:5; 1; 2g,

as previously. For

δ

¼ 0:5 the algorithm does not converge (the curves of the sandwich model do not meet fully) or the segmenta-tion results are as bad as the ones obtained with the Twin Snakes alone. For

δ

¼1, convergence is achieved for d0Z12, but the

segmentation is inaccurate at the right side of the HRC (Fig. 5

(b)).

δ

¼2 imposes again too strong a parallelism, whatever the choice for d0. Extending the parametrization proposed in [7], we defined then two different weighting values for the twin model ð

_δ

twin¼ 1Þ and the sandwich model ð

δ

sandÞ, so as to accelerate the

movement of the sandwich curves toward each other. Good segmentation results were then obtained for

δ

twin¼ 1,

δ

sand¼ 3

and d0¼ 8. However, they were slightly less accurate at the right

hand extremity of the HRC (Fig. 5(c)), compared to the Parallel Double Snake result (Fig. 3(b)).

Finally the Ribbon Snakes were also evaluated, taking back the initialization ofFig. 3(b). The main feature of this model is that the half thickness parameter is constrained by the same“tension” and “rigidity” forces as the two coordinate components. Thus, no additional weighting parameter has to be set. However, the algorithm does not converge to the solution (Fig. 6(a)), whatever the parameter setting (

α

,

β

,

γ

), probably because the central curve and the half thickness have different properties and cannot be processed in the same way in a general case. Without changing the de_{finition of the energy functional, we set a different viscosity} parameter

γ

wa

γ

in the evolution equation of the half thickness

w(s). We found the best results for

γ

w¼ 1 (Fig. 6(b)). A similar

result is obtained starting from an initialization inside the HRC. 3.3. Parametrization

The previous experiments show that the ROT model, the Ribbon Snakes and the PDS model can all segment the HRC, with a slightly more accurate result however for the Parallel Double Snakes. Moreover, the last two models can converge towards the solution from a large range of initializations, contrary to the Ribbon of Twins which requires an accurate initialization. We now study the robustness of the algorithms with respect to parametrization, which is important in our application, given the variability of the structures to be segmented. In the following

Table 1

Energy functional and specific parameters of the studied active contour models. The notation ðÞn_{refers to a specific parametrization used in our experiments, as a variant of}

the one proposed by the authors.

Method Model Specific parameters

Standard[10] v ¼ ðx; yÞ

EðvÞ ¼ EIntðvÞþEImageðvÞ

EIntðvÞ ¼ R1 0α v0ðsÞ 2 þβ v″_ðsÞ2 ds; EImageðVÞ ¼ R1 0PðvðsÞÞ ds Twin snakes[5] vi¼ ðxi; yiÞ, i ¼ f1; 2g δ, d0

ETðviÞ ¼ EIntðviÞþEImageðviÞþ

R1

0δ v1ðsÞv2ðsÞd0

 2

ds

Ribbon of twins (ROT)[7] vi¼ ðxi; yiÞ, iAf2; 1; 1; 2g ðδtwin¼ δsand¼ δ; d0Þ or δðtwin; δsand; d0Þ

internal curves inside the structure: iAf1; 1g external curves outside the structure: iAf2; 2g

ERðvi ¼  1; þ 1Þ ¼ EIntðviÞþEImageðviÞþEtwinwith Etwin¼

R1

0δtwinv 1ðsÞv1ðsÞd0

 2

ds ERðvi ¼7 2Þ ¼ EIntðviÞþEImageðviÞþEsandwith Esand¼

R1

0δsandv7 1ðsÞv7 2ðsÞ2ds

Ribbon snakes[6] v ¼ ðx; y; wÞ ðγwÞn: viscosity parameter applied

specifically on w EðvÞ ¼ EIntðvÞþEImageðvÞ with EImageðvÞ ¼

R1

07ð∇Iðv2ðsÞÞ∇Iðv1ðsÞÞÞ  nðsÞ ds

(sign depending on the relative mean intensity of the structure with respect to the background) where v1ðsÞ and v2ðsÞ are defined by vc¼ ðx; yÞ and w: v1ðsÞ ¼ vcðsÞþwðsÞnðsÞ, v2ðsÞ ¼ vcðsÞwðsÞnðsÞ

PDS v ¼ ðx; yÞ, b φ, γb

Eðv; bÞ ¼ EIntðvÞþEImageðv1ÞþEImageðv2ÞþRðv1; v2; bÞ

RðV1; V2; bÞ ¼

R1 0φðb

0_ðsÞÞ2_ds

where v1ðsÞ ¼ vðsÞþbðsÞnðsÞ and v2ðsÞ ¼ vðsÞbðsÞnðsÞ

Fig. 3. Sensitivity analysis of the classical active contours and our model with respect to the initialization, inside the HRC (a) or mostly outside (b). The proposed method provides stable results. (Figures are best seen in color on the web version of the paper.)

experiments, we kept

α

¼10,

β

¼0,

γ

¼10, unless otherwise stated, and made the specific parameters vary.

The ROT model requires 3 additional parameters,

δ

twin,

δ

sand and d0. Starting from the setting that visually led to the best result (

δ

twin¼ 1,

δ

sand¼ 3, d0¼ 8), we found that the segmentation is

stable for ð

δ

sand; d0ÞA½2; 4  ½8; 10.

For the Ribbon Snakes model, the specific parameter is the viscosity

γ

wrelated to the half thickness w(s). It has to be adjusted with respect to the viscosity

γ

applied to the coordinates, as stated before. We found that the model converges correctly for

γ

Z5 and

γ

wrγ5.

Finally, we studied the behavior of the Parallel Double Snakes with respect to the parametrization of the parallel constraint,

φ

,

γ

b. We varied the parameter

φ

(but still constant along the structure) while keeping the other parameter valuesfixed. The results are illustrated inFig. 7. With a large value of

φ

(here

φ

¼700) we avoid under-segmentation or over-segmentation that are observed with a low value (

φ

¼100). Setting

φ

to a higher value will make the energy term related to parallelism very high as compared to the rest of the energy terms, thus creating a strict parallelism (see the example with

φ

¼5000). We calculated the distance between the manual segmentation and the automatic segmentation according to the parameter

φ

. Our experiments showed that the results are very stable around the illustrated value

φ

¼700, in the interval between 400 and 1200, which is a remarkable property (see

Fig. 8). Moreover, this result is also stable for a large range of

γ

b

values chosen in an interval including

γ

. For example, for

γ

¼10, every value

γ

bZ2 leads to a very good segmentation.

4. Segmentation of retinal layers in OCT images acquired on pathological subjects

Optical coherence tomography (OCT) is a non-invasive imaging technology, based on the principle of the Michelson interferom-eter, which allows the observation of cross sections of biological tissues. It has revolutionized in the last two decades the practice of ophthalmology, since it enables clinicians to visualize the retinal layers with a high resolution. A typical retinal section, obtained by the OCT imaging, is displayed inFig. 9. The retina is composed of superimposed neuronal layers that transform the light to nerve impulses, transmitted to the brain via the optic nerve. Considering the spherical geometry of the eye, the layers close to the vitreous humor are generally called inner layers, whereas the others, towards the choroids, are called outer layers. In OCT images, the retinal layers appear as strips of different grey-levels due to the variability of their backscattering properties. The image resolution depends on the technology used. In our experiments, we pro-cessed images acquired with the FD-OCT of Heidelberg [21], having the following characteristics: horizontal resolution: 6

μ

m =pixel; vertical resolution: 4

μ

m=pixel, image size: 496  1537 pixels, and section width: 9 mm.

Fig. 4. Results with the twin snakes, for different parameter settings. (a)δ¼0.5, d0¼ 12. (b) δ¼2, d0¼ 12. (c) δ¼1, d0¼ 8. (d) δ¼1, d0¼ 20. (Figures are best seen in color on

the web version of the paper.)

Fig. 5. Results with the ROT model, for different parameter settings. (a) Initialization. (b)δtwin¼ δsand¼ 1, d0¼ 12. (c) δtwin¼ 1, δsand¼ 3, d0¼ 8.

In our previous work[4], we developed an automatic algorithm for the segmentation of OCT images acquired on healthy subjects, in the foveal and perifoveal area of the retina. This previous approach

was very accurate and robust given the image quality and the normal anatomical variability. However it could not be directly applied on images of patients affected by retinitis pigmentosa (RP), for several

Fig. 7. Comparison of the results obtained with the Parallel Double Snakes, from two initializations, and for different values ofφ and γb.

Fig. 8. (a) Evolution of the distance (in pixels) between the manual and the automatic segmentation according to the parameterφ; (b) zoom in the interval 0rφr2000.

reasons linked both to morphological changes and contrast loss. Indeed, this pathology tends to destroy the photoreceptors from the periphery towards the center of the fovea, leading to a fusion of the inner and outer retinal layers. The extent of the fusion is correlated to the disease progress (red circles in Fig. 10). As a consequence, the inner nuclear layer (INL) is no more well defined, especially at its extremities where it tends to join the inner limit of the hyper-reflective complex (HRC)[22]. We can also observe a thinning of the OPL layer and of the HRC at the periphery. For the most advanced stages of the RP, a thickening of the HRC occurs as well. Besides, from an image quality point of view, all these morphological changes result in a loss of contrast, especially on the outer border of the INL layer, as underlined previously. Moreover, the contrast is also low on the outer side of the HRC layer and a classical snake is disturbed by the artefacts inside the choroid (Fig. 10).

Thus, the adaptation of the segmentation approach to these pathological cases is not only a question of parameter tuning: new prior information has to be modeled and integrated, in order to compensate for these additional difficulties. The most interesting knowledge we have at our disposal is the approximate parallelism of the interfaces. This kind of structural information remains stable and relevant, whatever the progress stage of the disease. Unfortu-nately the Kalmanfilter, applied on healthy subjects[4], fails in RP cases, where the destruction of photoreceptors induces a loss of contrast and parallelism between the inner and outer retinal layers. In addition, this method exploits only local information of the image and thus lacks robustness. Therefore the proposed method, with approximate parallelism constraint, is appealing.

We now present the segmentation of eight layers in OCT retinal images for RP subjects. The globalflowchart is the same as in[4]

(Fig. 11). Here we detail the only segmentation steps that are

substantially new with respect to the method in[4], in particular involving the Parallel Double Snakes model.

The overall methodology is as follows:

(i) Image denoising, based on an anisotropicfilter[23], in order to reduce the image noise while preserving the edges; (ii) Segmentation of the most contrasted layers: Inner Limiting

Membrane (ILM) [4] and hyper-re_{flective complex (HRC)}

(Section 4.1). The retinal area is thus delimited and regions

of interest within this area can then be further analyzed; (iii) Segmentation of the photoreceptors, around the inner HRC

interface (Section 4.2);

(iv) Alignment with respect to the outer HRC interface[4]; (v) Segmentation of the inner layer interfaces, RNFL[4], ONL, INL

(Section 4.3). The regions of interest are each time deduced from

the previous segmentation results. As the interfaces may have a very low contrast, region-based approaches were preferred (k-means[24,25]with a Markovian regularization[26,27], under a Gaussian noise model assumption). The model parameters were set experimentally on some sample images. Then the same values were used for all images of the database.

The coordinates are defined by the origin at the top-left corner, the vertical x-axis and the horizontal y-axis.

4.1. Detection of the hyper-reflective complex (HRC)

A line inside the HRC isfirst determined[4]so as to roughly locate the median position of the HRC. The initialization of the inner side of the HRC is based on two classifications (k-means with Markovian regularization[24–27]), with respectively 3 (Fig. 12(a)) and 4 (Fig. 12(b)) classes. Indeed the case k¼ 3 provides generally a good segmentation of the HRC around the foveola, while the case k¼ 4 is more consistent in periphery. The inner boundary of the HRC is estimated from the regions of highest intensity crossed by the median line. Wefirst extract candidate points from the two images (in dark blue inFig. 12(a) and (b)) and apply the following fusion rule at every y-coordinate: at the center, both classifications are considered and the candidate point with minimal x-coordinate is retained, otherwise, the candidate point given by the classi fica-tion with k ¼4 is kept. The standard snake algorithm[10,20] is then applied in order to re_{fine this first result. We denote by l}inthe inner HRC border so obtained. As the HRC is well contrasted on its inner side, especially on the area of clinical interest, which is 5 mm wide and centered on the foveola, thefinal result is reliable and accurate. So, it will serve as a reference for the initialization of the

Fig. 10. Illustration of the various stages of the retinitis pigmentosa evolution: (a) early stage (at the beginning); (b) intermediate stage and (c) advanced stage. The regions indicated by red circles illustrate the limits between atrophic and non-atrophic retina. (For interpretation of the references to color in thisfigure caption, the reader is referred to the web version of this paper.)

Fig. 11. Flowchart of the proposed segmentation method. The main steps involving the Parallel Double Snakes model are the HRC, photoreceptor and inner layer segmentations.

HRC outer border. We refine the estimation of the foveola position, ðxF; yFÞ, by minimizing the distance between the ILM and lin.

The outer HRC boundary is much noisier, excepted near the foveola. Therefore, the initialization is performed from the estimation of the HRC thickness ðb0ðyFÞÞ at the foveola (using the classification

image with k¼3), the knowledge of the inner border lin(y) and the gradient magnitude image Igðx; yÞ. The local HRC thickness b0ðyÞ is

estimated from the center to the periphery by piecewise maximizing the mean gradient magnitude on a sliding window (Fig. 12(c)). We initialize the central curve of the Parallel Double Snake, V, by translating linby b0=2. The outer border of the HRC is thus accurately

located (Fig. 12(d)). The method takes advantage of the good contrast along the inner side to determine the outer side which is much noisier. A very good robustness with respect to initialization inaccu-racy has been observed on the entire database (see also the experi-mental results inSections 3.3 and 4.4).

4.2. Segmentation of the photoreceptor segments (IS, OS)

The segmentation method consists of the following two steps: detecting the photoreceptors and segmenting the ONL/IS and OS/ RPE interfaces.

Knowing that the thickness of the HRC layer increases from the image boundaries towards the center, we are able to detect two points Ph1and Ph2of thickness variation in this region (Fig. 13). The point Ph1 corresponds to thefirst detected point, from left to right, at which the HRC thickness increases by more than 10%, and similarly, from right to left, for the point Ph2. We approximate the OS/RPE interface in the center of the foveola region by connecting the two points Ph1and Ph2. From an anatomical point of view, the region located between these two points is the one where the photoreceptors are not yet destroyed. This is therefore the clinically relevant region for further analysis. The clivus is defined by the two highest points of the ILM found on both sides of the foveola ((xCl, yCl) and (xCr, yCr),Fig. 13). The inner interface of the RPEþ ChCap (OS/RPE) is then estimated by extrapolating between the points Ph1and Ph2. The next step consists in detecting the ONL/IS and OS/RPE boundaries between Ph1and Ph2points by applying the Parallel Double Snakes. We initialize the central curve, V(s), by the inner boundary of the HRC layer, and b(s) by the distance between this boundary and the approximation of the OS/RPE interface

(Fig. 14(a)). This method provides simultaneously good estimations of

both ONL/IS and OS/RPE interfaces. They are then refined separately by

applying the classical snake (ONL/IS) and the PDS algorithm (OS/RPE), the two side-curves V1and V2being this time respectively the OS/RPE and the HRC outer interface (Fig. 14(b)).

4.3. Inner nuclear layer segmentation (INL)

In order to locate the inner border of the INL layer (GCLþ IPL/INL), we apply a classification followed by a Markovian regularization, with K¼ 3. The detection of the outer side of the INL layer (INL/OPL), which is less contrasted, is performed in the same way, on a more restricted area deduced from the inner border, with K¼2. Both interfaces are then re_{fined by applying the Parallel Double Snakes.}Fig. 15shows the initialization (a) and thefinal interfaces (b).

Final segmentation examples are presented in Fig. 16. The results obtained on the database confirm that the proposed approach detects precisely the various interfaces in the different stages of the RP evolution. 4.4. Experiments and results

In this section we detail the measures performed to assess the accuracy of our segmentation approach and the contribution of our PDS model compared to other parametric snakes [10,5–7]. The database includes 95 images, selected by the medical doctors, acquired from 30 patients suffering from RP. 10% of the images show other abnormalities, such as cystic edema, thick fovea with incom-plete cleavage of the inner layers over the fovea, drusen-like deposits, epiretinal membrane, increasing further the image variability and the complexity of the segmentation task. For each eye, the medical experts have selected one or two 2D images (horizontal and/or vertical slices), according to their clinical relevance. This study was conducted in accordance with the French ethics regulation and all subjects gave informed consent to participate. The segmentation method was applied to all the images, and allowed determining 97.6% of all layer interfaces in the foveal area (5 mm wide around the foveola), which is the region of medical interest. These results were visually validated by an expert, whose experience in thefield of OCT image interpretation exceeds 12 years.

The evaluation protocol is composed of three steps, as in[4]: (1) direct evaluation of the precision of the interface locations by

calculating the mean square error (MSE) between two differ-ent segmdiffer-entations;

Fig. 12. HRC localization for RP subjects: (a) and (b) regularized classification results with respectively k¼3 and k¼4, with, in dark blue, the estimated inner HRC boundary; the vertical blue lines indicate the central and the peripheral zones involved in the fusion rule; (c) initialization of the Parallel Double Snakes (V1¼ linin red and V2¼ linþb0

in blue); (e)final HRC outer boundary segmentation, obtained with the PDS method. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.)

(2) direct evaluation of regions (i.e. retinal layers) by calculating similarity (S), specificity (SPEC) and sensitivity (SENS) indices between two different segmentations;

(3) evaluation of derived measures of thickness and horizontal distances of the main retinal layers, to provide data for physicians in order to study the morphology of the RP pathology.

To quantitatively evaluate our method, we compared the automatic segmentations to manual segmentations. Three physi-cians segmented fifteen images acquired on RP subjects. Five of them were segmented twice by every physician at two different

times, the procedure being separated by several weeks in order to eliminate the memory effect between both segmentations. The images were selected from the entire image database in order to ensure a good representativeness in terms of image quality, noise levels and disease progress.

4.4.1. Evaluation of the interface detection and localization For each interface we calculated the mean square error (MSE) between two different segmentations. Let us denote by Segði;jÞðyÞ the abscissa of the interface i at the ordinate y for the segmenta-tion performed by the physician j. The MSE measured between the

Fig. 13. Detection of Ph1and Ph2on the aligned image for a RP subject, and clivus definition.

Fig. 14. Localization of the OS/RPEþ ChCap and ONL/IS interfaces: (a) initialization of the Parallel Double Snakes; (b)final result obtained for RP subjects.

Fig. 15. Segmentation of the INL layer: (a) initialization of the Parallel Double Snakes, from the classification results; (b) final interfaces.

segmentations provided by the two physicians Phys1 and Phys2, expressed in pixels, is calculated by

MSEðSegði;Phys1Þ; Segði;Phys2ÞÞ ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi PyR

y ¼ yL minym

d2ððSegði;Phys1ÞðyÞ; yÞ; ðSegði;Phys2Þðy

mÞ; ymÞÞ yRyLþ1 v u u t _ð12Þ

where d is the Euclidean distance. In order to estimate the intra-physician variability, we calculated the mean square error between manual segmentations of the same image, made by the same physician. This estimation helps us to choose the physician who will be our reference for the comparison. As shown inTable 2, the physician Phys1has the lowest intra-physician variability. Based on these results, we chose this physician, who is also the most experienced one, as a reference because he performed the most stable segmentations. The maximum value of the mean square error is 1.53 pixels and its standard deviation does not exceed 1 pixel. Note that for other physicians, the mean square error in some cases exceeds 2 pixels.

After choosing our reference physician PhysRef, we calculated the inter-physician variability and the MSE between the automatic segmentation and the segmentation performed by the reference physician. The results are summarized inTable 3. This table shows a significant variability between physicians especially for the INL/ OPL interface and the outer HRC border. This is mainly due to their poor definition and the presence of noise. The mean square error between the automatic segmentation and the reference is lower than the inter-physician and intra-physician variability for three interfaces out of six, and similar for the others.

4.4.2. Comparison with other algorithms

These experiments aim at comparing quantitatively the perfor-mances of the PDS model against the standard snakes[10] and three other models integrating parallelism information: Twin Snakes[5], Ribbon Snakes[6], Ribbon of Twins (ROT)[7]. In order to do that, we replaced the PDS algorithm by each of these algorithms, wherever it is applied in the whole segmentation program. The parameters of the introduced snakes were set experimentally and independently, so as to optimize the number of correct segmentations in each case. The other parts of the method remain unchanged, notably the automatic initialization steps. We also developed a graphical interface that displays the

segmented images one by one and enabled the most experienced doctor to evaluate every interface and annotate it as correct or uncorrect (considering still the foveal area). Note that the doctor did not know which of thefive approaches had been used for the segmentation of the presented image. Table 4 summarizes the evaluation results thus obtained on the whole database for the hyper-re_{flective complex (HRC) and the Inner Nuclear Layer (INL).} These experiments show that the classical model cannot delineate the external interface of the HRC, while the introduction of parallel information improves noticeably the performances. The PDS algorithm leads to the best results with almost 96% of correct segmentations. The rates vary from 74% to 79% for the other models. For the Twin Snakes and ROT algorithms, the errors occur especially outside the inter-clivus area, which results from the compromise that has to be made on the parameter settings: increasing the weight of the parallelism constraint improves the results outside the inter-clivus area but damages the performances inside. The typical distance between the two curves is also a parameter that is difficult to set, since this distance may vary considerably, up to a factor 3. For the Ribbon Snakes model, the regularity properties imposed on the centerline through the weighting parameters (

α

,

β

) have positive effects on the segmen-tation outside the inter-clivus zone but do not allow detecting properly the external interface of the HRC at the center. Indeed, the local thickness may vary abruptly in the inter-clivus zone, especially for advanced stages of the RP pathology. This illustrates the interest of defining a distinct coupled energy term in the global energy functional. These global results confirm the observa-tions made inSection 3.

The segmentation of the INL layer is even more problematic, given its strongly varying thickness (almost 0 at the foveola) and the image noise. That is the reason why the Twin Snakes algorithm is not adapted at all to this particular case. The ROT model leads to better results thanks to the“sandwich” energy term. However, it suffers strongly from initialization difficulties, as the pre-segmentation is generally not accurate enough to set properly the four curves. The Ribbon Snakes are not appropriate either even if the results are slightly better. While the central line shows high regularity proper-ties, the half thickness, on the contrary, varies greatly, from 0

μ

m at the foveola to around 100

μ

m (25 pixels) at the clivus. As underlined above, the ribbon parametrization cannot manage both these contra-dictory structural features with the same parameter settings (

α

,

β

). The PDS model outperforms the other algorithms, with more than 87% of correct segmentations. However, experiments show that the performances are sensitive to the quality of the initialization since the INL layer is very noisy and often poorly contrasted. This explains the 13% of unsatisfactory cases.

Finally MSE values have been calculated for the five models, and are reported inTable 5. They show that our PDS model leads to the best accuracy.

4.4.3. Evaluation of region segmentation

We calculated the similarity S, sensitivity SENS and specificity SPEC indices for the INL, IS þOS and HRC regions (the ones relying on parallelism information for their segmentation). In the

Table 2

Intra-physician mean square error (MSE) for RP subjects, estimated from 5 images. Retinal interfaces Phys1=Phys1 Phys2=Phys2 Phys3=Phys3

GCL þ IPL/INL 1:5370:14 1:7570:13 1:6370:35 INL/OPL 1:2570:22 2:2070:48 1:8070:28 ONL/IS þOS 1:4070:29 1:6270:56 1:5170:23 IS þ OS/RPE 1:4170:93 1:2670:45 1:4070:46 Inner HRC 1:3470:22 1:7370:29 1:9270:56 Outer HRC 1:2370:15 2:1270:31 1:6170:28 Table 3

Inter-physicians and automatic/the most experienced physician mean square error (MSE) for the RP subjects, estimated from 15 images.

Retinal interfaces Mean inter-physicians Our method=PhysRef

GCL þ IPL/INL 1:6570:40 1:3070:33 INL/OPL 1:7170:51 2:1270:46 ONL/IS þOS 1:3570:45 1:8170:42 IS þ OS/RPE 1:4470:46 2:0671:30 Inner HRC 1:3570:42 1:1570:36 Outer HRC 2:2370:80 1:1970:36 Table 4

Percentages of correct segmentation of the HRC and the INL layers in the foveal area (5 mm wide, centered on the foveola), for thefive studied methods, estimated on the whole database.

Retinal structures Standard Twins Ribbon ROT PDS HRC 18.9 77.9 73.7 78.9 95.8 INL 11.6 8.4 35.8 23.2 87.4

following equations, jRegði;jÞj represents the cardinality of the set of pixels of the region i segmented by physician j:

SðRegði;jÞ; Regði;Ref Þ_{Þ ¼}2jRegði;jÞ\ Regði;Ref Þj

jRegði;jÞ_jþjRegði;Ref Þ_j ð13Þ

SENSðRegði;jÞ; Regði;Ref Þ_{Þ ¼}jRegði;jÞ\ Regði;Ref Þj

jRegði;Ref Þ_j ð14Þ

SPECðRegði;jÞ; Regði;Ref Þ_{Þ ¼}jRegði;jÞ\ Regði;Ref Þj

jRegði;jÞ_j ð15Þ

The results are summarized inTable 6(a)–(c). They show that our segmentation approach enables us to get very close to the intra- and inter-physician variability. The similarity indices are between 0.87 (INL) and 0.92 (HRC). No significant under- or over-segmentation is noticed for any of the three layers.

The indices of sensitivity, specificity and similarity obtained with the other methods (standard snake, Twins snakes, ROT model, Ribbon model) are also reported in Table 6(a)–(c). They confirm the observations made in the MSE study. The PDS algorithm achieves the best compromise between sensibility and specificity, which results in the best similarity indices.

4.4.4. Evaluation of retinal thickness measures

In this section, we evaluate the measures derived from the segmentation. The physicians defined nine thickness and horizon-tal measures (Fig. 17): foveal thickness ðD1FÞ, total thickness at both

clivus (D1Cland D1Cr), foveal thickness of the ONL þOPLþ INL layers

ðD2FÞ, ONLþOPL thickness at distances 0.5 and 1.0 mm from the

foveola (D2_0:5l, D20:5r, D21l, D21r), and the horizontal IS/OS distance

between points Ph1and Ph2ðD3Ph1  Ph2Þ. The results are reported in

Table 7. They show a very good agreement between thickness

measurements performed manually and those obtained with our

automatic segmentation. The relative error is less than 5% and does not exceed the inter- and intra-physician variability for all of them. Other algorithms[10,5–7]do not achieve such a high level of reliability. Our automatic segmentation method provides also a correct estimation of the horizontal distance D3Ph1  Ph2, in

accor-dance with the physician values (223271250, 255471515, 260171250), unlike the other automatic approaches.

Overall, and considering the results of this evaluation, we can conclude that the proposed approach accurately segments the different layers, for a wide range of images, acquired from patients affected by even advanced stages of RP. No over-segmentation or under-segmentation is observed. It is worth noting that the method of Parallel Double Snakes allowed detecting the HRC, ISþOS and INL layers with a relatively high accuracy. All evalua-tion criteria show that the proposed method provides similar results as the ones obtained by the reference physician.

5. Conclusion

We proposed in this paper a new deformable model, called Parallel Double Snakes, dedicated to the segmentation of approxi-mately parallel contours. Several variants of this model make it suitable for the detection of two parallel contours with a tunable thickness constraint expressed with respect tofixed or deformable

Table 5

MSE calculated on the subset of 15 images segmented manually by the medical experts.

Retinal structures Interfaces PDS Standard Twins ROT Ribbon INL GCLþ IPL/INL 1.3070.33 3.3272.32 2.0270.64 2.2671.82 2.5170.47

INL/OPL 2.1270.46 3.7871.34 3.4470.82 4.6372.38 3.5470.70 IS þOS ONL/IS þOS 1.8170.42 1.8370.89 1.9570.60 3.8270.97 2.1070.61 ISþ OS/RPE 2.0671.30 3.8876.63 3.2774.62 3.9272.39 3.4471.36 Outer HRC 1.1970.36 2.8571.17 2.3870.72 1.9871.01 1.9470.85

Table 6

Sensitivity (a), Specificity (b) and Similarity (c) for the RP subjects.

Retinal structures Mean inter-physicians Mean intra-physicians PDS=PhysRef Standard=PhysRef Twins=PhysRef ROT=PhysRef Ribbon=PhysRef

(a)

INL 0:8870:07 0:8370:04 0.8570.04 0:7570:12 0:7470:06 0:6170:19 0:6970:06 ISþ OS 0:9170:05 0:8970:06 0:9370:09 0:8770:24 0:7470:22 0:9670:08 0:9170:09 HRC 0:8570:08 0:9170:03 0:9070:05 0:7870:15 0:8970:04 0:9270:04 0:9170:04

Mean inter  physicians mean intra  physicians PDS=PhysRef Standard=PhysRef Twins=PhysRef ROT=PhysRef Ribbon=PhysRef

(b)

INL 0:8870:04 0:8970:05 0:9070:04 0:7570:12 0:8370:07 0:8470:10 0:8470:05 ISþ OS 0:9070:05 0:9170:06 0:8470:07 0:7870:23 0:8770:21 0:6270:11 0:7770:10 HRC 0:8970:07 0:8670:04 0:9470:03 0:9470:04 0:8970:07 0:8970:07 0:8870:07

Mean inter  physicians mean intra  physicians PDS=PhysRef Standard=PhysRef Twins=PhysRef ROT=PhysRef Ribbon=PhysRef

(c)

INL 0:8870:03 0:8670:02 0:8770:03 0:7470:10 0:7870:05 0:6970:16 0:7570:04 ISþ OS 0:9170:02 0:9070:03 0:8870:06 0:8270:23 0:7970:20 0:7470:08 0:8270:08 HRC 0:8670:05 0:8870:02 0:9270:03 0:8470:10 0:8970:04 0:9070:04 0:8970:03

centerline, or of one contour with respect to anotherfixed one, again with a tunable parallelism constraint. This method proved to converge fast and accurately towards the actual contours, with better properties than existing methods using a parallelism constraint.

The interest of this model was demonstrated on an application in medical imaging for the segmentation of retinal layers in OCT images. The parallelism constraint proved to be useful in particular in pathological cases. The evaluation was performed by comparing the results with those obtained from manual segmentations made by medical experts, leading to a similarity index above 0.87, sensitivity between 0.85 and 0.93 and speci_{ficity between 0.84} and 0.94, for the three retinal layers involved in this study. Noticeably, the results are within the range of intra- and inter-expert variability.

Extensions to other types of pathologies are planned for future work, as well as further clinical investigations. It is worth noting that the Parallel Double Snakes have been used also for the estimation of vessel diameters in standard photos of the eye fundus [9], and that the model with one curve evolving with respect to a fixed one has been successfully applied for the segmentation of retinal arteries in Adaptive Optics images [28_– 31]. Other applications can be anticipated, in differentfields (e.g. road or river detection in aerial images, automated analysis of cell migration dynamics [32]), taking benefit of the variants of the proposed model.

Conflict of interest None declared.

Acknowledgements

The authors would like to thank the medical team of the Clinical Investigation Center of the Quinze-Vingts hospital for contributing to the manual segmentations. This work was partially funded by a CIFRE Grant from FOVEA and the French Ministry of Research (Contract CIFRE 897-2008).

References

[1] E. Villéger, G. Aubert, L. Blanc-Féraud, Image disocclusion using a probabilistic gradient orientation, in: International Conference on Pattern Recognition, Vol. 2, 2004, pp. 52–55.

[2] M.C. Vanegas, I. Bloch, H. Maître, J. Inglada, Approximate parallelism between fuzzy objects: some definitions, in: WILF, 2009, pp. 12–19.

[3]M.C. Vanegas, I. Bloch, J. Inglada, Alignment and parallelism for the description of high-resolution remote sensing images, IEEE Trans. Geosci. Remote Sens. 51 (6) (2013) 3542–3557.

[4]I. Ghorbel, F. Rossant, I. Bloch, S. Tick, M. Paques, Automated segmentation of macular layers in OCT images and quantitative evaluation of performances, Pattern Recognit. 44 (8) (2011) 1590–1603.

[5]M. Kerschner, Twin snakes for determining seam lines in orthoimage mosaick-ing, Int. Arch. Photogramm. Remote Sens. 33 (2000) 454–461.

[6]I. Laptev, H. Mayer, T. Lindeberg, W. Eckstein, C. Steger, A. Baumgartner, Automatic extraction of roads from aerial images based on scale space and snakes, Mach. Vis. Appl. 12 (2000) 23–31.

[7]B. Al-Diri, A. Hunter, D. Steel, An active contour model for segmenting and measuring retinal vessels, IEEE Trans. Med. Imaging 28 (9) (2009) 1488–1497.

[8] Q. Wang, O. Ronneberger, H. Burkhardt, Using lateral coupled snakes for modeling the contours of worms, in: Proceedings of the 31st DAGM Sympo-sium on Pattern Recognition, Springer, Jena, Germany, 2009, pp. 542–551. [9] I. Ghorbel, F. Rossant, I. Bloch, M. Paques, Modeling a parallelism constraint in

active contours. application to the segmentation of eye vessels and retinal layers, in: IEEE International Conference on Image Processing (ICIP), 2011, pp. 445–448.

[10]M. Kass, A. Witkin, D. Terzopoulos, Snakes: active contour models, Int. J. Comput. Vis. 1 (4) (1988) 321–331.

[11] F.A. Velasco, J.L. Marroquín, Robust parametric active contours: the sandwich snakes, Mach. Vis. Appl. 12 (2001) 238–242.

[12]X. Zeng, L.H. Staib, R.T. Schultz, J.S. Duncan, Segmentation and measurement of the cortex from 3-D MR images using coupled-surfaces propagation, IEEE Trans. Med. Imaging 18 (10) (1999) 927–937.

[13]M. Lynch, O. Ghita, P.F. Whelan, Left-ventricle myocardium segmentation using a coupled level-set with a priori knowledge, Comput. Med. Imaging Graph. 30 (4) (2006) 255–262.

[14]Y. Zhu, X. Papademetris, A.J. Sinusas, J.S. Duncan, A coupled deformable model for tracking myocardial borders from real-time echocardiography using an incompressibility constraint, Med. Image Anal. 14 (3) (2010) 429–448. [15] A. Yazdanpanah, G. Hamarneh, B.R. Smith, M.V. Sarunic, Intra-retinal layer

segmentation in optical coherence tomography using an active contour approach, in: Proceedings of the 12th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI '09, 2009, pp. 649–656.

[16]T.F. Chan, L.A. Vese, Active contours without edges, IEEE Trans. Image Process. 10 (2) (2001) 266–277.

[17] M.K. Garvin, M. Abrámoff, R. Kardon, S. Russell, W. Xiaodong, M. Sonka, Intraretinal layer segmentation of macular optical coherence tomography images using optimal 3-D graph search, IEEE Trans. Med. Imaging 27 (10) (2008) 1495–1505.

[18] M. Haeker, X. Wu, M. Abrámoff, R. Kardon, M. Sonka, Incorporation of regional information in optimal 3-d graph search with application for intraretinal layer segmentation of optical coherence tomography images, in: Interational Con-ference on Information Processing in Medical Imaging (IPMI), 2007, pp. 607– 618.

[19]X. Chen, M. Niemeijer, L. Zhang, K. Lee, M.D. Abrámoff, M. Sonka, Three-dimensional segmentation offluid-associated abnormalities in retinal OCT: probability constrained graph-search-graph-cut, IEEE Trans. Med. Imaging 31 (8) (2012) 1521–1531.

[20] C. Xu, J.L. Prince, Snakes, shapes, and gradient vectorflow, IEEE Trans. Image Process. 7 (3) (1998) 359–369.

[21] Heidelbergengineering, Spectralis HRAþ OCT, December 2012,〈http://www. heidelbergengineering.com/〉.

[22] D. Hartong, E. Berson, T. Dryja, Retinitis pigmentosa, The Lancet 368 (2006) 1795–1809.

[23] J. Weickert, B.H. Romeny, M. Viergever, Efficient and reliable schemes for nonlinear diffusionfiltering, IEEE Trans. Image Process. 7 (3) (1998) 398–410. [24] R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, Wiley, New

York, 1973.

[25] H. Maître, Image Processing, ISTE-Wiley, London, UK, 2008.

[26] S. Geman, D. Geman, Stochastic relaxation, Gibbs distribution, and the Bayesian restauration of images, IEEE Trans. Pattern Anal. Mach. Intell. 6 (1984) 721–741.

[27] S.Z. Li, Markov Random Field Modeling in Image Analysis, Springer, 2009. [28] N. Lermé, F. Rossant, I. Bloch, M. Paques, E. Koch, Segmentation of retinal

arteries in adaptive optics images, in: International Conference on Pattern Recognition (ICPR 2014), Stockholm, Sweden, 2014.

Table 7

Measures derived from the manual and automatic segmentations (inμm).

Measure Manual PDS Standard Twins ROT Ribbon D1F 230731 227733 230733 223734 231734 244737 D1Cl 302738 304738 297743 303735 305738 304736 D1Cr 311739 310740 305746 311737 312740 309738 D2F 137727 132728 134727 144726 117728 132728 D20:5l 94715 90713 78714 88716 78714 83715 D20:5r 95719 90715 80716 92718 78714 83717 D21l 72718 74716 7279 80712 85712 86713 D21r 73718 76716 78712 81714 88713 87716 D3Ph1  Ph2 255471515 25307734 217471067 28897518 293271712 18747560

[29]E. Koch, D. Rosenbaum, A. Brolly, J.-A. Sahel, P. Chaumet-Riffaud, X. Girerd, F. Rossant, M. Paques, Morphometric analysis of small arteries in the human retina using adaptive optics imaging: relationship with blood pressure and focal vascular changes, J. Hypertens. 32 (2014).

[30] N. Lermé, F. Rossant, I. Bloch, M. Paques, E. Koch, Coupled parallel snakes for segmenting healthy and pathological retinal arteries in adaptive optics images, in: International Conference on Image Analysis and Recognition (ICIAR 2014), Vilamoura, Algarve, Portugal, 2014.

[31] M. Paques, A. Brolly, J. Benesty, N. Lermé, E. Koch, F. Rossant, I. Bloch, J.-F. Girmens, Venous nicking without arteriovenous contact: the role of the arteriolar microenvironment in arteriovenous nickings, JAMA Ophtalmol. in press,http://dx.doi.org/10.1001/jamaophthalmol.2015.1132.

[32] I. Ghorbel, N. Bertacchi, X. Gidrol, V. Haguet, Parallelized contact imaging and automated analysis of cell migration dynamics, in: International Meeting of the German Society for Cell Biology (DGZ), Regensburg, Germany, 2014.

Florence Rossant received her engineering degree from ISEP (1992), her Ph.D. degree from ENST (Telecom ParisTech, 2006) and the Habilitation degree from the University Paris 5 (2012). She is currently a Professor at ISEP. Her domains of interest are image processing and analysis, in various applicationfields (document analysis, biometrics, medical imaging). Her main contributions concern the integration of a priori information in the segmentation and recognition algorithms, especially the modelling of structural knowledge and of high-level rules. She has been in charge of the SITe (Signal, Image and Telecommunications) research team since 2007.

Isabelle Bloch is graduated from the Ecole des Mines de Paris, Paris, France, in 1986, she received the Master's degree from the University Paris 12, Paris, in 1987, the Ph.D. degree from the Ecole Nationale Supérieure des Télécommunications (Telecom ParisTech), Paris, in 1990, and the Habilitation degree from the University Paris 5, Paris, in 1995. She is currently a Professor with the Signal and Image Processing Department, Telecom ParisTech, in charge of the Image Processing and Understanding Group. Her research interests include 3D image and object processing, computer vision, 3D and fuzzy mathematical morphology, information fusion, fuzzy set theory, structural, graph-based, and knowledge-based object recognition, spatial reasoning, and medical imaging.

Itebeddine Ghorbel received the engineering degree from Tunisian University in 2007 and the Science and Technology master's degree from the University Pierre et Marie Currie (Paris 6) in Paris in 2008. In 2012 he received the Ph.D. degree from Télécom Paris Tech. He then moved to Grenoble as a postdoctoral fellow at Orange Labs in the Data and health team where he worked on smart access to medical images from PACS. His area of research is signal and image processing, compression and transmission of medical images, development of algorithms and methods for segmentation of medical images.

Michel Paques MD, Ph.D., is an ophthalmologist specialized in vitreoretinal imaging and diseases. He was trained in Paris, and is currently professor of ophthalmology at the Quinze-VIngts hospital. He coordinates several research projects on high resolution retinal imaging, addressing to OCT or to adaptive optics imaging.