Appearance Models

As presented in Sec. 2.5.1.2, two main kinds of features are available: region and boundary features. One of the most popular appearance models assumes that features follow a Gaussian distribution. Mean µ and covariance matrixΣX are thus estimated from the training features X. The goodness of fit of a feature u to this appearance model is then simply achieved by calculating the Mahalanobis distancedM: dM(u) = (u−µ)^⊺Σ⁻¹_X (u−µ). By building a PCA from the features, the Mahalanobis distance can be instead expressed as a function of the eigenvalues and the eigenvectors matrix [CHTH93]. Furthermore, by keeping the first k modes, a more robust appearance model to noise is devised. This appearance model is part of the Active Shape Models (ASM) of Cooteset al.[CT04]. Cooteset al.[CHTH93] and Behielset al.[BMVS02] used intensity and (normalized) gradient boundary profiles to build PCA models, and exploited the Mahalanobis distance. These two studies drew different conclusions with respect to the best choice of profile (intensity vs. normalized gradient), suggesting that choices strongly depended on the type of image modalities (video vs. X-rays) and structures (face vs. bone).

When region features are used, the most popular appearance model is Active Appearance Mod-els (AAM) [CET98b,CT04]. The AAM appearance model also uses PCA modeling to express the modes of variations of the shape interior appearance, usually denoted as texture. To decouple the appearance from the shape and to be able to apply PCA, textures need first to be transformed into a reference shape (commonly the mean shape of the shape PCA model) by using a warping approach. Various warping strategies exist, such as mesh-based and TPS [CT04,Rog01] or piece-wise affine [Ste02,Ste04] warpings. Linear [CT04] and non-linear [BML⁺02] texture intensity nor-malizations are usually required to cope with strong intensity variations among training images.

Also, alternative intensity features (e.g., edges, corners, and gradients [SCT03]) less sensitive to these intensity changes can be included into the appearance model. Similarly, multiple views of a structure can be considered in the AAM construction [OLU⁺03] to improve the appearance model robustness. Region-based models with AAM were successfully exploited in the literature [OLU⁺03,MLvdG⁺00,LvdGM⁺02,ÜFS⁺03,SFK⁺08], but AAMs have their shortcomings which may demand appropriate evolution strategies (e.g., Robust AAM [BBLS05]).

Similarly to shape models, non Gaussianity of region features can be addressed by building probabilistic atlases of appearance [Koi06,SCD⁺03]. The choice of the reference image can affect the results as bias is introduced [RFS03]. Various authors proposed to carefully select it based on distance and similarity criteria [PBHM05], or to remove it by achieving the simultaneous registration of all training images [JDJG04]. More details on atlas-based construction can be found in Sec. 2.5.4.4.

In case of boundary features, non-parametric approaches are available to build non-normally distributed appearance models. In particular, the k-Nearest Neighbor (k-NN) classifier [Cov68]

was used in a segmentation context [dBGVN03,BGNV03,HMMW07]. To achieve this, sets of boundary S and non-boundary ^⌝S training profiles are computed during the training phase to define the two classes border and^⌝border, respectively. During the segmentation, a profile u of dimension d belongs to a boundary if its probability P(border∣u)is over a threshold. The k closest “neighbor” profiles in^⌝S ∪ Stouare computed based on a profile distance, typically the Euclidean distance in d−dimension. The moderate k-NN probability [AK02] is then expressed as P(border∣u) = (K+1)/(k+2), where K is the number of neighbor profiles that belong to S.

Since all training profiles need to be constantly explored to compute the knearest neighbors, efficient (approximate)k-NN computation methods were proposed [War96,AMN⁺98,AASK08].

The use ofk-NN classifier is theoretically applicable with any type of appearance features. For instance, Van Ginnekenet al.[VGFS⁺02] replaced intensity profiles by 2D square grids centered at the landmark points. As reported by Heimann and Meinzer [HM09], any classifier returning a probability can be in general used and combined for segmentation purpose, as exemplified in [LZJ04, LI05, NSC⁺05]. Fusion of multiple classifiers is a topic which has been extensively studied in pattern recognition. We refer interested readers to [LS97,ACK01,KA03].

To improve appearance models and to minimize the effects of the curse of the dimensionality, clustering approaches can be adopted. In clustering approaches, appearance features are clus-tered into classes from which statistics are inferred, via e.g. PCA. This yields various advantages such as the increase of training samples number for each landmark (good for statistical infer-ence), the reduction of memory consumption and computational cost since during segmentation extracted profiles need only to be compared against the clusters. Brejl and Sonka [BS00] were the first to propose the clustering of appearance feature vectors for appearance models. They used the fuzzy C-means algorithm [BE⁺84] to estimateKclusters from all landmarks in all train-ing images. Similarly, Stoughet al.[SBPC07b,SBPC07a] also used the fuzzy C-means to cluster appearance features based on intensity quantile histograms. In [PEW05], features computed as the dot product between surface normals and image gradients were clustered and used in heart segmentation from CT cardiac images. In [HMMW07], the accuracy of the k-NN classifier was improved by using the k-Means clustering algorithm [HW79]. Chung and Delingette [CD09]

later on proposed an EM-based algorithm, initialized with fuzzy C-means, to achieve cluster-ing and classification of intensity profiles. In particular, their approach proposed an automatic selection of the best number Kof clusters. Moreover, a spatial smoothing was applied because appearance-based approaches did not consider neighborhood information, which might lead to a

non-spatially smooth distribution of the intensity profiles into the different clusters. The process is illustrated in Fig. 2.13.

(a) (b) (c) (d)

Figure 2.13:Intensity profiles clustering. (a)From a MRI segmented femur mesh,(b)intensity profiles are extracted and an(b)EM-based algorithm is used to achieve profiles clustering by optimally choosing the best number of clusters. (c)Finally a last smoothing processing is applied to account for neighborhood information. Images courtesy of François Chung.

Similarly, Ho and Gerig [HG04] considered a multi-scale spatial clustering to hierarchically gather profiles based on neighborhood information. Finally, various hybrid appearance models can be found in the literature. For instance, Yushkevichet al.[YZG05] exploited the continuous medial representation to combine appearance ASM- and AAM-like appearance models.

2.5.4 Correspondence

In the previous Sections it was assumed that the features were aligned and in correspondence.

Since some correspondence approaches are based on concepts of PCA, it appears as more nat-ural to detail now the various correspondence approaches. We present various correspondence approaches that are suitable for appearance correspondence (e.g., image-based correspondence Sec.2.5.4.4) with accent on shape correspondence method based on landmarks. Moreover, when shape correspondence is achieved, appearance correspondence might be directly available.

Approaches that exploit the correspondence of features across training samples require accurate and meaningful correspondence. Otherwise the resulting model will capture the errors intro-duced by the (absence of the) correspondence procedure. So far, in the literature no consensus has been reached on the choice of the best correspondence approach. Usually, the application context drives users’ choices. Human annotation is often regarded as a gold standard (like manual segmentation) to define correspondence but it is unpopular. This is mainly due to the time-consuming and tedious aspect of the approach (especially in 3D) [HM09] and the ubiqui-tous inter-users variability which affects the reproducibility of the results. Most correspondence methods are based on the registration of the features across the various shapes. Depending on the features’ nature, different correspondence strategies are more appropriate than others. Based on the recent review of Heimann and Meinzer [HM09], each correspondence category will be presented.