Shape and Appearance Priors

The use of prior knowledge expressed as statistical modeling of shape and appearance variations has brought many benefits in all the key stages of deformable models.

Initialization The constrained initialization has greatly benefited from the generative property of SSM which instantiates “valid” shapes from few control parameters (Eq. (4.3) in Sec. 4.3.1.3).

This shape parameterization, coupled with geometrical constraints derived from the hip joint kinematics (Sec. 5.2.3.3), yielded an efficient constrained initialization which drastically reduced the search space of the global minimization (Sec. 5.2.3.5). As a result, the robustness of the initialization was improved which explains the good results of the GI in experiment 1 (Sec.

5.5.2.2).

Evolution To attenuate the limitations from small FOVs, the use of SSM in the deformable mod-els was of significant importance. By bringing robustness in the shape regularization process, SSM-based segmentation was found to outperform conventional deformable models. In fact, the deformable model based on weak shape priors expressed as shape memory (SMEM) constraints (Sec.5.3.3.2) performed very poorly with the cVB and TF images in experiment 1 (Sec. 5.5.2.2) as overall ASSDs of 1.59±0.54 and 3.09±1.29 were computed, respectively. Furthermore, while large

FOV may alleviate the negative effects of poor initializations, we are convinced that the use of SSM remains essential. In fact, we observed that the deformable model based on the shape mem-ory prior was incapable to cope with the mediocre atlas-based initialization (ASSD of 5.86±2.15) in experiment 3 (Sec. 5.5.4.2).

Nevertheless, special care was required when choosing the multi-resolution SSM scheme (Sec.

4.3.1.3) in order to yield optimal results. The SSS scheme reported the best results with the VB datasets, while RAA scheme performed better with cVB and TF images. More exactly, we only observed statistical differences between SSS and RAA schemes when segmenting the hip bones in the TF datasets as depicted in Fig. 5.35. We observed that with insufficient accuracy in the initialization, combined with the absence of sufficient image cues or a poor appearance model, the SSS-based evolution was impeded. The use of a rigid alignment with the coarsest resolution in the RAA scheme prevented shapes from “over-stretching” when trying to fit the shape with the image, as exemplified in Fig. 5.35(c) and (g). From our observation, femurs were not subject to this problem, likely due to better image features, such as the strong image gradients around the shaft (Fig. 5.20(e)), and also better initializations.

(a) (b) (c) (d) (e) (f) (g) (h)

Figure 5.35: Right hip bone comparative results on TF dataset. (a) coronal slice with (b) constrained global initialization, (c) SSS scheme-based and (d) RAA scheme-based fine segmentation results overlaid in white.

(e)-(h) Comparative results on a sagittal slice. The RAA scheme improved the initialization and appears as better than the SSS scheme which tended to over-stretch the hip bone as depicted in (c) and (g).

Regarding the use of appearance priors (Sec. 4.4) in our experiments, the coupling of multi-variate Gaussian (MGD) priors with the Mahalanobis distance did not significantly outperform the use of reference profiles with the normalized cross correlation (NCC). As we explained, the robustness of the NCC measure with respect to linear intensity changes was a real asset. In fact, the possible absence of these intensity variations in the training dataset would prevent the MGD model from capturing them, affecting thus its efficiency. In all cases, appearance priors were complementary to the gradient features, especially when edge information was missing or non-reliable (e.g., fuzzy boundaries between articulations). As a result, they were essential in the segmentation approach and were thus always exploited in conjunction with the gradient-based forces. The NCC-gradient-based approach was in particular preferred to MGD priors for its lower complexity, which yielded faster segmentation algorithms (see timing results in Table5.1of ex-periment 1).

SSM construction and enlargement The limitation of creating and using incomplete shapes based on the image FOV has been efficiently addressed in this thesis by devising a robust evo-lution and SSM construction approaches that were able to cope with corrupted shapes. This im-proved automation and thus made possible the enlargement of a unique SSM per shape, which remained efficient in the segmentation context. Indeed, our evaluation results of experiment 2 (Sec.5.5.3.1) demonstrated that the segmentation results from the images with small FOV could be efficiently re-used to improve the SSM performance in subsequent segmentations. In situa-tions where the Generality of the SSM was poor (e.g., our sub-SSM experiment with 12 shapes), our methodology to enrich the SSM with corrupted shapes was shown to significantly improve the segmentation performance. Standard SSM performance measures (e.g. Compactness and Specificity presented in Sec. 4.5.1.3) were not used to compare sub- with mixed- or full-SSMs, as there were different numbers of models in the individual SSMs. Moreover, these measures do not necessarily reflect the segmentation performance and were also criticized by some studies [MDW08].

MRF-based Local Modeling The modeling of local variations based on MRF (Sec. 5.3.3.4) was beneficial when the Generality property of the SSM was not satisfactory due to the training dataset being too small. In experiment 2 (Sec.5.5.3.2), a multi-resolution refinement based on the MRF evolution improved the overall ASSD error of the sub SSM (trained with only 12 shapes) to about 15% (i.e. from 1.59 to 1.35 mm). This improvement in accuracy was achieved at the expense of an increase of 22% in the computation time. The MRF was suitable to delineate fine details with the condition of being close to the optimal solution. In fact, the MRF modeling required to define the reference shape X (Sec. 4.3.3) to control the amplitude of the local deformationsδi. Our solution consisted in setting the reference shape as the shape state at the end of the iterations of each resolution level.

A difficulty that we faced with the MRF-based forces was the setting of the parameters. In [KH98], no rules of thumb were provided to select these parameters and we needed to perform several trials to find appropriate values of βand the number of iterations. If there was a way to statistically model the reference shape X, a promising solution would consist in estimating by statistical inference these unknown parameters based on e.g. an EM algorithm.

5.6.1.3 Coupled Registration-Segmentation

The presented coupled registration-segmentation approach evaluated in experiment 4 (Sec. 5.5.5) appeared to notably improve segmentation results while simultaneously co-registering multiple images. The use of this knowledge expressed by the coupling of registration and segmentation tasks had an impact on the speed of the segmentation approach.

Furthermore, some preliminary experiments showed that a coupled registration-segmentation has also a positive impact on shape correspondence. Indeed, we designed an experiment involv-ing femurs to be co-registered and segmented from 10 images. An affine mappinvolv-ing was used since all images came from different subjects and the blending parameter λ was set to a low

value of 0.05 to not penalize the segmentation (Sec. 5.3.4.2). We ran the experiment in coupled (i.e. with the coupled registered-segmentation approach) and single (i.e. 10 independent seg-mentations) modes. We also applied the correspondence refinement based on landmark sliding (Sec. 4.3.1.2) on the results of these two modes for comparison purposes. We compared the SSM correspondence metrics (Sec. 4.5.1.3) with the single and coupled modes. Examples of the Compactness and Generality curves are shown in Fig. 5.36(a)and 5.36(b), respectively. These curves clearly show that (i) the coupled approach yielded much better performance metrics than a single approach and that (ii) the correspondence refinement based on landmark sliding could not improve the results of the coupled mode since those were already excellent.

single single + ldm coupled + ldm coupled

(a) Compactness

single single + ldm coupled + ldm coupled

(b) Generality

Figure 5.36:Impact of coupled registration-segmentation on shape correspondence illustrated with femurs.

Curves indicate (a)Compactness and(b) Generality measures. Axis denotes the number of modes and

“ldm” refers to a correspondence optimized by landmark sliding.

These results show a very promising usage of the coupled registration-segmentation approach.

Intuitively, these observations make sense since the coupled registration-segmentation approach enforces point correspondence across shapes via its registration and blending stages. However its similar performance with respect to the correspondence optimization using the landmark sliding is striking in this preliminary experiment. The coupled approach may hence be a good alternative (or at least a complement) to the correspondence refinement by landmark sliding.

However, despite the accuracy of the coupled approach (ASSD of 1.58±0.18) was better than in the single mode (ASSD of 1.65±0.36), like it was previously observed in experiment 4, this accuracy was not very satisfactory when compared to the other experiment results. This was due to the fact that the affine mapping could not cope with non-rigid inter-subject variations.

A promising solution would be hence to use an extension of the Procrustes alignment to non-rigid transforms (e.g., [CP10]). In any case, these are preliminary results and a more thorough evaluation is necessary.