Knowledge-based Deformable models Segmentation

Knowledge-based deformable models being very efficient and robust approaches, they have quickly become a great success in musculoskeletal modeling. Bone modeling has particularly benefited from them since bone variations among individuals are less important compared to those of soft organs, which allow the creation of satisfactory training datasets.

Musculoskeletal modeling from digitized data can be also considered as image-based modeling.

Digitized data are commonly obtained by reconstructing a structure surface by means of optical capture technology. Typically, bone surfaces are partially reconstructed in intra-operative setups where only part of the bones is visible in the operative field. SSMs expressed as PDMs are used in an optimization process which varies the shape parameters until the generated instance fits the digitized data optimally [FLJ99,RST⁺07,ZSG10].

Generally speaking, SSMs are appropriate to segment structures which are partially visible. This is valid for X-rays imagery which is a widely utilized modality for its ability to be used intra-operatively. Since X-rays images suffer from their inherent 2D projections SSMs shape priors are very useful. The idea is similar to digitized data, i.e. shape and pose parameters are optimized to have an instance which best matches the imaged structure in the X-rays image. This can be seen as a 2D (X-rays image) to 3D (instance of the SSM) registration process. Different strategies are thus possible based on the various registration components (Sec. 2.3). Dong et al.[DGZ07]

registered a 3D femur SSM to a 2D contour extracted by Bayesian inference. Similarly, Lamecker et al.[LWH06] extracted pelvis contours from X-rays images (simulated from CT in their experi-ments) with a Canny filte [Can86], and matched them with optimized instances of a pelvis SSM.

Cresson et al.[CCB⁺09] segmented pathological spines from X-rays images by using 2D regis-tration features extracted from the images. Signed distance maps built from the extracted 2D contours were also used in [KNO⁺09] to express the similarity criterion. 3D to 3D registration is also possible, Heinze et al.[HMK⁺02] used an ICP-driven registration to optimize shape and pose parameters in segmenting knee bones from CT images.

In [SCT07,HJ08,SFK⁺08], the use of contour or features extraction was avoided by registering the X-rays image with a simulated one, called Digitally Reconstructed Radiograph (DRR). The simulated image was built from a statistical model similar to AAM which encompassed shape and appearance obtained from CT data. Given a point source and calibration parameters, the DRR creation process mimics the principle of X-rays image formation. In [DLvB⁺10], DRRs were not created because only an SSM was available. However, a projective image to be registered

with the X-rays was created from the SSM following the same principle of ray casting. One of the difficulties of using 2D-3D registration approaches with X-rays images is that in some cases the calibration parameters (intrinsic and extrinsic parameters) of the X-rays acquisition device are unknown. As a result, additional parameters might need to be optimized, which increases the search space dimension, the computation time and the risks of being stuck in local minima.

Alternatively, X-rays and other types of modalities can be segmented with evolving statistical deformable models. To segment femurs in X-rays, Behiels et al. [BMVS02] used active shape models. In [DLH07], statistical implicit deformable models (i.e., levelsets) yielded the extraction of hip bones and femurs from X-rays. These approaches were strongly inspired by the seminal work of Leventonet al.[LGF00] which illustrated the efficient embedding of levelsets into SSM to segment femurs and vertebrae from MRI and CT, respectively. ASM and AAM evolutions were coupled in [KMB⁺03] to efficiently segment carpal bones in X-rays images. Knee structures were successfully segmented from MRI images based on ASM [FCWO07,FBCO07], although specific acquisition protocols were necessary. Pelvic bones were also successfully segmented from CT images by Lamecker et al. [LSHD04], who combined an SSM of the pelvis with an intensity profiles-based evolution, as shown in Fig.3.12.

Figure 3.12:CT pelvis segmentation based on SSM [LSHD04]. Gray contours indicate the final segmentation results (black ones should not be considered here). Images used by permission.

Seimet al.[SKH⁺08] extended this approach by adding in particular an automatic initialization relying on the generalized Hough transform. In [KLZH09] and [YOT⁺09] hip joint kinematics were moreover included into SSMs (Fig. 3.13(a)). Yokotaet al.[YOT⁺09] additionally considered pathological conditions of the hip joint (osteoarthrosis caused by hip displasia) where bones appeared to be fused in the images. To do this, a separate SSM of pathological articular areas was built and combined with the pose-shape statistical model (Fig. 3.13(b)). In both works, such enriched SSMs yielded significant improvements in hip bones segmentation, in particular in the difficult articular area, where bones are in close proximity. Seiseet al.[SMRW06] also tackled the issue of knee bones proximity in X-ray by devising shape and appearance models to be used in a Bayesian framework.

(a) (b)

Figure 3.13:SSM of the hip considering shape, pose and pathology [YOT⁺09]. (a)First mode of variations of SSM which clearly expresses relative pose between the hip bone and the femur.(b)Fourth mode mainly represents pathological deformations of osteoarthrosis caused by hip dysplasia. Images used by permission.

3.5 Discussion

This Chapter has underpinned the great complexity of the human musculoskeletal system. Dif-ferent soft and bony structures interact together to ensure the stability, protection and locomotion of the human body. This complex machinery is studied in vivo with acquisition modalities that have their advantages and drawbacks. In particular, MRI is the chosen modality in our work due to its capacity to simultaneously image soft and bony structures in an non-invasive manner.

We explained that computer-based musculoskeletal modeling is addressed in different ways de-pending on the desired requirements. If complex models need to be created without the con-straint of being subject-specific, CG-based modeling is a good solution. In fact, generic models can be carefully built with interactive tools and subsequently resized or altered to create different models. Coupled with some simulation capabilities, these models improve the realism of virtual humans. Similarly, these models can be used for teaching purpose to a targeted audience (e.g., biomechanics or medical students) as long as these models provide the desired degree of realism and complexity.

When medical diagnosis or assistance is in stake, subject-specific modeling is fundamental.

Image-based modeling approaches should hence be considered and chosen depending on the segmentation context. For instance, highly contrasted images might be correctly processed with direct segmentation approaches. However as previously explained in Chap. 2, many real prob-lems require the use of more robust approaches, such as those based on prior-knowledge. These approaches are very powerful but have their pitfalls such as the correspondence and training size problems. These factors strongly motivate our research in devising efficient and robust knowledge-based deformable models for musculoskeletal modeling.

As a summary, Table 3.2reports the cited image-based modeling approaches with used acqui-sition modalities and structures. The following abbreviations are used to describe the main techniques involved in each approach:

– DS: direct segmentation (Sec.2.2), where:

○ class.: classification

○ edge: edges-based

○ fitting: simple primitive fitting, like sphere

○ grow.: region growing

○ morph.: mathematical morphology

○ prot.: special acquisition protocol

○ thresh.: thresholding – Reg.: registration (Sec.2.3)

– DM: deformable model (Sec.2.4), where:

○ AC: active contour

– KDM: knowledge-based deformable model (Sec.2.5), where:

○ PDM: point distribution model, might be used or not with the active shape model search (ASM)

○ SDF: SSM built with signed distance functions

○ AAM: SSM built on appearance and using the active appearance model

○ DF: SSM built with a non-rigid deformation field

The termdigitizedrefers to digitized point clouds or partial surfaces of structures acquired with an optical technology.

Table 3.2:Image-based musculoskeletal modeling approaches.

Reference Modality Structure Techniques

[LEHE97] MRI Cartilage: knee DS: edge

[HM02] CT, MRI Bone: spine DS: thresh., morph., grow.

[KEK03] CT, MRI Bone: hip, skull DS: grow., morph.

[ZSN⁺04] MRI Bone: femoral head DS: grow., morph., fitting,

class.

[YAGE05] CT Bone DS: edge

[FOP⁺05] MRI Cartilage: knee DS: class.

[BFS⁺07] MRI Bone: knee DS: class., prot.

[DAWW07] MRI Bone DS: thresh., prot.

[FBCO07] MRI Bone, cartilage: knee DS: SVM class., prot., KDM:

PDM

[NLBA07] MRI Bone: shoulder DS: edge, grow.

[LRN⁺08] CT Bone DS: thresh., graph cut

[KVP⁺10] CT Bone: shoulder DS: thresh., special region

growing

[OEvJ99] US Bone: femoral head Reg.: sphere fitting

[EHW⁺00] CT Bone: hip Reg.: atlas affine, demons

[PKB06] CT Bone: hip Reg.: atlas non-rigid

[FCWO07] MRI Bone: knee Reg.: atlas affine, demons;

KDM: PDM

Table 3.2:Image-based musculoskeletal modeling approaches (continued)

Reference Modality Structure Techniques

[GP08] MRI Bone, muscle, ligament,

carti-lage: hip Reg.: shape matching

[SLS⁺09] MRI Muscle: action lines points Reg.: atlas non-rigid

[MLZ⁺10] CT Bone: thoracic vertebra Reg.: shape matching

[GXDX10] X-rays bone: femur Reg.: shape matching; DM:

AC

[SM99] CT Bone: proximal femur DM: AC

[KGG⁺03] Synthetic MRI Bone, cartilage: knee DM: AC

[BB03] X-rays Bone: hand DM: genetic AC

[STCK03] CT Bone: wrist DM: AC; DS: grow.

[LFG⁺98] MRI Bone: knee DM: geodesic AC

[RBH⁺00] MRI Bone: skull DM: levelset

[KHC05] MRI, CT Bone: knee DM: levelset

[KK08] CT Bone: spine, pelvis DM: levelset

[UPB08] X-rays, MRI Bone: hand, knee DM: geodesic AC

[SVG02] MRI, CT Bone: wrist DM: discrete

[GMMT06,GMT10] MRI Bone, cartilage, ligament,

muscle: hip DM: discrete

[FLJ99] Digitized data Bone: femur KDM: PDM

[LGF00] MRI, CT Bone: femur, vertebrae KDM: SDF

[HMK⁺02] CT Bone: knee KDM: PDM

[BMVS02] X-rays Bone: femur KDM: PDM

[KMB⁺03] X-rays Bone: hand KDM: PDM + AAM

[LSHD04] CT Bone: pelvis KDM: PDM

[LWH06] X-rays Bone: rib cage KDM: PDM

[SMRW06] X-rays Bone: knee KDM: PDM, appearance

mod-els

[DGZ07] X-rays Bone: femur KDM: PDM, appearance

mod-els

[RST⁺07] Digitized data Bone: femur KDM: PDM

[DLH07] X-rays Bone: hip KDM: SDF

[SCT07] X-rays Bone: pelvis Reg.; KDM: PDM

[FCWO07] MRI Bone: knee DS: prot.; KDM: PDM

[HJ08] X-rays Bone: femur Reg.; KDM: PDM, AAM

[SFK⁺08] X-rays Bone: femur KDM: PDM

[SKH⁺08] CT Bone: pelvis KDM: PDM

[KLZH09] CT Bone: hip KDM: SSM

[YOT⁺09] CT Bone: femur DS: edge, thresh.; KDM: PDM

[KNO⁺09] X-rays Bone: femur KDM: PDM

[CCB⁺09] X-rays Bone: spine KDM: PDM

Table 3.2:Image-based musculoskeletal modeling approaches (continued)

Reference Modality Structure Techniques

[ZSG10] X-rays Bone: femur KDM: PDM

[DLvB⁺10] X-rays Bone: rib cage KDM: PDM

[KMG⁺10] US Bone: spine KDM: DF

C onstruction of S hape and

Dans le document Knowledge-based deformable models for medical image analysis (Page 99-106)