Automatic whole heart segmentation in CT images based on multi-atlas image registration

HAL Id: hal-01777688

https://hal-univ-rennes1.archives-ouvertes.fr/hal-01777688

Submitted on 12 Feb 2020

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Automatic whole heart segmentation in CT images based on multi-atlas image registration

G. Yang, C. Sun, Y. Chen, L. Tang, H. Shu, Jean-Louis Dillenseger

To cite this version:

G. Yang, C. Sun, Y. Chen, L. Tang, H. Shu, et al.. Automatic whole heart segmentation in CT

images based on multi-atlas image registration. 8th International Workshop on Statistical Atlases and

Computational Models of the Heart, STACOM 2017, Held in Conjunction with MICCAI 2017, Sep

2017, Quebec City, Canada. pp.250-257, �10.1007/978-3-319-75541-0_27�. �hal-01777688�

Images based on Multi-atlas Image Registration

Guanyu Yang

, Chenchen Sun

, Yang Chen

, Lijun Tang

, Xiaomei Zhu

, Huazhong Shu

, and Jean-Louis Dillenseger

1

LIST, Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing, China

yang.list@seu.edu.cn

2

Univ Rennes, Inserm, LTSI - UMR1099, 35000 Rennes, France

3

Department of Radiology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China

4

Centre de Recherche en Information Biom´ edicale Sino-Fran¸ cais (CRIBs), France

Abstract. Whole heart segmentation in CT images is a significant pre- requisite for clinical diagnosis or treatment. In this work, we present a three-step multi-atlas-based method for obtaining a segmentation of the whole heart. In the first step, the region of the heart was detected by aligning the down-sampled patient CT with the low-resolution atlas im- ages. The detected region of heart was used to crop the original patient image. In the second step, the registration between high-resolution atlas images and cropped original patient images was performed to obtain the precise segmentation of the heart. In the third step, the registration was performed again by minimizing the dissimilarity within the heart region.

Finally, the labels of four cardiac chambers, aorta and pulmonary artery were generated according to the similarity between the deformed atlas images and the patient image. A leave-one-out experiment has been per- formed on the 20 training datasets of MM-WHS 2017 challenge. The average Dice coefficient between our segmentation results and the man- ual segmentation results is 0.9051. The mean and standard deviation of Dice coefficients of each structure (i.e. LV, RV, LA, RA, Myo, Ao, PA) are 0.9601 ± 0.0324, 0.9344 ± 0.0418, 0.9594 ± 0.0316, 0.8836 ± 0.0826, 0.8724 ± 0.0707, 0.9295 ± 0.0883, 0.7966 ± 0.1149 respectively.

Keywords: Whole heart segmentation, Heart, Segmentation, Multi-atlas, Car- diac chambers

1 Introduction

Cardiovascular disease, with increasing mortality and morbidity, has been threat-

ening human health globally. Whole heart segmentation in CT images is a sig-

nificant prerequisite for clinical diagnosis or treatment, providing not only the

anatomical morphology but also the functional information of each cardiac cham-

ber. Some approaches have been proposed for whole heart segmentation in CT

2 G. Yang et al.

images. Ecabert [1] introduced a method based on statistical shape model, with model adaption to increase segmentation accuracy. Kiri¸sli et al. [2] proposed a multi-atlas based method and did an evaluation with a leave-one-out strategy.

Recently, Zhuang [6] proposed a framework M

AS for whole heart segmentation of MRI. The method is based on a multi-scale patch strategy and a label fusion algorithm using both global and local weights.

In this work, we present a three-step multi-atlas-based method for obtaining a segmentation of whole heart, including left and right ventricles, left and right atrium, aorta, pulmonary artery in CT images. This method mainly relies on image registration between the patient image and multiple atlas images, which is to generate segmentation results in each step. Experimental results show that our proposed multi-atlas based method can segment whole heart accurately.

2 Methodology

2.1 A three-step multi-atlas-based whole heart segmentation

In this multi-atlas registration method, the patient dataset is aligned with each atlas image to map the reference labels of the atlas images to the patient image.

A coarse-to-fine segmentation result is obtained during the registration process.

Then, the de-formed labels from several atlas image, which is selected by a strat- egy, are fused to generate the final segmentation result according to a specified fusion criterion.

This multi-atlas registration method chiefly relies on the image registration process which can be described as an optimization problem to find an optimal transformation ˆ T between a fixed image I

(x) and a moving image I

(x), and the expression (1) illustrates this transformation.

T ˆ = arg min

T

C(I

(x), I

(T(x))) (1)

In this expression, T (sx) is a transformation function which deforms I

(x) to align with I

(x) spatially. C is the cost function to measure the dissimi- larity between I

(x) and deformed I

(T (x)). C is minimized iteratively by an optimization algorithm. In this work, the mutual information defined in [5] is adopted to measure the dissimilarity due to a large difference of intensity distri- bution between CT images. The adaptive stochastic gradient descent method [4]

is used to optimize the cost function. Owing to the high computational cost in multi-atlas registration, we applied some techniques to reduce the computation time. In each iteration, a randomly sampled subset of image voxels is selected to be measurement of the dissimilarity and the number of voxels in 3 the sampled subset is set to 2048. A multi-resolution strategy based on Gaussian pyramid is also used during the registration process.

In the cardiac CT images, the strong boundaries of tissues surrounding the

heart, such as lung and rib cage, the difference scanning range will lead to an

inaccurate transformation parameters of image registration. Therefore, the seg-

mentation results could be inaccurate if we register the whole CT image with the

atlas image as the method in [3] did. So, in this paper, we introduce a three-step method to generate the coarse-to-fine segmentation result, in order to generate more accurate results. Figure 1 describes the framework of the proposed method which consists of three steps: (1) patient image ROI definition, (2) precise seg- mentation within ROI, (3) similarity computation, no-grid registration and label fusion.

In the first step, a ROI detection based on registration is performed to locate the region of heart in the patient image. In this step, the atlas images and label images are down-sampled and resized to generate low-resolution atlas images I

(i = 1, . . . , N ) which are then aligned with the down-sampled patient image to obtain the coarse segmentation of heart. Both the atlas and patient images will be down-sampled to an isotropic image with the size of each axial slice of 128 × 128. The region of interest (ROI) is defined by the low resolution label images to locate the region of heart. In the second step, the heart is cropped from the original patient image by the ROI defined in the previous step. Then the cropped high-resolution patient image are registered with the high-resolution atlas images I

(i = 1, . . . , M ) to obtain the precise segmentation of heart. In this step, two registration methods are applied. Affine registration is used to align the patient CT image with atlas images roughly and this is followed by B-spline registration which is to refine the spatial transformation. In the third step, only B-spline non-rigid registration is re-performed with the mask of heart estimated in the previous step in order to refine the transformation from atlas images to patient images. The MI between affine registration results and patient image is computed to measure the dissimilarity. Then the atlas images is ranked according to the dissimilarity. To generate a final segmentation of the patient image, many methods adopted majority voting. Regarding some situation that several atlas images have low similarity with patient images, we employ a strategy to do a selection in these atlas images by comparing the similarity S(I

, I

) between its deformed results and patient image. The similarity S(X, Y ) is defined as S(X, Y ) = P

i=1

(X

− Y

)

, in which n is the total number of voxels in the ROI of the image. An atlas image will be selected if its similarity with the patient image S(I

, I

) is above the mean value S

= (1/M ) P

i=1

S(I

, I

).

According to this strategy, m atlas images will be selected and m is usually less than M/2 in our experiment. Then a global fusion is performed on the deformed labels of the selected atlas images to generate the final segmentation of whole heart. The global weight w

of each selected atlas is defined as:

w

= S(I

, I

)/

m

X

j

S(I

, I

) (2)

During the second and third step, registrations are executed on the ROI of

the images which is defined in the first step. Because the whole heart is the major

part of the ROI in images, it can be matched more accurately between patient

CT images and atlas images. In all the three steps, the limitation of iteration

times in the optimization of cost function C is fixed to 500. The cubic B-spline

interpolator is used to estimate the values of the voxels at non-voxel positions.

4 G. Yang et al.

Fig. 1. The framework of the proposed method for automatic whole heart segmenta-

tion. I

, I

: low resolution and high resolution patient images, I

, I

: low resolution

and high resolution atlas images, I

, I

: patient images and atlas images

cropped from high resolution images by the ROI.

The B-spline grid is defined by control points with 16 mm interval. To generate the segmentation results, the labels of 5 selected atlas images are then fused by the global weights which are defined according to the similarity.

2.2 Multiple atlas images

Sixty multi-modality cardiac images were used in the experiment, which are pro- vided by the Challenge Multi-Modality Whole Heart Segmentation(MM-WHS 2017). These data was acquired in real clinical environment, which indicates its various quality. Some images with poor quality were included to test the robustness of the algorithms in real clinical situation.

The cardiac CT/CTA images were obtained at Shanghai Shuguang Hospital, China, using routine cardiac CT angiography. Each data consists of the whole heart, covering the region from upper abdominal to the aortic arch. The slices were obtained in axial view and the inplane resolution is about 0.78 × 0.78 mm and the slice thickness is 1.60 mm on average.

The data comprises two types, which are training data (20 CT images) and test data (40 CT images). Training data were segmented manually into seven whole heart substructures [1, 2]:

– the left ventricular cavity ; – the right ventricular blood cavity;

– the left atrial cavity ; – the right atrial blood cavity;

– the myocardium of the left ventricle ;

– the ascending aorta trunk from the aortic valve to the superior level of the atria;

– the pulmonary artery trunk between the pulmonary valve and the bifurcation point.

These structures above in training data were manually labeled with different values. Due to labels of test data is not provided, we also employed training data for evaluation of this method, using a leave-one-out test.

As described in the last section, the proposed method is based on multi- atlas registration and processed images are used. The patient images and atlas images used in this method have two scales: low resolution whole images and high resolution cropped images. Original atlas images and label images were down- sampled to generate low resolution whole images. The axial sizes were different among these original images and they were resized to low resolution images whose axial size is 128 × 128. To acquire high resolution images, the whole heart was cropped from the original images using region of heart defined by the labels of heart. The same option is per-formed on patient images. Not all atlas images are suitable for segmenting a patient image because the dissimilarity between them is large. In this paper, we did a selection among the atlas images, N and M in I

i

(i = 1, . . . , N ) and I

i

(i = 1, . . . , M ) are fixed to be 8.

6 G. Yang et al.

Fig. 2. Segmentation results of whole heart by our proposed method. (a) cropped high resolution patient image. (b) segmentation result obtained in the cropped image

3 Experimental results

The three-step multi-atlas-based whole heart segmentation framework is imple- mented in Mevislab (http://www.mevislab.de/). The image registration is per- formed by an open-source package named ELASTIX. The images used in the experiment were provided and segmented by MM-WHS 2017 challenge.

First, we performed our method on the 40 test images to yield segmentation results and the test results were submitted to organizers of MM-WHS 2017 challenge for evaluation. Figure 2 illustrates the segmentation results from a test patient image. The results demonstrates that this proposed method has the capability to segment the whole heart accurately. The evaluation results of test datasets will be provided on the day of MM-WHS2017 challenge event. We have to provide quantitative evaluation of our proposed method with leave-one-out test of training datasets.

Second, we applied leave-one-out test with 20 labeled atlas images to evaluate our proposed segmentation method. Each atlas image was segmented by the proposed method with the other 19 atlas images. The segmentation result of each structure (i.e. LV, RV, LA, RA, Myo, Ao, PA) from a training image is displayed by Figure 3. At the same time, Dice similarity coefficient (DSC) between the segmentation results and reference atlas labels was calculated to evaluate the accuracy of this method.

DSC = 2(M ∩ R)

M + R (3)

The DSCs of each structure in 20 training images are displayed in Table 1.

The mean and standard deviation of Dice coefficients of each structure (i.e. LV, RV, LA, RA, Myo, Ao, PA) are 0.9601 ±0.0324, 0.9344±0.0418, 0.9594±0.0316, 0.8836 ± 0.0826, 0.8724 ± 0.0707, 0.9295 ± 0.0883, 0.7966 ± 0.1149 respectively.

These quantitative evaluations show that our method can generate accurate

whole heart segmentation in 3D CT images.

Fig. 3. The original training image and segmentation result of each structure (i.e. LV,

RV, LA, RA, Myo, Ao, PA)

8 G. Yang et al.

Table 1. Dice similarity coefficient between reference labels and segmentation results of each structure of heart

DSC of each structure of proposed method

Atlas LV RV LA RA Myo Ao PA

No.1 0.9738 0.9932 0.9985 0.9452 0.8949 0.9902 0.9845

No.2 0.9293 0.8244 0.9680 0.9600 0.7934 0.9859 0.9279

No.3 0.9895 0.9446 0.9508 0.8706 0.9227 0.9802 0.8177

No.4 0.9068 0.9241 0.9939 0.8951 0.8184 0.9732 0.8408

No.5 0.9915 0.9530 0.9079 0.6784 0.8860 0.9941 0.7487

No.6 0.8683 0.9428 0.9768 0.9651 0.8656 0.7442 0.9046

No.7 0.9586 0.9398 0.9284 0.8710 0.8698 0.9855 0.8127

No.8 0.9847 0.9987 0.9802 0.8444 0.8292 0.9910 0.7908

No.9 0.9585 0.9443 0.9361 0.8963 0.9039 0.9858 0.8074

No.10 0.9721 0.9526 0.9809 0.9748 0.9620 0.9593 0.8605

No.11 0.9930 0.8941 0.9796 0.9657 0.9248 0.7952 0.7995

No.12 0.9852 0.9630 0.9630 0.9008 0.8886 0.9943 0.7747

No.13 0.9636 0.9338 0.9872 0.8951 0.6786 0.9824 0.8124

No.14 0.9419 0.8957 0.9342 0.9224 0.7518 0.9095 0.7040

No.15 0.9673 0.8910 0.9513 0.8332 0.9053 0.9374 0.5936

No.16 0.9745 0.9646 0.9897 0.8909 0.8627 0.8373 0.7211

No.17 0.9928 0.9409 0.9568 0.8481 0.9030 0.7465 0.7805

No.18 0.9701 0.9039 0.9054 0.8344 0.9665 0.9816 0.5177

No.19 0.9222 0.9896 0.8999 0.6953 0.8741 0.8184 0.7377

No.20 0.9573 0.8928 0.9984 0.9850 0.9467 0.9979 0.9957

Avg. 0.9601 0.9344 0.9594 0.8836 0.8724 0.9295 0.7966

4 Conclusion

In this paper, an automatic method for segmenting whole heart in CT images is presented. The method mainly relies on a three-step multi-atlas image reg- istration to acquire the segmentation results. In the first step, down-sampled patient image is registered with low resolution atlas images to detect the region of heart which was used to crop the original patient image. In the second step, the registration between cropped original patient images and high-resolution at- las images was performed to obtain the precise segmentation of the heart. In the third step, the registration was performed again by minimizing the dissimilarity within the heart region. Finally, the labels of four cardiac chambers, aorta and pulmonary artery were acquired respectively according to the similarity between patient image and deformed atlas images. Ex-perimental results in 20 training images and 40 test images show that this method can generate an accurate whole heart segmentation.

References

1. Ecabert, O., Peters, J., Schramm, H., Lorenz, C., von Berg, J., Walker, M.J., Vem- bar, M., Olszewski, M.E., Subramanyan, K., Lavi, G., Weese, J.: Automatic model- based segmentation of the heart in CT images. IEEE Transactions on Medical Imag- ing 27(9), 1189–1201 (Sep 2008). https://doi.org/10.1109/TMI.2008.918330 2. Kiri¸sli, H.A., Schaap, M., Klein, S., Papadopoulou, S.L., Bonardi, M., Chen, C.H.,

Weustink, A.C., Mollet, N.R., Vonken, E.J., van der Geest, R.J., van Walsum, T., Niessen, W.J.: Evaluation of a multi-atlas based method for segmentation of cardiac CTA data: a large-scale, multicenter, and multivendor study. Medical physics 37, 6279–6291 (Dec 2010). https://doi.org/10.1118/1.3512795

3. Kiri¸sli, H.A., Schaap, M., Klein, S., Neefjes, L.A., Weustink, A.C., Van Walsum, T., Niessen, W.J.: Fully automatic cardiac segmentation from 3D CTA data: a multi- atlas based approach. In: Medical Imaging 2010: Image Processing. vol. 7623, p.

762305. International Society for Optics and Photonics (2010)

4. Klein, S., Pluim, J.P., Staring, M., Viergever, M.A.: Adaptive stochastic gradient descent optimisation for image registration. International journal of computer vision 81(3), 227 (2009)

5. Thevenaz, P., Unser, M.: Optimization of mutual information for multiresolution image registration. IEEE Transactions on Image Processing 9(12), 2083–2099 (Dec 2000)

6. Zhuang, X., Shen, J.: Multi-scale patch and multi-modality atlases for whole heart

segmentation of MRI. Medical image analysis 31, 77–87 (2016)