Organ Segmentation in CT Images With Weak Annotations: A Preliminary Study
Rosana El-Jurdi
(1,2), Caroline Petitjean
(1), Paul Honeine
(1), Fahed Abdallah
(2,3)(1)LITIS, Universit´e de Rouen, France (2)Universit´e Libanaise, Liban (3)UTT, Troyes, France
Abstract
Medical image segmentation has unprecedented challenges, compared to natural image segmentation, in particular because of the scarcity of annotated datasets. Of particular interest is the ongoing 2019 SegTHOR competition, which consists in Segmenting THoracic Organs at Risk in CT images. While the fully supervised framework (i.e., pixel-level annotation) is considered in this competition, this paper seeks to push forward the competition to a new paradigm: weakly supervised segmentation, namely training with only bounding boxes that enclose the organs. After a pre-processing step, the proposed method operates a GrabCut algorithm that transforms the images into pixel-level annotated ones. And then a deep neural network is trained on the medical images, where several segmentation loss functions are examined. Experiments show the relevance of the proposed method, providing comparable results to the ongoing fully supervised segmentation competition.
Framework
SegTHOR Dataset
Constituted of 60 patients with lung cancer [1]
Contains groundtruth segmentations of 4 organs at risk Focus in this work is on heart segmentation
Challenges
Soft organs are hard to detect and share same gray scale intensities with neighboring tissues
Reasonable class imbalance is evident due to the vari- ation of organ size with respect to background
Low contrast in CT images
Outline of the method
Step 1: Data preprocessing
Step 2: From bounding boxes to label segments Step 3: Refining label estimates with deep learning
Proposed method
From bounding boxes to pixel annotations:
Grabcut method: an iterative intensity graph based algorithm that gener- ates label segments from bounding boxes, based on estimating Gaussian mixture models and optimizing graphs built from image pixels [2]
Training the segmentation model with U-net :
– Encoder: VGG16 backbone with its full convolutional layers removed, pre- trained with ImageNet weights.
– Decoder: deconvolutional layers that extract and concatenate feature maps to generate coarse grained labels.
– Skip grams combine feature maps from both encoder and decoder layers for accurate organ localization.
Addressing class imbalance using loss functions:
– Balanced cross entropy loss [3]: −
PN
i=1 βigi log(pi) – Dice loss [4]: 2 PN
i=1 pigi + ǫ PN
i=1 pi + PN
i=1 gi + ǫ – Tversky loss [5]:
PN
i=1 pigi + ǫ PN
i=1 pigi + α PN
i=1 pi(1 − gi) + β PN
i=1(1 − pi)gi + ǫ
Results and Analysis
Dataset Distribution: training (26 patients: 1477 slices), validation (4 patients:
155 slices), testing (20 patients: 693 slices)
Results: Dice similarity, with the mean, extrema and standard deviation values mean % std % max % min % Pre-trained only (with ImageNet) 10.09 6.42 32.19 0.15 Trained with cross entropy loss 71.87 12.88 89.78 0.0 Trained with balanced cross entropy loss 84.67 3.94 90.57 67.86
Trained with Dice loss 41.07 18.49 74.63 0.0
Trained with Tversky loss 28.94 10.97 53.89 1.28
Trained with Full Supervision 93.18 4.54 98.18 72.12
Analysis:
Cross entropy losses achieve the highest scores with a Dice accuracy of 84.67%.
Proper convergence of model with dice loss is dependent on smoothing variable ǫ Grabcut label estimates are poor representatives of labels
Conclusion
Proposed a weakly supervised framework for organ segmentation in CT images Examined multiple loss functions with balanced cross entropy resulting in best performance
Future work
Integrating to multi-class organ segmentation
Proposing a smoothing function that varies w.r.t training iterations
Proposing a training procedure that alternates between losses, taking into account the strengths of each loss function to handle class imbalance
References
[1] R. Trullo, C. Petitjean, S. Ruan, B. Dubray, D. Nie, and D. Shen, “Segmentation of organs at risk in thoracic ct images using a sharpmask architecture and conditional random fields,” in 14th IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1003–1006, 2017.
[2] A. Khoreva, R. Benenson, J. H. Hosang, M. Hein, and B. Schiele, “Weakly supervised semantic labelling and instance segmentation,” CoRR, vol. abs/1603.07485, 2016.
[3] Y. S. Aurelio, G. M. de Almeida, C. L. de Castro, and A. P. Braga, “Learning from imbalanced data sets with weighted cross-entropy function,” Neural Processing Letters, Jan 2019.
[4] F. Milletari, N. Navab, and S. Ahmadi, “V-net: Fully convolutional neural networks for volumetric medical image segmentation,” CoRR, vol. abs/1606.04797, 2016.
[5] S. S. M. Salehi, D. Erdogmus, and A. Gholipour, “Tversky loss function for image segmentation using 3d fully convo- lutional deep networks,” CoRR, vol. abs/1706.05721, 2017.
Work supported by the National Council for Scientific Research of Lebanon, Agence Fran¸caise de la Francophonie, and ANR-18-CE23-0014
