IMIV, Inserm, CEA, Universit´e Paris-Sud, CNRS, Universit´e Paris-Saclay, 91400, Orsay
† MAP5, CNRS, T´el´ecom SudParis, Universit´e Paris-Saclay, 91011, Evry, France
Abstract. In this paper, we propose a novel 3Dmethodformultiplesclerosissegmentation on FLAIR Magnetic Resonance images (MRI), based on a lesion context-based criterion performed on a max-tree repre- sentation. The detection criterion is refined using prior information from other available MRI acquisitions (T2, T1, T1 enhanced with Gadolinium and DP). The method has been tested on fifteen patients su↵ering from multiplesclerosis. The results show the ability of the method to detect almost all lesions. However, the algorithm also provides false detections. Keywords: Max-tree, multiplesclerosis, MRImultimodal segmenta- tion.
Index Terms— Graph Cut, Expectation-Maximization, multiplesclerosis, tissue classification
Manual and semi-automatic segmentation methods are very time consuming and can show a high variability among man- ual delineations, especially on longitudinal data. To solve this issue, we present a fully automated methodfor MS le- sions segmentation based on the combination of graph cut and robust EM tissues segmentation using multiple sequences of Magnetic Resonance Imaging (MRI). The algorithm is based on several previous segmentation algorithms. A fully auto- mated method implies that it does not include user interac- tions. Our process is applied to the ISBI 2015 challenge data for longitudinal MS lesion segmentation. Only one parameter set per patient is used. No training steps are involved in the workflow and we do not use the longitudinal information to obtain the segmentation of a given time point.
Over the past years, various approaches for semi-automatic and automatic segmentation of MS lesions have been proposed. In these methods, different im- age features, classification methods and models have been tried, but they usu- ally suffer from high sensitivity to the imaging protocols and so usually require tedious parameter tuning or specific normalized protocols . More recently, sparse representation has evolved as a model to represent an important variety of natural signals using few elements of an overcomplete dictionary. Many pub- lications have demonstrated that sparse modeling can achieve state-of-the-art results in image processing applications such as denoising, texture segmentationand face recognition [4, 5]. In , given multiple images of individual subjects under varying expressions and illuminations, the images themselves were used as dictionary elements, for classification. Such amethod uses dictionary learning to analyze image as a whole. Mairal et al  proposed to learn discriminative dictionaries better suited for local image descrimination tasks. In medical imag- ing, local image analysis is of prime importance and it could be interesting to see the performance of sparse representation and dictionary learning based classification methods in the context of disease detection. Some researchers have reported works on segmentation of endocardium and MS lesions using dictionary learning [7, 8]. Weiss et al. proposed an unsupervised approach for MS lesion seg- mentation, in which a dictionary learned using healthy brain tissue and lesion patches is used as basis for classification .
are registered and intensity normalized on a set of controls. Vector images are created to combine T2-w and FLAIR sequences, on one hand, from MS patient images, and on the other hand, from control subject images. These vector images are then compared with the method described in section 3.2. Intersection of T2-w and FLAIR is interesting as it generates theoretically less false lesionsdetection as T2-w or FLAIR segmentation alone. However, this way to combine images has a drawback: in some cases, it is possible that a lesion intensity profile is close to the one of a tissue ina modality, which may reduce the Mahalanobis distance and induce the non-detection of a lesion even if its intensity profile is far from tissues intensity profiles in the other modalities.
We proposed a new framework for MS lesion detection based on differences between multi-channel MRI of patients and controls. This framework fused with intensity standardization was applied to the study of MS and highlighted the great interest of quantitative MRI measurements made on MRIfora better understanding and characterization of the disease. The efficacy of our method was evaluated though detection with and without intensity correction. Compared to un-normalized images, detection with intensity correction is a better choice for MS lesion analysis thanks to its ability to preserve the intensity variations caused by pathological changes while normalizing healthy tissue intensities. The resulting system is both efficient and accurate. This performance suggests that it can provide valuable assistance in detecting MS lesionsin clinical routine with high reliability. Our models are already capable of detecting highly variable lesion patterns, but we would like to move towards richer models. The framework described here allows for exploration of additional MR sequences with or without contrast agents. For example, one can consider infusing T1-w Gadolinium.
Abstract. In most of the current approaches to automatic segmenta- tion of multiplesclerosis (MS) lesions, the segmentation methods are not optimized with respect to all relevant evaluation metrics at once. An ob- stacle is that the computation of relevant metrics is three-dimensional (3D). The high computational costs of 3D metrics make their use im- practical as learning targets for iterative training. Ina companion paper, we propose an oriented training strategy that targets cheap 2D metrics as surrogates for costly 3D metrics. We applied the evaluation-oriented training with surrogate learning targets to optimize a simple multilayer perceptron (MLP) network at the core of asegmentation pipeline. Ina previous competition, our segmentation strategy achieved a perfor- mance that was comparable to state-of-the-art methods. In this paper, by the opportunity of the MS Segmentation Challenge 2016, we apply the proposed method on a larger and more diverse dataset. We expect to compare the proposed strategy to other methods concerning segmen- tation quality and runtime on the computational cloud provided by the challenge organizers.
Context: The spinal cord is a key component of the central nervous system, which contains neurons responsible for complex functions, and ensures the conduction of motor and sensory information between the brain and the peripheral nervous system. Damage to the spinal cord, through trauma or neurodegenerative diseases, can lead to severe impairment, including functional disabilities, paralysis and/or pain. Inmultiplesclerosis (MS) patients, the spinal cord is frequently affected by atrophy and/or lesions. Conventional magnetic resonance imaging (MRI) is widely used by researchers and clinicians to non-invasively assess and characterize spinal cord microstructural changes. Quantitative assessment of the structural damage to the spinal cord (e.g. atrophy severity, lesion extent) is essential for the diagnosis, prognosis and longitudinal monitoring of diseases, such as MS. Furthermore, the development of objective biomarkers is essential to evaluate the effect of new therapeutic treatments. Spinal cord and intramedullary MS lesionssegmentation is consequently clinically relevant, as well as a necessary step towards the interpretation of multi-parametric MR images. However, manual segmentation is highly time- consuming, tedious and prone to intra- and inter-rater variability. There is therefore a need for automated segmentation methods to facilitate the efficiency of analysis pipelines. Automatic lesion segmentation is challenging for various reasons: (i) lesion variability in terms of shape, size and location, (ii) lesion boundaries are most of the time not well defined, (iii) lesion intensities on MR data are confounding with those of normal-appearing structures. Moreover, achieving robust segmentation across multi-center MRI data is challenging because of the broad variability of data features (e.g. resolution, orientation, field of view). Despite recent substantial developments in spinal cord MRI processing, there is still no method available that can yield robust and reliable spinal cord segmentation across the very diverse spinal pathologies and data features. Regarding the intramedullary lesions, a thorough search of the relevant literature did not yield available method of automatic segmentation.
the detection of macroscopic WM lesions, however it is not specific of a particular biological abnormality, and generally fails to identify cortical demyelination due to weak spatial resolution and sparse cortical network of fibers in the cortex resulting in poor contrast for myelin. Diffusion-weighted MRI is based on the diffusion of water molecules, which can be restricted by membranes, and can thereby provide differential measurements of myelin and axonal integrity ( Song et al., 2002; Bodini et al., 2015 ). Several MRI studies attempted to visualize demyelination and remyelination in the WM of mice treated with CPZ using different MRI techniques such as T 2 -weighted imaging, which
Multiplesclerosis (MS) is a demyelinating and inflammatory disease of the central nervous system anda major cause of disability in young adults [ Compston, 2008 ]. MS has been characterized as a white matter (WM) dis- ease with the formation of WM lesions, which can be visualized by magnetic resonance imaging (MRI) [ Paty, 1988 ; Barkhof, 1997 ]. The fluid-attenuated inversion recovery (FLAIR) MRI pulse sequence is commonly used clinically andin research for the detection of WM lesions which appear hyperintense compared to the normal appearing WM tissue (NAWM). Moreover, the sup- pression of the ventricular signal, characteristic of the FLAIR images, allows an improved visualization of the periventricular MS lesions [ Woo, 2006 ], and can also suppress any artifacts created by CSF. In addition, the decrease of the dynamic range of the image can make the subtle changes easier to see. Typical MRI pulse sequences used ina clinical setting are shown in Fig. 2.1 . WM lesions (red rectangles) characteristic of MS are clearly best seen on FLAIR pulse sequences. However, ina clinical setting, some MRI pulse sequences can be missing because of limited scanning time or patients’ interruptions in case of anxiety, confusion or severe pain. Hence, there is a need for predicting the missing FLAIR when it has not been acquired during patients’ visits. FLAIR may also be absent in some legacy research datasets, that are still of major interest due to their number of subjects and long follow-up periods, such as ADNI [ Mueller, 2005 ]. Furthermore, the automatically synthesized MR images may also improve brain tissue classification andsegmentation results as suggested in the works of Iglesias et al. [ Iglesias, 2013 ] and Van Tulder and Bruijne [ Van Tulder, 2015 ], which is an additional motivation for this work.
The present study highlights for the first time in MS that the 3D MP2RAGE method is able to detect cervical cord lesions with higher confidence and higher sensitivity than the recommended conventional MAGNIMS set. Given the essential diagnostic and prognostic significance of SC lesionsin MS, these results could have some critical added value for clinical practice, as previously demonstrated using phase-sensitive inversion recovery or double inversion recovery contrast 9-11 .
automated multimodal Graph Cut
Jeremy Beaumont Olivier Commowick Christian Barillot
VisAGeS U746 INSERM / INRIA, IRISA UMR CNRS 6074, Rennes, France
Abstract. In this paper, we present an algorithm forMultipleSclerosis (MS) lesion segmentation. Our method is fully automated and includes three main steps: 1. the computation of a rough total lesion load in order to optimize the parameter set of the following step; 2. the detection of lesions by graph cut initialized with a robust Expectation-Maximization (EM) algorithm; 3. the application of rules to remove false positives and to adjust the contour of the detected lesions. Our algorithm will be tested on the FLI 2016 MSSEG challenge data.
In recent years, various methods for medical image enhancement and synthe- sis using deep neural networks (DNN) have been proposed, such as reconstruction of 7T-like T1-w MRI from 3T T1-w MRI [ 4 ] and generation of FLAIR from T1-w MRI [ 5 ]. Several works have proposed to predict PET images from MRI or CT images [ 6 , 7 , 8 ]. A single 2D GAN has been proposed to generate FDG PET from CT for tumor detectionin lung [ 6 ] and liver region [ 7 ]. However, they do not take into account the spatial nature of 3D images and can cause discontinuous predictions between adjacent slices. Additionally, a single GAN can be difficult to train when the inputs become complex. A two-layer DNN has been proposed to predict FDG PET from T1-w MRI [ 8 ] for AD diagnosis. However, a 2-layer DNN is not powerful enough to fully incorporate the complex information from multimodalMRI. Importantly, all these works were devoted to the prediction of FDG PET. Predicting myelin information (as defined by PIB PET) is a more difficult task because the signal is more subtle and with weaker relationship to anatomical information that could be found in T1-w MRI or CT.
In addition, in the works of Sikka et al. ( 2018 ); Li et al.
( 2014 ) and Pan et al. ( 2018 ), only a single MRI pulse sequence is used for prediction, for example Sikka et al. ( 2018 ) and Li et al. ( 2014 ) only use T1-w MRI as the input. However, we showed improved performances can be achieved by including more modalities as inputs. Using MTR +RD instead of only MTR can dramatically increase the myelin content prediction results especially in MS lesions. Adding AD and FA only marginally improved the results compared to MTR +RD. How- ever, AD, FA and RD are all computed from a single DTI acqui- sition. Therefore, adding AD and FA does not require acquisi- tion of more MRI sequences and does not increase the scanning time. We thus recommend using MTR +DTI since this leads to the best results, even though the improvement is small com- pared to MTR+RD. In fact, using multiple modalities for image synthesis andsegmentation has also been studied in Chartsias et al. ( 2018 ) and Havaei et al. ( 2016 ). In their works, multichan- nel neural networks have been used. During the inference step, each modality is provided independently to convolutional neu- ral networks. After encoding each modality into latent repre- sentations, multiple fusion strategies such as the mean-variance fusion ( Havaei et al. , 2016 ) or the max fusion ( Chartsias et al. ,
Several automatic segmentation methods for MS lesions have been presented that can be classified in two categories: supervised or data-driven. Supervised methods employ a test database of previously segmented images to learn the charac- teristics of MS lesions , , , . The results of the supervised methods depend on the way the test database has been segmented and on the MR protocol of the database which may limit the interest of these approaches in multi-center trials. Data-driven methods avoid the use of any sample database, extracting all the necessary information directly from the images , , , . The majority of these data- driven methods models the distribution of the image intensities using a Gaussian mixture model (GMM), where each Gaussian law represents a tissue: e.g. cerebrospinal fluid (CSF), gray matter (GM) or white matter (WM). The GMM enables characterization of the image intensities with a reduced num- ber of parameters. In healthy subjects , these parameters have been estimated using a maximum likelihood estimator (MLE) with an optimization method such as the Expectation- Maximization (EM) algorithm . The EM algorithm has been widely used in this context because it is very easy to implement and always converges to a local maximum (or saddle point) of the data likelihood.
Abstract— The problem addressed in this paper is the automatic segmentation of stroke lesions on MR multi- sequences. Lesions enhance differently depending on the MR modality and there is an obvious gain in trying to account for various sources of information ina single procedure. To this aim, we propose amultimodal Markov random field model which includes all MR modalities simultaneously. The results of the multimodalmethod proposed are compared with those obtained with a mono-dimensional segmentation applied on each MRI sequence separately. We constructed an Atlas of blood supply territories to help clinicians in the determination of stroke subtypes and potential functional deficit.
2 Material and methods
2.1 MS lesion detection framework
The a-contrario approach The a-contrario approach is a locally multivari- ate procedure which uses the size of a local excursion set as statistic . An a-contrario framework was previously presented to extract patterns of abnormal perfusions in individual patients . Its general steps can be summarized as fol- lows: i) a voxel-wise probability map is computed under a background model (i.e. the null-hypothesis in statistical decision theory ), ii) a locally multivariate probability is estimated, and iii) a correction formultiple testing is performed. We propose to apply the a-contrario approach to the segmentation of MS lesions from multimodalMRI as follows.
Figure 4: Results for test images. For each measure the value is given in two different metric: real value in the left anda normalized value in the right.
5.1 Definition of MS lesion inMRI
Compared to the segmentation validation of other targets, MS lesion segmentation have an increased com- plexity. Today there is no consensus on a precise definition of MS lesion inMRI. This situation leads to a very high variability, in first place, in the detection. In second place, and once all the lesions have been detected, there is a still a high variability in contour of each lesion [ 9 , 17 , 7 ], as MS lesions not rarely lack a sharp border, but also because MS lesion size and border vary according to the MR sequence used [ 7 ].
We propose a new method to detect abnormalities by comparing the ori- entation distribution functions (ODF) at the voxel level between a patient anda population of controls, ina similar fashion to Commowick et al. (2008). It relies on a statistical comparison, combined to a principal component anal- ysis (PCA) for robustness, of the diffusion probability values sampled on the sphere from virtually any model that has a probability density function de- fined on the sphere. We applied this method first to simulated data with lesions to evaluate the detection power of ODF-based scores with respect to DTI-based scores. Then, we applied our pipeline to the longitudinal de- tection of differences between a database of CIS patients with scans at two time points, to evaluate the ability of ODF-based scores to highlight early differences that may lead to the appearance or disappearance of lesions at the second time point.
Throughout this study we focused on lesionsin the cervical cord, which although showed interesting results, excludes lesions existing in the brain. The PAM50 spinal cord tem- plate has been aligned with the ICBM152 space (De Leener et al., 2018), enabling combination of brain and spinal MRI, and therefore prompting further investigation into inﬂuence of brain and cervical spine lesion location on clinical status. Recent progress in automatic multiplesclerosis lesion seg- mentation in the spinal cord (Gros et al., 2018), which has shown good performance across highly variable MRI data, may also be of interest in follow-up studies. Addressing other key pathological mechanisms, such as atrophy and myelopathy, may increase the strength of correlation be- tween lesion measures and disability, as has been shown previously (Lukas et al., 2013; Rocca et al., 2013; Kearney et al., 2015a). Finally, performing a similar longitudinal study is likely to considerably improve our understanding of the association between lesion location and clinical status.