HAL Id: hal-02146580
https://hal.archives-ouvertes.fr/hal-02146580
Submitted on 4 Jun 2019
HAL
is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire
HAL, estdestinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Validation of Equilibrated Warping-image registration with mechanical regularization-on 3D ultrasound images
Lik Chuan Lee, Martin Genet
To cite this version:
Lik Chuan Lee, Martin Genet. Validation of Equilibrated Warping-image registration with mechanical regularization-on 3D ultrasound images. FIMH 2019 - International Conference on Functional Imaging and Modeling of the Heart, Jun 2019, Bordeaux, France. pp.334-341, �10.1007/978-3-030-21949-9_36�.
�hal-02146580�
Validation of Equilibrated Warping—image registration with mechanical regularization—on
3D ultrasound images
Lik Chuan Lee1[0000−0002−1123−6428] and Martin Genet2,3[0000−0003−2204−201X]
1 Department of Mechanical Engineering, Michigan State University, East Lansing, Michigan, USA
lclee@egr.msu.edu
2 Laboratoire de M´ecanique des Solides, ´Ecole Polytechnique / CNRS / Universit´e Paris-Saclay, Palaiseau, France
3 M3DISIM team, INRIA / Universit´e Paris-Saclay, Palaiseau, France martin.genet@polytechnique.edu
Abstract. Image registration plays a very important role in quantifying cardiac motion from medical images, which has significant implications in the diagnosis of cardiac diseases and the development of personalized cardiac computational models. Many approaches have been proposed to solve the image registration problem; however, due to the intrinsic ill- posedness of the image registration problem, all these registration tech- niques, regardless of their variabilities, require some sort of regulariza- tion. An efficient regularization approach was recently proposed based on the equilibrium gap principle, named equilibrated warping. Compared to previous work, it has been formulated at the continuous level within the finite strain hyperelasticity framework and solved using the finite element method. Regularizing the image registration problem using this principle is advantageous as it produces a realistic solution that is close to that of an hyperelastic body in equilibrium with arbitrary boundary tractions, but no body load. The equilibrated warping method has al- ready been extensively validated on both tagged and untagged magnetic resonance images. In this paper, we provide full validation of the method on 3D ultrasound images, based on the 2011 MICCAI Motion Tracking Challenge data.
Keywords: Image registration · Finite element method·Equilibrium gap regularization.
1 Introduction
Image registration plays a very important role in quantifying cardiac motion from medical images, which has significant implications in the diagnosis of car- diac diseases [21] and the development of personalized cardiac computational models [9, 5] to understand the pathophysiological mechanisms of heart diseases [2, 20] and the effects of treatments [16, 14]. While image registration is still
2 L. C. Lee and M. Genet
largely performed as a separate step in the personalization of models [9, 6], it can be integrated and coupled with data assimilation techniques [13], also called integrated correlation [10], to estimate model parameters in a robust and efficient manner [2].
Many approaches have been proposed to solve the image registration prob- lem, and they can be broadly categorized into local and global approaches. In the latter category, the different image registration techniques vary depending on (i) the displacement field support (image-basedvs.mesh-based),(ii) the inter- polation scheme (e.g., linearvs.splines vs.polynomials), (iii)the chosen image similarity metric (e.g., L2vs.mutual information) and/or(iv)the minimization algorithm [1, 17]. Due to the intrinsic ill-posedness of the image registration prob- lem, however, all these registration techniques, regardless of their variabilities, require some sort of regularization [3, 4, 18, 12, 19, 17, 7]. Various regularizers have been proposed, from pure mathematical (laplacian smoothing) to geomet- rical to complex mechanical regularizer enforcing that the solution satisfies basic physical principles.
To address this issue, an efficient regularization approach was recently pro- posed based on the equilibrium gap principle [4], named equilibrated warping [7]. Compared to previous work [4, 10], it has been formulated at the continuous level within the (nonlinear) finite strain hyperelasticity framework, discretized consistently, and solved using the finite element method. Regularizing the image registration problem using this principle is advantageous as it produces a real- istic solution that is close to that of an hyperelastic body in equilibrium with arbitrary boundary tractions (but no body load). It could also be potentially formulated and integrated with data assimilation techniques to estimate bound- ary tractions (e.g., cavity pressure) that has important clinical implications [7].
Here, we propose to validate the equilibrated warping method based on the 2011 MICCAI Motion Tracking Challenge data [17] on both tagged (3DTAG) and untagged (SSFP) magnetic resonance and 3D ultrasound (3DUS) images.
2 Methods
2.1 Image Registration Problem
I˜0& ˜I are two images representing the same bodyB at two instantst0&t:
I˜0:
(0→R
X7→I˜0(X) , I˜:
(→R
x7→I˜(x) , (1) where0 &are the image domains att0&t, which are usually identical. The domains occupied by the body B at t0 & t are denoted Ω0 &Ω, respectively.
The problem is to find the smooth mapping Φbetween materials points in the reference and deformed domains:
Φ:
(Ω0→Ω
X7→x=Φ(X) , (2)
whereX &xdenote the position of a given material point in the reference and deformed configurations. Equivalently, one can search for the smooth displace- ment fieldU:
U :
(Ω0→R3
X 7→U(X) =Φ(X)−X . (3) Due to its intrinsic ill-posedness, the problem is formulated as a regularized minimization problem:
findΦsol= argmin{Φ}{(1−β)Ψcor(Φ) +βΨreg(Φ)}, (4) where Ψcor is the image similarity metric, or “correlation energy”, Ψreg is the regularization energy, andβdefines the regularization strength. The correlation energy is assumed to be convex, at least in the neighborhood of the solution, though it is in general not quadratic. Similarly, the regularization energy is assumed to be convex in the neighborhood of the solution.
2.2 Image intensity-based global image registration
In image intensity-based global approaches, the following correlation energy is generally used:
Ψcor(Φ) =1 2
Z
Ω0
I˜(Φ(X))−I˜0(X)2
dΩ0. (5) Other metrics have been proposed; however, we retain this one notably because it can be differentiated straightforwardly.
It can happen that the body to track is partly out of the images, at the reference frame and/or at the registered frames. This is especially true for cardiac echocardiography, where the ventricle barely fits within the imaging cone. In this case, in order to prevent the body parts outside the images to play a role in the registration, we propose the following augmentation of the correlation energy:
Ψcor(Φ) = 1 2
Z
Ω0
1(X)1(Φ(X))
I˜(Φ(X))−I˜0(X)2
dΩ0, (6) where 1 denotes the indicative function of the imaging domain, which corre- sponds to the entire image box in MRI, and only to the imaging cone in 3DUS.
2.3 Equilibrium gap regularization
In order to describe the equilibrium gap regularization that was first introduced in [4] and first formulated at the continuum level within the framework of non- linear finite strain hyperelasticity in [7], let us first recall that mechanical equi- librium, i.e., conservation of momentum, in absence of body load and inertia, can be expressed as:
Div F·S
= 0
tS=S ∀X ∈Ω0, (7)
4 L. C. Lee and M. Genet
where S is the second Piola-Kirchhoff stress tensor, andF = ∂X∂Φ is the trans- formation gradient [8]. These relations correspond to the conservation of linear and angular momentum, respectively. We also recall that the second principle of thermodynamics requires that:
S= ∂ρ0ψ
∂E , (8)
where ρ0ψ is the free energy density, and E = 12 C−1
the Green-Lagrange strain tensor, withC=tF·F the right Cauchy-Green dilatation tensor [8]. The Green-Lagrange strain tensor is symmetric, such that when computed through relation (8), the second Piola-Kirchhoff stress tensor is necessarily symmetric and the conservation of angular momentum is automatically verified. However, the conservation of linear momentum still needs to be enforced. In principle one could useΨreg=12
Div F·S
2
L2(Ω0). However, Problem (4) is discretized using standard Lagrange finite elements, so thatF andSbelong toL2(Ω0), but not toH(div;Ω0). Thus, the following equivalent norm is used instead:
Ψreg=X
K
1 2
Div F·S
2
L2(K)+X
F
1 2h
F·S·N2
L2(F), (9) whereKdenotes the set of elements,F the set the interior faces with normalN, andha characteristic length of the finite element mesh. See [7] for more details.
2.4 Implementation and Workflow
The method has been implemented based on FEniCS4 [11] and VTK5 [15] li- braries, and the code is freely available6. In practice, to register the images, one only needs a segmentation of the object of interest, which can then be automat- ically meshed. From there the registration is entirely automatic.
3 Results & Discussion
In this section we present the validation of the equilibrated warping method for 3DUS cardiac images registration (results for SSFP & 3DTAG images were already presented in [7]), using the publicly available dataset of the Cardiac Motion Analysis Challenge that was held at the 2011 MICCAI workshop7and is described in details in [17]. Briefly, the dataset consists of untagged (SSFP) and tagged (3DTAG) magnetic resonance images as well as 3DUS images acquired from a dynamic phantom (PHANTOM) and fifteen healthy volunteers (V1-V2,
4 https://www.fenicsproject.org
5 https://www.vtk.org
6 https://gitlab.inria.fr/mgenet/dolfin_dic
7 https://www.cardiacatlas.org/challenges/motion-tracking-challenge
V4-V16) with corresponding segmentations and markers that were manually tracked by experts (i.e., ground truth, GT). Results, in the form of tracked markers, from the challenge competitors (INRIA, IUCL, MEVIS, UPF) are also provided in the dataset.
Equilibrated warping was applied to all sets of 3DUS images. For all cases, a regularization strength of 0.1 was used, which was shown to be a good com- promise in [7]. For the sake of illustration, Figure 1 shows the result of the registration for V1. One can see the relatively good tracking of the ventricle despite the low image quality.
Early-systole End-systole End-diastole
Fig. 1. Sequence of 3DUS images (V1 case) with superimposed warped mesh (top), and sliced mesh in roughly short axis (middle) and long axis (bottom).
After successful computation of the displacement fields over the mesh at all time frames, the displacements were interpolated onto the markers, which were then warped as well. In order to quantitatively assess the quality of the registration, the following normalized error on the markers trajectory was used:
err = 1 nmarkers
nmarkers
X
m=1
Pnframes
f=1
Xm(f)−Xm,GT(f) Pnframes−1
f=1
Xm,GT(f+ 1)−Xm,GT(f)
, (10)
6 L. C. Lee and M. Genet
where nmarkers is the number of “valid” markers (i.e., markers that lie within the mesh in the reference configuration and can thus be tracked), nframes is the number of frames, Xm,GT(f) is the ground truth position of marker m at frame f, and Xm(f) is the tracked position of marker m at frame f. See [7]
for more details. A comparison of the normalized error between equilibrated warping and other challenge competitors is shown on Figure 2. One can draw similar conclusions as for SSFP & 3DTAG images: equilibrated warping performs as well, if not better, than established registration methods.
For the sake of completeness, tracked strain components are also shown, on Figure 3. Radial thickening, and circumferential and longitudinal shortenings are well captured, though underestimated, as already noted in [17].
Fig. 2.Normalized markers error, for 3DUS images and for all cases, as well as nor- malized markers error mean over all cases (last column). EW stands for Equilibrated Warping [7], while the other columns represent challenge competitors [17].
4 Conclusion
Equilibrated warping provides a good registration in three different types of images (i.e. tagged and untagged magnetic resonance images as illustrated in [7], and 3DUS images as presented here for the first time) with a normalized marker error that is comparable, if not better, than other established methods from the challenge competitors. Besides its ability to correctly register material points in medical images as we have shown here, the main advantage of having a mechanically-sound regularization in equilibrated warping ensures that the displacement and strain fields are proper physical fields that can be used to derive physiologically relevant biomarkers [21, 5].
Fig. 3.Radial, circumferential and longitudinal strain components tracked from 3DUS images, for all cases as well as the average value.
References
[1] M. Bornert et al. “Digital Image Correlation”. In:Full-Field Measurements and Identification in Solid Mechanics. Ed. by M. Gr´ediac et al. John Wiley
& Sons, Inc., 2012. doi:10.1002/9781118578469.ch6.
[2] R. Chabiniok et al. “Estimation of Tissue Contractility from Cardiac Cine- MRI Using a Biomechanical Heart Model”. In:Biomechanics and Modeling in Mechanobiology (2012).doi:10.1007/s10237-011-0337-8.
[3] G. E. Christensen et al. “Deformable Templates Using Large Deformation Kinematics”. In:IEEE transactions on image processing : a publication of the IEEE Signal Processing Society (1996).doi:10.1109/83.536892.
[4] D. Claire et al. “A Finite Element Formulation to Identify Damage Fields:
The Equilibrium Gap Method”. In: International Journal for Numerical Methods in Engineering (2004).doi:10.1002/nme.1057.
[5] H. Finsberg et al. “Efficient Estimation of Personalized Biventricular Me- chanical Function Employing Gradient-Based Optimization”. In:Interna- tional Journal for Numerical Methods in Biomedical Engineering (2018).
doi:10.1002/cnm.2982.
[6] M. Genet et al. “A Novel Method for Quantifying Smooth Regional Varia- tions in Myocardial Contractility Within an Infarcted Human Left Ventri- cle Based on Delay-Enhanced Magnetic Resonance Imaging”. In:Journal of Biomechanical Engineering (2015).doi:10.1115/1.4030667.
8 L. C. Lee and M. Genet
[7] M. Genet et al. “Equilibrated Warping: Finite Element Image Registration with Finite Strain Equilibrium Gap Regularization”. In: Medical Image Analysis (2018).doi:10.1016/j.media.2018.07.007.
[8] G. A. Holzapfel. Nonlinear Solid Mechanics: A Continuum Approach for Engineering. Chichester: Wiley, 2000.
[9] A. Krishnamurthy et al. “Patient-Specific Models of Cardiac Biomechan- ics”. In: Journal of computational physics (2013).doi:10.1016/j.jcp.
2012.09.015.
[10] H. Leclerc et al. “Integrated Digital Image Correlation for the Identifica- tion of Mechanical Properties”. In: Computer Vision/Computer Graphics Collaboration Techniques. Ed. by A. Gagalowicz et al. Berlin, Heidelberg:
Springer Berlin Heidelberg, 2009.doi:10.1007/978-3-642-01811-4_15.
[11] A. Logg et al., eds.Automated Solution of Differential Equations by the Fi- nite Element Method: The FEniCS Book. Lecture Notes in Computational Science and Engineering. Heidelberg: Springer, 2012. 723 pp.
[12] T. Mansi et al. “iLogDemons: A Demons-Based Registration Algorithm for Tracking Incompressible Elastic Biological Tissues”. In:International Journal of Computer Vision (2011).doi:10.1007/s11263-010-0405-z.
[13] P. Moireau et al. “Joint State and Parameter Estimation for Distributed Mechanical Systems”. In: Computer Methods in Applied Mechanics and Engineering (2008).doi:10.1016/j.cma.2007.08.021.
[14] M. K. Rausch et al. “A Virtual Sizing Tool for Mitral Valve Annuloplasty”.
In:International Journal for Numerical Methods in Biomedical Engineer- ing (2017).doi: 10.1002/cnm.2788.
[15] W. Schroeder et al. The Visualization Toolkit: An Object-Oriented Ap- proach to 3D Graphics. 4. ed. Clifton Park, NY: Kitware, Inc, 2006. 512 pp.
[16] M. Sermesant et al. “Patient-Specific Electromechanical Models of the Heart for the Prediction of Pacing Acute Effects in CRT: A Preliminary Clinical Validation.” In:Medical Image Analysis (2012).doi:10.1016/j.
media.2011.07.003.
[17] C. Tobon-Gomez et al. “Benchmarking Framework for Myocardial Track- ing and Deformation Algorithms: An Open Access Database”. In:Medical Image Analysis (2013).doi:10.1016/j.media.2013.03.008.
[18] A. I. Veress et al. “Measurement of Strain in the Left Ventricle during Diastole with Cine-MRI and Deformable Image Registration”. In:Journal of Biomechanical Engineering (2005).doi:10.1115/1.2073677.
[19] H. Wang et al. “Cardiac Motion and Deformation Recovery from MRI:
A Review”. In: IEEE transactions on medical imaging (2012).doi: 10.
1109/TMI.2011.2171706.
[20] C. Xi et al. “Patient-Specific Computational Analysis of Ventricular Me- chanics in Pulmonary Arterial Hypertension”. In: Journal of Biomechan- ical Engineering (2016).doi:10.1115/1.4034559.
[21] H. Zou et al. “Quantification of Biventricular Strains in Heart Failure With Preserved Ejection Fraction Patient Using Hyperelastic Warping Method”.
In:Frontiers in Physiology (2018).doi:10.3389/fphys.2018.01295.
