Patch-based field-of-view matching in multi-modal images for electroporation-based ablations

HAL Id: hal-02868718

https://hal.archives-ouvertes.fr/hal-02868718

Submitted on 15 Jun 2020

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of

sci-entific research documents, whether they are

pub-lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destinée au dépôt et à la diffusion de documents

scientifiques de niveau recherche, publiés ou non,

émanant des établissements d’enseignement et de

recherche français ou étrangers, des laboratoires

publics ou privés.

images for electroporation-based ablations

Luc Lafitte, Rémi Giraud, Cornel Zachiu, Mario Ries, Olivier Sutter, Antoine

Petit, Olivier Seror, Clair Poignard, Baudouin Denis de Senneville

To cite this version:

Luc Lafitte, Rémi Giraud, Cornel Zachiu, Mario Ries, Olivier Sutter, et al.. Patch-based field-of-view

matching in multi-modal images for electroporation-based ablations. Computerized Medical Imaging

and Graphics, Elsevier, 2020, 84, �10.1016/j.compmedimag.2020.101750�. �hal-02868718�

ele troporation-based ablations L Latte a , R Giraud b , CZa hiu , M Ries d , O Sutter e,f , A Petit e,f , O Seror e,f , C Poignard a

and B Denis de Senneville a, ,

∗

a

UniversityofBordeaux,IMB,UMRCNRS5251,INRIAProje tteamMon ,Talen e,Fran e,F-33405Talen eCedex, Fran e

b

UniversityofBordeaux,IMS,CNRSUMR5218,F-33405 Talen eCedex,Fran e

DepartmentofRadiotherapy,UMCUtre ht,Heidelberglaan100,3584CX,Utre ht,TheNetherlands d

ImagingDivision,UMCUtre ht,Heidelberglaan100,3584CX,Utre ht,TheNetherlands e

Interventional radiologyunit,HpitauxUniversitairesParisSeineSaintDenis,HpitalAvi enne,Assistan ePublique HpitauxdeParis,BobignyFran e

f

UniversityofParis13,S ien esMédi aleetBiologieHumaine,Bobigny,Fran e

ARTICLE INFO Keywords:

Multi-modalimageregistration pat h-basedmat hing

interventionalpro edures

ABSTRACT

Variousmulti-modalimagingsensorsare urrentlyinvolvedatdierentstepsofan interventionaltherapeuti work-ow.Conebeam omputedtomography(CBCT), omputedtomography(CT)orMagneti Resonan e(MR)imagestherebyprovides omplementaryfun tionaland/orstru turalinformationofthetargetedregionand organs atrisk. Merging thisinformationrelies ona orre tspatialalignmentof the observed anatomy between the a quired images. This an be a hieved by the meansofmulti-modaldeformableimageregistration (DIR),demonstrated to be apableofestimatingdenseandelasti deformationsbetweenimagesa quired bymultipleimagingdevi es. However,duetothetypi allydierenteld-of-view (FOV)sampleda rossthevariousimagingmodalities,su halgorithmsmayseverely failinndingasatisfa torysolution.

Inthe urrentstudyweproposeanewfastmethodtoaligntheFOVin multi-modal3Dmedi alimages. Tothisend,apat h-basedapproa hisintrodu edand ombinedwitha state-of-the-artmulti-modalimagesimilaritymetri inorderto opewithmulti-modalmedi alimages. Theo urren eofestimatedpat hshiftsis omputedforea hspatialdire tionandtheshiftvaluewithmaximumo urren e is sele tedand usedto adjust the imageeld-of-view. Theperforman eof the proposed method  in terms of both registration a ura y and omputational needs  isanalyzed in the pra ti al ase of on-line irreversible ele troporation pro edures. Intotal,30pairsofpre-/per-operativeIREimagesare onsideredto illustratethee ien yofouralgorithm.

Weshowthataregionalregistration approa husingvoxelpat hesprovidesa goodstru tural ompromisebetweenthevoxel-wiseandglobalshiftsapproa hes. hemethodwastherebybene ialforCTtoCBCTandMRItoCBCTregistration tasks, espe ially when highly dierent image FOVs are involved. Besides, the benet of the method for CT to CBCTand MRI to CBCTimage registration is analyzed,in luding the impa t of artifa ts generated by per utaneous needle insertions. Additionally,the omputationalneedsusing ommodityhardwareare demonstratedtobe ompatiblewith lini al onstraintsinthepra ti al aseof on-linepro edures. Theproposedpat h-basedworkowthusrepresentsanattra tive assetforDIRatdierentstagesofaninterventionalpro edure.

⋆

Experimentspresentedinthispaperwere arriedoutusingthePlaFRIMexperimentaltestbed,supportedbyInria,CNRS (LABRIandIMB),UniversitédeBordeaux, BordeauxINPandConseilRégionald'Aquitaine(seehttps://www.plafrim.fr/). The authors thank the Laboratory of Ex ellen e TRAILANR-10-LABX-57for funding. This studyhas been arried out withthe nan ial supportof the Fren h NationalResear hAgen y(ANR) inthe frameofthe Investmentsfor thefuture ProgrammeIdExBordeaux-CPU (ANR-10-IDEX-03-02). Thisresear hhasbeenpartlygranted bythe PlanCan erproje t NUMEP(Inserm11099),ledbyC.P.

∗Corresponding

author or id(s):

1. Introdu tion

Multiple imaging devi es an be involved at dierent stages of an interventional pro edure, su h as image-guidedradiotherapy(IGRT)[9℄, irreversibleele troporation(IRE)[7℄orhyperthermiaablation [12℄ [18℄. In parti ular, one-beam omputed tomography (CBCT), omputed tomography(CT) or Magneti Resonan e(MR) images are re ently beingemployed at nearlyall stages of the therapy: i.e., pre-, intra-andpost-operatively. Oneofthebenetsthatemployingmultipleimagingsensorsprovides,istheabilityto extra t omplementaryfun tionaland/or stru turalinformation ofthetargeted regionand organs-at-risk. For example, as shown in [11℄, novel diagnosti indi ators an thereby be al ulated by fusing pre- and post-operativeimagedata. Similarly,multi-modalimagingmayalsobebene ialduring theinterventional pro edureitself[26℄. Ofnoteisthatthequalityandtheamountofdatathat anbea quiredintra-operatively isgenerallylimitedbypra ti al lini al onsiderations:CBCTguidan eisforexampleparti ularlybene ial duetothelowamountofimaging-relatedradiationdeliveredtothepatient omparedtoa onventionalCT s an. However, this often leads to the intra-operative images being subje t to low ontrast, low signal-to-noise ratio and artifa ts. Thus, it would be of lini al benet if su h images would be augmented by pre-operativedata[7℄. A ommon pre-requisiteis that organlo ationsmust beset in a ommon frameof referen e. To thisend, previousstudies propose severalmulti-modal deformableimage registration (DIR) algorithms dedi ated tothe estimationof denseand elasti deformationsbetweenimages [10,20, 6℄. This remainsa hallengingtasksin esu halgorithmshavetobefast(tomeet lini allya eptabledurations)and automati (theusemustnotbelimitedtoa ase-by- asebasisandamanualre alibrationisnotpreferable), espe iallywhenthepatientisontheinterventionaltable [21℄[27℄.

Aparti ular hallenge ariseswhenhighlydierentelds-of-view(FOV)aresampledwithin theimages. For example, while the FOV within intra-operative CBCT images is typi ally restri ted to the targeted organanditsimmediatesurroundings,the orrespondingpre-operativehigh-resolutionCTimagegenerally overstheentireabdomenandpartofthethorax. Asimilarsituationariseswhenapatientiss reenedvia both CT/CBCT and MR imaging, with the resulting a quisitions typi ally having onsiderably dierent FOVs. This anseverelyhampertheperforman eofimageregistrationalgorithms,espe iallywheniterative optimizationstrategiesareemployed: thealgorithmislikelytogettrappedintolo aloptimaiftheapparent lo ationoftheanatomy-of-interestistoofarapartwithinthetwoimages. Insu ha ase,adire temployment ofDIRmethodsmaybehardlyfeasibleandapreliminarymat hing oftheimageFOVs(i.e., ompensation ofthe3Dglobalshiftbetweenimages)isne essary.

While registration solutions optimizing a translational model may perform well for estimating rigid displa ements,theymayalsobe omesub-optimalwhenelasti deformationsarepresentbetweentheimages. Moreover,su h methods typi ally implyhigh omputational demands andmanual tuning of severalinput parameters,whi h limitstheiruseina lini alsetting[16℄.

Alternatively,aregionalregistrationapproa husingpixel/voxelpat hes,mayprovideagoodstru tural ompromisebetween thevoxel-wiseand global shifts approa hes. Several pat h orblo k-mat hing algo-rithms havebeenpreviouslyproposed,dedi atedtovarious appli ations[15℄. Theaim oftheseapproa hes isto onsiderea h pixelbyitssquareneighborhood,to hara terizeitslo al ontext. Mat hingalgorithms may then beused to nd lo al orresponden es between images. Nevertheless, these methods are highly time onsuming, espe ially when dealing with an important number of pat hes and when the sear h for orresponden esmustbeperformedinalargewindowsear h(i.e.,sear hingforlargepat hdispla ements). A signi ant breakthrough has been obtained with the so- alled Pat hMat h algorithm [3℄, whi h was initially proposedfor ndingpixel pat h orresponden esbetween2Dimages in digitalphotography. The ideabehindthis approa his that somegoodpat h mat hes anbefoundbyrandom sampling,whi h an subsequentlybeallo atedtosurroundingareasaswell,relyingontheassumptionthatneighboringareas typ-i allyhavesimilardispla ements. Thefast onvergen eofthepro essenablestoqui klyndgoodmat hes, evenwhenthese arelo atedfar fromea hother in theimagespa es. This approa hhasbeensu essfully employedto a hievenumerous imageanalysis and editing tasks su h as: stereomat hing [4℄, opti al ow omputation[2℄,regioninpainting[19℄,or3Dmedi alimage segmentation[8℄.

Inthe urrentstudy,our ontributionisfour-fold:

1. Anewfastmethod usingasastarting-pointa3DmodiedPat hMat halgorithm isproposed toaligntheFOVin medi al3Dimages. A user-denedmask surroundingaregion/organofinterest

in oneof theimages an beprovided as aninput sothat a globalshift anbeestimated relyingon imageinformationfromthisspe i region.

2. The modied Pat hMat h algorithm is ombined with a well-adapted multi-modal image similarity metri inorder to opewithmulti-modalmedi alimages.

3. Theperforman e  in terms of registration a ura y and omputational needs  is analyzed, and demonstrated to be ompatible with lini al onstraints in the pra ti al ase of on-line irreversible ele troporation pro edures. In total, 30 pairs of pre-/per- operative IRE images are onsidered to illustratethee ien y ofouralgorithm.

4. The benet of proposed approa h is evaluated for a potential pre- onditioning of a more omplex multi-modalDIRalgorithm.

2. Materials and Methods 2.1. Proposedmethod

Let

I

and

J

betwo3Dimages. Inthes opeofthis study,let

I

and

J

beapre-andanintra-operative image, respe tively. We seek X-, Y- and Z- translation omponents between

I

and

J

in order to mat h the position anorgan ofinterestmanually delineated in

I

. We re allthat theestimation of elasti organ deformationsbyitselfisoutsidethes opeofthestudy: theproposedworkowissolelyintendtostandardize eld-of-view in multi-modal images for a potential pre- onditioning a more omplex multi-modal elasti registrationalgorithm.

Theproposedmethod(detailed inFigure1)in ludes thefollowingthreemainsu essivesteps:

1. The Pat hMat h (PM) algorithm is adapted and ombined with a multi-modal metri in order to omputepat h orresponden esbetween

I

and

J

(seese tion2.1.3). Themulti-modalmetri aimsat evaluatingedgealignments(EA)withinpat hes.

2. Theo urren eofestimatedpat hshifts,i.e., thedispla ementbetweenthepat hpositions in

I

and their orresponden ein

J

,is omputedforea hspatial dire tion(seese tion2.1.4).

3. Forea hspatialdire tion,theshiftvaluewithmaximumo urren eissele tedandusedtoadjustthe imageeld-of-view(seese tion2.1.5).

The proposed method is referred to asPM-EA (Pat hMat h-Edge Alignment) in the s ope of this study.

Figure1: Datapro essingsequen edesignedforthefaststandardizationofeld-of-viewinmulti-modalimagesusingthe proposedpat h-basedframework.

2.1.1. Manual delineation ofthe organof interest.

Thepre-operativeimage

I

isrst used to manually segmentthe targeted regionof interest. A binary mask(denotedby

M

)is onstru ted. Voxelsoftheimageinside themaskhaveavalueofone,andoutside avalueofzero. Weunderlinethat,in thes opeofthisstudy,thispro esswasdoneusingthepre-operative image

I

only,inordertodemonstratethatthemethodis ompatiblewithanautomati useduringan intra-operativesession. Notethatthisdelineation stepisoftenperformedanywayduring theplanningsessionof thetherapyandthusdoesnotputextraburdenonthemedi alsta.

2.1.2. Prepro essing of inputdata.

I

,

J

and

M

wereresampledontoa ommongridwithavoxelsizeof

1 × 1 × 1

millimetersusingatrilinear interpolation.

2.1.3. ImplementedPat hMat h algorithm.

Pat hMat his aniterativealgorithm designedto qui klyestimate pat h orresponden esbetweentwo given2D images[3℄. Inour study, werst extendthis algorithm forthe mat hing of3D pat hes. Hen e, a pat h onsists in a ubi subset of the image domain, denoted by

Γ

, entered on one singlevoxel. Let

~

r

= (x, y, z) ∈ Ω

be the spatial lo ation of the enter voxel,

Ω

the image domain and

(x, y, z)

the voxel oordinates.

ˆ Initialization.Aninitialguessisrst omputed: ea hpat hfromimage

I

isinitiallyrandomlymat hed withapat h fromimage

J

. Subsequently, atea hiterationof Pat hMat h, voxelsare s annedfrom leftto right(X-axis), head to foot (Y-axis), frontto ba k (Z-axis). Forea hvoxelexamination, the orrespondingpat hundergoesapropagation step followedbyarandomsear h step,asdes ribed inthe seminalpaper[3℄: Theoutputofouralgorithm isapat hshift map

V

~

, dened forea hvoxel in

Ω

.

ˆ Propagation step. During this step, in order to nd better orresponden es, the pat h shift of the urrentvoxel

~

r

in

I

in

I

is onsideredtobesimilartotheonesofitsthreealreadyexaminedneighboors in ea h dire tion (6- onnexity)(i.e., the three voxelsat lo ations

~

r

(−1,0,0)

= (x − 1, y, z)

,

~

r

(0,−1,0)

=

(x, y − 1, z)

and

~

r

(0,0,−1)

= (x, y, z − 1)

).

Let

V

~

= (u, v, w)

bethepat hshiftsthatweseek(

(u, v, w)

beingthevoxelwisepat hshift oordinates). Let

~

r

1

and

~

r

2

betwogivenspatial lo ationin

Ω

and

D(~r

1

, ~

V

(~r

2

))

thedistan ebetweenthepat h at lo ation

r

~

1

in

I

andthepat hatlo ation

r

~

1

+ ~

V

(~r

2

)

in

J

.

~

V

(~r)

isupdatedasfollows:

~

V

(~r) = argmin

_V

~

{D(~r, ~

V

(~r)), D(~r, ~

V

(~r

(−1,0,0)

)), D(~r, ~

V

(~r

(0,−1,0)

)), D(~r, ~

V

(~r

(0,0,−1)

))}

(1)

ˆ Randomsear hstep. Thisstepattemptstoimprove

~

V

(~r)

by omputingasetof andidateshifts(noted

~

V

i

(~r)

)atanexponentiallyde reasingspatial distan efrom

V

~

(~r)

:

~

V

i

(~r) = ~

V

(~r) + wα

i

R

i

(2)

R

i

beingauniformrandom in

[−1, 1]

3

,

w

themaximumsear h distan e(set to themaximumimage dimension),and

α

axed ratio betweensear h windowsizes (wetook

α

= 0.5

, assuggestedin [3℄). Pat hesfor

i

=

0,1,2andsoon areexamineduntilthe urrentsear h distan e

wα

i

falls belowone voxel.

As reported in Barnes et al seminal paper, Pat hMat h provides satisfa tory results using a xed number of iterations (5 iterations max) and that the algorithm onvergesmost rapidly in the rst iterations[3℄. Inthe urrentstudyweusedtwoiterations,sin eitwasfoundtobeagood ompromise betweena ura yand omputational osts.

Inourimplementation, theinput imagesare down-sampled before Pat hMat h: whilelower ompu-tationtimesareexpe ted usingdown-sampled versions ofinputimages,it shouldalsoimpa toverall registrationresults. Thedown-samplingfa toristhusanimportantinputparameterforthealgorithm anditsimpa twill be arrefullyanalysedanddis ussedbelow.

To further redu e the omputational burden, the sear h window were a re tangular bounding box in ludingboth

J

andvoxelswith avalueof onein

M

.

ˆ Aggregationof multiple PM estimations. As for exemplar-based segmentation [8℄, our method an benet frommultiple PMestimations. Pat h-shiftestimates indeed relyonrandom andidate sele -tionandseveralindependentpro essesmayprovidedierent orresponden es. AlthoughPat hMat h inherently relies on serial operations and annot benet from parallel ar hite tures in its urrent form,multiple realisationsof Pat hMat h aneasily be al ulatedusing separate CPUthreads. Let

V

j

_{(~r) = (u}

j

_{(~r), v}

j

_{(~r), w}

j

_(~r))

beonerealisationatspatiallo ation

~

r

,

j

beingtherealisationindex. For ea hvoxel,theobtained3Dshiftrealisationswerethen ombinedintoasingle3Dmedianve tor

V

(~r)

[1℄asfollows:

V

(~r) = argmin

V

(~

r)∈{V

j

_(~

_r)}





X

j



V

(~r) − V

j

(~r)









(3)

Inthis manner, outliersrealisationsweredis ardedwithout any additionalpenaltyonthe omputa-tionalburden.

2.1.4. Proposed multi-modal metri .

IntheoriginalPat hMat hpaper,thedistan ebetweenpat hes

D(.)

istheSumofSquaredDieren es (SSD, also referred to as L2-norm) applied on the image intensity. While for mono-modal registration algorithmstheSSDapplieddire tlyontheimagesmightbesu ient[13℄[5℄, su hameasureisunsuitable forregisteringa rossmodalities. Amodalityindependentsimilaritymeasureisthusne essary. Inthe urrent paperweusedanexistingmulti-modalmetri whi hfavorsedgealignments(EA)in bothpat hes[14℄[24℄.

Let

~

∇

I

and

~

∇

J

bethegradientofthereferen eimage

I

andtheimageto register

J

, respe tively. The distan e

D(~r

1

, ~

V

(~r

2

))

betweenapat hofinterest

Γ

in

I

( enteredonthevoxello atedin

~

r

1

)anditspotential orresponden ein

J

( enteredonthevoxello atedin

~

r

1

+ ~

V

(~r

2

)

)wasdened asfollows:

D(~r

1

, ~

V

(~r

2

)) = −

R

Γ







∇

~

I

(~r

1

) · ~

∇

J



~

r

1

+ ~

V

(~r

2

)





 d

~

r

R

Γ

∇

~

I

(~r

1

)

2

∇

~

J



~

r

1

+ ~

V

(~r

2

)



2

d~r

(4)

where

k · k

2

istheEu lideannorm.

Pra ti ally,thes alarprodu tinthenumeratorismaximizedwhentheedgesin

Γ

arealignedwithedges in the potential orresponding pat h in

J

. Note that the numeratoris maximized regardlessany possible ontrastreversals: duetotheabsolutevalue,thenumeratorismaximizedforbothparallelandanti-parallel edges. Inaddition,thes alarprodu tinthenumeratorfavorsstrongedgespresentinbothmodalities. The denominator,foritspart,a tsasanormalisationfa tor. Ultimately,sin eaminimization of

D

is required to omputepat h orresponden esinEq. (1),anegativesignhasbeensetbehingthefra tionalterminEq. (4).

2.1.5. Mat hingimageFOVs from estimated pat h orresponden es.

Atthispointwehaveavoxelwise3Dshiftmaps

V

~

(~r)

,

~

r

∈ Ω

. Theobje tiveisnowto simpliy

V

~

down to asingle3Dimageshift. Forthis purpose,weindividually analysedshifto urren esin ea h omponent of

~

ahistogramwas al ulatedandtheshiftvaluewiththehighesto urren ewassele ted. Theobtained3D imageshiftwassubsequentlyused toadjusttheFOVof

J

withrespe tto theoneof

I

.

Note that lower and upper bounds on estimated shift values aswell asa numberof binsneeds to be determined forthe onstru tion ofthehistograms. The lowerand upperbounds were setto -100and 100 millimeters, respe tively, whi h wasfound to be su ient in our tests. The number of bins is an input parameterforthealgorithmthat willbeanalysedbelow.

2.2. Experimental evaluation 2.2.1. Datasets.

For the evaluation of the method we used data a quired during IRE pro edures whi h are routinely performedattheUniversityHospitalJeanVerdieratBondyinFran e. Thisretrospe tivestudyisin a or-dan e withethi al prin ipalsoftheDe larationofHelsinki andhasbeenapprovedby thelo al ommittee onhumanresear hoftheUniversityHospitalJVerdier.

The lini alworkowin ludedthefollowingsessions:

ˆ Pre-operativesession. Thissession,performedseveraldaysbeforetheinterventionalpro edure,allowed identifyingthetumorandthemainliverstru turesusingeitheraCT-s an(voxelsize=

[0.67 − 0.88] ×

[0.67 − 0.88] × [1.25 − 2]

mm

3

, FOV=

[341 − 450] × [341 − 450] × [182 − 506]

mm

3

) or a MR-s an (T1-weighting,voxelsize=

1.72 × 1.72 × [2.5 − 3]

mm

3

,FOV=

440 × 440 × [180 − 200]

mm

3

).

ˆ Interventional session. The day of the pro edure, an IRE ablation was performed under general anesthesia. Theneedlesareper utaneouslyinsertedaroundthetumorbytheinterventionalradiologist with a free-hand te hniqueunder ombination of real-time ultrasound (US) and 3D Virtual Target Fluoros opi Display su h that the ele tri eld overs the target region [25℄. A 3D CBCT (voxel size=

0.45 × 0.45 × 0.45

mm

3

,FOV=

230 × 230 × [192 − 256]

mm

3

)imagingwasperformedtovisualise liverandneedlelo ations. Duringtheinterventionalsession,aregistrationwithpre-operativedatais intendtoaugmenttheCBCTwithtumor/liverstru turessegmentations. Thepursuedobje tiveisto improvethetargetingandtoallowdosemodeling,asdes ribedin [7℄.

Intotal,weanalysedaset of30pairsofpre-/intra-operativeimagesdistributed overthefourfollowing groups:

1. 8pairsofCT/CBCTimagesobtainedon8patients,respe tively. CBCTswerea quiredbeforeneedle insertion.

2. 8pairsof CT/CBCT images. Patients andCTswere those used in (i). CBCTswere a quired after needleinsertion.

3. 7pairsofMR/CBCTimagesobtainedon8patients,respe tively. CBCTswerea quiredbeforeneedle insertion.

4. 7pairsofMR/CBCTimages. PatientsandMRswerethoseusedin(iii). CBCTswerea quired after needleinsertion.

2.2.2. Performan eassessment.

TheDi eSimilarityCoe ient(DSC)wasemployedtodeterminethe ontouroverlapoftheliver:

DSC =

2 |A ∩ B|

|A| + |B|

(5)

where

A

and

B

are two manually dened ROIs en ompassingthe liverin the referen eand the orre ted image, respe tively.

A

∩ B

is theirinterse tion and

|·|

denotes the ardinalityofaset (i.e., thenumberof voxels).

DSC

mean and standard deviation were omputed over the 4 sets of image pairs individually (i.e., CT/CBCTnoneedle,CT/CBCTneedles,MR/CBCTnoneedle,MR/CBCTneedles,asdened inse tion

2.2.1). A Wil oxonpairedtest was arriedout inorder to study whether

DSC

dieren esare statisti ally

2.2.3. Calibration of theproposedPM-EA algorithm.

Atthispointitisimportanttounderlinethatfourmaininputparametersmayinuen etheperforman e of the proposed approa h. The proposed PM-EA algorithm was hallengedagainst various modi ations applied tothese alibrationparameters:

Inputparameter

#1

: down-samplingofinputimages(se tion 2.1.3).

DSC

and omputationtimewere al ulatedusing

I

,

J

and

M

atoriginalimagedimensionandusingdown-sampling(DS)fa tors

2×

,

4×

and

8×

in ea hspatial dire tion. Adefaultdown-samplingfa torof

8×

wasused.

Input parameter

#2

: pat h size (se tion 2.1.4).

DSC

and omputation time were al ulated using pat hesofsize

3 × 3 × 3

,

5 × 5 × 5

,

7 × 7 × 7

,

9 × 9 × 9

and

11 × 11 × 11

voxels. Adefaultsize of

9 × 9 × 9

voxelswasused.

Input parameter

#3

: number of histogram bins (se tion 2.1.5).

DSC

were al ulated for numberof histogrambinsof10, 30,50,70 and90. The omputationtime wasnotevaluatedsin eamarginalimpa t isexpe ted here. Adefaultvalueof50wasused.

Input parameter

#4

: manual delineation errors of the targeted organ (se tion 2.1.1). To analyse potentialerrorsarisingfrom themanualdelineationpro ess,

M

wasiterativelyeroded(resp. dilated)using a

5 × 5 × 5

kernel. Atea hiteration,thevolumeoftheeroded(resp. dilated)maskwas al ulatedaswellas the orrespondingDSC afterimage alignement. Hereagain, the omputation timewasnotevaluatedsin e amarginalimpa tisexpe ted.

2.2.4. Tested algorithms.

The PM-EA's ability to estimate aglobal 3D translation between images was hallengedagainst two ompeting approa hes. We also evaluated the benetof using PM-EA as astartingpointfor an existing more omplex multi-modal elasti registration algorithm. The above-mentioned 30 pairs of images were pro essedusingPM-EA andusingthefollowingsele tionofimageregistrationsolutions:

Elastix. Wehavesele tedin theOpensour eElastixregistrationsoftware[16℄[22℄ aregistrationsolution whi h employs a3Dtranslation transformationmodeland whi h maximizes thenormalizedmutual infor-mation[23℄betweentheimagestoberegistered. ThisregistrationsolutionisreferredtoasElastix inthe following.

PM-L2. The proposed Pat hMat h registration framework is here employed using a L2-norm for pat h omparison, asitwasintrodu edin theoriginal paper[3℄. Thisregistrationsolutionisreferredto as PM-L2 hereafter.

Evo. TheEVolutionalgorithm(whi hisabbreviatedasEvo inthes opeofthisstudy)wasemployedto estimatetheelasti deformation

V

~

astheminimizerofthefollowingenergy

E

:

E(~

V

) =

Z

Ω

exp(D(~

V

)) +

α

2



k ~

∇u k

2

2

+ k ~

∇v k

2

2

+ k ~

∇w k

2

2



d~r,

(6)

D(~

V

)

being the multi-modal metri of Eq. (4) and

α

a weighting fa tor designed to link both the data delityterm(left partof Eq. (6))andthemotioneld regularity(rightpartofEq. (6)). Notethat

D(~

V

)

is omposedwithanexponentialfun tioninordertoensurethatthedatadelitytermisapositive-denite fun tion. Additionaldetails on erningthemannerinwhi htheEvofun tionalisminimizedtogetherwith adetailedanalysisofthealgorithmperforman e anbefoundin[6℄. Anumeri alimplementationdesigned fortheredu tionof omputational osts anbefoundin [17℄.

PM-EA+Evo. Theabove-mentionedmulti-modalregistrationalgorithm Evois hereemployedafterFOV standardizationusingproposedPM-EAalgorithm. Theregistrationworkow, omprisedofPM-EAfollowed byEvo,isreferredtoasPM-EA+Evothroughouttherestofthemanus ript.

Ea hofElastix,PM-L2andPM-EAaimstoestimateaglobal3Dtranslationbetween

I

and

J

withinthe shortestpossibletime. A ommonfa torof

8×

in ea h spatialdire tion (i.e. thedefault valueforPM-EA givenin se tion2.2.3)wasusedtodown-sampletheinputdata(i.e.,

I

,

J

and

M

).

Ontheother hand,elasti registrationisknownto bea omplextask. Using Evo,adown-samplingof input data (i.e.,

I

,

J

and

M

) bya fa tor

4×

wasused in order to maintain registration a ura y of the outputsand omputationtimes ompatible withour lini al onstraints(below30se onds).

2.3. Hardware and implementation

OurtestplatformwasanIntel2.5GHzi7workstation(8 ores)with32GBofRAM.Theimplementation wasperformedin C++andparallelizedthroughmulti-threading(onethread per ore).

3. Results

Figure 2 provides a visual assessment of hallenges arising from the use of CBCT during the intra-operative IRE session. Middle transversal, sagittal and oronal sli es are illustrated for CBCT images a quired onthesamepatient before andafter insertionof4needles. First,onlyvoxels ontainedwithin a ylinderhavenon-zerovalues: apartial ir ularFOVinthetransveralplaneisobservableintherst olumn (see2aand2d). Inaddition,alow ontrast-to-noiseratioisobservableonbothimages. Also,anin reasing amountofstreakingartifa tsisobservableafterneedleinsertions.

AnexampleofCT/CBCTandMR/CBCTregistrationresultsareshowningures3and4,respe tively. Forboth ases,the CBCTimage (rst olumn) wasa quiredintra-operativelyafter needleinsertionsand wasemployedasareferen eforimageregistration. Thepre-operativeimageisdisplayedbeforeregistration (se ond olumn), after PM-EA (third olumn) and after PM-EA+Evo (fourth olumn). The o urren e of pat h shifts is reported for ea h spatial dire tion in panels (mo): for ea h histogram, the shift with maximal o urren eis shownby thered dashedline. Forpanels(al), aROI manually dened onthe CBCTimage/en ompassingtheliverisshownusingreddashlines. Ourvisualizationshowsanimproved orresponden e of the ontour of the liver with the manually dened liver boundary when the PM-EA solutionisemployed(see3( ,g,k)and4( ,g,k)). Moreover,anevenbetter orresponden e ofthe ontouris observableusingthePM-EA+Evosolution(see3(d,h,l)and4(d,h,l)).

Figure5analyzesthe sensitivityto thedown-samplingparameter(i.e., theinputparameter

#1

ofthe proposed PM-EA method, see se tion 2.2.3). A great DSC improvement together with a huge speed-up of thealgorithmwasobtainedforin reasingdown-samplingfa tors. This tendan ywasobservedforboth CT/CBCT(5a)andMR/CBCT(5b). Bestresultswereobtainedusingthedefaultdown-samplingfa torof

8×

. Foralltested s enarios,dieren esbetweenDSC obtainedbeforeandafter needleinsertionswere not statisti ally signi ant(

p

≥ 0.2

forCT/CBCT,

p

≥ 0.3

forMR/CBCT).

Figure6analyzestheimpa tofthe pat h size (i.e., theinputparameter

#2

ofPM-EA). An improved registration a ura y with minimal losses in terms of omputation time (several tenth of se onds) was obtainedforanin reasingpat hsize. Thistendan ywasobservedforbothCT/CBCT(6a)andMR/CBCT (6b). Best resultswereobtainedusing the defaultpat h size of

9 × 9 × 9

voxels. Forall tested s enarios, dieren esbetweenDSCobtainedbeforeandafterneedleinsertionswerenotstatisti allysigni ant(

p

≥ 0.2

forCT/CBCT,

p

≥ 0.22

forMR/CBCT).

Dieren esbetweenDSC for ea h pairof tested numbersof histogram bins (i.e., the input parameter

#3

of PM-EA)werenotstatisti ally signi ant(

p

≥ 0.08

for bothCT/CBCTandMR/CBCT)(see gure

7). This wasobservable for bothCT/CBCT (7a) and MR/CBCT (7b). Here again, dieren es between DSC obtainedbefore andafter needleinsertionswerenotstatisti allysigni ant(

p

≥ 0.11

forCT/CBCT,

p

≥ 0.16

forMR/CBCT).

Figure8analyzesthesensitivityoftheproposedPM-EAalgorithmagainstmanualdelineationerrorsof theorganofinterest(i.e., theinputparameter

#4

ofPM-EA).A signi antnegativeimpa tisobservable for eroded versions of

M

, espe ially for MR/CBCT (8b) (

p

≤ 0.01

). Using dilated versions of

M

, DSC dieren eswerenotstatisti allysigni ant(

p

≥ 0.3

). Hereagain,dieren esbetweenDSCobtainedbefore andafterneedleinsertionswerenotstatisti allysigni ant(

p

≥ 0.46

forCT/CBCT,

p

≥ 0.3

forMR/CBCT). Figure 9 omparesthe registration a ura y obtainedusing alltested algorithms (see se tion 2.2.4for details). Regarding solutions designed to estimate the global 3D shift of the liver (i.e., Elastix, PM-L2

Beforeneedle insertion (a) (b) ( ) Afterneedle insertion (d) (e) (f)

Figure2:Typi alCBCTimagesobtainedduringanIREpro edure. Comparedtotheimagea quiredbeforeneedleinsertion (toprow),theimagea quiredaftertheinsertionoffourneedles(bottomrow)isvisiblyalteredbystreakingartifa ts. The latterintrodu eintensityvariationswhi hobstru tsanddegradesnerdetailsoftheanatomy. ThepartialimageFOVis also observable: onlydata ontainedwithina ylinder areavailable(seethe ir ularFOV inthetransveralplaneinthe rst olumn).

and PM-EA), best results where a hieved using the proposed PM-EA approa h forboth CT/CBCT (9a) and MR/CBCT (9b). PM-EA outperformed signi antly a standard registration strategy implemented using the Elastixtoolbox(

p

= 0.02

). The use of theproposed multi-modalmetri improvedsigni antly the DSC obtained using the L2-norm used in the original Pat hMat h paper [3℄ (

p

= 0.01

). Regarding solutionsdesignedto estimatetheelasti deformationoftheliver(i.e., EvoandPM-EA+Evo), theuseof PM-EAimprovedsigni antlytheperforman eofthetestedmulti-modalelasti registrationalgorithmEvo (

p

= 0.01

). For all tested solutions, dieren esbetween DSC obtainedbefore and after needle insertions werenotstatisti allysigni ant(

p

≥ 0.2

forCT/CBCT,

p

≥ 0.08

forMR/CBCT).It isinterestingtonote that the omputationaldemand remainedherebelow30se ondsforthesu essivea hievementofPM-EA andEvoalgorithmswiththeusedhardwareforbothCT/CBCTandMR/CBCTpairs.

4. Dis ussion

In the urrent study, we designed anovel method to estimate the global 3D translation between two multi-modalimages. Weretrospe tivelyevaluatetheproposedPM-EA algorithmunder arealisti lini al s enarios. Wefo usonthespe i interventionalpro edureofirreversibleele troporation(IRE)ablationfor livertumors. IRE te hniqueprovidesaninterestingalternativetostandardablativete hniques,espe ially for tumor lo ated near vital stru tures as detailed in [7℄. Moreover, it gathers the main omputational hallenges in terms of medi al image registrations, that haveto be addressedto improve the pro edures. Indeed,thepro eduresrelyuponmultimodalmedi alimaging: preoperativeCT-s anorMRIto dete tthe target ablation region, and preoperative CBCT without and with needles to position the needles and to verifythepositioning. Importantly,theneedlespositioninggenerateanelasti deformationoftheliver,that hasbea ountedforaspreviouslyshownin[7℄,andthusnonrigidmultimodalalgorithmasEVolution[6℄has

Trans. [X-Y℄ (a) (b) ( ) (d) Sag. [X-Z℄ (e) (f) (g) (h) Cor. [Y-Z℄ (i) (j) (k) (l) (m) (n) (o)

Figure3: ExampleofaCT/CBCTregistrationresults.TheCBCTimage,usedasareferen eforregistration,wasa quired immediatelyafterinsertionof4needles. Transversal(a-d),sagittal(e-h)and oronal(i-l) ross-se tionsarereportedfor: CBCT (rst olumn),CT before(se ond olumn) andafterregistrationusingPM-EA(third olumn) andPM-EA+Evo (fourth olumn). HistogramsofX-,Y-andZ-shifts arereportedin(m),(n)and(o),respe tively(maximumo urren e inreddashedline).

tobeused. Asfarasweknow,the urrentnonrigidregistrationalgorithmneededaninitialmanualtuning step to superimpose rouglhythe FOVs of twoimages of dierent modality. Importantly, the registration hasbeperformed duringthepro edure,as demonstratedin [7℄ in orderto provideanumeri alassessment

Trans. [X-Y℄ (a) (b) ( ) (d) Sag. [X-Z℄ (e) (f) (g) (h) Cor. [Y-Z℄ (i) (j) (k) (l) (m) (n) (o)

Figure4:ExampleofaMR/CBCTregistrationresults. TheCBCTimage,usedasareferen eforregistration,wasa quired immediatelyafterinsertionof3needles. Transversal(a-d),sagittal(e-h)and oronal(i-l) ross-se tionsarereportedfor: CBCT(rst olumn),MRIbefore(se ond olumn)andafterregistrationusingPM-EA(third olumn)andPM-EA+Evo (fourth olumn). HistogramsofX-,Y-andZ-shifts arereportedin(m),(n)and(o),respe tively(maximumo urren e inreddashedline).

of thetherapyto thephysi ians. There istherefore a ru ial needto automatize theimage prepro essing of FOV alignement in any ele troporation ablation pro edures. In the urrent study, the use of various imagingsensors(CT/CBCT,MR/CBCTimagepairs,CBCTsbeinga quiredintra-operatively)isanalysed

Computation time [s]

Image down-sampling

DSC [a.u]

No needle

Needles

(a)

Computation time [s]

Image down-sampling

DSC [a.u]

No needle

Needles

(b)

Figure5: Analysisoftheimpa tofthedown-samplingofinputdataontheperforman eoftheproposedPM-EAmethod. DSC(leftY-axis)and omputationtimes(reddashedline/rightY-axis)arereportedfortheregistrationofCT/CBCT(a) andMR/CBCT(b)pairsfordown-samplingfa tors

2×

(DS-2),

4×

(DS-4)and

8×

(DS-8).Were allthatthepat hsize washerexedto

9 × 9 × 9

voxels. Thenumberofhistogrambinswasxedtoavalueof50.

CT/CBCTregistration

2

2.5

3

3.5

4

Computation time [s]

3x3x3

5x5x5

7x7x7

9x9x9

11x11x11

Patch size

0

0.2

0.4

0.6

0.8

1

DSC [a.u]

No needle

Needles

(a) MR/CBCTregistration

2

2.5

3

3.5

4

Computation time [s]

3x3x3

5x5x5

7x7x7

9x9x9

11x11x11

Patch size

0

0.2

0.4

0.6

0.8

1

DSC [a.u]

No needle

Needles

(b)

Figure6:Analysisoftheimpa tofthepat hsizeontheperforman eoftheproposedPM-EAmethod. DSC(leftY-axis) and omputationtimes(reddashedline/rightY-axis)arereportedfortheregistrationofCT/CBCT (a)andMR/CBCT (b)pairs. Were allthatthedown-samplingfa torofinputdatawasherexedto

4×

. Thenumberofhistogrambinswas xedtoavalueof50.

as well astheimpa tofneedle insertionsduring IRE pro edures. Usingtheproposedexperimental setup, theregistration pro essis hamperedbythe useofdierent imageFOVs,espe ially whenusing CTduring thepre-operativesession (seethelowDCS obtainedin gure9beforeregistration whenusing CTinstead of MRI). Moreover, partial FOVs, ross- ontrast variations, appearing/disappearing (anatomi al or not) stru turesarealsoinvolvedbetweentheimagetoregisterandthereferen eone.

Usingsu h data sets, optimizing asimple translationalmodel, as implemented in the Elastixtoolbox, wasfound to beinsu ient. The proposed regional registrationapproa h using voxel pat hesprovided a good stru tural ompromisebetween the voxel-wise (as done with Evo) and global shifts (as done with Elastix in the s ope of this study) approa hes. Moreover, ontrary to optimization methods whi h are

Histogram bins [#]

DSC [a.u]

No needle

Needles

(a)

Histogram bins [#]

DSC [a.u]

No needle

Needles

(b)

Figure7: Analysisofthe impa tofthenumberofhistogrambinsontheperforman eof theproposedPM-EA method. DSC(leftY-axis)and omputationtimes(reddashedline/rightY-axis)arereportedfortheregistrationofCT/CBCT(a) andMR/CBCT (b)pairs. We re allthatthedown-samplingfa torof inputdatawashere xedto

4×

. Thepat hsize washerexedto

9 × 9 × 9

.

CT/CBCTregistration

Masking sensibility [%]

DSC [a.u]

No needle

Needles

(a) MR/CBCTregistration

Masking sensibility [%]

DSC [a.u]

No needle

Needles

(b)

Figure8: Analysisoftheimpa toferrorso urredinthetargetedorgandelineationpro essperformedonthepre-operative image

I

.

inherentlysensitivetolo alminima,PM-EAisabletodealwithlargetranslationamplitude,sin epotential pat h mat hesare onsideredwithin the ompleteFOV, asdes ribedinse tion2.1.3. Wehavealsoshown that theproposed multi-modal image similarity metri , whi h favors edge alignements irrespe tiveof the gradientdire tion,outperformstheL2-normproposedintheoriginalPat hMat hpaper[3℄. Ultimately,we haveshownthatPM-EAmaygreatlyimprovetheperforman eofanexistingmulti-modalelasti registration algorithm(Evoin thes opeofthisstudy).

As expe ted, omputation times were greatlyredu edusing down-sampled versions ofinput data (see gure 5). This down-sampling step also a ts as an inherent low-pass lter applied on

I

and

J

whi h improved theregistration a ura y in ourtests. Using the proposed default user-dened parameter (i.e., down-samplingfa torof

8×

),theaverageDSC withPM-EA ex eeded0.6forbothCT/CBCT,MR/CBCT

DSC [a.u]

No needle

Needles

(a)

DSC [a.u]

No needle

Needles

(b)

Figure9: SummaryofDSCs oresobtainedfortheregistrationofCT/CBCT(a)andMR/CBCT(b)images,usingtested solutionsdetailedinse tion2.2.4. Standarddeviationsoverthepatientsaregivenbythesizeofthebla kerrorbars. We re all thatthedown-samplingfa torof inputdatawas herexedto

4×

. Thepat hsizewasxed to

9 × 9 × 9

voxels. Thenumberofhistogrambinswasxedtoavalueof50.

imagepairstogetherwitha omputationtime ostbelow3se ondsona ommodityhardware.

The proposed multi-modal metri of Eq. (4) performs a weighted average over pat hes of the edge alignements ore. Consequently, the pat h size is an input parameterwhi h anbein reased in order to mitigatethefa tthattheobservedimagefeaturesmightnotbedis riminativeenough. In reasingthepat h size maythusimprovetherobustnessagainstanatomi alstru turewithout ounterpartbetweentheimage to register and the referen e one. This benet was a hievable with a moderate negative impa t on the omputationtime. However,tosomeextent,in reasingthepat hsizemaybeunableto opewith omplex lo al tissuedeformations. Agood ompromisein the hoi eof thepat hsizeisthusessentialforareliable anda uratepat hmat hing. Usingtheproposeddefaultuser-denedparameter(i.e.,pat hsizeof

9 × 9 × 9

voxels),PM-EAattainedthebestresultsintermsofDSCforalltestedexperimental onditions(CT/CBCT, MR/CBCTregistration,needleinsertions),as showningure6.

It anbenoti ed thatthe numberof histogrambinshad noimpa ton theoverall results,asshown in gure7. Thedefaultuser-denedparameter(i.e.,50bins)wasthuswellsuitedforallpresentedresults.

Inour data, the liverundergoes omplexdeformations betweenthe images being registered. The reg-istration a ura y de reasedfor eroded versions of

M

, asshownin gure8. Liverboundaries,whi h are needful ontrastregions,are nottakeninto a ount insu h a ase. Moreover,theoverall shiftestimateis likelytodierfromthegloballiverdispla ementifthemanuallydened mask

M

onlyin ludesasubsetof theliver. Alternatively,nosigni antimpa tontheregistrationa ura ywasobservedfordilatedversions of

M

in ourtests. Therefore,theguidelineisthat

M

mustin lude atleastthetargetedorgan.

5. Con lusion

Thesu essfull ompletionofaninterventionaltherapeuti workowoftenreliesonestablishingaspatial oheren e betweenimagesa quired byvarioussensorsat dierentstages. Theproposed PM-EAalgorithm was validated in several omplementary experiments. It was demonstrated that it outperforms existing registration solutionsfor theestimation of aglobal 3Dtranslation betweentwomulti-modalimages. The method anbeusedasapre- onditioningstepforamore omplexmulti-modalelasti registrationalgorithm. Themethodwastherebybene ialforCTtoCBCTandMRItoCBCTregistrationtasks,espe iallywhen highlydierentimageFOVsareinvolved. Inaddition,thiswasa hievabletogetherwitha omputationtime ostbelow3se ondsona ommodityhardwareusingourexperimentalproto ol.Theproposedpat h-based

workowthusrepresentsanattra tiveassetforDIRat dierentstagesofaninterventionalpro edure.

Referen es

[1℄ J.Astola,P.Haavisto,andY.Neuvo. Ve tormedianlters. Pro eedingsoftheIEEE,78(4):678689,1990.

[2℄ L.Bao,Q.Yang,andH.Jin. Fastedge-preservingpat hmat hforlargedispla ementopti alow. InPro eedingsofthe IEEEConferen eonComputerVisionandPatternRe ognition,pages35343541,2014.

[3℄ C.Barnes,E.She htman,A.Finkelstein,andD.B.Goldman.Pat hMat h:Arandomized orresponden ealgorithmfor stru turalimageediting.ACMTransa tionsonGraphi s(Pro .SIGGRAPH),28(3),2009.

[4℄ M. Bleyer, C.Rhemann,andC.Rother. Pat hmat h stereo-stereomat hingwithslanted support windows. InBmv , volume11,pages111,2011.

[5℄ B.DenisdeSenneville,M. Ries,G.Ma lair,andC.Moonen. MR-guidedthermotherapy ofabdominalorgans usinga robustPCA-basedmotiondes riptor. IEEETransa tionsonMedi alImaging,30(11):19871995,2011.

[6℄ B.Denis deSenneville,C. Za hiu, M.Ries, and C.T.W.Moonen. Evolution: an edge-basedvariational method for non-rigidmulti-modalimageregistration. Physi sinMedi ineandBiology,61(20):7377,2016.

[7℄ O.Gallinato,B.DenisdeSenneville,O.Seror,and C.Poignard. Numeri alworkow ofirreversibleele troporation for deep-seatedtumor. Physi sinMedi ineandBiology,64(5):055016,2019.

[8℄ R.Giraud,V.-T.Ta,N.Papadakis,J.V.Manjón,D.L.Collins,P.Coupé,andtheAlzheimer'sDiseaseNeuroimaging Ini-tiative. AnoptimizedPat hMat hformulti-s aleandmulti-featurelabelfusion.NeuroImage,124:770782,2016. [9℄ M.Gu kenberger,A.Ri hter,J.Boda-Heggemann,andF.Lohr.Motion ompensationinradiotherapy. Criti alReviews

andtrade;inBiomedi alEngineering,40(3):187197,2012.

[10℄ M.Heinri h,M.Jenkinson,M.Bhushan,T.Matin,F.Gleeson,S.Brady,andJ.S hnabel.MIND:Modalityindependent neighbourhooddes riptorformulti-modaldeformableregistration. Medi alImageAnalysis,16(7):14231435,2012. [11℄ A. Ho quelet, H. Trillaud,N.Frulio,P.Papadopoulos, P.Balageas,C.Salut, M. Meyer, J. F.Blan , M.Montaudon,

andB.DenisdeSenneville. Three-dimensionalmeasurementofhepato ellular ar inomaablationzonesandmarginsfor predi tinglo altumorprogression. JournalofVas ularandInterventionalRadiology,27(7):10381045.e2,2016. [12℄ A.B.Holbrook,J.M.Santos,E.Kaye,V.Rieke,andK.ButtsPauly. Real-timeMRthermometryformonitoringHIFU

ablationsoftheliver.Magneti Resonan einMedi ine,63(2):365373,2009.

[13℄ B.HornandB.S hun k. Determiningopti alow.Arti ial intelligen e,17:185203,1981.

[14℄ M.IraniandP.Anandan.Robustmulti-sensorimagealignment. InIEEEComputerVision,SixthInternational Confer-en e,pages959966,1998.

[15℄ M.JakubowskiandG.Pastuszak.Blo k-basedmotionestimationalgorithmsasurvey.Opto-Ele troni sReview,21(1), 2013.

[16℄ S. Klein, M. Staring, K. Murphy, M. Viergever, and J. Pluim. elastix: Atoolbox for intensity-based medi al image registration. IEEETransa tionsonMedi alImaging,29(1):196205,2010.

[17℄ L. Latte, C. Za hiu, L. G.W. Kerkmeijer, M. Ries, and B. Denis de Senneville. A elerating multi-modal image registrationusingasupervoxel-basedvariationalframework. Physi sinMedi ineandBiology,63(23):235009,2018. [18℄ C.Mougenot,B.Quesson,B.DenisdeSenneville,P.deOliveira,S.Sprinkhuizen,J.Palussiere,N.Grenier,andC.T.W.

Moonen.Three-dimensionalspatialandtemporaltemperature ontrolwithMRthermometry-guidedfo usedultrasound (MRgHIFU). Magneti Resonan einMedi ine,61:603614,2009.

[19℄ A.Newson,A.Almansa,M.Fradet, Y.Gousseau,andP.Pérez. Videoinpaintingof omplexs enes. SIAMJournalon ImagingS ien es,7(4):19932019,2014.

[20℄ H. Rivaz, Z.Karimaghaloo, and D. L.Collins. Self - similarityweighted mutualinformation: Anewnonrigid image similaritymetri .MedImageAnal,18(2):343358,2014.

[21℄ M. Rubeaux, A. Simon, K. Gnep, J.Colliaux, O.A osta,R. deCrevoisier, and P.Haigron. Evaluation of non-rigid onstrainedCT/CBCTregistrationalgorithmsfordelineationpropagationinthe ontextofprostate an erradiotherapy. InMedi alImaging2013: Image-guided pro edures,roboti interventionsandmodeling,volume8671.SPIEPro eedings, 2013.

[22℄ D.Shamonin.Fastparallelimageregistrationon puandgpufordiagnosti lassi ationofalzheimer'sdisease.Frontiers inNeuroinformati s,7,2013.

[23℄ C.Studholme, D.Hill,andD. Hawkes. Anoverlap invariantentropymeasure of3d medi alimagealignment, pattern re ognition. 32(1):7186,1999.

[24℄ C.Sutour,J.F.Aujol,C.A.Deledalle,andB.DenisdeSenneville.Edge-basedmulti-modalregistrationandappli ation fornightvisiondevi es.Journal ofMathemati alImagingandVision,53:131150,2015.

[25℄ O.Sutter,A.Fihri,R.Ourabia-Belka em,N.Sellier, A.Diallo,and O.Seror. Real-time3Dvirtualtargetuoros opi displayfor hallenginghepato ellular ar inomaablationsusing onebeamCT.Te hnologyin an erresear h&treatment, 17:1533033818789634,2018.

[26℄ C.Za hiu, B.DenisdeSenneville,I.D.Dmitriev,C.T.W.Moonen,andM.Ries. Aframeworkfor ontinuous target tra kingduringMR-guidedhighintensityfo usedultrasoundthermalablationsintheabdomen. JournalofTherapeuti Ultrasound,5(1):27,2017.

[27℄ C.Za hiu,B.DenisdeSenneville,R.H.N.Tijssen,A.N.T.J.Kotte,H.A.C.,L.G.W.Kerkmeijer,J.J.W.Lagendijk, C.T.W.Moonen,and M.G.Ries. Non-rigidCT/CBCTto CBCTregistrationfor onlineexternalbeamradiotherapy guidan e. Physi sinMedi ineandBiology,63(1):015027,2017.