Physically interpretable machine learning algorithm on multidimensional non-linear fields

(1)

OATAO is an open access repository that collects the work of Toulouse

researchers and makes it freely available over the web where possible

Any correspondence concerning this service should be sent

to the repository administrator:

tech-oatao@listes-diff.inp-toulouse.fr

This is a publisher’s version published in:

https://oatao.univ-toulouse.fr/2

7580

To cite this version:

Mouradi, Rem-Sophia and Goeury, Cédric and Thual, Olivier

and Zaoui,

Fabrice and Tassi, Pablo Physically interpretable machine learning algorithm on

multidimensional non-linear fields. (2021) Journal of Computational Physics, 428.

110074. ISSN 0021-9991.

Official URL:

(2)

Contents lists available atScienceDirect

Journal

of

Computational

Physics

www.elsevier.com/locate/jcp

Physically

interpretable

machine

learning

algorithm

on

multidimensional

non-linear

ﬁelds

Rem-Sophia Mouradi

a,b,

∗

,

Cédric Goeury

a

,

Olivier Thual

b,c

,

Fabrice Zaoui

a

,

Pablo Tassi

a,d

a_EDF_R&D,_National_Laboratory_for_Hydraulics_and_Environment_(LNHE),₆_Quai_Watier,₇₈₄₀₀_Chatou,_France

b_Climate,_Environment,_Coupling_and_{Uncertainties}_research_unit_(CECI)_at_the_European_Center_for_Research_and_Advanced_Training_in ScientiﬁcComputation(CERFACS),FrenchNationalResearchCenter(CNRS),42AvenueGaspardCoriolis,31820Toulouse,France c_Institut_de_Mécanique_des_Fluides_de_Toulouse_(IMFT),_Université_de_Toulouse,_CNRS,_Toulouse,_France

d_Saint-Venant_Laboratory_for_Hydraulics_(LHSV),_Chatou,_France

a

r

t

i

c

l

e

i

n

f

o

a

b

s

t

r

a

c

t

Articlehistory:

Received25May2020

Receivedinrevisedform6November2020 Accepted10December2020

Availableonline7January2021

Keywords:

Data-DrivenModel(DDM)

ProperOrthogonalDecomposition(POD) DimensionalityReduction(DM) PolynomialChaosExpansion(PCE) MachineLearning(ML)

Geosciences

In an ever-increasinginterestfor MachineLearning (ML) and afavorable data develop-ment context, we here propose an original methodology for data-based prediction of two-dimensional physical fields. Polynomial Chaos Expansion (PCE), widely used inthe UncertaintyQuantificationcommunity(UQ),haslongbeenemployedasarobust represen-tationforprobabilisticinput-to-outputmapping.IthasbeenrecentlytestedinapureML context,andshowntobeaspowerfulasclassicalMLtechniquesforpoint-wiseprediction. Some advantagesareinherenttothemethod,suchasitsexplicitnessand adaptabilityto smalltrainingsets,inadditiontotheassociatedprobabilisticframework.Simultaneously, DimensionalityReduction(DR)techniquesareincreasinglyusedforpatternrecognitionand datacompressionandhavegainedinterestduetoimproveddataquality.Inthisstudy,the interestofProperOrthogonalDecomposition(POD)fortheconstructionofastatistical pre-dictivemodelisdemonstrated.BothPODandPCEhaveamplyprovedtheirworthintheir respective frameworks.The goalof thepresent paperwas tocombine themfora field-measurement-basedforecasting. Thedescribedstepsare alsousefultoanalyzethe data. Somechallengingissuesencounteredwhenusingmultidimensionalfieldmeasurementsare addressed,forexamplewhendealingwithfewdata.ThePOD-PCEcouplingmethodology ispresented,withparticularfocusoninputdatacharacteristicsandtraining-setchoice.A simplemethodologyforevaluatingtheimportanceofeachphysicalparameterisproposed forthePCEmodelandextendedtothePOD-PCEcoupling.

1. Introduction

Deep Learning techniques(DL [1])and more generally Machine Learning (ML [2]), andtheir applications to physical problems(ﬂuidmechanics[3];plasmaphysics[4];quantummechanics[5],etc.)havemadeapromisingtake-offinthelast few years.Thishasbeenparticularlythecaseforﬁeldswherethemeasurement potentialhasdramaticallyincreased(e.g. GeoscienceData[6]).Inthiscontext,learningtechniquesareofinteresttoestablishnon-linearphysicalrelationships from

*

Correspondingauthorat:EDFR&D,NationalLaboratoryforHydraulicsandEnvironment(LNHE),6QuaiWatier,78400Chatou,France.

E-mailaddress:remsophia.mouradi@gmail.com(R.-S. Mouradi).

https://doi.org/10.1016/j.jcp.2020.110074

(3)

Fig. 1. Representation of the POD-PCE ML approach.

the databya combinationofsteps,inparticularusingtransformation functions,to capturethe complexityofthe system [2].

In particular, multi-layer NeuralNetworks (NN) [7] are widely used for physical applications.Their popularitycomes fromthiscomplexstructure,whichmakesthemadaptableforvariousapplications[8,9].However,somelimitationsprevent theuseofNNforphysicalapplications:(i)itisdiﬃculttoprovideanexplicitinput-to-outputformulation,duetothe com-binationsofstepsinvolvedinthelearning(ActivationFunctions,HiddenLayers [1]).Physicalinterpretationoftheconstructed model is therefore tedious[10]; (ii) too manyhyper-parameters and choicesare involved, depending on the number of neuronsandlayers(curseofdimensionality)[11];(iii)nogeneralproofforthetheoreticalabilityofapproximatingarbitrary functionsisavailable,excepttheUniversalApproximationTheorem anditsextensions[12,13] forparticularcases.

Toovercometheselimitations,wepropose analternativeML method,basedonacouplingbetweenProperOrthogonal Decomposition (POD) [14] and Polynomial Chaos Expansion (PCE) [15,16]. This approach is proposed for the prediction ofspatially-distributed physicalﬁeldsthat varyin time.The ideais tousePOD toseparate thespatial patternsfromthe temporalvariations,thatarerelatedtotheconditioningparametersusingPCE.TocorrespondtocommonNNparadigms,an adequate representationofthisideaisgiveninFig.1.Inparticular, PODisusedforbothEncoding and Decoding whereas

PCEisusedasanActivationFunction intheLatentRepresentation [1]. TheproposedPOD-PCEaddressesthesedrawbacksofML.

(

i

)

Itisexplicitandsimpletoimplement, asitconsistsoftheassociationoftwo lineardecompositions.PODisalinear separationof the spatiotemporal patterns [17], shown to be accurate for both linearand non-linearproblems [18], combiningsimplicityandrelevance.PCEisawell-establishedmethodinUncertaintyQuantiﬁcation(UQ)[19,20],widely used for the study of stochastic behavior in physics [21,22]. It is a linear polynomial expansion that allows non-linearitiestobegradually addedto themodelby increasingthepolynomialdegree. Thelinearityandorthonormality ofthePODandPCEcomponentsandtheprobabilisticframeworkofPCEmaketheoutput’sstatisticalmomentseasier tostudy[23],enablingstraightforwardphysicalinterpretationofthemodel[24].

(

ii

)

Itonlyhastwo hyper-parameters:anumberofPODcomponents,andaPCEpolynomial degree.Both canbe chosen according to quantitative criteria[14,25]. All other forms of parameterization (choice of the polynomial basis) can be achieved with robust physical and/or statistical arguments [26], as assessed in the present paper. Furthermore, theorthonormality ofthe PODand PCEbases minimizesthe numberof componentsnecessary to captureessential variationsindata.Additionally,thePODmodescapturemoreenergythananyotherdecomposition[27],PCEisknown toexponentiallyconvergewithpolynomialdegree[16],andthecardinalityofthelattercanbereducedbysparsebasis selection[28].

(

iii

)

Itcan beconsidered asa universalexpansion forphysicalfield approximation: aphysical fieldhasa finitevariance, whichimpliesthatitbelongstotheHilbertspaceofrandomvariableswithfinitesecondordermoments.There there-foreexists a numerable setoforthogonal random variables, thatform thebasis of thisHilbertspace, on whichthe fieldofinterestcan beexpanded(strictequality,notapproximation)[20].Amathematicalsettingforbasis construc-tion based on input was established by Soize and Ghanem [26] for the general case of dependent variables with arbitrarydensity,providedthatthesetofinputsisfinite.

Intheliterature,associatingregressiontechniquestoReducedOrderModels(ROM), thatincludePOD,isnot novel[29,

30].Thecitedstudies,however,focusedondimensionalityreduction,whereas theexplicitformulationandapplicabilityto complex physicalprocessesare emphasized inthe presentstudy.Secondly,coupling PCEtoROM was recentlyaddressed

(4)

[31,32] and theuseofPCEasML isconsistentwiththe workofTorreet al. [33],wherethe authorsshowedthat PCEis aspowerfulasclassicalMLtechniques.However,neitherspatiotemporalfieldsnorphysicalinterpretabilitywereaddressed. Thedatainthesestudieswereeitherobtainedfromnumericalexperiments,emulatedfromanalyticalbenchmarkfunctions such asSobolorIshigami,orbasedonone-dimensionaldatasets[33].Incontrast,theproposedPOD-PCE methodologyis herein assessedon two-dimensionalphysicalfields.Inparticular, atoy examplewheresyntheticdataare emulatedusing an analytical function (groundwater perturbations due to tidal loadings [34]), and a real data set (high-resolution field measurementsofunderwatertopography)areused.Althoughsimilarfromalearningpointofview,thesetwoapplications are characterized withdifferences. In particular, thetoy problemispurely parametric and controllable,whereas the real data concern temporaldynamics andare of limitedsize. The casesare therefore complementary, in the sense that they allow demonstrating differentpropertiesof the proposed methodology. Hence, usingthe particularities ofeach case, the studyconsistsin:i)theevaluationofthecombineduseofPODandPCEasMLforpoint-wiseprediction;ii)therobustness ofthemethodologytonoise;iii)theapplicationtofielddatawiththeinherentchallengesnotencounteredwithnumerical data (e.g.paucity); iv) a focuson model explicitnessas a key condition forphysical understanding andv) theinfluence of forcing variablesstudy, basedon a classicalmeasure ofimportance (Garson weights [35]) directlycomputedwith the POD-PCEexpansioncoefficients.

Thepaperisorganizedasfollows.Section2givesadetailedexplanationofthemethodology,withaproposalforphysical importance measures inSubsection2.2.2.Section 3deals withthe assessmentofthe methodologyonsynthetic data,for bothpredictionandphysicalinterpretation.Inparticular,therobustnessoftheapproachtonoiseisevaluatedinSubsection

3.3.The modelis then deployed onﬁeld measurements in Section 4.The study caseanddataare described in 4.1. POD andPCEperformancesare thendemonstratedindependentlyin4.2witha deepphysicalanalysis. Theperformance ofthe POD-PCEpredictorisdiscussedin4.3.Asummaryofthestudyandperspectivesoftheproposedmethodologyarepresented inSection5.

2. Theoreticalframework

Inthissection,theobjectiveistodefinetheframeworkoftheproposedPOD-PCEMachineLearningmethodology,along withphysicalinfluenceindicatorsforthe inputs.Thisis theobjectofSubsection2.3,butfirst,areminder oftheexisting PODandPCEtheoreticalbasesispresentedin2.1and2.2respectively.

2.1. ProperOrthogonalDecomposition

PODisa dimensionalityreductiontechnique [17] thatiswell documentedinliterature [14,18].Theoretical detailsand demonstrationscanbefoundin[27,36].Forclarity’ssake,theessentialelementsofPODaresummarizedbelow.

The goal of POD is to extract the main patterns of continuous bi-variate functions. These patterns, when added and multipliedbyappropriatecoeﬃcients,explainthedynamicsofthevariableofinterest:areal-valuedphysicalﬁeld.

Letu

:

× T → D

beacontinuous functionoftwovariables

(

x

,

t

)

∈

× T

.Thefollowingrelationshipsandproperties holdforany

× T

andHilbertspace

D

characterizedbyitsscalarproduct

(. ,

.)

D andinducednorm

||.||

D.However,as isthecaseforamajorityofphysicalﬁelds,weshallconsider

asasetofspatialcoordinates(e.g.

R

2 _or

_R

3_),

_T

_an_event

space(e.g.parametersspace

R

V _with _V

_{∈ N}

∗_,_or_a _temporal_subset

_[

₀

_,

_T

_]

_{⊆ R}

+_),_and

_D

_as_a_set_of_scalar_real_values_or vector realvalues(e.g.

R

or

R

2_). _POD_consists_in_an _{approximation}_of _u

₍

_x

_,

_t

₎

_at_a_given_order_d

_{∈ N}

_(Lumley_[₁₇_])_as_in

Equation(1), u

(

x

,t)

≈

d

k=1 vk

(t)φ

k

(

x

) ,

(1)

where

{

vk

(.)

}

d_k₌₁

⊂

C(T,

R)

and

{φ

k

(.)

}

kd=1

⊂

C(,

D)

,with

C(A,

B)

denotingthe spaceof continuous functionsdeﬁned

over

A

andarrivingat

B

.TheobjectiveofPODistoidentify

{φ

k

(.)

}

dk=1 thatminimizesthedistanceoftheapproximation

fromthetruevalue u

(.,

.)

,overthewhole

× T

domain,withanorthogonalityconstraintfor

{φ

_k

(.)

}

d_k₌₁ usingthescalar product

(. ,

.)

D.Thiscanbedeﬁned,intheleast-squaressense,asaminimizationproblem.

The minimization problem is deﬁned for all orders d

∈ N

, so that the members

φ

k are ordered according to their

importance.Inparticular,fororder1,

φ

1isthelineargeneratorofthesub-vectorspacemostrepresentativeofu

(

x

,

t

)

in

D

.

For

D

=

Im

(

u

)

,thefamily

{φ

_k

(.)

}

d_k₌₁iscalledthePODbasisof

D

ofrankd.Thesolutiontothisproblemhasalreadybeen established in literature [17,37]. The theoretical aspects ofPOD and demonstrations of mathematical properties can, for example,befoundin[27]:thePODbasisof

D

oforderd istheorthonormalsetofeigenvectorsofanoperator

R

: D → D

deﬁnedas

Rφ = (

u

, φ)

D

×

u

T,iftheeigenvectorsaretakenindecreasingorderofthecorrespondingeigenvalues

{λ

k

}

dk=1.

Forthisexpansion,anaccuracyrate,alsocalledtheExplainedVarianceRate(EVR),denoteded atrankd,canbe calcu-latedasinEquation(2).EVRtendsto1(perfectapproximation)whend

→ +∞

.

ed

=

k≤d

λ

k

_+∞ k=1

λ

k

.

(2)

(5)

Inpractice,for

D

_{= R}

,whenu

(.,

.)

isadiscretesample onasetofm

∈ N

spacecoordinates

X = {

x1

,

. . . ,

xm

}

andfor n

∈ N

measurement events

T = {

t1

,

. . . ,

tn

}

(e.g.realizationsof theparameters, timecoordinates, etc.),the available data

setisarrangedinamatrixU

(

X ,

T )

= [

u

(

xi

,

tj

)

]

i,j

∈ R

m×n,calledthesnapshotmatrix,soastobeabletoworkinadiscrete

space.ThePODproblemformulatedinEquation(1) canbewritteninitsdiscreteformasU

(

X ,

T )

=

(d)

(

X )

V(d)

_{(T )}

_,_where

(d)

₍

_{X )}

_{:= [φ}

j

(

xi

)

]

i,j

∈ R

m×d andV(d)

(

T )

:= [

vi

(

tj

)

]

i,j

∈ R

d×n.Theproblemcanthereforebeviewedasifworkingwitha

newfunctionU

(

X ,

.)

= [

u

(

xi

,

.)

]

i∈{1,...,m}

:

T → D = R

M.Then,theaverageover

T

canbe deﬁnedasthestatisticalmean

overthesubset

T

,andthescalarproduct

(. ,

.)

Dasthecanonicalproductover

R

m.ThePODoperator

R

canbewrittenas inEquation(3),

R

φ(X ) =

1 n n

j=1 U

(

X ,

tj

)

T

(

X )

U

(

X ,

tj

)

=

1 nU

(

X ,

T

)

U

(

X ,

T

)

T

₍

_{X ) ,}

₍₃₎

where U

(X ,

tj

)

= [

u

(

xi

,

.)

]

i∈{1,...,m} isthecolumnnumber j of thematrixU

(

X ,

T )

(i.e realizationtj ofthemeasurement

over

X

),and

(

X )

= [φ(

xi

)

]

i∈{1,...,m}.AsﬁndingthePODbasisisequivalenttoidentifyingtheorthonormalsetof eigenvec-torsoftheoperator

R

,thenforthisdiscreterepresentationtheproblembecomesequivalent tosolvingtheeigenproblem ofthematrixR

:=

_n1U

(

X ,

T )

U

(

X ,

T )

T_,_called_the_covariance_matrix._A_number_d

_{∈ N}

_of_eigen_vectors

₍

_{X )}

_are_identiﬁed

andstoredinthecolumnsofthematrix

(d)

(

X )

.FortheeigenvaluesofthecovariancematrixR denoted

{λ

k

}

dk=1,the

ex-pansioninEquation(1) canalsobewrittenasinEquation(4),where

{φ

k

(.)

}

dk=1togetherwith

{

ak

(.)

}

d k=1arebi-orthonormal, andvk

(.)

=

ak

(.)

√

n

× λ

k. u

(

x

,t)

≈

d

k=1 ak

(t

)

n

× λ

k

φ

k

(

x

) .

(4)

BydeﬁningthematrixA(d)

_{(T )}

_{:= [}

_a

i

(

tj

)

]

i,j

∈ R

d×n andtheoperatorD(d)

(λ

1

,

...,

λ

d

)

correspondingtothediagonalmatrixof

elements

λ

i,wehaveU

(

X ,

T )

=

(d)

(

X )

D(d)

(

√

n

× λ

1

,

...,

√

n

× λ

d

)

A(d)

(

T )

.ThereforethetransposedformisU

(

X ,

T )

T

=

A(d)

(T )

TD(d)

(

√

n

× λ

1

,

...,

√

n

× λ

d

)

(d)

(

X )

T. Thanksto theorthonormalityof

{

ak

(.)

}

d_k₌₁,the covariancematrixreadsR

=

1

n

(

d)

₍

_{X )}

_D(d)

₍

_n

_{× λ}

1

,

...,

n

× λ

d

)

(d)

(

X )

T

=

(d)

(

X )

D(d)

(λ

1

,

...,

λ

d

)

(d)

(

X )

T.

Whenn

<<

m,itismorecomputationallyeﬃcienttosolvetheeigenproblemofRT insteadoftheeigenproblemofR as

highlightedbySirovich [37].Thisisoftenthecasewhenalimitednumberofoccurrencesismeasuredforatwo-dimensional physicalﬁeld,asisthecaseencounteredfortheapplicationdescribedinSection4.

When an order d

<<

min

(

m

,

n

)

corresponds to a high EVR as deﬁned in Equation (2), we speak of dimensionality reduction, because the data are projected in a sub-space that is of much smaller dimension than

R

m×n_. _When _diverse

enoughrecordsareavailableforthevariableunderstudy,wemayconsiderthat

{φ

_k

(

X )}

d_k₌₁

= {[φ

_k

(

xi

)

]

i∈{1,...,m}

}

dk=1,i.e.the

resulting POD basis,is a generator ofall possiblestates. Predictingthe associated expansion coeﬃcients

{

ak

(

t

)

}

dk=1 fora

giveneventt wouldthereforebeenoughtopredictthewholestate.Hence,weproposetousethePODasabasisextractor. Thiswouldﬁrstenable studyofthedynamicsofthevariableofinterestandeventually extractionofphysicalinformation, as shownin theapplications Sections 4and 3. Then, the basis can be used asa generator forthe predictionof diverse states.Thisimpliespredicting

{

ak

(

t

)

}

dk=1,forwhichwepropose tousePolynomialChaosExpansion(PCE),asdescribed in

thefollowingSection2.2.

2.2. PolynomialChaosExpansion

AreminderofthetheoreticalbaseofPCEispresentedinSubsection2.2.1.Theoreticaldetails,demonstrationsand inter-esting referencescanbe foundin [23,19]. Then,a simpleindicator isproposed inSubsection2.2.2forthe analysisofthe variablesinﬂuenceontheoutputvalue.ThelatterislatergeneralizedforPOD-PCEinSection2.3.

2.2.1. Learning

The idea behind Polynomial ChaosExpansion (PCE)is to formulate an explicitmodel that linksa variable ofinterest (output)toconditioningparameters(inputs),bothinaprobabilityspace.Thisenablesthepropagationpathofprobabilistic information(uncertainties,occurrence frequencies)tobe mappedfromthe inputto theoutput space. Thevariable of in-terest,Y,andtheinputparameters

= (θ

1

,

θ

2

,

...,

θ

V

)

arethereforeconsideredrandomvariables,characterizedbyagiven

ProbabilityDensityFunction(PDF)denoted f.ItshouldbekeptinmindthattheoutputsofourproblemarethePOD ex-pansioncoeﬃcientsY

= [

ak

(

t

)

]

k∈{1,...,d},andthattheinputscorrespondtophysicalforcings,asdescribedlaterinSection2.3.

The objective isto derive the variations of the PODcoeﬃcients asthe outcome ofthe forcings.Let usnow recall some fundamentalsofthemathematical probabilisticframework, takingtheexampleofaone dimensionalreal-valued variable. Thedeﬁnitionscanbeeasilyextendedto

R

M_.

Let

(,

F

,

P )

be a probability space, where

is the eventspace (space ofall the possible events

ω

) equipped with

σ

-algebra F (someeventsof

)anditsprobabilitymeasure

P

(likelihoodofagiveneventoccurrence).Arandomvariable deﬁnesanapplicationY

(

ω

)

:

→

DY

⊆ R

,withrealizationsdenotedby y

∈

DY.ThePDF ofY isa function fY

:

DY

→ R

(6)

thatveriﬁes

P (

Y

∈

E

⊆

DY

)

=

_E fY

(

y

)

dy.Thekth moments ofY aredeﬁnedas

E

_[

Yk

]

:=

_D

Y y

k_fY

₍

_y

₎

_dy,_the_ﬁrst_being_the

expectationdenoted

E

[

Y

]

.Inthesamemanner,wedeﬁnethekth centralmoments ofY as

E

[(

Y

− E[

Y

])

k

_]

_,_the_ﬁrst_being

0 andthe secondthe varianceof Y denotedby

V

[

Y

]

.The covarianceoftworandomvariables isdeﬁnedascov

(

X

,

Y

)

=

E

_[(

X

− E[

X

])(

Y

− E[

Y

])]

andaresultingpropertyis

V

_[

Y

]

=

cov

(

Y

,

Y

)

.

ReturningtothePCEconstruction,inputs

= (θ

1

,

θ

2

,

...,

θ

V

)

areconsideredtoliveinthespaceofrealrandomvariables

withﬁnitesecondmoments(andﬁnitevariances).Thisspaceisdenotedby

L

2_R

(,

F

,

P

;

R)

andisaHilbertspaceequipped with the inner product

(θ

1

,

θ

2

)

_L2

R

:= E[θ

1

θ

2

]

=

θ

1

(

ω

)θ

2

(

ω

)

d

P (

ω

)

and its induced norm

||θ

1

||

_L2 R

:=

E

[θ

2

1

]

. The PCE

objectiveistomaptheoutputspacefromtheinputspacewithamodel

M

asinEquation(5):

Y

=

M

()

=

_I_⊆{₁_,...,_V_}

M

_I

(θ

_I

)

=

M

0

+

V i=1

M

i

(θ

i

)

+

1≤i<j≤V

M

i,j

(θ

i

, θ

j

)

+ ... +

M

1,...,V

(θ

1

, θ

2

, ..., θ

V

) ,

(5)

where

M

0 isthe expectation of Y and

M

_I⊆{1,...,V} represents thecommon contribution ofthe variables

I ⊆ {

1

,

...,

V

}

to the variation in Y . For the PCE model, these contributions have a polynomial form. We shall deﬁne, for each input variable

θ

i,an orthonormalunivariate polynomial basis

ξ

_β(i)

(.), β

∈ [|

0

,

p

|]

where p

∈ N

is a chosen polynomialdegree and

ξ

_β(i)

(.)

isof degree

β

.The orthonormality isdeﬁned withrespect to the inner product

(.,

.)

_L2

R. If we introduce the

multi-indexnotation

α

= (

α

1

,

...,

α

V

)

∈ N

V sothat

|

α

|

=

Vi=1

α

i,wecandeﬁne amultivariatebasis

ζ

α

(.),

|

α

| ∈ [|

0

,

p

|]

as

ζ

_α

(θ

1

,

θ

2

,

...,

θ

V

)

:=

iV=1

ξ

(i)

αi

(θ

i

)

.Therefore,themodelinEquation(5) canbewrittenas:

Y

=

M

()

=

|α|≤P

cα

ζ

α

(θ

1

, θ

2

, ..., θ

V

) ,

(6)

wherecα

∈ R

are deterministiccoeﬃcientsthat canbe estimatedthanksto differentmethods.Itcan be formulatedasa minimization problem, andregularization methodscan be usedwhen dealingwith smalldatasets. Inthepresentstudy, weusedtheLeastAngleRegressionStagewisemethod(LARS)inordertoconstructanadaptivesparsePCE.Itisaniterative procedure,consistingonanimprovedversionofforwardselection.Thealgorithmbeginsbyﬁndingthepolynomialpattern, heredenoted

ζ

i forsimplicity,that isthemostcorrelatedtotheoutput.Thelatterislinearlyapproximatedby

i

ζ

i,where

i

∈ R

. Coeﬃcient

i is not setto its maximal value, butincreasedstarting from0,until another pattern

ζ

j is found to

be ascorrelatedto Y

−

i

ζ

i,andsoon.Inthisapproach,acollectionofpossiblePCE,orderedbysparsity,isprovidedand

an optimumcanbechosenwithanaccuracyestimate.Itwasperformedinthisstudyusingcorrectedleave-one-outerror. The reader can refer to the work ofBlatman and Sudret [25] forfurther details on LARS andmore generallyon sparse constructions.

Thechoiceofthebasisiscrucialandisdirectlyrelatedtothechoiceofinputvariablemarginals,viatheinnerproduct

(.,

.)

_L2

R.Chaospolynomialswereﬁrstintroducedin[38] forinputvariablescharacterizedbyGaussiandistributions.The

or-thonormalbasiswithrespecttothismarginalistheHermitepolynomialsfamily.Later,otherAskeyschemehypergeometric polynomial families were associatedto some well-knownparametric distributions [39]. Forexample,theLegendre family is orthonormalwithrespecttothe Uniformmarginals.This iscalled g P C (generalizedPolynomialChaos) whenvariables ofdifferentPDFsareusedasinputs.Inpracticehowever,theinputdistributionsofphysicalvariablescanbedifferentfrom usualparametricmarginals.Insuchcases,themarginalscanbeinferredbyempiricalmethodssuchastheKernelSmoother (see[40] fortheoreticalelements).Inthiscase,anorthonormalpolynomialbasiswithrespecttoarbitrarymarginalscanbe builtwithaGram-Schmidtorthonormalizationprocessasin[41] orviatheStieltjesthree-termrecurrenceprocedureasin [42].

Tohighlighttheimportanceofthemarginalsandchoiceofpolynomialbasisforthelearningprocess,several conﬁgura-tions areattempted inSection4.Different inputsetsanddistributions (Gaussian,Uniform,inferred byKernelSmoothing) weretested.TheinﬂuenceofthepolynomialbasisontheMLisinvestigatedinSection4.2.2.

2.2.2. Physicalimportancemeasures

OncethePCEconstructionisachieved,aphysicalinterpretationcanbeperformed.ItisnotablethatclassicalNN indica-torscanbeused[35].ThePCEcanberepresentedintheFeedforwardNNparadigmasinFig.2.Suchnetworksareclassically composed,inadditiontotheinputandoutputlayers,ofsuccessivehiddenlayers.Eachhiddenlayeriscomposedofneurons

that transformthe variables of the previous layer (outputs of the previous neurons)into a new set of variables. Thisis donebycombiningalineartransformation,givingdifferentweights tothepreviousneurons,andatransformationfunction, calledActivationFunction (AF). Thissuccession of layers iscalled thelatentrepresentation. Fora numberof hiddenlayers

L

≥

1,theNNcan be formallywritten asY

≈

fout

(

AL fL

(. . .

A2 f2

(

A1 f1

(

Ain

))))

where

{

Ak

}

k∈[|1,L|] and

{

fk

}

k∈[|1,L|] are thehiddenlayerweightmatricesandAFs,Ain istheinput-to-hiddenconnectionmatrixand fout istheﬁnalhidden-output

transformation[2].

The PCE-basedNNrepresentedinFig.2isasingle layerfeedforward,composed ofl

∈ N

neurons,that canbe written asY

≈

fout

(

A1 f1

(

Ain

))

.TheﬁrstmatrixAin istheinput-to-hiddenconnectionmatrixofdimensionV

×

V ,thatlinksthe

inputlayertothePCEhiddenlayercontainingthemultivariatepolynomials

ζ

α

,

α

∈ {

α

1

, ...,

α

l

}

(7)

Fig. 2. Representation of the PCE learning in the NN paradigm.

inputsandthemultivariateindexes

{

α

1

,

...,

α

l

}

areconditionedbythechosenpolynomialdegree p suchas

∀

i

∈ [|

1

,

l

|]

0

≤

|

α

i

|

≤

p,andby thenumberofselectedfeatures ifa sparsepolynomial isconstructed,asinthepresentcaseusingLARS

[28]. Matrix Ain represents thecontributions of the V variables to the multivariate polynomials

ζ

α

,

α

∈ {

α

1

, ...,

α

l

}

.It is a diagonal matrix such that

[

Ain

]

j,j∈[|1,V|]2 is 0 if

∀

i

∈ [|

1

,

l

|] (

α

i

)

j

=

0 and 1 if not. The ﬁrst multi-dimensional AF f1 is a vector ofmultivariate functionsthat transforms theset of selectedinputs corresponding to

[

Ain

]

i,i∈[|1,V|]2

=

1 to

the multivariate polynomials of the chosen basis (Hermite, Legendre, etc.) by tensor product over the univariate basis. The hiddenlayerweightmatrixA1 givesdifferentweightsto theconstructedpolynomial features.It isadiagonal matrix composedofthePCEexpansioncoeﬃcientssuchas

[

A1

]

i,j∈[|1,l|]2

= [

cα_i

]

i∈[|1,l|].

The ﬁnal hidden-output transformation fout is a summation.Fig. 2 can alsobe presented differently:another hidden layer can beadded to thePCElatent representationas Y

≈

fout

(

A2 f2

(

A1 f1

(

Ain

))

.The ﬁrst AF f1 wouldrepresenta

transformation ofeach input variableto a list ofmonomialsof degrees1 to p (here, Ain isidentity). The second AF f2

thereforerepresentsthetensorproductthattransformsthedifferentmonomialstomultivariatefeatures,withA1

appropri-atelyﬁlledwithzerosandones,and

[

A2

]

_i_,_j_∈[|₁_,_l_|]2

= [

cα_i

]

i∈[|1,l|].

To capture the importance of each feature, the Garson relative Weights (GW) deﬁned in Equation (7) are a classical measure to quantify the relative importance of each neuron of the last hiddenlayer, andtherefore of each polynomial pattern,fortheoutputvalue[35,43].

w_ζ α

=

|

cα

|

0≤β≤1

|

cβ

|

.

(7)

This measure can be used to understand the importance given by the NN algorithm to the variables andtheir possible interactions, especially when using feature selection algorithms asLARS: “feature interactions [...] are created at hidden unitswithnonlinear activationfunctions, andthe inﬂuencesofthe interactionsare propagated layer-by-layerto theﬁnal output”[43].Intheparticularcaseofapolynomialexpansion,theinterpretationisstraightforward,theimportanceofeach variablealonecorrespondstoitsmonomials,andtheimportanceofitsinteractionswithothervariablescorrespondstothe multivariatepolynomialsinwhichitisinvolved.

FortheparticularcaseoftheorthonormalbasisprovidedbyPCE,theGWdeﬁnedin(7) canbeinterpretedintermsof Pearson’scorrelationsbetweenoutputY andthepolynomialbasiselements

ζ

α denoted

ρ

Y, ζ

α,with

α

= (

0

,

...,

0

)

.Indeed, Pearson’scorrelations

ρ

_Y_{, ζ}

α aredeﬁnedasinEquation(8),

ρ

_Y_{, ζ} α

=

E

(Y

− E(

Y

))(ζ

_α

− E(ζ

_α

))

V (

Y

)

V (ζ

α

)

=

cα 1≤|β|≤pc2β

,

(8)

thankstotheorthonormalityofthebasiswithrespecttothescalarproduct

(. ,

.)

_L2

R thatguarantees:

• E

ζ

_α

=

ζ

_α

, ζ

_β₌₍₀_,...,₀₎

=

1

L2 R

=

0;

• E

[Y ]

=

₀_≤|β|≤_pcβ

ζ

_β

, ζ

_β₌₍₀_,...,₀₎

L2 R

=

cβ=(0,...,0);

• E

Y

, ζ

_α

=

₀_≤|β|≤_pcβ

ζ

_β

, ζ

_α

L2 R

=

cα;

• V

ζ

_α

= E

ζ

_α

− E

ζ

_α

2

= E

ζ

_α

2

= ||ζ

α

||

2_L2 R

=

1;

• V

[Y ]

= E

Y

− E

[Y ]

2

=

1≤|β|≤pcβ

ζ

β

,

1≤|β|≤pcβ

ζ

β

L2 R

=

1≤|β|≤pc2β.

Therefore, the weights w_ζ

α can alsobe computedas

|

ρ

Y, ζα

|/

1≤|β|≤p

|

ρ

Y, ζ

β

|

. Thismeans that they measure the

(8)

ofthe latter.These“relativePearson’s correlations”can beseen asaphysicalcontributionsince thePCEmodelisstrictly linear.ThesumoftheGW w_ζ

α forallthepolynomialfeaturesequals1.Thismeansthattheyallow

{ζ

α

}

|α|≤p toberanked in terms ofrelative contribution to the output Y . The contributions can be analyzed eitherfor each polynomial pattern separately,orforasinglevariable

θ

i byaddingallthepolynomialsharesrelatedtothisvariablealone,orbyaddingallthe

polynomialsharesrelatedtothisvariableanditsinteractions(Sobol’indicesanalogy[23]).

2.3. POD-PCEbasedpredictor

PODandPCEwere introduced separatelyinSubsections 2.1and2.2respectively.We arenow fullyequippedwiththe adequate theoretical basis andmathematical notations,topresentthe POD-PCE ML methodologyfor adata-based model learning of a multidimensional physical field. In this Subsection, we will first summarize the proposed approach, then theformal details ofthecouplingwillbe givenwiththe definitionofadequate accuracymeasures. Finallythepreviously discussedimportancemeasureswillbegeneralizedforthePOD-PCEphysicalstudy.

TheproposedPOD-PCEMLconsistsofsteps,inalearningandapredictionphase,summedupasfollows:

•

Learningphase:

∗

PODbasis construction:givenasetofmeasurements U

(

X ,

T )

= [

u

(

xi

,

tj

)

]

i,j

∈ R

m×n (snapshotmatrix), constructa

spatial POD basis accordingly.Variable tj can represent time in caseof temporal dynamics,or more generallyan

occurrenceofU

(

X ,

.)

.Then,ingeneral,

T

wouldbeaneventspace;

∗

PCE learning: constructPCE models that map each POD coefficient, obtained inthe previous step along withthe spatial basis, to a set ofinputs. In theparticular caseof temporaldynamics, previous valuesof the physicalfield, representedby previous PODcoefficients, can bepartof thelearning inputs. Forexample,onecould usean initial fieldvalue U

(

X ,

tj

)

tolearnafutureﬁeldU

(

X ,

tj+1

)

fromasetofphysicalparametersthatcondition theevolution

over

[

tj

,

tj+1

]

.Thelattercanconsistintimeseriesofphysicalvariables,representativestatisticsofthelatter,physical

constants,etc.andcanbedenoted

(

tj

→

tj+1

)

;

•

Predictionphase:

∗

Givena newrealizationofthe inputs,predict thenewPODcoeﬃcientsusingthelearnedPCEmodels,then recon-struct the newestimate U

(

X ,

tk

)

on thePOD basis. As previously explained, an initial value to the physicalﬁeld,

denotedU

(

X ,

tk−1

)

,maybe partoftheinputsfortemporal dynamics.Inparticular, itsreducedform, consistingin

temporal PODcoeﬃcients, isused. Inthis case, an additionalstep is needed:U

(

X ,

tk₋1

)

is projected onthe

con-structedPODbasisinordertoretrievethevaluesofassociatedPODcoefficientswhicharethenusedasPCEinputs. The learning andpredictionset-upsaremore complextoestablishfortemporal evolutionproblems,because thefield informationatprevioustimesarerequired.Therefore,forthesakeofclarity,thestepsareexplicitlydevelopedinthe follow-ingSubsection2.3.1.TheaccuracyofthemethodologyislaterdemonstratedonbothaparametrictoyprobleminSection3, and a field measurements-based temporal problem in Section 4. Thesetwo can be considered as complementary appli-cations, anddemonstratethat thePOD-PCE MLcanbe appliedindifferentlearning set-upsofmulti-dimensionalphysical fields.Similaritiesinthetreatmentofbothproblemscanbenoticed,buttheirparticularitiesarealsousedtodemonstrate differentpropertiesofthePOD-PCElearning,thatareshortlydescribedatthebeginningofeachsection.

2.3.1. Machinelearningmethodology

Here, the formal hypothesis behind the POD-PCE ML reasoning and its mathematical formulation are discussed. Let

U

(

X ,

.)

= [

u

(

xi

,

.)

]

i∈{1,...,m}beaﬁeldofinterestdeﬁnedonasetofm

∈ N

spacecoordinates

X = {

x1

,

. . . ,

xm

}

.Let

(.)

=

(θ

1

(.),

θ

2

(.),

...,

θ

V

(.))

be a vector of the inputs supposed to condition the evolution of U

(

X ,

.)

over time. The dynamic

model,denoted

H

,that givesan estimationofafuture state U

(

X ,

tj+1

)

fromapast state U

(

X ,

tj

)

andan estimationof

(

tj

→

tj+1

)

overthetimeinterval

[

tj

,

tj+1

]

,wheretj

<

tj+1

∈ R

+,isformulatedasinEquation(9).

U

(

X ,

tj+1

)

≈

H

U

(

X ,

tj

),

tj+1

−

tj

, (t

j

→

tj+1

)

.

(9)

If theﬁeld of interest hasbeen recorded overa set of pasttimes

T = {

t1

,

. . . ,

tn

}

⊂ R

+, wheretj

<

tj+1,a POD

ba-sis can be constructed as in Section 2.1, consisting of d

∈ N

vectors of dimension m stored in a matrix as

(d)

(

X )

=

(

₁(d)

(

X ),

. . . ,

_d(d)

(

X ))

∈ R

m×n_,_and_can_be _seen_as_a_generator_of_all _possible_states_if_enough_records_are_available._If_so,

anyfuturestateU

(

X ,

tj

)

canbeexpandedonthisPODbasisandtheassociatedtemporalcoeﬃcientsaresimplytheweights

ofU

(

X ,

tj

)

onthePODbasis.Theyarethereforeobtainedusingthecanonicalscalarproductover

R

m,asinEquation(10). U

(

X ,

tj

)

≈

dk=1ak

(t

j

)

√

n

× λ

k

(_kd)

(

X )

≈

d k=1

(

U

(

X ,

tj

),

_k(d)

(

X ))

Rm

(d) k

(

X )

≈

d k=1U

(

X ,

tj

)

T

_k(d)

(

X )

(_kd)

(

X ) .

(10)

Hence,thevariablepartofU

(

X ,

tj

)

isfullyexpressedinthetemporalcoeﬃcientsak

(

tj

)

.TheﬁeldofinterestU

(

X ,

tj

)

can

(9)

byarandomprocess“inthesensethatnaturehappenswithoutconsiderationofwhatcouldbethebestrealizationsforthe learning algorithm” [2].Therefore,the coeﬃcientsak

(

tj

)

canalsobe seenasthe jth realizationofarandom variable Ak.

WecanthereforeconstructaPCEapproximation

H

k thatmaps Ak fromitsinput space.Thelatteristakenasacollection

of randomvariables, composed fromthe set

(

A1

,

...,

Ad

)

at aprevious time, theduration of thedynamic, andthe input

variables

(

tj

→

tj+1

)

.ThisisformulatedasaclassicaldynamicmodelinEquation(11).

ak

(t

j+1

)

≈

H

k

a1

(t

j

), . . . ,

ad

(t

j

),t

j+1

−

tj

, (t

j

→

tj+1

)

.

(11)

Themodel

H

inEquation(9) isapproximatedasinEquation(12).

H

U

(

X ,

tj

),

tj+1

−

tj

, (t

j

→

tj+1

)

≈

d

k=1

H

k

a1

(t

j

), . . . ,

ad

(t

j

),t

j+1

−

tj

, (t

j

→

tj+1

)

n

× λ

k

(_kd)

(

X ) .

(12)

SomelimitationstotheintroducedformulationsinEquations(11) and(12) canbehighlighted.Afirstlimitationconcerns discontinuities thatcan bemetinphysicalfields.Thiscanoccureitherinthecomplete spatialfield U

(.,

.)

,inits reduced version representedbythe PODcoeﬃcientsak

(.)

,orintheinputs

.Intheﬁrstcase, theclassicallinearapproximations

as POD may be inefficient [44]. One solution developed by Taddei [44], called RePOD (Registration POD), consists in a parametrictransformationoftheinterestdiscontinuousfieldintoasmootheroneforlineartransformations.Inthesecond case, wherediscontinuity happensinthePOD temporalcoefficients, thiswouldimpact thelearning withPCE.Innovative solutions were identified toapply PCEwhen the output’s space ischaracterized with rapidvariations ordiscontinuities, for instance near a critical point in the inputs space. As an example, a method calledadaptive Multi-Element PCE was developed for Legendrepolynomialsin [45] andextended toarbitrarymeasures in[42]. The inputsspaceisdecomposed toaunionofsubsets, andtheoutputvariableislocallyexpandedoneachsubset.Thefinalsolutionisthenacombination ofPCEsub-problems.Inthelast discontinuitycasethatconcernsthe inputs

,theprevious splittingtechniquescanalso beused.Forexample,thesub-intervalsintheinputsspacecanbeconstructedinsuch waytoavoidthediscontinuity.PCE sub-problemswouldthereforebetreatedasusual.

Asecondlimitationconcernsthechoiceofinputvariablesforregressionmodels,andisanongoingresearchquestionin statistics[46].Asapracticalillustration,thedynamicalproblemwritteninEquation(11) canincorporateadditionalinputs, forexamplethe informationatprevious timestj−1,tj−2,etc.However, whena large setofinputscanbe used andonly

asmallsetofrealizationsisavailable forlearning,awell-posednessproblemoccurs.Onesolutionconsistsintransforming thelargesetofinputstoa reducedversion,forexamplewiththehelpofPCA[47] forDR.Thisapproachwasnot studied hereandwill be thetopicofa futurestudy.However, differentinput conﬁgurations willbe evaluated, toinvestigatethe inﬂuenceofvariableselection ontheproposedlearning. Forexample,thehypothesis ofdependencebetweentherandom variables

(

A1

,

. . . ,

Ad

)

could be relaxed. Thiswould implywritingthe approximation in Equation(11) in a relaxedform

as

H

k

ak

(

tj

),

tj+1

−

tj

, (

tj

→

tj+1

)

.Inthatcaseasimplermodel

H

,underthestrongindependenceassumption,canbe formulatedasinEquation(13).

H

U

(

X ,

tj

),

tj+1

−

tj

, (t

j

→

tj+1

)

≈

d

k=1

H

k

ak

(t

j

),t

j+1

−

tj

, (t

j

→

tj+1

)

n

× λ

k

(kd)

(

X ) .

(13)

Both alternativesare testedinSection4.Toinvestigatetheinﬂuenceofinputselection onlearning accuracy,a quanti-tative evaluationofthe hypothesisis needed.Moregenerally, whetherforthe above-mentionedsimpliﬁcationsorforthe approximatedformofthemodelingeneral,accuracyestimatorsareneeded.Thesearepresentedbelow.

2.3.2. Accuracytestsfortheapproximation

There are two determining parts in the POD-PCE learning process. Firstly, the PCE learning

H

k

(.)

of each mode Ak

should beasaccurateaspossible.Secondly,thereconstructedﬁeld

d_k₌₁

H

k

(.)

√

n

× λ

k

(kd)

(

X )

foragivenrankd should

beasclosetotherealﬁeld U

(

X )

aspossible.

The distance between each mode and its PCE approximate can be evaluated using the generalizationerror, denoted

δ(

Ak

,

H

k

)

anddeﬁnedasinEquation(14).

δ(A

k

,

H

k

)

= E

(

Ak

−

H

k

(.))

2

.

(14)

Forthe modeldeﬁnedinEquation(13),thiserrorcan be estimated,on aset ofpairedrealizations

(

ak

(

t1

),

. . . ,

aj

(

tn

))

and

((

t0

→

t1

),

. . . ,

(

tn−1

→

tn

))

,asin Equation(15) asexplained by Blatman [28].This approximatedversion of the

generalizationerror iscalledtheempiricalerror.

δ(A

k

,

H

k

)

≈ δ

emp

(

Ak

,

H

k

)

:=

1 n n

j=1

ak

(t

j

)

−

H

k

ak

(t

j−1

),

tj+1

−

tj

, (t

j−1

→

tj

)

2

.

(15)

(10)

Itsrelativeestimatedenoted

emp

(

Ak

,

H

k

)

canbedeﬁnedasinEquation(16).

emp

(

Ak

,

H

k

)

:=

δ

emp

(

Ak

,

H

k

)

V

[

Ak

]

.

(16)

OncethePCElearningscanbetrusted,thedistanceattimetj betweenthetruestateU

(

X ,

tj

)

andthePOD-PCE

approx-imation

H

U

(

X ,

tj

),

tj+1

−

tj

, (

tj

→

tj+1

)

canbe deﬁned.It mightbe estimatedusing therelativeRoot MeanSquared Error (relativeRMSE), denoted r

[

U

,

H](

tj

)

and calculated asin Equation (17), where h

(

xi

,

tj

)

refers to the value of the

POD-PCEapproximationatcoordinatexi andtimetj.

r

[

U

,

H

](

tj

)

:=

mi=1

u

(

xi

,t

j

)

−

h

(

xi

,t

j

)

2

m i=1

u

(

xi

,

tj

)

2

.

(17)

AmeanvalueoftherelativeRMSEiscalculatedoverasetofrealizationscorrespondingtoasetoftimes

T = {

t1

,

. . . ,

tn

}

.

Itisdenotedr

[

U

,

H]

(T )andestimatedasinEquation(18).

r

[

U

,

H

]

(T )

:=

1 n n

j=1 r

[

U

,

H

](

tj

) .

(18)

OncetheaccuraciesofthePCElearningsandthe POD-PCEcouplinghavebeenevaluated,aﬁnal model,whichwillbe themostaccurateone,canbechosen.Thismodelwould, forourMLset-up,bethebestrepresentationofthedependence structurebetweeninputsandoutputs.Itisusedtoshedlightontheunderlyingphysicalrelationships.Thereforetheinputs arerankedintermsofphysicalinﬂuence,usinganappropriaterankingindicator,presentedinthefollowingSubsection.

2.3.3. PhysicalinﬂuenceofinputsbasedonthePOD-PCEmodel

TheGWinﬂuencemeasurespresentedforthePCEmodelsinSubsection2.2arehereextendedforthePOD-PCEcoupling. These indicatorsare adequate for theanalysis ofeach PCE model

H

k,i.e.for interpreting thecontribution ofthe inputs

toeachrandomvariable Ak separately.However,calculatingthecontributionstoeach Ak independentlyprecludesputting

them in perspective according to the importance of Ak in the ﬁnal reconstructed model

H

that approximates U

(

X ,

.)

. Hence,adaptedindicatorsshouldbecalculated.

LetU

(

X ,

.)

betherandomspatiotemporal ﬁeldapproximatedby thePOD-PCE ML,forpredictionfromtimetj totime tj+1 andlet

H

kbethePCEapproximationatdegree p(k)thatmapstherandomPODtemporalcoeﬃcient Akfromasetof

input variables,usingtheexpansiononthemultivariate polynomialbasis

ζ

α(k)

(.)

|α|≤p(k).ThePOD-PCEmodelformulated

inEquation(12) iswrittenasinEquation(19):

U

(

X , .) ≈

d

k=1 Ak

n

× λ

k

_k(d)

(

X ) ≈

d

k=1

⎛

⎝

|α|≤p(k) c(_αk)

ζ

α(k)

(.)

⎞

⎠

n

× λ

k

(_kd)

(

X ) .

(19)

Thankstoitslinearity,thePOD-PCEMLcanberepresentedasasingle-layeredNN,asshowninFig.3. Therefore,anewindicator,GeneralizedGarsonWeights (GGW),denotedW_ζ(k)

α ,iscomputedandsimplyre-evaluatedfrom thePCEGarsonweights(GW),heredenotedw

ζα(k),asinEquation(20). W ζα(k)

:=

|

c(_αk)

|

√

n

× λ

k

d e=1

|β|≤p(e)

|

c(_βe)

|

√

n

× λ

e

=

|β|≤p(k)

|

c(_βk)

|

w ζα(k)

√

λ

k

d e=1

|β|≤p(e)

|

c_β(e)

|

√

λ

e

=

⎛

⎝

|β|≤p(k)

|

c(_βk)

|

√

λ

k

d e=1

|β|≤p(e)

|

c(_βe)

|

√

λ

e

⎞

⎠

w ζ_α(k)

.

(20)

TheseGGWindicatorsshowthatthecontributionofthepolynomials

{ζ

α(k)

}

|α|≤p(k)ofAkareenhancedwiththeeigenvalue

λ

k,whichisdirectlylinkedto theimportanceofthePODmode

_k(d)

(

X )

(EVR inEquation(2)).Ananalogycanbedrawn

withthegeneralizedsensitivityindicesforareducedordermodel[48].The

d_k₌₁

_|α|≤_p(k)W

ζα(k)

=

1 propertyholds.This means that the indices allow

{{ζ

_α(k)

}

_|α|≤_p(k)

}

k∈{1,...,d} to be ranked altogether in terms of contribution to output U. The

(11)

Fig. 3. Representation of the POD-PCE ML approach in the NN paradigm.

3. Applicationonaparametrictoyproblem

The theoretical framework ofthe proposed POD-PCE learning was presented in the previous Section 2, including the detailedcouplingformulation, accuracyestimatorsandphysicalinﬂuencemeasures inSubsection2.3.Inthe latter,itwas highlightedthatthereisaslightdifferenceinthelearningandpredictionstepsbetweentemporalproblemsandparametric problems. In this section, the POD-PCE ML is applied to a parametric toy problem, for which the analytical solution is introduced inSubsection3.1.Theproblemissimpleandcontrollable,andallowsdemonstratingthelearning performance, theconsistencyofphysicalinterpretationsincomparisonwiththeanalyticalinformation,andtherobustnessofthelearning tonoiseinthedata.Subsection3.2thereforedealswiththeapplicationofthePOD-PCEmethodologyforphysicalanalysis andprediction,whileinSubsection3.3,robustnesstodifferentnoiselevelsisinvestigated.

3.1. Problemdescription

The chosen toy problem deals with the representation of groundwater flow in a confined aquifer. Such a flow can be complex todescribe andisgenerally representedusingthe depth-averaged groundwaterflow equations[34]. Analyti-cal solutions for theseequationscan be found forparticular configurations. Forexample,a solution was identified by Li et al. [34] incaseofa semi-infinitecoastalaquifersubjectto oscillatingboundary conditions,resultingfromoceanicand estuarine tidal loadings. The solution is givenforthe particular casewhere the estuary andcoastline are perpendicular. The oceanicBC(alongcoastline) istakenasasingle andspatiallyuniformtidalharmonicconstituent Acos

(

ω

t

)

,where A

and

ω

arethe tidalamplitudeandpulsation respectively.The corresponding BC alongthe estuaryis anon-uniformtidal loading Aexp

(

−

κ

erx

)

cos

(

ω

t

−

κ

eix

)

, where

κ

er and

κ

ei are theestuary’s tidaldamping coeﬃcientandtidal wave number

respectively,thatrepresentchangesintheamplitudeandphasealongtheestuary.

Thisforcing resultswithfluctuationsinthewatertable,that isdefinedasthelevelseparatingthewaterandsaturated groundfromtheremainingupperunsaturated ground.The fluctuations,denoted f ,can becalculatedusingtheanalytical solutiondefinedin[34] asinEquation(21).

⎧

⎪

⎨

⎪

⎩

f

(x,

y,t

)

=

f0

(x,

t)

+

f1

(x,

y,t

)

f0

(x,t)

=

Aexp

−

ω 2Dx

cos

ω

t

−

ω 2Dx

f1

(x,

y,t)

=

A

×

Re

t 0[g

(k

1

,

x)

−

g(k1

,

−

x)

−

g(k2

,

x)

+

g(k2

,

−

x)] dt0

(21)

whereconstant D isthediffusivityoftheaquifer[34],t isthetime variable,and

(

x

,

y

)

are thecartesiancross- and long-shore coordinates,corresponding tothedistancefromoceanandestuaryrespectively.The operator Re

{

z

}

denotesthereal