Bayesian selection of graphical regulatory models International Journal of Approximate Reasoning

(1)

Contents lists available atScienceDirect

International Journal of Approximate Reasoning

www.elsevier.com/locate/ijar

Bayesian selection of graphical regulatory models

^✩

Silvia Liverani^a^,^∗,Jim Q. Smith^b

aDepartmentofMathematics,BrunelUniversityLondon,UK bDepartmentofStatistics,UniversityofWarwick,Coventry,UK

a r t i c l e i n f o a b s t r a c t

Articlehistory:

Received15October2015

Receivedinrevisedform5May2016 Accepted31May2016

Availableonline15June2016

Keywords:

Clustering Bayesianinference Causality

Time-coursemicroarrayexperiments

We define a new class of coloured graphical models, called regulatory graphs. These graphs have their own distinctive formal semantics and can directly represent typical qualitativehypothesesaboutregulatoryprocesseslikethosedescribedbyvariousbiological mechanisms.Theyadmitanembellishmentintoclassesofprobabilisticstatisticalmodels and sostandard Bayesian methodsofmodel selection canbeused tochoose promising candidate explanations of regulation. Regulation is modelled by the existence of a deterministicrelationshipbetweenthelongitudinalseriesofobservationslabelled bythe receivingvertexandthedonatingone.Thisclasscontainslongitudinalclustermodelsasa degenerategraph.Edgecoloursdirectlydistinguishimportant featuresofthemechanism likeinhibitionandexcitationandgraphs areoftencyclic.Withappropriatedistributional assumptions,becausetheregulatory relationshipsmapontoeachotherthroughagroup structure,it ispossible todefineaconditionalconjugate analysis.Thismeansthateven whenthe model spaceishuge itisneverthelessfeasible, usingaBayesian MAPsearch, to a discover regulatory network with a high Bayes Factor score. We also show that, liketheclassofBayesianNetworks,regulatorygraphsalsoadmitaformalbutdistinctive causalalgebra.Thetopologyofthegraphthenrepresentscollectionsofhypothesesabout thepredicted effectofcontrollingthe processbytearingoutmessagepassers orforcing themtotransmit certainsignals. We illustrateourmethodsonamicroarrayexperiment measuring the expression of thousands of genes as a longitudinal series where the scientificinterestliesinthecircadianregulationoftheseplants.

©²⁰¹⁶^TheÂuthor(s).^Published^byÊlsevierÎnc.^Thisîsânôpenâccessârticleûnder^the CCBY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

There has been a recentexplosion in the development of newtechniques in exploratory dataanalysis forhuge data setsfuelled,forexample,byintenseactivitiesinthestudyofgenomesequencesandDNAmicroarrayanalyses.Examplesof such exploratorytoolsarethefastBayesian methods[7,21,8],designedtocluster tensofthousandsoflongitudinalseries ofmeasurements.Theoutputs fromthesemethods,especially thedisplaysofthecluster means(seee.g. Fig. 1)ofa MAP estimatedmodel, haveproved to be particularlyhelpfulto biologistswhen they explore hypotheses aboutwhichsets of genesmightbepassingamessagetoanother.

HowevertheseBayesianclustermethodsbeginwiththeassumptionthatclustersexpressindependentlyofoneanother.

Thisisclearly notusuallyacrediblehypothesisforregulation.Forunderahypothesisthatoneclusterregulatesanotherit

✩ ThispaperispartofthevirtualspecialissueonStatisticalandcomputationalmethodsforgenomicsandintegrativegenomics,editedbyPaolaRancoita andCassioCampos.

* Correspondingauthor.

E-mailaddress:[email protected](S. Liverani).

http://dx.doi.org/10.1016/j.ijar.2016.05.007

0888-613X/©²⁰¹⁶^TheÂuthor(s).^Published^byÊlsevierÎnc.^Thisîsânôpenâccessârticleûnder^the^CC^BY-NC-ND^license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

(2)

istypicallybelievedthattheshapeoftheproﬁleofthereceivingclusterwillberelatedtotheproﬁleofthesendingcluster inarelativelypredictableway.

Various authorshavethereforemodelledthelikely dependencesbetweenthetime coursesofgeneexpressionproﬁles usingavarietyofstandardgraphicalmodels:oftenvariantsoftheBayesianNetwork(BN)[9].Thus[12],acknowledgingthe consequentcomputationalandmethodologicallimitationsoftheirtechniques,useddynamicBNstomodelsubsetsofseries like those we model here. Albeitin a non-exploratory context[19] analysed variations ofa baseline model representing knowndependencestructures.Othergraphicalstudiesofregulatorymodelsinclude[4].

Althoughthesegraphicalmodelsareclearlyveryuseful,particularlyatlaterstagesoftheanalysisofagivenprocess,they arehardlyeverintegratedintoanearlyexploratorydataanalysis[2,13,20].Furthermorewearguebelowthatthesemantics oftheBNisnotwellmatchedtomodellingdependencesinducedbyregulation.Ratherthanrepresentingconditionalinde- pendence relationships,ideallywe wouldlikeourgraphtobe explicitlyregulatory:i.e.thatadirectededgefromaparent node v₁isconnectedtoachildv₂ ifandonlyifthemodelembodiesthehypothesisthattheobjectv₁regulates v₂.Clearly thisisnotingeneraltrueofaBN.Inparticularthereisnowayitsacyclicgraphcouldembodyanycyclicalregulation,the mostimportanttypeofrelationshipinourrunningexample.

InthispaperwedevelopanewgraphicalmodelfordepictingregulatoryrelationshipswithinaGaussianfamily,building on aclassofBayesianclustermodels thathavebeenwidelyandsuccessfullyimplemented [3,7].Themodelclassembeds the class ofBayesian longitudinalcluster models asaspecial case. Weare aware ofno similarwork in theliterature, in scopenordesign.Wedemonstratehowthemethodprovides:

1. afeasibleexploratorytechniqueforregulatorynetworksoflongitudinalhigh-dimensionaldata;

2. aformalsemanticsthatembodiesmostoftheusefulpropertiesoftheBN;

3. aframeworkforaconditionalconjugateanalysis;

4. aformal graphicalrepresentationofa MAPmodelwhichcan representthetypesofdependencesthe scientistmight hypothesise; thisgraphicalrepresentation willbe ina formwhich closelycorresponds tothe lessformal graphs she mightcurrentlyusetodepictthatmodel’sregulatorymechanism;

5. anassociatedcausalalgebraenabling thescientist,undercertainassumptions, togeneratepredictionsoftheeffectof certaintypesofcontrolonthesystem.

Thisworkismotivatedbyourcollaborationwithbiologistssoourrunningexampleconcernsthestudyofthestatistical modelsoftheregulationmechanismofthecircadianclockinplants.However,theapplicabilityofthisnewclassofmodels extendsbeyondgeneticsandbioinformatics.

InSection 1.1we introducethedatausedinthepaperandinSection2webriefly reviewtheformofBayesian cluster analysis which we plan to generalise. Then in Section 3 we use a group action which is hypothesised to embody the relationshipoftheprofileshapeofareceivingclusterandtheprofileofitsregulatingcluster.Thisgroupdefinesequivalence classes ofclusterscalled supraclusters.These supraclusterswill define the disconnectedcomponents ofthe graphof our process.InSection4wediscussgraphicalrepresentationsforsupraclusters.Wecompleteourmethodologicaldevelopment in Section 5wherewe providea causalinterpretation forourgraphs ofsupraclusters.We thendemonstrate ourmethod usingtwoexamplesinSection6:asimulatedscenarioandarealsystem.Inthiswayweillustratenotonlythatourmethods arefeasibleandevocativebutalsohowtheoutputgraphsprovideahelpfulandfamiliarframeworkthroughwhichresults canbefedbacktothescientist.

1.1. Data

Wedemonstratethatourmethodsuccessfullyidentiﬁesdependencebetweenclustersofgenes intwoexamples:a sim- ulatedrunningexampleandamoredetailedanalysisofagenuinemicroarrayexperimentontheplantArabidopsisthaliana.

These experiments were designedto detect genes whose expression levels, andhencefunctionality, might be connected withcircadianrhythms.Theaimwastoinvestigatethepossibleidentityandroleofthosegenesinvolvedintheregulation ofthecircadianclockofaplant. Forourrunningexamplewesimulated5clustersof30geneseach,ofthetype discussed in[5].HerethegeneexpressionwasmeasuredatT=13 timepointsovertwodays.Constantwhitelightwasshoneonthe plantsfor26hoursbeforetheﬁrstmicroarraywastaken,withsampleseveryfourhours.Thus,therearetwocyclesofdata foreachoftheArabidopsismicroarraychip.Thedata y_z_i wassimulatedfrom

y_i=λz_iB(θz₁)Rz_iβ+ε_i

wherethesubscriptzi= ^j^if^individualⁱ^belongs^to^cluster ^j^with ^j=¹,. . . ,5 and εi∼^Normal(0,4).ThematrixB(θz_i)isa Fourierbasisfunction(seeEquation(2)formoredetails)andRz_i isarotationmatrix,thatis,ablockdiagonalmatrixwhere eachblockisamatrixM(k,θzi)as

M(k, θ_z_i)=

cos(kθ ) −^sin(kθ ) sin(kθ ) cos(kθ )

(3)

withkcorrespondingtotheindexoftheFouriercoeﬃcients.Thegeneratingvaluesofλzi,θzi andβ aregiveninSection6.

Forthesecondexampleweanalysedthisdatasetinfull (22,810genes whoseexpressionsareeach observedover13time point)aswellasotherdatasets,aswediscussinSection6.

2. Clustermodelsforlongitudinaldata

ThestudyofthefamilyofregulatorygraphsproposedinthispaperbeganwithourworkonBayesianclustermodels.In theapplicationsoftheproposedmethodologythereare N –usuallytensofthousands–longitudinalsetsofmeasurements oftheactivityofunitstakenatparticulardiscretepointsovertime.Tounderstandtheunderlyingprocessesitistherefore importanttolearnwhichoftheseprocessesarecopiesofeachother,asevidencedbythefactthat theirproﬁlesappearto bereplicates.Inthiswaywecanreducethetensofthousandsoftimeseriesintoamuchsmallernumberof K clusters.

TypicallyinourapplicationsK dependsonhowwesetthehyperparametersofourmodels.Howeverforourrealdataset thereareusuallyaboutonehundredlongitudinalclusters:astilllargebutmoremanageablenumber.Eachoftheselongitu- dinalclustersisthentreatedasacandidateregulator.Theunderlyingdependencestructurecanberepresentedgraphically anditcanbefurtherelaboratedtodescribehypothesesabouttheeffectsofthetypesofcontrolseludedtoabove.

Wefirstneedtosetupsomenotation.LetC{C1,C2, . . .CK}denoteapartitionofindices{1,2, . . . ,N}, N≥K,where N_k denotes the cardinalityof cluster C_k.Let Y_i, i=¹,2,. . . ,N, denotea set of N T-vectors ofreal valuedobservations of the continuous time development ofa unit at T given time points – henceforth calleda profile. Let DC, the T ×^Nj matrixofvaluesbedefined by D_C= {^Yi:ⁱ∈^C}^for^some^subset ^C ôf{¹,2, . . . ,N}^.^Let^thedistributionof{^Y1,Y₂, . . . ,Y_N} be parametrised by θ∈, andfixed hyperparameters ϕ_∈.Let θ=(θ1,θ2, . . .θK) whereθ_k,for k=¹,2,. . . ,K isthe parametervectorspecific tothek-thclusterwithN_k elements.Notethatthen_K

k=¹N_k=^N.^The^ﬁrst^Bayesian^model^we reviewisonewhichadmitsnodependenceorcausalstructurebetweenanyoftheobjectsofinterest–theclusterproﬁles.

Definition1.ABayesclustermodelC ôfâ^setôf^profiles{^Y1,Y2, . . . ,YN}âssumes^that:

1. proﬁlesindifferentclustersandtheirassociatedparametersareindependentgiventhehyperparametersi.e.

_k^K₌₁ D_C_k,θk

|ϕ

2. withineachclustertheproﬁlesYi|θk,ϕareidenticallydistributed,i=¹,2,. . . ,Nk,k=¹,2,. . . ,K andconditionallyon eachcluster’sparametervectorθkandthehyperparameters ϕ theseobservationsaretakenindependently:i.e.

^N_i₌^k₁^Yi|θ_k,ϕ

TherenowareseveralsuchBayesianclustermodelswhichhavebeensuccessfullyandusefullyimplemented[7,16,5,17].

ManytaketheformthatforclusterCk thetimeseriesofobservationsismodelled byalinearregression

Y⁽_i^k⁾=^X⁽^k⁾β⁽^k⁾+ε⁽_i^k⁾ (1)

fork=¹,. . . ,K andi=¹,. . . ,N_k whereβ⁽^k⁾∈R^p îs ^the^vector ôf^parameters ^with ^p≤^T^, ^whereε⁽_i^k⁾ are independent identicallydistributedmeasurementerrorswithvariance σ_k² andthedesignmatrix,orbasisfunction,X⁽^k⁾,amatrixofsize N_kT×^p,îs ^customised^to ^the hypothesisedunderlying process. Inour running examplewhere interest liesin regulators actingoveratwentyfourhourcycle,anappropriatebasisfunctionisFourier.Sothen,forexamplewhen piseven,

y⁽_it^k⁾=β₁⁽^k⁾+

p/2

i=¹

β_2i⁽^k⁾cos(2πt(2i)/T)+

p/2

i=¹

β_2i⁽^k₊⁾₁sin(2πt(2i)/T)+ε_it⁽^k⁾. (2) Here theparameters θ_k associatedwithcluster C_k are thecorresponding regressionparameters β_k andan errorvariance parameterlinkedtothewithinclusterdiversity.

Withintheseclassesitisoftenpossible,usinganappropriate conjugateanalysis,toconstructascorefunctionbasedon thecorrespondingBayesFactor–aMAPscorewhichcanbederivedinclosedformconditionalonthehyperparametersfor eachpartitionoftheproﬁles.ThusiferrorsandpriorparameterdistributionsareassumedGaussian,andthenoisevariance aprioriinverseGammadistributedthenconditionalonaﬁxednoise-to-signalratiomatrix,themarginallikelihoodofeach possibleclusterpartition isaproduct ofmultivariate t densities.Forthedetails ofhow thesemethods workseee.g.[3,7, 17]and[10].Ithasnowbeenestablishedthat theseandrelatedmethodologiescansuccessfullyandfeasiblyidentifywell supportedmodelswithinthesetypicallyhugesearchspaces.

Themethodsdiscussed inthissection arewidelyusedintheliterature onclusteringofmicroarray experiments.Fig. 1 depictstheexpressionproﬁleswithin afewtypicalclusterstogether withgraphsoftheclusters’ meanFouriercoeﬃcients intheanalysisgivenin[5],oneofthemanyanalysesperformedusingthismethodology.

Ourexperience usingthese methodshave demonstratedthat withthe carefulsettingof hyperparametersthe outputs ofsuch modelsare remarkablyrobust,both formallyandpractically, whenit comesto theestimationofparameters that mightbelinkedtoregulation[5,17,10,11].

(4)

Fig. 1.Fouroftheclustersobtainedclusteringthedatain[5],ourrealdataapplication.Thebluelinerepresentstheposteriormean,thethickercoloured linesarewellknowngenesandthebarplotontheleftaretheposteriorestimatesoftheFouriercoeﬃcients.(Forinterpretationofthereferencestocolour inthisﬁgurelegend,thereaderisreferredtothewebversionofthisarticle.)

3. Supraclusteringandregulatorymodels

Thistype ofcluster analysiscan beextended tomodeldirectlythedependences betweenclustersifaregulatory rela- tionship betweenthem canbespecifiedasfollows.Atransmissionfunctionisdefinedwhichmapstheparametersofone unit’s profileon tothe parameters oftheprofile it regulates.These functionsarethen coded asa groupof transmission matrices,customisedtomirrorthetypesofdependencesinducedbyaparticularhypothesisedregulatingmechanism.

In this section we fix the value of the cluster hyperparameter ϕ_∈ and for clarity suppressthis parameter in the indexing.Then forthetransitionhyperparametersφ∈,therearetwo groupsusedtomap theshapeofoneprofileonto another(L^∗⁾ândône^cluster’s^parametersôntoânother⁽L^).

Forsomeintegerm letL^∗

L^∗(φ):φ∈⊆R^m ^beâ^familyôf^T×^T ^transition^matricesûsed^to^defineâ ^regulatory relationshipfromaprofileYi₁ ofagivenuniti1toasecondunitwiththeprofileYi₂,wherethetransitionhyperparameter φ indexeseach matrixin thisfamily.AssociatedwithL^∗ îsâ ^set L

L(φ):φ∈⊆R^m ^of(p+¹)×(p+¹)matrices mappingtheparametersθj₁,i∈^Cj₁ totheparametersoftheregulatedunitθj₂,i2∈^Cj₂,suchthat

θj2^L(φ_j₁_→_j₂)θj1

ThefamilyoftransitionfunctionsLparametrisedbyφj1→^j2mustfirstbeelicitedfromthedomainexpertsincethismust be determinedfromthose featuresinthedata shebelieveswillbeindicativeofa regulatoryrelationship.Inour running examplesuch relationshipswerecommunicatedtous:“Thetime profileofexpressionofaregulatedgeneusually appears tobeashorttranslationoftheprofileoftheregulatingclusterofgenes,usuallyrescaledandsometimes,ifinhibitedrather than excited, reversed”.This determineda 3parametertransitionfunction witha parametercapturingthephase change, another expressing apositive rescaling anda third representinginhibition, which correspondedtoa simple signchange.

In thispaper, bothforsimplicity andbecausethe subclassseemed toﬁt well, we conﬁnedourselves tosearch over this parameterclass,althoughotherofourmorecomplexmodelscouldalsomodeldampingeffects.Ofcourseinothersettings andwithdifferentinterpretationsofbasisfunctionsthegroupoftransformationscouldbeverydifferent.

For thepurposes of thispaperwe will assume, asinour runningexample, that both L^∗ ând L^formâ ^group ûnder matrixmultiplication.Writei2ⁱ1 wherei2∈^Cj₂,i1∈^Cj₁ andCj₁,Cj₂∈C îff∃φ_j₁_→_j₂∈suchthat