Time-frequency Granger causality with application to nonstationary brain signals

(1)

Thesis

Reference

Time-frequency Granger causality with application to nonstationary brain signals

CEKIC, Sezen

Abstract

This PhD thesis concerns the modelling of time-varying causal relationships between two signals, with a focus on signals measuring neural activities. The ability to compute a dynamic and frequency-specific causality statistic in this context is essential and Granger causality provides a natural statistical tool. In Chapter 1 we propose a review of the existing methods allowing one to measure time-varying frequency-specific Granger causality and discuss their advantages and drawbacks. Based on this review, we propose in Chapter 2 an estimator of a linear Gaussian vector autoregressive model with coefficients evolving over time. Estimation procedure is achieved through variational Bayesian approximation and the model provides a dynamical Granger-causality statistic that is quite natural. We propose an extension to the à trous Haar decomposition that allows us to derive the desired dynamical and frequency-specific Granger-causality statistic. In Chapter 3 we propose an application of the model to real experimental data.

CEKIC, Sezen. Time-frequency Granger causality with application to nonstationary brain signals. Thèse de doctorat : Univ. Genève, 2015, no. GSEM 18

URN : urn:nbn:ch:unige-789688

DOI : 10.13097/archive-ouverte/unige:78968

Available at:

http://archive-ouverte.unige.ch/unige:78968

Disclaimer: layout of this document may differ from the published version.

1 / 1

(2)

Time-Frequency Granger Causality

with Application to

Nonstationary Brain Signals

by

Sezen Cekic

A thesis submitted to the

Geneva School of Economics and Management, University of Geneva, Switzerland,

in fulfillment of the requirements for the degree of PhD in Statistics

Members of the thesis committee:

Prof. Olivier Renaud, Adviser, University of Geneva Prof. Didier Grandjean, Adviser, University of Geneva Prof. Maria-Pia Victoria-Feser, Chair, University of Geneva Prof. Anthony Davison, École Polytechnique Fédérale de Lausanne

Thesis No. 18 December 2015

(3)

(4)

(5)

(6)

Acknowledgements

A l’ach`` evement de ce travail de thèse, je suis évidemment convaincue que le cheminement est loin d’avoir été solitaire. Je n’aurais jamais pu réaliser ce travail sans le soutien d’un grand nombre de personnes dont la générosité, la bonne humeur et la patience à mon

´

egard m’ont permis d’avancer et de progresser dans ce tr`es curieux environnement mi- jungle mi-d´esert qu’est celui du travail doctoral.

Tout d’abord, je tiens à remercier chaleureusement mes deux directeurs de thèse et mentors académique.

Le Professeur Olivier Renaud, qui m’a fait confiance et m’a pris sous son aile dès le début de mon travail de master et qui sans relâche a été à mes côtés, bienveillant et pilier de ce travail de thèse. Son caractère doux et conciliant, sa sympathie, sa profonde humilité ainsi que ses immenses qualités scientifiques m’ont donné confiance et m’ont permis d’avancer jusqu’ici.

Le Professeur Didier Grandjean ensuite, qui a cru en moi dès le début en me pro- posant un challenge impossible pour mon travail de master et qui m’a proposé ensuite de poursuivre avec une thèse. Son énergie, son inventivité et sa très grande intelligence ont beaucoup contribué à rythmer ce travail et je lui suis extrêmement reconnaissante de m’avoir mise sur les voies de la carrière académique.

J’ai eu beaucoup de chance et de plaisir à travailler aux côtés de mes deux directeurs de thèse et j’espère n’en être qu’au début de nos collaborations scientifiques.

Je remercie la pr´esidente du jury, la Professeure et Doyenne Maria-Pia Victoria-Feser.

Je la remercie tout d’abord chaleureusement d’avoir soutenu ce projet dès ses tout débuts en acceptant d’en être la co-directrice. Je la remercie ensuite de m’avoir tendu la main au moment critique où j’en avais besoin en m’ouvrant les portes de la nouvelle GSEM.

Je remercie sincèrement le Professeur Anthony Davison de nous avoir fait l’honneur d’être le jury de ce travail de thèse. Je le remercie pour sa lecture extrêmement attentive et ses précieux commentaires qui ont sans aucun doute beaucoup amélioré la qualité du manuscrit.

Je remercie enfin mon collègue Roberto Molinari d’avoir accepté de relire entièrement ma thèse et d’y apporter les corrections nécessaires à mon anglais de basse-cour.

L’achèvement de cette thèse me fait évidemment penser à mes anciens collègues de master, William, James et Dravasp qui m’ont énormément aidée à combler mes très nom- breuses lacunes au début de ma carrière en statistique. Merci à eux pour leur aide sans laquelle je n’aurais très certainement pas survécu à ce moment-là.

Ces années de thèse et les différentes équipes côtoyées m’ont permis de rencontrer bon nombre de gens que je tiens à remercier ici. Tout d’abord mes collègues du NEAD et du CISA, Kim, Alison, Julie, Valérie, Léo, Andy, Nico et tous les autres pour ces premières

(7)

années de thèse remplies de bonne humeur, de congrès et de soirées en tous genre. Merci ensuite à mes collègues de la GSEM, à Élise, Rose, Roberto, Samuel, Kustrim, Marc–

Olivier et Mattia de m’avoir chaleureusement accueuillie au sein de leur équipe et du Rob’in Café. Merci `a eux pour leur bonne humeur et leur camaraderie qui ont fait beaucoup à la période doctorale à laquelle je me trouvais au moment où je suis arrivée chez eux. Merci finalement à mes adorables collègues du MAD, à Eda, Émilie, Emma, Nadège, Catherine, Jaromil, Boris, Marc, Julien et Paolo pour leur accueil au sein de leur équipe.

Je ne pouvais pas esp´erer mieux que leur compagnie pour finir ma th`ese.

Je ne serai bien-sûr jamais venue à bout de ce travail sans l’aide et le soutien de tous mes proches qui me sont si chers et que je tiens à remercier ici. Tout d’abord merci à mes amis, particulièrement à Melia, Vanille, Clara, Dragana, Antonin, Léo et à tous les autres bien-sûr, pour leur amitié indéfectible, leur soutient et surtout pour tous ces moments vitaux partagés avec eux tout au long de ces années.

Merci ensuite à ma belle-famille, à Fabienne, Alan, Carla, Léonard, Marine, Thomas, César et James pour leur présence si chère et si importante dans ma vie.

Un merci particulier à mes deux merveilleuses amies Aurélie et Malvina pour leur incroyable présence, leur amour et leur soutient pendant toutes ces années. Merci à elles de s’intéresser pareillement à ce que je fais et d’avoir compris, supporté et remonté tous mes états d’âme depuis mes tout débuts dans ce monde de la statistique.

Merci ensuite à ma famille, à mon oncle Ariel et ma grand-mère Gabriele, de m’avoir appris tant de choses, d’être à mes côtés et de croire en moi depuis depuis toutes ces années.

Merci à ma sœur Sevline, à mon beau-père Francis et à ma maman Oria d’avoir toujours été là pour moi, d’avoir cru en moi dès le début, de m’avoir laissée être moi, faire mes propres choix sans jamais les juger mais en tentant plutôt de comprendre en étant présents et aimants à mes côtés. Merci à ma maman d’avoir été un tel exemple de vie et de m’avoir appris très vite que quand on veut, on se donne les moyens, et on peut.

Finalement merci à mon amour Arthur d’être là depuis le début, merci d’avoir soutenu,

´

ecouté, recadré, encouragé, calmé, reboosté et j’en passe! Merci d’avoir tout supporté, d’avoir compris ce que c’était et ce que cela impliquait de faire une thèse et surtout merci de m’avoir aidée tout au long de cette aventure à sans cesse remettre les vraies choses à leur vraie place. Merci à lui d’avoir vécu tout cela à mes côtés et de m’avoir donné tant de force à travers sa bonne humeur, son énergie et sa joie de vivre hors du commun.

(8)

Abstract

This PhD thesis concerns the modelling of time-varying causal relationships between two signals, with a focus on signals measuring neural activities. Indeed, in an experimental context, data are recorded during an experimental situation where stimuli are presented at fixed times and are expected to induce a reaction. The causal links between recorded signals may therefore vary in time and be frequency-specific. The ability to compute a dynamic and frequency-specific causality statistic is essential. Granger causality provides a natural statistical tool allowing us to investigate possible causal relationships between two signals, its properties being assessed a few decades ago (Granger,1969).

In Chapter 1 we propose a review of the existing methods allowing one to measure time- varying frequency-specific Granger causality and discuss their advantages and drawbacks.

Based on this review, we propose in Chapter 2 the use of an adaptive Kalman filter type of estimator of a linear Gaussian vector autoregressive (LGVAR) model with coefficients evolving over time. Estimation procedure is achieved through variational Bayesian approximation (Beal, 2003). This Bayesian State Space model (BSS model), proposed in Cassidy (2002) provides a dynamical Granger-causality statistic that is quite natural.

The Bayesian nature of the model provides a criterion for model order selection and allows us to include prior knowledge. We propose to extend the BSS model to include the

`

a trous Haar decomposition. We call this new model the multiscale Bayesian state space model (MSBSS model). This wavelet-based forecasting method based on a multiple reso- lution decomposition of the signal using the redundant `a trous wavelet transform allows us to capture short- and long-range dependencies between signals and further allows us to derive the desired dynamical and frequency-specific Granger-causality statistic.

Finally in Chapter 3 we propose an application of the MSBSS model to intracranial electroencephalogram data recorded in the amygdala and medial orbitofrontal cortex brain regions during experimental emotional auditory stimuli recognition. We present preliminary results for this work, which is in progress.

(9)

(10)

R´ esum´ e

L’objet de recherche de cette thèse de doctorat concerne la modélisation des relations causales dynamiques entre deux signaux avec une application spécifique aux signaux mesurant une activité cérébrale.

Dans un contexte expérimental, les stimuli sont présentés à instants fixes et l’on s’attend à ce qu’ils provoquent une réaction chez le sujet. Pour les chercheurs, être en mesure de décrire le lien causal existant entre les signaux enregistrés dans les différentes régions du cerveau est d’un intérêt premier. Les liens de causalité entre les signaux enreg- istrés pouvant varier dans le temps et être spécifiques à certaines bandes de fréquences, une statistique de causalité dynamique et fréquence spécifique est essentielle. La causalité de Granger vient naturellement à l’esprit comme outil statistique solide permettant de tester des relations causales entre deux signaux (Granger,1969).

Nous proposons tout d’abord dans le Chapitre 1 une revue des méthodes existantes permettant d’obtenir une statistique de causalité de Granger dynamique et fréquence spécifique et en discutons leurs avantages et inconvénients.

Suite à ce travail de revue, nous proposons dans le Chapitre 2 d’adopter une mé- thodologie basée sur la représentation “state space” et le filtre de Kalman afin d’estimer un modèle linéaire Gaussien vecteur auroregressif avec des coefficients dynamiques par une méthode d’approximation dite Bayesienne variationnelle (Beal, 2003). Ce modèle Bayesian (BSS model) d’abord proposé dansCassidy(2002), permet de dériver une statistique de causalité dynamique en estimant en même temps toutes les autres quantités nécessaires. La méthode d’approximation Bayesienne variationelle offre naturellement un critère de sélection de modèle et permet d’inclure les connaissances a priori dans le modèle en évitant ainsi les problèmes de surparamétrisation.

Nous proposons d’étendre le modèle proposé à la méthodologie Haar à trous. Cette méthode de prédiction par ondelettes est basée sur une décomposition multiple du signal qui utilise la transformée par ondelettes redondantanteà trouset qui permet de modéliser les dépendances à court comme à long terme entre les signaux. Cette extension du modèle

`

a la méthodologie à trous permet de dériver une statistique de causalité de Granger qui soit à la fois dynamique et fréquence spécifique.

Dans le Chapitre 3 nous proposons une application de la méthodologie proposée à des données réelles d’électroencéphalographie intracrâniennes enregistrées dans les régions de amygdale et du cortex orbitofrontal médian lors d’une tâche expérimentale de reconnais- sance émotionnelle auditive. Nous présentons dans ce chapitre les résultats préliminaires de ce travail en cours.

(11)

(12)

Introduction

The core of this thesis consists of three chapters. The first two are self-contained with their own short introduction and discussion, with supplementary material in the appendix.

The first chapter proposes a systematic methodological review and objective criticism of existing methods enabling the derivation of time-varying Granger-causality statistics in neuroscience. The increasing interest and the huge number of publications related to this topic calls for this systematic review which describes the very complex methodological aspects. The capacity to describe the causal links between signals recorded at different brain locations during a neuroscience experiment is of primary interest for neuroscientists, who often have very precise prior hypotheses about the relationships between recorded brain signals that arise at a specific time and in a specific frequency band. The ability to compute a time-varying frequency-specific causality statistic is therefore essential.

Two steps are necessary to achieve this: the first consists of finding a statistic that can be interpreted and that directly answers the question of interest. The second concerns the model that underlies the causality statistic and that has this time-frequency specific causality interpretation. In this chapter, we will review Granger-causality statistics with their spectral and time-varying extensions.

Based on this review, we present a new estimation and prediction methodology for multivariate nonstationary time series, with the aim of deriving a dynamical causal statistic that catches the nonstationary causal brain dynamics between two recorded signals.

Granger causality is a statistical tool capable of probing possible causal relationships between two signals, its properties being assessed by Granger(1969). We propose to derive this statistic from a linear Gaussian vector autoregressive model with coefficients evolving in time according to a linear dynamical system. As this model contains many parameters, we propose to make use of the variational Bayes methods (Cassidy, 2002; Beal, 2003) in order to estimate this dynamical linear Gaussian vector autoregressive (VAR) model. This methodology allows us to estimate all the necessary quantities and to deal with datasets containing several trials (or epochs). Neuroscience data are recorded during an experiment and therefore several replications of the same experiment must be taken into account in a single estimate. The Bayesian nature of the model offers a natural criterion, the free energy, for model order selection, which is very important in time-series modelling. We propose an extension of this model via the `a trousmultiscale wavelet transform. This approach has never been used in the context of time-varying VAR coefficient estimate, and we will show that it better captures short- and long-range dependencies between recorded signals while providing a very simple way to deal with time-frequency uncertainty bounds.

The resulting Bayesian dynamical Granger-causality statistic will then be derived from the time-varying estimated VAR coefficients.

The major contribution of this research is to provide a valid statistical methodology that can answer complex hypotheses linked to dynamical causality within a neuroscience framework. We builds on several previous works from engineering, statistics and neuro-

(19)

science.

Chapter 3 contains an application of the proposed methodology to intracranial electroencephalogram data. The seminal experimental framework of the study is the evalua- tion of the functional principles related to emotional signal processing. For this purpose, researchers investigated local field potentials (LFPs) in epileptic pharmaco-resistant pa- tients with recordings within the amygdala and medial orbito-frontal cortex in order to study the complex dynamics of neuronal processes within and between these regions in response to emotion exposure (Christen and Grandjean, 2010).

Based on the results in Grandjean et al. (2005), we propose to investigate to what extent brain oscillations recorded within the amygdala and the medial orbito-frontal cortex are causally related in the sense of Granger during specific emotional prosody exposures.

Preliminary results are presented for this work.

Finally, in Chapter 4, we give some arguments concerning the consistency of the variational Bayesian estimator proposed in Chapter 2 for a simpler model.

(20)

Chapter 1 Time, Frequency & Time-Varying Causality Measures in Neuroscience

1.1 Introduction

The investigation of the dynamical causal relationships between neuronal populations is a very important step towards the overall goal of understanding the links between functional cerebral aspects and their underlying brain mechanisms. This investigation requires statistical methods able to capture not only functional connectivities (e.g., symmetrical relationships), but also, and probably more importantly, effective connectivities (e.g., di- rectional or causal relationships) between brain activities recorded during a specific task or stimuli exposure.

The question of how to formalize and test causality is a fundamental and philosophical problem. A statistical answer, which relies on passing from causality to predictability, was provided in the 1960’s by the economist Clive Granger and is known as “Granger causality”. According to Granger (1969), if a signal X is causal for another signal Y in the Granger sense, then the history of X should contain information that helps to predictY above and beyond the information contained in the history of Y alone. It is the axiomatic imposition of a temporal ordering that allows us to interpret such dependence as causal: “The arrow of time imposes the structure necessary” (Granger, 1980, p. 139).

The presence of this relation between X and Y will be referred to “Granger causality”

throughout the text.

Granger(1969) adapted the definition of causality proposed by Wiener (1956) into a practical form and, since then, Granger causality has been widely used in economics and econometrics. It is however only during the past few years that it has become popular in neuroscience (see Pereda et al. (2005) and Bressler and Seth (2011) for a review of Granger causality applied to neural data).

Since its causal nature relies on prediction, Granger causality does not necessarily mean “true causality”. If the two studied processes are jointly driven by a third one, one might reject the null hypothesis of non-Granger causality between signals although manipulation of one of them would not change the other, which contradicts what “true causality” would have implied.

Granger causality may also produce misleading results when the true causal relation- ship involves more variables than those that have been selected and so the accuracy of its causal interpretation relies on a suitable preliminary variable selection procedure (Pearl, 2009).

(21)

If we concentrate on just two signals, the problem is twofold: the first part is the choice of a suitable causality statistic that can easily be interpreted and that answers the question of interest. This said, the statistic needs to rely on a model which intrinsically includes this prediction or Granger-causality principle, and so the second part of the problem is to define and properly estimate this fundamental statistical model. A wrong statistical model indeed may lead to a wrong causality inference.

The scope of this chapter is to review and describe existing Granger-causality statistics in the time and frequency domains and then to focus on their time-varying extensions. We will describe existing estimation methods for time-varying Granger-causality statistics, in order to give the reader a global overview and some insight on the pertinence of using a given method depending on the research question and the nature of the data.

In Sections 1.3 and 1.4, we will present time and frequency-domain Granger-causality statistics in the stationary case. In Section 1.5, we will discuss their time-varying extensions in terms of time-varying causal model estimation. In Section 1.6, we will outline existing toolboxes allowing us to derive time-varying frequency-specific Granger-causality statistics and then discuss the limitations and the potential application of these statistics in neuroscience in Section 1.7.

To our knowledge, there is no systematic methodological review and objective criticism of existing methods that lead to time-varying Granger-causality statistics. The increasing interest reflected by the number of publications related to this topic in neuroscience justifies this literature review undertaken from a statistical viewpoint.

1.2 Stationarity

Many Granger-causality models rely on the assumption that the system analyzed is covariance stationary. Covariance stationarity (also known as weak- or wide-sense stationarity) requires that the first moment and the covariance of the system do not vary with respect to time.

A random process Z_t is covariance stationary if it satisfies the following restrictions on its mean function:

E[Z(t)] =mZ, ∀t ∈R, (1.1)

and on its autocovariance function:

E[(Z(t₁)−m_Z)(Z(t₂)−m_Z)] =C_Z(t₁, t₂) =C_x(τ), where τ =t₁−t₂, ∀t₁, t₂ ∈R. (1.2) The first property implies that the mean function m_Z is constant with respect to t.

The second property implies that the covariance function depends only on the difference between t₁ and t₂. The variance is consequently constant as well.

1.3 Time Domain Causality

We will first discuss the simplest case of Granger causality which is defined in the time domain. It is important to note that it requires that the data are stationary.

As mentioned in the introduction, Granger causality is based on prediction and its fundamental axiom is that “the past and present may cause the future but the future cannot cause the past” (Granger, 1969). The origin of Granger-no-causality was stated by Wiener in 1956 and then adapted and defined into practical form by Granger. As

(22)

1.3. Time Domain Causality 5 we will see, Granger restates Wiener’s principle in the context of autoregressive models (Granger,1969). In particular, the main idea lies in the fact that if a signalX is causal for another signal Y in the Granger sense, then past values ofX should contain information that helps to predict Y better than merely using the information contained in past values of Y (Granger,1969).

This concept of predicting better with an additional variable can be linked to significance tests in multiple linear regression, where an independent variable is declared significant if the full model explains (predicts) the dependent variable better than the model that does not contain this variable. In many fields these tests are called marginal and are linked to the so-called “Type III sum of squares” in ANOVA.

The general criterion of causality is: if the prediction error of a first series given its own past is significantly bigger than its prediction error given its own past plus the past of a second series, then this second series causes the first, in the Granger sense (Granger, 1980, 1969; Ding et al.,2006).

As Chamberlain (1982), Florens (2003) and Chicharro (2011) point out, the most general criterion of Granger non-causality can be defined based on the equivalence of two conditional densities:

f_t(Y_t|Y_t−1^t−p) =f_t(Y_t|Y_t−1^t−p, X_t−1^t−p), (1.3) where Xt andYtare the two recorded time series, Y_t−1^t−p and X_t−1^t−p denote the history from timet−1 tot−pofY andX respectively (i.e. [Yt−1, . . . , Yt−p], and [Xt−1, . . . , Xt−p]), and pis a suitable model order. This general criterion is expressed in terms of the distributions only, so it does not rely on any model assumptions (Kuersteiner,2008). Note that in this general definition, f_t(.) can be different for each time, and therefore the general criterion in equation (1.3) includes nonstationary models.

Any existing method for assessing Granger causality can be viewed as a restricted estimation procedure allowing us to estimate the two densities in equation (1.3) and to derive a causality statistic in order to test their difference.

For linear Gaussian autoregressive models, the assumptions are Gaussianity, homoscedas- ticity and linearity, which implies stationarity in most cases. The quantities in equation (1.3) become an autoregressive model of order p (AR(p)) for the left-hand side:

f_t(Y_t|Y_t−1^t−p) = φ(Y_t;µ=

p

X

j=1

ϑ_1(j)Yt−j, σ² = Σ₁), (1.4) and a vector autoregressive model of order p (VAR(p)) for the right-hand side:

f_t(Y_t|Y_t−1^t−p, X_t−1^t−p) =φ(Y_t;µ=

p

X

j=1

ϑ_11(j)Yt−j +

p

X

j=1

ϑ_12(j)Xt−j, σ² = Σ₂), (1.5) where φ stands for the Gaussian probability density function.

In the next sections, we will present the two widely used approaches for testing hypotheses (1.3) in the linear Gaussian context. The first one is based on an F statistic expressed as the ratio of the residual variances of models on equations (1.4) and (1.5) (Geweke, 1982). The second one is based on a Wald statistic and tests the significance of the causal VAR coefficients (Hamilton, 1994;L¨utkepohl, 2005).

1.3.1 Granger-causality criterion based on variances

The original formulation of Granger causality (Granger, 1969) is expressed in terms of comparing the innovation variances of the whole (equation (1.5)) and the restricted

(23)

(equation (1.4)) linear Gaussian autoregressive models (Geweke,1982; Ding et al., 2006).

Granger (1969) proposed the following quantity to quantify this variance comparison:

FX→Y = ln(Σ₁

Σ₂). (1.6)

In Hesse et al.(2003) and Goebel et al.(2003) this quantity is estimated by replacing the two variances by estimates. A test based on resampling this statistic is used for assessing the significance.

Geweke made several other important statements for (1.6) (Geweke,1984a,1982). He showed first that the total interdependence between two variables can be decomposed in terms of their two reciprocal causalities plus an instantaneous feedback term. Secondly, he showed that under fairly general conditions, FX→Y can be decomposed additively by frequency (see Section 1.4). Lastly, he pointed out that it is possible to extend Granger causality to include other series. Based on the conditional densities, the null hypothesis would write

f_t(Y_t|Y_t−1^t−p,W^t−p_t−1) =f_t(Y_t|Y_t−1^t−p, X_t−1^t−p,W^t−p_t−1), (1.7) whereW^t−p_t−1 represents a set of variables that are controlled for when assessing the causality from X to Y. In the literature, this extension bears the name conditional Granger causality (Ding et al., 2006).

As explained inBressler and Seth(2011) andGeweke(1982), comparing the innovation variances of the whole and restricted linear Gaussian autoregressive models amounts to evaluating the hypothesis

Ho: Σ₁ = Σ₂, (1.8)

which can be assessed through the statistic

F =

RSS_r−RSS_ur m RSS_ur T −2m−1

. (1.9)

RSS_r and RSS_ur are the residual sum of squares of the linear models in equations (1.4) and (1.5), needed to estimate Σ₁ and Σ₂ respectively, and T is the total number of observations used to estimate the unrestricted model.

This statistic follows approximately anF distribution with degrees of freedomm and T −2m−1. A significant F may reasonably be interpreted as an indication that the unrestricted model provides a better prediction than does the restricted one, and so that X causes Y in the Granger sense.

1.3.2 Granger-causality criterion based on coefficients

Another way to test for causality between two series under the same conditions as in Section 1.3.1 is to estimate model (1.5) only and to directly test the significance of the VAR coefficients of interest (Hamilton, 1994; L¨utkepohl, 2005). Let us first define the complementary equation of equation (1.5)

f_t(X_t|X_t−1^t−p, Y_t−1^t−p) =φ(X_t;µ=

p

X

j=1

ϑ_22(j)X_t−j+

p

X

j=1

ϑ_21(j)Y_t−j, σ² = Σ₃), (1.10)

(24)

1.3. Time Domain Causality 7 and the variance-covariance matrix of the whole system

Σ= Σ₂ Γ₂₃ Γ₂₃ Σ₃

!

, (1.11)

where the off-diagonal elements may or may not be equal to zero. Testing whether X causes Y in the Granger sense amounts to testing the hypotheses

ϑ₁₂₍₁₎ =ϑ₁₂₍₂₎ =ϑ₁₂₍₃₎=· · ·=ϑ_12(p) = 0, (1.12) and testing whether Y causes X in the Granger sense amounts to testing

ϑ₂₁₍₁₎ =ϑ₂₁₍₂₎ =ϑ₂₁₍₃₎=· · ·=ϑ_21(p) = 0. (1.13) In the context of linear Gaussian autoregressive models, the two null hypotheses (1.8) and (1.12) are equivalent.

We can observe that the approach using hypothesis (1.8) requires the computation of two models (an AR model and a VAR model), whereas a single VAR model is sufficient for the approach using hypothesis (1.12).

Under joint normality and finite variance-covariance assumptions, the Wald statistic is defined as

W = (ϑˆ12)⁰var(ϑˆ12)⁻¹(ϑˆ12), (1.14) whereϑ12contains all the parametersϑ12(j), forj = 1, . . . , p. AsT increases, this statistic asymptotically follows a χ² distribution with p degrees of freedom (L¨utkepohl, 2005). A significant Wald statistic suggests that at least one of the causal coefficients is different from zero, and, in that sense, that X is causal for Y in the Granger sense. SeeSato et al.

(2006) for an example of application of this statistic in neuroscience.

The time-domain Granger-causality statistics in equations (1.9) and (1.14) are derived from AR and VAR modelling of the data. Their relevance therefore relies on the quality of the fitted models. The first issue is the selection of the model order p. Traditional criteria used in time series are the Akaike information criterion (Akaike, 1974) and the Bayesian information criterion (Schwarz, 1978). For the first statistic, in equation (1.9), it is advisable to select the same p for the two models. The second issue is probably often overlooked but of utmost importance. In practice, and particularly for neuroscience data, the plausibility of the assumptions behind these models must be checked before interpreting the resulting tests. This includes analysis of the residuals from the fitted model.

1.3.3 Transfer entropy

Transfer entropy (TE) is a functional statistic developed in information theory (Schreiber, 2000). It can be used to test the null hypothesis (1.3) in terms of the distributions themselves, and thus does not rely on the linear Gaussian assumption. It is defined as the Kullback–Leibler distance between the two distributionsf(Y_t|Y_t−1^t−p) andf(Y_t|Y_t−1^t−p, X_t−1^t−p):

TX→Y =

Z

· · ·

Z

f(y_t|y_t−1^t−p, x^t−p_t−1) lnf(y_t|y_t−1^t−p, x^t−p_t−1)

f(y_t|y_t−1^t−p) dy_tdy^t−p_t−1dx^t−p_t−1

=KLⁿf(y_t|y^t−p_t−1)kf(y_t|y^t−p_t−1, x^t−p_t−1)^o,

(1.15)

(25)

where the integrals over y^t−p_t−1 andx^t−p_t−1 are both of dimensionp, and so the overall integral in equation (1.15) is of dimension {2p+ 1}.

An even more general definition would allow the distributionsf(.) to depend on time, letting the transfer-entropy statistic be time-dependent.

It has been shown that for stationary linear Gaussian autoregressive models (1.4) and (1.5), the indices (1.15) and (1.6) are equivalent (Barnett et al., 2009; Chicharro,2011).

In its general form, TE is a functional statistic, free from any parametric assumption on the two densities f(Y_t|Y_t−1^t−p) and f(Y_t|Y_t−1^t−p, X_t−1^t−p). See for example Ch`avez et al.

(2003),Garofalo et al. (2009), Vicente et al. (2011), Wibral et al. (2011), Lizier et al.

(2011), Besserve and Martinerie (2011) and Besserve et al. (2010) for applications of TE in neuroscience. Difficulties arise when trying to estimate and compute the joint and marginal densities in equation (1.15). In principle, there are several ways to estimate these two quantities non-parametrically, but the performance of each strongly depends on the characteristics of the data. For a general review of non-parametric estimation methods in information theory see Vicente et al. (2011) and Hlav´aˇckov´a-Schindler et al.

(2007). For simple discrete processes, the probabilities can be determined by computing the frequencies of occurrence of different states. For continuous processes, which are those of interest for neuroscience, it is more delicate to find a reliable non-parametric density estimation. Kernel-based estimation is among the most popular methods; see for example Victor (2002), Kaiser and Schreiber (2002), Schreiber (2000) and Vicente et al. (2011). The major limitation of non-parametric estimation is due to the dimension and the related computational cost. In the present case, the estimation of f(Y_t|Y_t−1^t−p) and f(Y_t|Y_t−1^t−p, X_t−1^t−p) presents two major limitations due to the curse of dimensionality induced by the model order p: a computational limitation, as it implies integration in dimension 2p+ 1 in equation (1.15), and the huge number of observations required to non-parametrically estimate the densities, as this number grows exponentially with the dimension. Typically,Schreiber (2000) proposes to choose the minimalp, meaningp= 1, for computational reasons (Schreiber, 2000, p.462).

A toolbox named TRENTOOL provides the computation of TE and the estimation of f(Y_t|Y_t−1^t−p) and f(Y_t|Y_t−1^t−p, X_t−1^t−p) through kernel estimation (Lindner et al.,2011). This toolbox enables us to estimate a supplementary parameter, called the embedding delay (τ), which represents the lag in time between each observation of the past values of variables X and Y. Equation (1.15) then becomes

TX→Y =

Z

· · ·

Z

f(yt|y_t−1τ^t−pτ, x^t−pτ_t−1τ) lnf(y_t|y_t−1τ^t−pτ, x^t−pτ_t−1τ)

f(y_t|y^t−pτ_t−1τ) dytdy^t−pτ_t−1τdx^t−pτ_t−1τ. (1.16) The model order p (called the embedding dimension in this context) is optimized simultaneously with the embedding delay τ through two implemented criteria. The first is the “Cao criterion” (Cao, 1997), which selects τ on an “ad hoc” basis and p through a false neighbour criterion (Lindner et al., 2011). The second is the “Ragwitz criterion”

(Schreiber, 2000), which selects τ and p simultaneously by minimising the prediction error of a local predictor. As discussed in Lindner et al. (2011), the choice of the order p and of the embedding delay τ is quite important. Indeed, if p is chosen too small, the causal structure may not be captured and thus the TE statistic will be incorrect. On the other hand, using an embedding dimension which is higher than necessary will lead to an increase of variability in the estimation, in addition to a considerable increase in computation time. Typically, Wibral et al. (2011) select the value of p as the maximum determined by the Cao criterion from p = 1 to 4, and choose the value of τ following a

(26)

1.4. Frequency Domain Causality 9 popular ad hoc option as the first zero of the autocorrelation function of the signal.

TRENTOOL allows us to compute the distribution of the transfer entropy statistic under the null hypothesis through a permutation method. The data are shuffled in order to break the links between the signals and then the transfer entropy statistic is recomputed on each surrogate dataset (e.g., Wibral et al., 2011, use 1.9×10⁵ permutations for assessing the significance of the TE statistic). Analyses with TRENTOOL are limited so far to bivariate systems.

The formulation of causality based on the conditional independence in equation (1.3) was later used and theoretically refined in Chamberlain (1982) and Florens (2003). Al- though less general, the statistics given in equations (1.6) and (1.14) are much easier to implement and are testable. This probably explains why they have received considerably more attention in applied work.

1.4 Frequency Domain Causality

1.4.1 Geweke–Granger-causality statistic

As mentioned in Section 1.3.1, an important advance in developing the Granger-causality methodology was to provide a spectral decomposition of the time-domain statistics (Geweke, 1982, 1984b).

For completeness, we give below the mathematical details of this derivation. The Fourier transform of equations (1.5) and (1.10) for a given frequency ω (expressed as a system of equations) is

ϑ₁₁(ω) ϑ₁₂(ω) ϑ₂₁(ω) ϑ₂₂(ω)

! Y(ω) X(ω)

!

= ε₁(ω) ε₂(ω)

!

, (1.17)

where Y(ω) and X(ω) are the Fourier transforms of Y₁^T and X₁^T at frequency ω, and ε₁(ω) andε₂(ω) are the Fourier transforms of the errors of the models (1.5) and (1.10) at frequency ω. The components of the matrix are

ϑ_lm(ω) = δ_lm−

p

X

j=1

ϑ_lm(j)e^(−i2πωj), where

( δ_lm = 0, l =m,

δ_lm = 1, l 6=m, , l, m= 1,2.

Rewriting equation (1.17) as Y(ω) X(ω)

!

= H₁₁(ω) H₁₂(ω) H₂₁(ω) H₂₂(ω)

! ε₁(ω) ε₂(ω)

!

, (1.18)

we have

H₁₁(ω) H₁₂(ω) H₂₁(ω) H₂₂(ω)

!

= ϑ₁₁(ω) ϑ₁₂(ω) ϑ₂₁(ω) ϑ₂₂(ω)

!−1

, (1.19)

where H is the transfer matrix. The spectral matrix S(ω) can now be derived as

S(ω) =H(ω)ΣH^∗(ω), (1.20)

where the asterisk denotes matrix transposition and complex conjugation. Σis the matrix defined in equation (1.11) (Ding et al., 2006). The spectral matrix S(ω) contains cross- spectra terms (S₁₂(ω), S₂₁(ω)) and auto-spectra terms (S₁₁(ω), S₂₂(ω)). IfX and Y are independent, the cross-spectra terms are equal to zero.

(27)

Let us now write the auto-spectrum ofY as

S(ω)₁₁=H(ω)₁₁Σ₂H^∗(ω)₁₁+ 2Γ₂₃Re(H(ω)₁₁)H^∗(ω)₁₂) +H(ω)₁₂Σ₃H^∗(ω)₁₂. (1.21) In the following derivation, we will suppose that Γ₂₃, the off-diagonal element of the Σ matrix in equation (1.11), is equal to zero. In the case where this condition is not fulfilled, a more complex derivation is required (see Ding et al., 2006, for further details). If this independence condition is fulfilled, the auto-spectrum reduces to two terms,

S(ω)₁₁ =H(ω)₁₁Σ₂H^∗(ω)₁₁+H(ω)₁₂Σ₃H^∗(ω)₁₂. (1.22) The first term, H(ω)₁₁Σ₂H^∗(ω)₁₁, only involves the variance of the signal of interest and thus can be viewed as the intrinsic part of the auto-spectrum. The second term H(ω)₁₂Σ₃H^∗(ω)₁₂ only involves the variance of the second signal and thus can be viewed as the causal part of the auto-spectrum.

In Geweke’s spectral formulation, the derivation of the spectral measuref_X→Y requires the fulfillment of several properties. The measures have to be non-negative, and the sum over all frequencies of the spectral Granger-causality components has to equal the time- domain Granger-causality quantity (1.6):

1 2π

Zπ

−π

f_X→Y(ω)dω =FX→Y. (1.23)

The two conditions together imply the desirable property

FX→Y = 0⇔fX→Y(ω) = 0, ∀ω. (1.24)

The third condition is that the spectral statistics have an empirical interpretation. The spectral Granger-causality statistic proposed by Geweke fulfills all three requirements.

For a given frequency ω and scalar variables X and Y, it is defined as f_X_→Y(ω) = S₁₁(ω)

H₁₁(ω)Σ₂H₁₁^∗ (ω), (1.25) where Σ₂ is the variance defined in equation (1.5), S₁₁(ω) is the autospectrum of Y and H₁₁(ω) is the (1,1) element of the transfer matrix in equation (1.19). The form of equation (1.25) provides an important interpretation: the causal influence depends on the relative size of the total power S₁₁(ω) and the intrinsic power H₁₁(ω)Σ₂H₁₁^∗ (ω). Since the total power is the sum of the intrinsic and the causal powers (see equation (1.22)), the spectral Geweke–Granger-causality statistic is zero when the causal power is zero (i.e. when the intrinsic power equals the total power). The statistic increases as the causal power increases (Ding et al., 2006). Given the requirements imposed by Geweke, the measure f_X→Y(ω) has a clear interpretation: it represents the portion of the power spectrum asso- ciated with the innovation process of model (1.5). However, this interpretation relies on the VAR model because the innovation process is only well-defined in this context (see Brovelli et al. (2004),Chen et al. (2009),Chen et al. (2006) andBressler et al.(2007) for examples of application in neuroscience).

The estimation of the parameters and the model order selection procedure is the same as in Section 1.3.2, because the frequency-domain VAR model in equation (1.17) is directly derived from the time-domain VAR model. The model order selection has to be

(28)

1.4. Frequency Domain Causality 11 performed within the time-domain model estimation procedure (Brovelli et al.,2004;Lin et al., 2009).

Lin et al. (2009) showed that under the null hypothesis fX→Y(ω) = 0 and based on (1.25), one can derive a statistic that follows an F distribution with degrees of freedom (p, T − 2p) when the number of observations tends to infinity (it was first derived in Brovelli et al., 2004; Gour´evitch et al., 2006).

1.4.2 Directed transfer function and partial directed coherence

The directed transfer function (DTF) and the partial directed coherence (PDC) are al- ternative measures also derived from VAR estimated quantities that are closely related to the Geweke–Granger-causality statistic.

The DTF is a frequency-domain measure of causal influence based on the elements of the transfer matrix H(ω) in equation (1.19). It has both normalized (Kami´nski et al., 2001) and non-normalized (Kami´nski, 2007) forms. The PDC (Baccal`a and Sameshima, 2001) is derived from the matrix of the Fourier-transformation of the estimated VAR coefficients in equation (1.17). It provides a test for non-zero coefficients of this matrix.

See Schelter et al.(2009) for a renormalized version of PDC andSchelter et al. (2006) for an example of application in neuroscience.

The DTF is expressed as

DTFX→Y(ω) =

v u u t

|H₁₂(ω)|²

|H₁₁(ω)|²+|H₁₂(ω)|², (1.26) where H₁₂(ω) is the element (1,2) of the transfer matrix in equation (1.19). The PDC is defined as

PDCX→Y(ω) = ϑ₁₂(ω)

ϑ^∗₂(ω)ϑ₂(ω), (1.27)

whereϑ₁₂(ω) represents the Fourier transformed VAR coefficient (i.e. the causal influence from X toY at frequency ω), andϑ₂(ω) represents all outflows from X.

The PDC is normalized, but in a different way from the DTF. Indeed, the PDC represents the outflow from X toY, normalized by the total amount of outflows fromX.

The normalized DTF however represents the inflow fromX toY, normalized by the total amount of inflows to Y.

Comparisons between the Geweke–Granger-causality statistic, the DTF and the PDC are discussed in Eichler (2006),Baccal`a and Sameshima(2001),Gour´evitch et al.(2006), Pereda et al. (2005), Winterhalder et al. (2005), Winterhalder et al. (2006) and more recently in the context of information theory inChicharro(2011). As shown inChicharro (2011), the causal interpretation of the DTF and the GGC, at least in the bivariate case, relies on Granger’s definition of causality. For the PDC, a causal interpretation is different, as it relies on Sim’s definition of causality (Sims,1972). SeeChamberlain(1982) and Kuersteiner (2008) for a global overview and comparison of these two definitions of causality. Finally, Winterhalder et al. (2005) conducted a simulation-based comparison of the DTF and the PDC (and other statistics) in a neuroscience context.

Unlike the original time-domain formulation of Granger causality, statistical properties of these spectral measures have yet to be fully elucidated. For instance, the influence of signal pre-processing (e.g., smoothing, filtering) is not well established.

(29)

Assessment of significance

Theoretical distributions for DTF and PDC have been derived and are listed below. They are all based on the asymptotic normality of the estimated VAR coefficients. Therefore, they can be used and interpreted only if the assumptions behind this model hold. Schelter et al.(2006) showed that the PDC statistic asymptotically follows aχ² distribution with 1 degree of freedom. Furthermore, Schelter et al.(2009) showed that a renormalized form of PDC can be related to aχ² distribution with 2 degrees of freedom. Finally, Winterhalder et al. (2005) provide simulations that suggest that this χ² distribution even works well if the true model order is strongly overestimated.

Eichler (2006) showed that the DTF quantity can be compared to a χ² distribution with 1 degree of freedom. This property is also based on the asymptotic normality of estimated VAR coefficients and its accuracy is evaluated through simulations.

For the PDC as well as for the DTF asymptotic distributions, Schelter (2005) and Eichler (2006) state that a major drawback is that there are a lot of tests – one for each frequency. It is well known that when many tests are produced, caution has to be taken in interpreting those that are significant. For example, even under the null hypothesis of no information flow, there is a high probability that for a few frequencies the test will be significant.

1.5 Time-Varying Granger Causality

Neuroscience data are nonstationary in most cases. The specificity (task or stimulus related) of the increase or decrease and/or local field potential implies this nonstationarity which is of primary interest. A Granger-causality statistic in the time- or the frequency- domain is desirable as it would capture the evolution of Granger causality through time.

Since the original statistics are based on AR and VAR models, and therefore on assumptions assuming that the autocorelation does not vary along the time, these models have to be extended to cases assuming changing autocorelation structure in order to suitably extract a Granger-causality statistic.

Practically, getting a statistic to assess the causality between two series for each time requires the estimation of the densities f_t(Y_t|Y_t−1^t−p) and f_t(Y_t|Y_t−1^t−p, X_t−1^t−p) separately for each timet. There are two additional difficulties to keep in mind. The first is the necessity of an objective criterion for time-varying model order selection and the second is the difficulty of incorporating all the recorded data (meaning all the trials) in the estimation procedure.

1.5.1 Non-parametric statistics

Wavelet-based statistic

In the context of neuroscience,Dhamala et al. (2008) proposed to bypass the nonstationarity problem by non-parametrically estimating the quantities which allow us to derive the spectral Geweke–Granger-causality (GGC) statistic (1.25). They derived an evolutionary spectral density through the continuous wavelet transform of the data, and then derived a quantity related to the transfer function (by spectral matrix factorization). Based on this quantity, they obtain a GGC statistic that can be interpreted as a time-varying version of the GGC statistic defined in (1.25).

(30)

1.5. Time-Varying Granger Causality 13 This approach bypassed the delicate step of estimatingf_t(Y_t|Y_t−1^t−p) andf_t(Y_t|Y_t−1^t−p, X_t−1^t−p) separately for each time. However this method presents several drawbacks in terms of interpretation of the resulting quantity. The GGC statistic is indeed derived from a VAR model and its interpretation directly follows from the causal nature of the VAR coefficients. The non-parametric wavelet spectral density however does not have this Granger-causality interpretation. Therefore attention must be paid when interpreting this proposed evolutionary causal GGC statistic derived from spectral quantities which are not based on a VAR model.

Local transfer entropy

Lizier et al.(2008,2011) and Prokopenko et al.(2013) proposed a time-varying version of the transfer entropy (1.15), in order to detect dynamical causal structure in a functional magnetic resonance imaging (fMRI) study context. The “global” transfer entropy defined in equation (1.15) can be expressed as a sum of “local transfer entropies” at each time:

TX→Y = 1 T

T

X

t=1

ft(yt|y^t−p_t−1, x^t−p_t−1) ln f_t(y_t|y_t−1^t−p, x^t−p_t−1)

f_t(y_t|y^t−p_t−1) , (1.28) where each summed quantity can be interpreted as a single “local transfer entropy”:

tx→y(t) = lnf_t(y_t|y_t−1^t−p, x^t−p_t−1)

f_t(y_t|y_t−1^t−p) . (1.29) The step from equation (1.15) to equation (1.28) is obtained by replacing the joint density f(Y_t, Y_t−1^t−p, X_t−1^t−p) with its empirical version. This simplification seems difficult to justify in a neuroscience context, considering the continuous nature of the data. In fact, the sampling rate used in neuroscience data acquisition is often very high. As such, this local transfer entropy does not seem to be a suitable time-varying causality statistic for an application in neuroscience. Moreover, even if the overall quantity in equation (1.15) can be suitably expressed as a sum of orthogonal parts as in equation (1.29), its causal nature does not necessarily remain in each part. As such, we cannot directly interpret these parts as causal, even if the sum of them gives an overall quantity that has an intrinsic causal meaning. Finally, Prokopenko et al.(2013) orLizier et al.(2008, 2011) do not provide an objective criterion for model order selection.

1.5.2 Time-varying VAR model

As seen before in equations (1.9), (1.14), (1.25), (1.26) and in (1.27), parametric Granger- causality statistics in the time- and frequency-domains are derived from AR and VAR modelling of the data (equations (1.4) and (1.5) respectively). One way to extend these statistics to the nonstationary case amounts to allowing the AR and VAR parameters to evolve in time. In addition to the difficulties related to model order selection and the fact that we have to deal with several trials, time-varying AR and VAR models are difficult to estimate since the number of parameters is most of the time considerable compared to the available number of observations. To overcome the dimensionality of this problem, Chen (2005) propose to make one of the three following assumptions, local stationarity of the process (Dahlhaus, 1997), slowly-varying nonstationary characteristics (Priestley, 1965) and slowly varying parameters for nonstationary models (Ledolter,1980). In practice, it

Time-frequency Granger causality with application to nonstationary brain signals

Thesis

Reference

Time-frequency Granger causality with application to nonstationary brain signals

Time-Frequency Granger Causality

with Application to

Nonstationary Brain Signals

Sezen Cekic

Acknowledgements

Abstract

R´ esum´ e

Contents

Introduction

Chapter 1

Time, Frequency & Time-Varying Causality Measures in Neuroscience

1.1 Introduction

1.2 Stationarity

1.3 Time Domain Causality

1.3.1 Granger-causality criterion based on variances

1.3.2 Granger-causality criterion based on coefficients

1.3.3 Transfer entropy

1.4 Frequency Domain Causality

1.4.1 Geweke–Granger-causality statistic

1.4.2 Directed transfer function and partial directed coherence

1.5 Time-Varying Granger Causality

1.5.1 Non-parametric statistics

1.5.2 Time-varying VAR model