On the spectral analysis of second-order Markov chains

ANNALES

DE LA FACULTÉ DES SCIENCES

Mathématiques

PERSIDIACONIS, LAURENT MICLO

On the spectral analysis of second-order Markov chains

Tome XXII, n^o3 (2013), p. 573-621.

<http://afst.cedram.org/item?id=AFST_2013_6_22_3_573_0>

© Université Paul Sabatier, Toulouse, 2013, tous droits réservés.

L’accès aux articles de la revue « Annales de la faculté des sci- ences de Toulouse Mathématiques » (http://afst.cedram.org/), implique l’accord avec les conditions générales d’utilisation (http://afst.cedram.

org/legal/). Toute reproduction en tout ou partie de cet article sous quelque forme que ce soit pour tout usage autre que l’utilisation à fin strictement personnelle du copiste est constitutive d’une infraction pénale. Toute copie ou impression de ce fichier doit contenir la présente mention de copyright.

cedram

Article mis en ligne dans le cadre du

Centre de diffusion des revues académiques de mathématiques

pp. 573–621

On the spectral analysis of second-order Markov chains

Persi Diaconis⁽¹⁾, Laurent Miclo⁽²⁾

ABSTRACT.—Second order Markov chains which are trajectorially re- versible are considered. Contrary to the reversibility notion for usual Markov chains, no symmetry property can be deduced for the correspond- ing transition operators. Nevertheless and even if they are not diagonal- izable in general, we study some features of their spectral decompositions and in particular the behavior of the spectral gap under appropriate per- turbations is investigated. Our quantitative and qualitative results con- firm that the resort to second order Markov chains is an interesting option to improve the speed of convergence to equilibrium.

R^´ESUM´E.—On consid`ere des chaˆınes de Markov du second ordre, sup- pos´ees r´eversibles trajectoriellement. Contrairement `a la notion de r´eversi- bilit´e pour les chaˆınes de Markov usuelles, les op´erateurs de transition cor- respondants ne v´erifient pas de propri´et´e de sym´etrie et ne sont parfois mˆeme pas diagonalisables. On ´etudie n´eanmoins certains aspects de leurs d´ecompositions spectrales et en particulier le comportement de leurs trous spectraux sous des perturbations appropri´ees. Les r´esultats quantitatifs et qualitatifs obtenus confirment que le recours aux chaˆınes de Markov du second ordre peut se r´ev´eler int´eressant pour am´eliorer la vitesse de convergence `a l’´equilibre.

(∗) Re¸cu le 19/07/2012, accept´e le 03/03/2013

(1) Stanford University, Department of Mathematics, Building 380, Sloan Hall, Stan- ford, California 94305, USA

(2) Institut de Math´ematiques de Toulouse, Universit´e Paul Sabatier, 118, route de Narbonne, 31062 Toulouse Cedex 9, France

miclo@math.univ-toulouse.fr

Article propos´e par Mireille Capitaine.

1. Introduction

Resorting to nonreversible Markov chains or to second order Markov chains may present some important advantages from a practical simulation point of view. This feature was emphatized by Neal [10] who showed how the asymptotic variance can be diminished for empirical estimators of ex- pectations, when one uses stochastic chains which have a tendency to not return back where they were just coming from. Our main results here go in the same direction, but we will rather be interested in the asymptotic speed of convergence to equilibrium, measured through the spectral gap. This will lead us to study the whole spectrum of some second order Markov chains and its behavior under perturbations designed to improve them.

A stochastic chain (Xn)n∈Nis said to be a second order Markov chain if at any timen∈N, its future only depends on its past through the present position and the previous one, namely if we have the following almost sure equality for the conditional laws

L((Xm)mn+1|(Xm)m∈[[0,n]]) = L((Xm)mn+1|Xn, Xn−1) (by convention X₋1 = X0). Equivalently, this means that the stochastic chain (Xn, Xn+1)_n∈N is Markovian.

We will only consider time-homogeneous second order finite Markov chains, i.e. the Xn, for n ∈ N, take values in a finite state space V and for any n∈N\ {0} andx, x, x∈V, the quantity

M(x, x;x) := P[Xn+1=x|Xn=x, Xn−1=x] (1.1) does not depend on n(when it is well-defined, i.e. when the event {Xn = x, Xn−1=x}is not negligeable with respect to the underlying probability P). Conversely, if we are given a Markov kernelM := (M(x, x;x))x,x,x∈V

fromV²toV (a kernel from a finite setSto another finite setS just refers to a S×S matrix) and a probability distributionm0 = (m0(x, x))x,x∈V

onV², there exists a unique (in law) stochastic chain (Xn)n∈N onV such that the equality (1.1) holds and such that the initial law of (X0, X1) ism0. The probability distributionm onV²is said to be trajectorially reversible with respect to the kernel M, if the corresponding second order Markov chain starting with the initial lawmsatisfies for anyn∈N, the identity in law

L(X0, X1,· · ·, Xn) = L(Xn, Xn−1,· · ·, X0) (1.2) This notion of reversibility is different from the usual reversibility for the Markov chain (Xn, Xn+1)_n∈N, which is not satisfied in general. As shown

by Bacallado and Pande [2], it is sufficient to check the equality (1.2) for n= 1 andn= 2. In particular the distributionmis symmetrical in its first and second variables. Let us write

∀x, x∈V, m(x, x) = ν(x)L(x, x)

:= (ν×L)(x, x) (1.3) where ν is the law of X0 and L is a Markov kernel corresponding to the conditional law undermofX1whenX0is given (it is unique ifνis positive).

By symmetry, ν is reversible (in the usual sense) with respect to L. Let (Xn)_n∈N be a Markov chain on V, starting with ν as initial distribution and whose transition kernel is given by L. It is not difficult to see that the distributionmis invariant for the Markov chain (Xn, Xn+1)_n∈Nand by consequence we are insured of the identities in law

∀ n∈N, L(Xn, Xn+1) = L(Xn,Xn+1)

In particular, appropriate assumptions of aperiodicity and irreducibility on LandM will insure that whatever the initial distributions,

nlim→∞L(Xn, Xn+1) = m

= lim

n→∞L(Xn,Xn+1)

and our main goal in this paper is to compare the corresponding speeds of convergence. They will be measured via spectral gaps, which give the asymptotical convergence rates to equilibrium. Reinterpreting M as the transition kernel fromV²toV²of the Markov chain (Xn, Xn+1)_n∈Nenables us to seeM as a right operator on the spaceF(V²) of real-valued functions defined onV² (or on the spaceF(V²,C), when it will be more convenient to consider complex-valued functions) and as a left operator on the space M(V²) of signed measures onV². We have for instance

∀F ∈ F(V²),∀(x, x)∈V², M[F](x, x) :=

x∈V

M(x, x;x)F(x, x)(1.4) By invariance ofm, the subspaceF∗(m,C) :={F ∈ F(V²,C) : m[F] = 0} is stable underM, so let Θ_∗(M)⊂Cbe the spectrum ofM_∗, the restriction ofM to F∗(m,C). The spectral gap ofM is

λ(M) := 1−max{|θ| : θ∈Θ_∗(M)} ∈ [0,1]

Then, if · is a norm onM(V²), we have for any initial distributionm0

onV²,

nlim→∞

1

nln (m0Mⁿ−m) ln(1−λ(M))

and there exists an initial point (x, x)∈V² such that

nlim→∞

1

nlnδ(x,x)Mⁿ−m = ln(1−λ(M))

(where δ(x,x) stands for the Dirac measure at (x, x)). Similar definitions can be given for the spectral gap of L, by replacing the state space V² byV. One can alternatively consider the second order Markov kernelM⁽⁰⁾ naturally associated toL:

∀x, x, x∈V, M⁽⁰⁾(x, x;x) := L(x, x) (1.5) (it corresponds to the transition kernel of the Markov chain (Xn,Xn+1)_n∈N).

We will check that the spectrum of M⁽⁰⁾ is just the spectrum of L (with the same multiplicities, except for the eigenvalue 0) plus the eigenvalue 0, so that λ(L) = λ(M⁽⁰⁾). Note that since ν is reversible with respect to L, this operator is diagonalizable in L²(ν) and we have Θ_∗(M⁽⁰⁾) ⊂ R, but in general M⁽⁰⁾ won’t be diagonalizable: we will see that it contains 2×2-Jordan blocks if 0 is an eigenvalue ofL.

Our first result states that except in the cases whereL =ν (where as usual the probability measureν is seen as the Markov kernel whose all rows are given by ν) or where L is periodic (which by reversibility means that the graph induced byL is bipartite), it is always possible to find a second order Markov chainM which admitsν×Las trajectorial reversible measure and which has a larger spectral gap than L. Note that the largest possible spectral gap is 1 and that the equalityL=ν is equivalent to the fact that λ(L) = 1, so in this situation the spectral gap cannot be improved by any means. In the periodic case, the spectral gap cannot be improved either: let V and V be proper subsets of V such that V =VV and such that the V×V and V×V entries of L are null. Then if M admits ν×L as trajectorial reversible measure, necessarily M is also periodic, since the Markov chain (Xn, Xn+1)n∈Ngoes from the setV×Vto the setV×V and conversely.

Indeed, let M⁽¹⁾ be a second order Markov kernel admitting ν×L as trajectorial reversible measure and satisfying

∀ x, x, x∈V, M⁽¹⁾(x, x;x)> L(x, x), ifx=x,L(x, x)>0 andL(x, x)>0

(in particularM⁽¹⁾has less tendency to come back where it just came from thanM⁽⁰⁾). It is not difficult to see that such kernels exist. For anya∈[0,1], define

M^(a) := (1−a)M⁽⁰⁾+aM⁽¹⁾

which is a second order Markov kernel still admitting ν×Las trajectorial reversible measure. Then we have

Theorem 1.1. — Assume thatL is irreducible, that L =ν and that L is aperiodic. The mapping[0,1]a→λ(M^(a))is differentiable at0and we have

dλ(M^(a))

da (0) > 0

In fact we will show a better result, saying that under the above assump- tions the whole spectrum of L outside 0 and 1 has an initial tendency to improve (in the sense that the absolute values of the eigenvalues different from 0 and 1 begin by decreasing, thus contributing to increase the speed of convergence to equilibrium of the kernel): let [0, a0] a → θ(a) be a differentiable function such that θ(a) ∈ Θ_∗(M^(a)) for a ∈ [0, a0] (where a0 ∈(0,1) will be small enough in practice) and such thatθ(0)= 0. Since θ(0) ∈ Θ_∗(L), it is real. Then we will prove that dθ(a)/da(0) is real and that its sign is the opposite sign of θ(0), namely initially, when astarts to increase from 0,θ(a) is going in the direction of 0. Of course, the eigenvalues initially in 0 can only have the tendency to leave 0, behavior which is less favorable to a fast convergence to equilibrium.

But if instead of improving the whole spectrum outside 0 and 1, one only wants to improve the spectral gap, there is a more efficient way to do it, which uses the ordering of the space created by an eigenfunction associ- ated to the eigenvalue of L corresponding to the spectral gap (under the assumption that such an eigenfunction is unique up to a factor) as it will be explained in detail in Section 3. Nevertheless, it should be kept in mind that to get quantitative bounds on the distance to equilibrium (for instance in the chi-square sense), not just an asymptotical result, the knowledge of the whole spectrum is preferable to the spectral gap.

In order to get a global picture of how the spectral gap (and indeed the whole spectrum) depends on the parametera∈[0,1], we consider a family of homogeneous-space simple examples. Assume that V is a group G. Let (Xn)_n∈N be a second order Markov chain with kernel M and define the corresponding speed process (Yn)_n∈N by

∀n∈N, Yn := X_n⁻¹Xn+1

The position-speed process (Xn, Yn)_n∈Nis Markovian and its transition ker- nel can be written as

∀(x, y),(x, y)∈G², P((x, y),(x, y)) =δxy(x)Kx(y, y) (1.6)

where the Markov kernel of transition from speed y to speed y above the pointx is given by

Kx(y, y) := M(xy⁻¹, x;xy) (1.7) In the examples we will consider, the group G is Abelian (in which case we adopt an additive notation) and the speed variable only takes values in a symmetrical subset S ⊂Gof generators of G, which will be small in comparison to G. Furthermore the situation will be assumed to be homo- geneous: the kernels Kx do not depend on the support pointx∈G. Then under additional hypotheses (of trajectorial reversibility and a kind of irre- ducibility, see next section), the probability measure ν is uniform and the transition kernelLcorresponds to the Markov kernel of a reversible random walk: there exists a symmetric probability measure µ whose support is S such that

∀x, x∈G, L(x, x) = µ(x−x) (1.8) In this context, the second order Markov kernel M⁽⁰⁾ corresponds to the Markov kernelK⁽⁰⁾ for the speed variables given by

∀ y, y∈S, K⁽⁰⁾(y, y) := µ(y)

To define a natural perturbation, assume furthermore thatµis the uniform distribution over S (so our framework is parametrized by the data (G, S) consisting of a finite Abelian group and a symmetric subset of generators).

Consider the Markov kernelK⁽¹⁾which forbids reversing speed and transfers the corresponding weight to all the other directions:

∀ y, y∈S, K⁽¹⁾(y, y) :=

1

|S|−1 , ify=−y 0 , ify=−y

Of course, this definition makes sense only if|S|2 (due to the symmetry ofS, |S|= 1 corresponds to two trivial situations:G=S =Z1 or G=Z2

andS={1}). Define as well the interpolating kernels:

∀a∈[0,1], K^(a) := (1−a)K⁽⁰⁾+aK⁽¹⁾ (1.9) We associate to them the second order Markov kernelsM^(a) for a∈[0,1], which are all trajectorially reversible with respect to the probability measure

∀ x, x ∈G, m(x, x) :=

1

|G||S| , ifx−x∈S

0 , otherwise (1.10)

Then we have the following behavior for the spectral gap of M^(a) as a function ofa∈[0,1]:

Theorem 1.2. — Assume thatGis a finite Abelian group and thatSis a generating set of Gwith |S|2. Letµ be the uniform distribution onS and define

a0 := (|S| −1)1−

λ(2−λ) 1 +

λ(2−λ) 0 (1.11)

whereλ:=λ(L)is the spectral gap of the random walk transition kernel de- fined in (1.8). Then under the framework described above,λ(M^(a))is nonde- creasing fora∈[0, a0∧1]and nonincreasing on[a0∧1,1]. So the best choice of a∈[0,1]to get the largest possible spectral gap Λ := max_a∈[0,1]λ(M^(a)) isa=a0∧1 and we have, if a01,

Λ = 1− 1−

λ(2−λ) 1 +

λ(2−λ) while if a01 (the two formulas coincide ifa0= 1),

Λ = 1− |S| 2(|S| −1)



1−λ+ |S| −2

|S| 2

−λ(2−λ)



 (1.12)

The example treated in Section 4 will show that when |S| = 2, the improvement of the spectral gap can be important and erases the diffusive effect which usually slows down the convergence to equilibrium of Monte- Carlo methods. Unfortunately, as soon as|S|3, the improvement of the spectral gap is quite weak, as we will see in Section 5. In fact our analysis will also furnish all the eigenvalues of the operatorsM^(a), fora∈[0,1], see Section 5 for details.

When a0 > 1, Formula (1.12) is a particular case of a result due to Alon, Benjamini, Lubetzky and Sodin [1]: they consider a random walk on a connected, non bipartite and d-regular graph V, with d 3, whose spectral gap is still denoted by λ (be careful of the changes of notations with the article [1], what they callλ,ρanddcorrespond here respectively to

|S|(1−λ), 1−Λ and|S|), and they want to compare it with the second order random walk (Xn)n∈N which does not backtrack: at any timen∈N\ {0} and for any positionsy, xn, xn−1, ..., x0,

P[Xn+1=y|Xn=xn, ..., X0=x0] = 1

d−111_y∈N(xn)\{xn−1}

where N(xn) is the set of neighbors ofxn (for n= 0, their convention is thatP[X1=y|X0=x0] = ¹_d11y∈N(x0), but this is not very important). Then

they show that whatever the initial condition, if V is a Ramanujan graph (which means thatλ1−2√

d−1/d, namelya01), lim sup

n→∞ max

u∈V

P[Xn=u]− 1 n

1/n

= 1−Λ (1.13)

where Λ is defined as in (1.12) (the value given by this formula is attained fora= 1, so it does amount in our restricted framework to forbid backtrack- ing, note also that our statement is slightly more precise, since we get the asymptotic rate of convergence for the pair (Xn, Xn+1)). Their approach is completely different from ours and it is based on the recurrence relations satisfied by the numbers of nonbacktracking paths of a given length and going from a particular point to another particular point. These recurrence relations enable the authors to write explicitely the matrix of these numbers in terms of Chebyshev polynomials (applied to the adjacency matrix) and an asymptotic analysis of some linear combinations of these polynomials leads to (1.13). This result of Alon, Benjamini, Lubetzky and Sodin [1] is valid in a more general setting and its proof is more straightforward (admitting only a limited knowledge of Chebyshev polynomials) than ours, but they do not consider the case of weak forbidden backtracking (corresponding to 0< a <1), nor are they interested in the behavior of the whole spectrum.

In particular, they do not always exhibit a faster chain. But more impor- tant for us is the following conjecture, which cannot be stated ford-regular graph, since the sentence “keep going in the same direction” has then no (at least immediate) meaning.

Indeed, we believe that the qualitative part of Theorem 1.2 concerning the variation of the spectral gap could be extended to more general situ- ations. More importantly, we think that it is always possible to strongly improve the spectral gap, under the assumptions of Theorem 1.2, by con- sidering a more appropriate perturbation.

Conjecture 1.3. — LetGbe a finite Abelian group andS be a gener- ating set ofGwith|S|2. LetK⁽¹⁾ be the identityS×S matrix (seen as a speed transfer kernel, it forces the chain to keep going always in the same direction) and consider again the interpolating kernels given in (1.9), as well as the associated second order Markov kernelsM^(a)fora∈[0,1], which are also trajectorially reversible with respect to the probability measuremde- fined in (1.10). There exists a0∈[0,1] such that λ(M^(a)) is nondecreasing for a ∈ [0, a0] and nonincreasing on [a0,1]. Furthermore there exists two values 0< c1< c2(perhaps depending on|S|), such that

c1

λ(L) λ(M^(a0)) c2

λ(L)

The main interest of second order Markov chains is maybe that they enable the introduction of a notion of speed in a discrete setting. In ad- dition to the fact that a global reversibility property can be imposed on them, this feature could be the reason why they provide good models for molecular dynamics as suggested by the results of Bacallado and Pande [2].

In this context, a general motivation for us was to investigate the possible spectral implications of the trajectorial reversibility assumption. The results presented here should be seen as first attempts in this direction.

The plan of the paper is as follows: in next section we will study the structure of second order Markov kernels and the trajectorial reversibility property, in the general case and in the position-speed point of view for groups. In Section 3, we prove Theorem 1.1 and we present some links with the work of Neal [10]. In Section 4, we investigate the particular homoge- neous example of the discrete circleZ^N, with N ∈N\ {0,1}, when the set of allowed speeds is S={−1,+1}. This will lead us to revisit the analysis of a similar model done by Diaconis, Holmes and Neal [3], see also Gade and Overton [5]. In Section 5, we extend these results to prove Theorem 1.2 and we give some examples.

2. Trajectorial reversibility

We investigate here the trajectorially reversible second order Markov kernels, in order to be able to easily perturb them in the sequel. Their structure recalls the framework of differentiable manifolds, feature which should not be so surprising, since second order Markov chains should be seen as discrete position-speed Markov processes (at any time, the knowledge of the position and of the speed of the chain is sufficient to determine the law of the future of the trajectory). In the homogeneous Abelian group case, we also exhibit a skew Kronecker product property of the second order Markov kernels which simplifies their spectral decomposition.

Let M be a second order Markov kernel on a finite state space V as in the introduction. We want to characterize the fact thatM is trajectory reversible with respect to some probability measure m on V². As it was mentioned in the introduction, necessarilymis of the form (1.3), whereLis reversible with respect toν. LetV:={x∈V : ν(x)>0}be the support of ν. By reversibility, ifx∈V andx∈V satisfiesL(x, x)>0, thenx ∈V. Forx ∈V, considerV_x :={x∈V : L(x, x)>0}, andMx the Markov kernel defined by

∀x, x∈V, Mx(x, x) := M(x, x;x)

Let us check that the restriction ofMx toV_x×V_x is still a Markov kernel (which will also be denoted byMx). Indeed, the trajectorial reversibility of M withmcan be written has

∀x, x∈V,∀x∈V, ν(x)L(x, x)Mx(x, x) =ν(x)L(x, x)Mx(x, x) (2.1) so it appears that if x ∈ V_x and x ∈ V_x, then we have Mx(x, x) = 0. Thus only what happens on V is relevant: in the sequel we assume that V =V and we remove the primes from the notation. By considering (2.1) for x ∈ V and x, x ∈Vx and by dividing this identity by ν(x), it follows that the kernelMxmust be reversible with respect to the probability measureL(x,·) onVx. Let denote byR(L(x,·)) the convex set consisting of the kernels on Vx which are reversible with respect to L(x,·) and let R(L) be the “kernel-bundle”

x∈V R(L(x,·)) overV. The above arguments show the direct implication in the following characterization and the reverse implication is proven in the same way through (2.1).

Lemma 2.1. — The second order Markovian kernel M is trajectorially reversible with respect tom=ν×Lif and only if(Mx)x∈V ∈ R(L), namely ifM is a section of the bundleR(L)and from now on this will be indicated by the notationM ∈ R(L).

This result makes it clear that it is easy to perturb a second order Markov kernel while keeping it trajectorially reversible with respect to a fixed mea- surem=ν×L, since this can be done independently in the different fibers ofR(L). Note also that the particular second order Markov kernelM⁽⁰⁾con- sidered in the introduction corresponds to the section (L(x,·))x∈V ∈ R(L).

Contrary to reversibility, the notions of invariance, irreducibility and aperiodicity relative to second order Markovian kernel M will be the usual ones, whenM is interpreted as a transition kernel fromV² toV²via (1.4).

Nevertheless usually V² is not an appropriate state space and we restrict M to a subset ofV², for instance in the previous situation the natural state space is {(x, x) ∈ V² : m(x, x) >0} (or a recurrent component of this subset). We now check an assertion made in the introduction:

Lemma 2.2. — If the second order Markovian kernelM is trajectorially reversible with respect tom, then mis invariant for M.

Proof. —By definition a probability measure m onV² is invariant for M if

∀(x, x)∈V²,

x∈V

m(x, x)M(x, x;x) = m(x, x)

Equivalently, for anyF ∈ F(V²),

E[F(X1, X2)] = E[F(X0, X1)] (2.2) where (Xn)n∈Nis a second order Markov chain withM as kernel and start- ing from the initial distributionm. But ifm is reversible, we have

E[F(X1, X2)] = E[F(X1, X0)]

= E[F(X0, X1)]

where the second equality comes from the symmetry ofm.

Remark 2.3. — One can give a characterization similar to Lemma 2.1 of second order Markov kernels which leave a measure m invariant. First note that ifmis left invariant by a second order Markov kernelM, then its marginals on the first and second factor ofV² are equal. Indeed, just apply (2.2) to a functionF only depending on the first variable. Let us callν this common marginal distribution on V and as before assume that its support isV. LetL1 andL2 the Markov kernels defined by

∀x, x∈V, m(x, x) = ν(x)L2(x, x) = ν(x)L1(x, x) (ν is invariant forL1 andL2).

As above, representM as a family of Markov kernels (Mx)_x∈V. It follows from the invariance ofm byM, that for any x∈V, Mx can be seen as a Markov kernel fromV_x⁻ toV_x⁺, where

V_x⁻ := {x∈V : m(x, x)>0} V_x⁺ := {x∈V : m(x, x)>0}

Then it is not difficult to see thatmis invariant forM if and only if for any x∈V, Mx transports the probability measureL1(x,·) into the probability measureL2(x,·). In particular Lemma 2.1 enables to see that a probability measure m is trajectorially reversible with respect to M if and only it is invariant and symmetric.

Assume again that the second order Markov kernelM is trajectorially reversible with respect to m and consider M^∗ the dual operator of M in L²(m). Sincemis invariant forM, it is well-known thatM^∗is Markovian in the abstract sense: it is nonnegativity preserving and the function 11 taking everywhere the value 1 is preserved. But as we compute in next result, in generalM^∗ does not have not the form of a second order Markov kernel:

Lemma 2.4. — Define the tilde operation on F(V²) as the exchange of coordinates:

∀F ∈ F(V²),∀ x, x ∈V, F(x, x ) := F(x, x)

Then we have

∀F ∈ F(V²), M^∗[F] = M[F]

or more explicitly,

∀x, x∈V, M^∗[F](x, x) =

x∈V

M(x, x;x)F(x, x)

Proof. —Consider again (Xn)_n∈Na second order Markov chain withM as kernel and starting from the initial distributionm. For anyF, G∈ F(V²), we have

m[F M[G]] = E[F(X0, X1)G(X1, X2)]

= E[F(X2, X1)G(X1, X0)]

= E[G(X 0, X1)F(X 1, X2)]

= E[G(X 0, X1)M[F](X0, X1)]

= m[GM [F]]

= m[GM[F]]

= m[GM[F]]

where the last but one equality comes from the symmetry ofm.

We now come to the situation where the state space is a group Gand we consider the family of Markov kernels (Kx)x∈G associated to M as in (1.7). In addition define the Markov kernels

∀x, y, y ∈G, Kx(y, y) := Kx(y⁻¹, y)

which will be called the modified speed kernels associated toM. If a sym- metric probability measure m = ν ×L is given on V² with ν giving a positive mass to all the elements ofG, we consider the family of probability distributions (µx)x∈G given by

∀x, y∈G, µx(y) := L(x, xy) Then Lemma 2.1 leads to the following result:

Lemma 2.5. — The second order Markovian kernel M is trajectorially reversible with respect to the symmetric distribution m if and only if for any x∈ G, Kx is reversible with respect to µx. If furthermore the kernels Kx do not depend on x∈G and admit a unique recurrence classS which generatesG, thenν is the uniform distribution andLis the Markov kernel of the random walk on Gwhose increments follow the symmetric law µon S which is reversible with respect to K.

Proof. —The first part follows immediately from the fact that

∀ x, y, y∈G, Kx(y, y) = Mx(xy, xy)

For the second part, assume that for allx∈G, Kx =K, where K admits a unique recurrence class S which generates G and that K is reversible with respect to a probability measureµ. Then the support of µisS and by uniqueness of the invariant measure on a recurrence class, we have

∀ x, x ∈G, L(x, x) = µx(x⁻¹x) = µ(x ⁻¹x)

So it appears thatLis the Markov kernel of the random walk onGwhose increments follow the law µ. Since S generatesG, Lis irreducible and ad- mits a unique invariant probability measure which is necessarily the uniform distribution ν, because, asL, it must be invariant by translations. By re- versibility ofν with respect toL, we get that µis symmetric:

∀y∈S, µ(y ⁻¹) = µ(y) and in particularS must be symmetric.

Note that in practice, it is convenient to rather start with a Markov kernel K reversible with respect to a symmetric probability measure µ. It is then immediate to check that the associated second order Markov kernel M defined by

∀x, x, x∈G, M(x, x;x) := K((x )⁻¹x,(x)⁻¹x) is trajectorially reversible with respect to the measuremdefined by

∀x, x∈G, m(x, x) = 1

|G|µ(x ⁻¹x)

The second order Markov kernelM will then be said to be (spatially) ho- mogeneous (this situation can be extended to state spaces which are not groups, under appropriate homogeneity assumptions, but we won’t enter in the description of such a setting here).

Remark 2.6. — In generalK may have several recurrent classes and the link with the recurrent classes of M is not so clear. Consider for instance the case whereG=Z^N, withN ∈N\ {0,1}and where the modified speed kernel K is given by the matrix ¹₂

0 1 1 0

on the set S := {−1,1} ⊂ Z^N. The speed kernel K is then the identity kernel, which means that once a speed has been chosen in {−1,1}, it will never change. SoM has

two recurrent classes: {(x, x−1) : x ∈Z^N} and {(x, x+ 1) : x∈ Z^N}. Conversely, if we take for K the identity matrix, then all the sets of the form{(x, x+ 1),(x+ 1, x)}, where x∈Z^N, are recurrent classes forM. Indeed, even the link between the recurrent classes ofKand those ofK can be quite involved. Nevertheless, if a setI is symmetric and invariant forK (namelyx∈I and y∈I imply thatK(x, y) = 0), then it is also invariant forK.

We now concentrate on the homogeneous case. We will always denote by S the support of the measure µentering in the definition of this situation.

The transition kernels K and K will be understood as S ×S matrices.

Without loss of generality, we assume that G is generated by S (replace otherwiseG by the subgroup generated by S). By considering the change of variables

G²(x, x) → (x, y) := (x, x⁻¹x)∈G²

we transform the transition kernelM of the Markov chain (Xn, Xn+1)_n∈N into the position-speed variables transition kernel P given in (1.6). Note that the natural state space for this kernel is rather G×S instead ofG². In the next result we exhibit the spectral decomposition of kernels whose structure generalizes that ofP.

Proposition 2.7. — Let S and V be finite sets with respective cardi- nalities s and v. Let K be a s×s matrix and for any y ∈ S, let Qy be a v×v matrix. Assume there is a basis (ϕl)l∈[[v]] of F(V,C) consisting of eigenfunctions forQy, independent ofy∈S (where[[v]] :={1,2, ..., v}). The corresponding eigenvalues are allowed to depend on y∈S and are denoted by (σl(y))l∈[[v]]. For fixed l∈[[v]], considerKlthe s×smatrix defined by

∀ y, y ∈S, Kl(y, y) := σl(y)K(y, y)

Assume that it is diagonalizable (inC) and denote by(θl,k)k∈[[s]]and(ψl,k)k∈[[s]]

its eigenvalues and corresponding eigenvectors.

Next considerP the (V ×S)×(V ×S)matrix defined by

∀ (x, y),(x, y)∈V ×S, P((x, y),(x, y)) = K(y, y)Qy(x, x) (this is the skew Kronecker product ofQ by K). Then P is diagonalizable, its eigenvalues are the θl,k for(l, k)∈[[v]]×[[s]]and a corresponding family of eigenvectors is (ϕl⊗ψl,k)(l,k)∈[[v]]×[[s]].

Proof. —This result is in fact more difficult to state than to prove. For (l, k) ∈ [[v]]×[[s]], consider Fl,k := ϕl ⊗ψl,k. Since the family (ϕl)l∈[[v]]

is linearly independent and the same is true for the family (ψl,k)_k∈[[s]] for any fixed l ∈ [[v]], the family (Fl,k)(l,k)∈[[v]]×[[s]] is also linearly independent and is thus a basis of F(V ×S,C). Next for given (l, k) ∈ [[v]]×[[s]] and (x, y)∈V ×S, we compute that

P[Fl,k](x, y) =

x∈V,y∈S

K(y, y)Qy(x, x)Fl,k(x, y)

=

y∈S

K(y, y)ψl,k(y)

x∈V

Qy(x, x)ϕl(x)

=

y∈S

K(y, y)ψl,k(y)σl(y)ϕl(x)

= ϕl(x)

y∈S

Kl(y, y)ψl,k(y)

= ϕl(x)θl,kψl,k(y)

= θl,kFl,k(x, y)

so Fl,k is an eigenvector for the eigenvalueθl,k relatively toP.

Coming back to our homogeneous setting, we would like to apply the above proposition withV =Gand the translation kernels (Qy)y∈S defined by

∀ x, x ∈G, Qy(x, x) := δxy(x)

If they were jointly diagonalizable, they would commute and since G is generated byS, we would end up with

∀y, y ∈G, QyQy = QyQy

and these identities imply that G is Abelian. Conversely, if Gis Abelian, then we can find r ∈ N\ {0} and N1, ..., Nr ∈ N\ {0,1} such that G is isomorphe to the group product

l∈[[1,r]]ZNl, so a generical elementx∈G will be written in the form (xl)l∈[[1,r]], with xl ∈ Z^Nl for l ∈ [[r]]. For k = (kl)_l∈[[1,r]]∈

l∈[[1,r]][[0, Nl−1]], consider the mappingρk ∈ F(G,C) defined by

∀ x= (xl)l∈[[1,r]]∈G, ρk(x) = exp



2πi

l∈[[1,r]]

klxl/Nl





Since the mappings ρk, fork∈

l∈[[1,r]][[0, Nl−1]], correspond to the irre- ducible representations ofG, we get that for anyy∈S,

Qy[ρk] = ρk(y)ρk

So it appears that (ρk)_k∈

l∈[[1,r]][[0,Nl−1]]is a basis ofF(G,C) which enables the simultaneous diagonalization of theQy, fory∈S, and we are in position to exploit Proposition 2.7. To do so, we introduce some convenient notations:

G^∗ :=



ρk : k∈

l∈[[1,r]]

[[0, Nl−1]]



 (2.3)

and for ρ∈G^∗, (ρ) is the s×s diagonal matrix whose diagonal entries are given by (ρ(y))_y∈S. We also consider the s×s matrix Aρ := (ρ)K.

Then Proposition 2.7 implies the following result:

Proposition 2.8. — LetGbe a finite Abelian group and M a homoge- neous second order Markov kernel onGwith speed transfer matrixK. If all the matricesAρ, withρ∈G^∗, are diagonalizable, thenM is diagonalizable and its spectrum is the union of the spectra of the Aρ, with ρ∈ G^∗ (with multiplicities).

One can go further: if one of the matrices Aρ is not diagonalizable, one can consider its decomposition into Jordan blocks and an immediate extension of Proposition 2.7 enables to transfer these Jordan blocks into Jordan blocks forM. In particular the spectrum of M is always the union of the spectra of the Aρ, with ρ ∈ G^∗ (with obvious relations between the dimensions of the corresponding Jordan blocks). These observations do not require M to be trajectorially reversible, but we believe that the latter property could be helpful to understand the spectral structure of the matrices Aρ, withρ ∈ G^∗. Nevertheless this reversibility condition won’t be sufficient to insure that they are diagonalizable, even under the stronger assumption made in Lemma 2.9 below (see Section 4).

We end this section with a discussion of the structural properties which can be deduced from a natural reinforcement of the trajectorial reversibil- ity. Indeed, letM be a trajectorially reversible homogeneous second order Markov kernel, with modified speed kernelKreversible with respect toµ. By definition ofK, we have for anyy, y ∈S,µ(y)K(−y, y) =µ(y)K(−y, y), so taking into account the symmetry of µ, we get µ(−y)K(−y, y) =

µ(−y)K(−y, y). Summing over y ∈S, it follows that µ(−y) =µK( −y), namelyµis invariant forK. So the assumption of the next result is not so farfetched, note that it is in particular satisfied by the families of examples considered in Theorem 1.2 and Conjecture 1.3.

Lemma 2.9. — Assume that µis also reversible with respect toK, then the matrices Aρ, with ρ ∈ G^∗, are conjugate to symmetric and centro- Hermitian matrices.

Recall that aS×S matrixB (whereS is a symmetrical set) is said to be centro-Hermitian if B=B, where

∀y, y∈S, B(y, y ) := B(−y,−y)

see for instance Lee [8] or Pressman [12]. Weaver [14] gave a nice review of the corresponding notion of a (real) centro-symmetric matrix, property which is satisfied by the speed transition kernel under the above assumption.

Proof. —Under the hypothesis of this lemma, we simplify the notations and writeµinstead ofµ. For ρ∈G^∗, letρ^1/2:= (ρ^1/2(y))_y∈S be a vector of square roots of (ρ(y))_y∈S, chosen in such a way thatρ^1/2(−y) =ρ^1/2(y) (re- call that we haveρ(−y) =ρ(y) and that this complex number has modulus 1). Define next

Bρ := (√µρ^−1/2)Aρ(ρ^1/2/√µ)

= (√µρ^1/2)K(ρ^1/2/√µ) (2.4) (where for any vector v, (v) is the diagonal matrix with corresponding entries). SinceAρ is conjugate to Bρ, both matrix have the same spectral structure, butBρ has two practical advantages: it is symmetric and centro- Hermitian. Indeed, we have

∀y, y∈S, Bρ(y, y) =

µ(y)

µ(y)K(y, y)ρ^1/2(y)ρ^1/2(y) so the symmetry ofBρis a consequence of the reversibility ofKwith respect toµ. The centro-Hermitian property comes from the fact that

∀y∈S, µ(−y) =µ(y) andρ^1/2(−y) =ρ^1/2(y)

A consequence of the above result is that the spectrum ofAρ, forρ∈G^∗, is stable by conjugacy, since this is true for any centro-Hermitian matrixB (see for instance Lee [8]): to anyf = (f(y))y∈S∈ F(S,C) (seen as a column vector), associate the vector f := (f(−y))_y∈S ∈ F(S,C). An immediate computation shows that

∀f ∈ F(S,C), Bf = Bf

Thus ifB is centro-Hermitian and iff is an eigenvector ofB associated to eigenvalue θ∈C, thenfis an eigenvector of B associated to eigenvalue ¯θ.

Note that we knew a priori that the spectrum ofM is stable by conju- gacy, because its entries are real. But this property corresponds to the fact

that the spectrum of Aρ is conjugate to the spectrum ofAρ¯=Aρ, for any ρ∈G^∗. This leads to the following observation: ifθis a non-real eigenvalue of Aρ, with ρ= ¯ρ∈ G^∗, then its multiplicity as an eigenvalue of M is at least the double of its multiplicity as an eigenvalue ofAρ.

Remark 2.10. — In view of the above considerations, it is not clear how to deduce spectral informations from the trajectorial reversibility assumption.

Nevertheless in the homogeneous case, one can use it to deduce information on the convergence to equilibrium in the following way. For simplicity con- sider the Abelian case. Let (Xn, Yn)_n∈Nbe the position-speed Markov chain associated to the second order Markov kernelM, whose modified speed ker- nel is denoted K as above. Consider the Markov chain (Yn)n∈N defined by

∀n∈N, Yn := (−1)ⁿYn

This chain hasK as transition kernel and thus is reversible with respect to

µ, so to understand its convergence to equibrium, we can resort to the usual spectral techniques. The chain (Xn, Yn)n∈N can be deduced from (Yn)n∈N

via a deterministic procedure: for anyn∈N, we have Xn = Y0−Y1+Y2− · · ·+ (−1)ⁿ⁻¹Yn−1

Yn = (−1)ⁿYn

So (Xn)n∈N appears as a time-inhomogeneous, but periodic, additive func- tional of (Yn)n∈N. To come back to the situation of a time-homogeneous functional, it is better to consider the Markov chain (Y2n,Y2n+1)n∈N, whose convergence to equilibrium can easily be deduced from that of (Yn)n∈N. Then one can resort to the well-known large time behavior of additive func- tionals (see for instance Maxwell and Woodroofe [9] and the references given therein), at least on the covering lattice ofG, based on the central limit the- orem for martingales. Next it remains to “project” this result onG: once the variance grows to cover a fundamental domain forG, the stochastic chain (Xn)_n∈Nwill be close to equilibrium, namely uniformly distributed.

But we won’t develop this approach here, because it is not in the spirit of the paper.

3. Infinitesimal improvement of spectrum

Our purpose here is to show that the spectral gap (and even the whole spectrum outside 0) of a reversible Markov chain can always be improved (except in the situation of an i.i.d. sequence) by resorting to related second order Markov chains, which “have a little less tendency to hesitate in their

way to equilibrium” than the original chain. The rigorous formulation of this heuristic is enclosed in Theorem 1.1, and this section is mainly devoted to his proof. But we will also investigate the best way to improve only the spectral gap, when the underlying first order kernel L is the transition matrix of a random walk on a regular graph and under an additional assumption of multiplicity one for the spectral gap.

Our starting point is a Markov kernelLwhich admits a reversible proba- bility measureν, positive on the finite setV. Since we will only be interested in second order Markov chains which are trajectorially reversible with re- spect to the probability measureν×L(defined in (1.3)), up to a restriction of the state space V, there is no loss of generality in assuming that L is furthermore irreducible. We associate to L the second order Markov ker- nel M⁽⁰⁾ defined in (1.5). We begin by verifying an assertion made in the introduction:

Proposition 3.1. — The spectrum of the transition operator M⁽⁰⁾ is equal to the spectrum ofLwith the same multiplicities, except for the eigen- value 0, whose multiplicity isN²−N, whereN =|V|. In particularM⁽⁰⁾is diagonalizable if and only if0is not an eigenvalue ofL. When 0 is an eigen- value ofL, letdbe the dimension of the corresponding eigenspace. Then the canonical form of M⁽⁰⁾ containsd2×2-Jordan blocks

0 1 0 0

.

It follows that (M⁽⁰⁾)²is always diagonalizable. This is not true forM⁽⁰⁾, as we will see on an example in next section.

Proof. —Sinceν has positive weight everywhere and is reversible with respect to L, the latter operator is diagonalizable with real eigenvalues.

Let (θi)i∈[[N]] be its spectrum and (fi)i∈[[N]] be a corresponding family of eigenvectors. Assume thatθ1=θ2=· · ·=θd= 0, whered= dim(Ker(L)).

For i∈[[N]], define Fi := 11⊗fi ∈ F(V²) and fori ∈[[d]], Gi :=fi⊗11∈ F(V²). We compute that

M⁽⁰⁾[Fi] = 11⊗L[fi] = θiFi

M⁽⁰⁾[Gi] = 11⊗(fiL[11]) = Fi

First we deduce that the spectrum ofLis included in the spectrum ofM⁽⁰⁾ and next that for i ∈[[d]], the 2-dimensional vector space generated by Fi

and Gi is stable by M⁽⁰⁾ and that the matrix associated to the operator M⁽⁰⁾ in the basis (Gi, Fi) is the 2×2-Jordan block

0 1 0 0

.

Note that the vectorsF1, F2, ..., FN are independent and belong to the image ofM⁽⁰⁾, so the rank of M⁽⁰⁾ is at leastN. This rank is exactlyN, because

for anyF ∈ F(V²),M⁽⁰⁾[F] only depends on the second variable. It follows that the kernel ofM⁽⁰⁾ is of dimensionN²−N.

In the above proof the reversibility ofLwas not really necessary, it was sufficient to assume that it is diagonalizable. One can even go further by revisiting the above proof:

Remark 3.2. — Let L be any Markov kernel on S and construct the kernelM⁽⁰⁾as in (1.5). Then as above the spectrum ofM⁽⁰⁾ is equal to the spectrum ofLplus the value 0. More precisely, the Jordan blocks associated to an eigenvalueθ= 0 ofLare in exact correspondance with those ofM⁽⁰⁾. Concerning the eigenvalue 0, letdbe the number of 1×1-Jordan blocks for L. All of them are transformed into 2×2 blocks for M⁽⁰⁾. But for r 2, all the r×r-Jordan blocks of L are transformed intor×r-Jordan blocks forM⁽⁰⁾. And there is furthermore the creation ofN²−N−d1×1-Jordan blocks forM⁽⁰⁾. This comes from the fact that the (r+ 1)×(r+ 1) matrix Adefined by

∀i, j∈[[r+ 1]], A(i, j) :=





1 , ifj =i+ 1 andi∈[[r−1]]

1 , ifi= 0 andj=r+ 1 0 , otherwise

admits a decomposition into a 1×1 block and ar×rblock (both associated to the eigenvalue 0).

In Proposition 3.1 we have considered the spectrum ofM⁽⁰⁾ as an op- erator on F(V²), but in general we are only interested on its action on a subspace ofF(V²): since the probability measureν×Lis invariant forM⁽⁰⁾, we prefer to seeM⁽⁰⁾ as an operator onL²(ν×L), whose dimension is the cardinality of the set{(x, y)∈V² : L(x, y)>0}. By our assumptionν >0, we have F(V) = L²(ν) and there is no ambiguity for the spectrum of L.

The proof of Proposition 3.1 can then be extended to this situation: the spectrum ofM⁽⁰⁾is the spectrum ofLplus maybe the eigenvalue 0. Indeed the eigenvalue 0 is necessary in the spectrum ofM⁽⁰⁾if dim(L²(ν×L))> N.

Note that under our reversibility and irreducibility assumptions onL, the only situations when this inegality is not satisfied is when|V|= 1 or|V|= 2 and L is given by the matrix

0 1 1 0

. Similar observations also hold in the more general setting of the above remark (but assume furthermore that Lis irreducible, so that it admits a unique positive invariant measure).

In order to go in the direction of Theorem 1.1, consider a second order Markov kernelM⁽¹⁾ admittingν×Las trajectorial reversible measure and

satisfying for anyx, x, x∈V,

M⁽¹⁾(x, x;x)> L(x, x), ifx=x,L(x, x)>0 andL(x, x)>0 (3.5) Such transition kernels exists, since according to Lemma 2.1, the only re- quirement is that for any fixed x ∈ V, the kernel (M_x⁽¹⁾ (x, x))x,x∈V is reversible with respect to the probabilityL(x,·).

Next, as in the introduction, for anya∈[0,1], we consider the interpolation kernel betweenM⁽⁰⁾ andM⁽¹⁾:

M^(a) := (1−a)M⁽⁰⁾+aM⁽¹⁾

which is a second order Markov kernel still admitting ν×Las trajectorial reversible measure. We need some information about the behavior of the spectral decomposition ofM^(a)as a function ofa, at least for smalla. From Chapter 2§1.8 of Kato [7], we easily deduce the following result:

Lemma 3.3. — There exists a0 ∈(0,1) such that the number s of dis- tinct eigenvalues of M^(a) does not depend on a∈ (0, a0). It is possible to parameterize these eigenvalues by (θl(a))l∈[[s]] in such a way that for any l ∈ [[s]], the mapping (0, a0) a → θl(a) is analytic. Furthermore these mappings admit continuous extension to[0, a0)and the set{θl(0) : l∈[[s]]} coincides with the spectrum of M⁽⁰⁾ (note nevertheless that the cardinality of this spectrum can be stricly less than s, when a splitting phenomenon of some eigenvalues at 0 occurs). Moreover for any l ∈[[s]], we can find a continuous mapping[0, a0)a→Fl(a)∈ F(V²,C)\ {0} such thatFl(a)is an eigenfunction associated to the eigenvalue θl(a) of M^(a), and such that the mapping(0, a0)a→Fl(a)∈ F(V²,C)\ {0} is analytic.

In fact the above lemma holds for general second order Markov kernel M⁽¹⁾. But the assumption (3.5) is need for the next result. See also Theorem 2.3 in Chapter 2 §2.3 of Kato [7] for general differentiability properties of the eigenvalues.

Proposition 3.4. — With the notations introduced in the previous lem- ma, assume thatl∈[[s]]is such thatθl(0)= 0. Then the limitlima→0+θ_l(a) exists and belongs to R. By consequence θ_l(0) exists and coincides with the previous limit. Furthermore if |θl(0)| = 1, then the sign of θ_l(0) is the op- posite sign of θl(0) (recall that θl(0) ∈ R\ {0}). If |θl(0)| = 1, we have θ_l(0) = 0.

Proof. —By definition, we have for anya∈[0, a0), M^(a)Fl(a) = θl(a)Fl(a)

and we can differentiate this equality fora∈(0, a0) to get

(M⁽¹⁾−M⁽⁰⁾)Fl(a) +M^(a)F_l(a) = θ_l(a)Fl(a) +θl(a)F_l(a) (3.6) where the primes represent differentiation with respect toa∈(0, a0).

Our goal is to get rid of the termF_l(a) that we don’t control asais going to 0+. Lemma 2.4 implies that the functionFl(a) defined in Lemma 2.4 is an eigenvector of (M^(a))^∗associated to the eigenvalueθ(a). Indeed, we have

(M^(a))^∗[Fl(a)] = M^(a)[Fl(a)]

= θ(a)Fl(a)

= θ(a)Fl(a)

Multiply (3.6) byFl(a) and integrate the obtained equation with respect to m, to get

m[Fl(a)(M⁽¹⁾−M⁽⁰⁾)Fl(a)] +m[Fl(a)M^(a)F_l(a)]

=θ_l(a)m[Fl(a)Fl(a)] +θl(a)m[Fl(a)F_l(a)]

By definition of the dual operator (M^(a))^∗we have

m[Fl(a)M^(a)F_l(a)] = m[(M^(a))^∗[Fl(a)]F_l(a)]

= θ(a)m[Fl(a)F_l(a)]

Thus we deduce the following equation where the term F_l(a) has disap- peared:

m[Fl(a)(M⁽¹⁾−M⁽⁰⁾)Fl(a)] = θ_l(a)m[Fl(a)Fl(a)]

We can now use the continuity properties put forward in Lemma 3.3 to see that

a→0lim+

m[Fl(a)(M⁽¹⁾−M⁽⁰⁾)Fl(a)] = m[Fl(0)(M⁽¹⁾−M⁽⁰⁾)Fl(0)]

a→0lim+

m[Fl(a)Fl(a)] = m[Fl(0)Fl(0)]

so the limit lima→0+θ_l(a) exists if we havem[Fl(0)Fl(0)]= 0. To compute this term, we note that since Fl(0) is an eigenvector of M⁽⁰⁾ associated to the eigenvalueθl(0), according to the proof of Proposition 3.1, there exists an eigenfunctionf ∈ F(V) ofLassociated to the eigenvalueθl(0) such that

∀x, x∈V, Fl(0)(x, x) = f(x)