HAL Id: hal-01273085
https://hal.archives-ouvertes.fr/hal-01273085
Submitted on 11 Feb 2016
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Reciprocal classes of random walks on graphs
Giovanni Conforti, Christian Léonard
To cite this version:
Giovanni Conforti, Christian Léonard. Reciprocal classes of random walks on graphs. Stochastic Processes and their Applications, Elsevier, 2017, 27 (6), pp.1870-1896. �hal-01273085�
RECIPROCAL CLASSES OF RANDOM WALKS ON GRAPHS
GIOVANNI CONFORTI AND CHRISTIAN LÉONARD
Contents
Introduction 1
1. Preliminaries 6
2. Main results 9
3. Proofs of the main results 12
4. More results 18
5. Examples 21
Appendix A. Shared bridges 30
Appendix B. Closed walks 31
References 33
Abstract. The reciprocal class of a Markov path measure is the set of all mixtures of its bridges. We give characterizations of the reciprocal class of a continuous-time Markov random walk on a graph. Our main result is in terms of some reciprocal characteristics whose expression only depends on the jump intensity. We also characterize the reciprocal class by means of Taylor expansions in small time of some conditional probabilities.
Our measure-theoretical approach allows to extend significantly already known results on the subject. The abstract results are illustrated by several examples.
Introduction
This article answers the question: “When does a continuous-time random walk on a graph share its bridges with a given Markov walk?”, both in terms of their jump intensities and of Taylor expansions in small time of probabilities of conditioned events. The precise answers are stated at Theorem 2.4 and Corollary 2.6 which are the main results of the article.
The set of all path measures which share the bridges of a given Markov measure is called its reciprocal class. In contrast with most of the existing literature about recipro-cal classes, i.e. shared bridges, which relies on transition probabilities, in this paper we adopt a measure-theoretical approach: our main objects of interest are path measures, i.e. probability measures on the path space, rather than transition probability kernels. It turns out that this is an efficient way for solving our problem and allows to extend significantly already known results on the subject.
Date: Version 6. April 26, 2015.
2010 Mathematics Subject Classification. 60J27,60J75.
Key words and phrases. Random walks on graphs, bridges of random walks, reciprocal characteristics, Schrödinger problem.
CL is partially supported by the French ANR projects ANR-11-BS01-0007 - GeMeCoD and ANR-12-BS01-0019 - STAB.
Notation. Some notation is needed before bringing detail about the above question and its answers. For any measurable space Y,P(Y ) denotes the set of all probability measures on Y. We denote the support of a probability measure p by supp p. On a discrete space A, we have supp p = {a ∈ A : p(a) > 0} and for any probability measures p and q, p is absolutely continuous with respect to q: p q, if and only if supp p ⊂ supp q.
The support of a function u ∈ RA is defined as usual by supp u := {a ∈ A : u(a) 6= 0} .
Functions with a finite support will be useful to define Markov generators without extra assumptions on the jump intensity.
We consider random walks from the unit time interval [0, 1] to a countable directed graph (X , A→) where only jumps along the set A→ ⊂ X2 of the arcs of the graph are
allowed. The set of all the sample paths is denoted by Ω⊂ X[0,1]. As we adopt a measure
theoretical viewpoint, it is worth identifying the random processes and their laws on the path space. Consequently, any path measure P ∈ P(Ω) is called a random walk.
The canonical process on Ω is denoted as usual by (Xt; 0≤ t ≤ 1). For any random walk
P ∈ P(Ω) we denote
• P0(dx) := P (X0 ∈ dx) ∈ P(X ), its initial marginal;
• P01(dxdy) := P (X0 ∈ dx, X1 ∈ dy) ∈ P(X2), its endpoint marginal;
• Px := P (· | X
0 = x)∈ P(Ω), the random walk conditioned to start at x ∈ supp P0;
• Pxy := P (· | X
0 = x, X1 = y)∈ P(Ω), its xy-bridge with (x, y) ∈ supp P01.
Aim of the article. We take some Markov random walk R ∈ P(Ω) with the jump intensity j : [0, 1]× A→→ [0, ∞) and we assume that its initial marginal R0 ∈ P(X ) has
a full support. This random walk is our reference path measure.
For comparison with the results of this article, let us consider for a little while the set M(j) := ( X x∈X µ0(x) Rx; µ0 ∈ P(X ) ) ⊂ P(Ω),
assuming that for any x, Rx is uniquely well defined. Obviously, for any P ∈ P(Ω), the three following statements are equivalent:
(1) P ∈ M(j);
(2) Px = Rx, for all x∈ supp P 0;
(3) For any x∈ supp P0, Px is Markov with intensity j.
Rather than the collection of all the random walks Rx conditioned by their starting point x∈ X , our interest is in the bridges of R. We define
R(j) := ( X x,y∈X π(x, y) Rxy; π ∈ P(X2) : supp π ⊂ supp R01 ) ⊂ P(Ω) (1)
to be the convex hull of all these bridges. This set is called the reciprocal class of the intensity j. Since the reference random walk R is Markov, so are its bridges Rxy. But, in general a mixture of such bridges fails to remain Markov. However, any element P of R(j) still satisfies the reciprocal property which extends the Markov property in the following way.
Definition 0.1 (Reciprocal walk). A random walk P ∈ P(Ω) is said to be a reciprocal walk if for any 0≤ u ≤ v ≤ 1, P (X[u,v]∈ · | X[0,u], X[v,1]) = P (X[u,v]∈ · | Xu, Xv).
The reciprocal property of a path measure is defined in accordance with the usual notion related to processes. Basic material about reciprocal walks is collected at Appendix A.
(1)’ P ∈ R(j);
(2)’ Pxy = Rxy, for all (x, y)∈ supp P 01.
The aim of the article is to provide an analogue of statement (3) above. Indeed, Theorem
2.4 states that (1)’ and (2)’ are equivalent to (3)’ For any x∈ supp P0, Px is Markov and
χ[Px] = χ[j], (2)
where for any random walk P , χ[P ] = χ[k] only depends on the intensity k of P and is described at Definition 2.3. For this reason, χ[j] is called the characteristic of the reciprocal class R(j).
Markov or not Markov? There are other random walks P than R inR(j) that are Markov. In this case, the intensities(kx; x∈ supp P
0) are such that kxdoes not depend on x. When
kx depends explicitly on x, P is not Markov; this is the case for most of the elements of R(j).
Variational processes. Besides the interest in its own right of the description of the convex hull of the bridges of some Markov dynamics, there exists a stronger motivation for investigating the reciprocal class of a Markov process. Suppose that you observe two large samples of non-interacting particle systems with two distinct endpoint distributions (i.e. empirical measures of the couples of initial and final positions). Are these two random systems driven by the same force field? In mathematical terms, you want to know if the dynamics that drive these two particle systems belong to the same reciprocal class.
This question is rooted into a problem addressed by Schrödinger in the early 30’s in the articles [Sch31, Sch32].
Schrödinger problem. Consider a large number N of independent particles labeled by 1 ≤ i ≤ N and moving according to some Markov dynamics described by the reference path measure R ∈ P(Ω). In Schrödinger’s papers, R is the law of a Brownian motion on X = Rn and Ω = C([0, 1], Rn). Suppose that at time t = 0, the particles are distributed
according to a profile close to some distribution µ0 ∈ P(X ). In modern terms, this means
that the empirical measure N1 P
1≤i≤NδXi(0) of the initial sample is weakly close to µ0, where δa is the Dirac measure at a and t 7→ Xi(t) describes the random motion of the
i-th particle. Suppose that at time t= 1 you observe that the whole system is such that its distribution profile N1 P
1≤i≤NδXi(1) is weakly close to some µ1 ∈ P(X ) far away from the expected profile µ0eL ∈ P(X ) which is predicted by the law of large numbers. Here, L
is the Markov generator of R. Schrödinger asks what is the most likely trajectory of the whole particle system conditionally on this very rare event. As translated in modern terms by Föllmer in [Föl88] using large deviations technics, the empirical measure N1 P
1≤i≤NδXi weakly tends inP(Ω) to the unique solution of entropy minimization problem
H(P|R) → min; P ∈ P(Ω) : P0 = µ0, P1 = µ1, (3)
where H(P|R) := EP log(dP/dR)∈ [0, ∞] is the relative entropy of P with respect to the
reference measure R. A recent review of the Schrödinger problem is proposed in [Léo14]. A stochastic analogue of Hamilton’s principle. In the case where R is a Brownian mo-tion, any P ∈ P(Ω) with H(P |R) finite is the solution of a martingale problem associ-ated with some adapted drift field βP such that E
P R [0,1]|β P t |2dt < ∞ and H(P |R) = EP R [0,1]|β P
and the minimization problem (3) appears to be a stochastic generalization of the usual Hamilton variational principle, see for instance [Léo14] and the references therein.
A natural extension of Schrödinger’s problem. Schrödinger’s problem (3) also admits the following natural extension
H(P|R) → min; P ∈ P(Ω) : P01 = π (4)
with π ∈ P(X2) a prescribed endpoint distribution. In some sense, the entropy minimiza-tion problem (4) is the widest stochastic extension of the classical Hamilton least action principle.
The connection with the reciprocal classR[j]. To see the connection with the bridges of R, let us go back to our discrete set of verticesX and note that both Schrödinger’s problem (3) with the prescribed marginals µ0 = δx and µ1 = δy and problem (4) with π = δ(x,y),
admit the unique solution
P = Rxy.
This is a simple consequence of the additive decomposition formula H(P|R) = H(P01|R01) +
X
x,y∈X
P01(x, y)H(Pxy|Rxy)
applied with P01 = δ(x,y). One can interpret the bridge Rxy as the stochastic analogue
of a minimizing geodesic between x and y. We also see with the additive decomposition formula that the unique solution of (4) with a general endpoint distribution π ∈ P(X2)
such that H(π|R01) <∞ is
P = X
x,y∈X
π(x, y) Rxy.
Therefore, the reciprocal class R(j) appears to be essentially the set of all solutions of the stochastic variational problem (4) when π ∈ P(X2) describes all possible endpoint distributions. In addition, we see with (2) that all bridges of R satisfy
χ[Rxy
] = χ[j], ∀(x, y) ∈ supp R01.
This indicates that the reciprocal characteristic χ[j] encrypts the underlying stochastic Lagrangian associated with the stochastic action minimization problem (4).
This point of view is developed in a diffusion setting by Zambrini and the second author in [LZ]. It will be explored in the present setting of random walks on graphs in a forthcoming paper.
At the present time very little is known about the solutions of the variational problems (3) and (4) in the setting of random walks on graphs. This paper is a contribution in this direction.
Literature. One year after Schrödinger’s article [Sch31], Bernstein introduced in [Ber32] the reciprocal property as a notion that extends Markov property and is respectful of the time reversal symmetry. It was further developed four decades later by Jamison [Jam74, Jam75]. Relying on Jamison’s approach to reciprocal processes, Clark proved in [Cla91] a conjecture of Krener [Kre88] who proposed a characterization of the reciprocal class of a Brownian diffusion process in terms of an identity of the type χ[βP] = χ[0] where
βP is the drift of the Markov diffusion process P (recall the discussion below Eq. (3) for
prefer naming “reciprocal characteristics” the analogous quantities χ[k].1 In view of the
previous discussion about variational processes, it is not surprising that the reciprocal characteristics play a distinguished role when looking at reciprocal processes as solutions of second order stochastic differential equations. This is investigated by Krener, Levy and Thieullen in [KL93,Thi93, Kre97].
Characterization of reciprocal classes can also be stated in terms of stochastic integra-tion by parts formulas, often called duality formulas. This was investigated by Roelly and Thieullen in [RT04, RT05] for diffusion processes. In the specific context of counting random walks, this is done in [CLMR15], a paper by Murr, Roelly and the authors of the present article. This was extended by Dai Pra, Rœlly and the first author in [CDPR] for compound Poisson processes and in [CR] for random walks on Abelian groups. A main idea of [CDPR, CR] is to exploit the translation-invariant structure of the underlying graph to characterize the reciprocal classes through integration by parts formulas where the derivation measures the variation when adding a random closed walk to the canonical process, see [CDPR, Thm. 3.3] and [CR, Thm. 13]. In all these cases, reciprocal charac-teristics play a major role. If the graph is not assumed to be invariant with respect to some group transformations, such as translation-invariance and time homogeneity, there is no way of thinking of a natural differentiation. Since we do not assume any invariance in the present article, the integration by parts approach is not investigated.
It is worthwhile to note that, except for [RT04] and the recent papers [CLMR15,CDPR,
CR,LZ], in the whole literature on the subject, only the Markov members of the reciprocal class, that is the solutions of the original Schrödinger problem (3) as the marginal con-straints vary, are characterized. In the present article, following a strategy close to [LZ]’s one, we give a characterization of the whole reciprocal class, that is the set of solutions of (4) as π varies, under very few restrictions on the reference random walk.
Outline of the paper. Next Section 1 is devoted to some preliminaries about directed graphs, Markov walks and their intensities. We also state our main hypotheses which are Assumption 1.1 and Assumption 1.5 and define carefully the reference path measure R. Our main results are stated at Section 2. They are Theorems2.4 and 2.5, together with their Corollary2.6. Theorem2.4 is a rigorous version of statement (3)’ while Theorem2.5
provides an interpretation of the reciprocal characteristics in terms of Taylor expansions in small time of some conditional probabilities. Their proofs are done at Section 3. The key preliminary result is Lemma 3.2 whose proof is based on the identification of two expressions for the Radon-Nikodym derivative dP/dR of any element P of the reciprocal class with respect to the reference Markov measure R. Some more results about the ele-ments of the reciprocal class are proved at Section4. In particular, we give at Proposition
4.1 another characterization of R(j) in terms of the shape of the jump intensity of any element of the class. The characteristic equation (2) seen as an equation of the unknown Px where j is given is also solved at Theorem4.2. Several examples are treated at Section
5. We have a look at: directed trees, birth and death processes, some planar graphs, the hypercube, the complete graph and some Cayley graphs. We calculate their reciprocal characteristics and in some cases solve the associated characteristic equation. Finally, there are appendix sections devoted to reciprocal random walks and to closed walks on a directed graph.
1In the contexts of classical, quantum and stochastic mechanics (the latter being taken in its wide
meaning including Euclidean quantum mechanics [CZ08] or hydrodynamics where the evolution is de-terministic but the initial state is described by a probability measure), the term “invariant” refers to conserved quantities as time varies.
1. Preliminaries
This section is devoted to some preliminaries about directed graphs, Markov walks and their intensities. Our main hypotheses are stated below at Assumption 1.1 and Assumption 1.5. Much of the material in this section is required for the definition of the reciprocal characteristics and the statements of our results at Section 2.
Directed graph. Let X be a countable set. Any subset A ⊂ X2 of the product space
X2 defines a structure of oriented graph onX by means of the relation → which is defined
for all z, z0 ∈ X by z → z0 if and only if (z, z0) ∈ A . We denote (X , →) this directed
graph, say that any (z, z0)∈ A is an arc and write (z → z0)∈ A instead of (z, z0)∈ A .
Since(X , →) and A carry the same information, any subset of X2 will called a directed
graph. When we want to emphasize the role of the vertex set, we sometimes write(X , A ). Assumption 1.1. The directed graph (X , →) satisfies the following requirements.
(1) It is locally finite, meaning that any vertex z ∈ X has finitely many neighbors, i.e. for all z ∈ X , the outer degree of z: deg(z) := # {z0 ∈ X : z → z0} , is finite. (2) It has no loops, meaning that (z → z) 6∈ A for all z ∈ X .
Random walk on a graph. The countable setX is equipped with its discrete topology. We look at continuous-time random paths on (X , →) with finitely many jumps on the bounded time interval[0, 1]. The corresponding path space Ω⊂ X[0,1] consists of all càdlàg
piecewise constant paths ω = (ωt)0≤t≤1 onX with finitely many jumps such that ω1− = ω1 and for all t ∈ (0, 1), ωt− 6= ωt implies that ωt− → ωt. It is equipped with the canonical σ-field generated by the canonical process.
Definition 1.2. We call any probability measure on Ω a random walk on (X , →). This is not the customary usage, but it turns out to be convenient. As a probability measure, it specifies the behavior of a piecewise constant continuous-time random process that may not be Markov.
Notation related to random walks. As usual, the canonical process X = (Xt)t∈[0,1] is
defined for each t∈ [0, 1] and ω = (ωs)0≤s≤1 ∈ Ω by Xt(ω) = ωt ∈ X . For any T ⊂ [0, 1]
and any random walk P ∈ P(Ω), we denote XT = (Xt)t∈T and the push-forward measure
PT = (XT)#P. In particular, for any 0 ≤ t ≤ 1, Pt = (Xt)#P ∈ P(X ) denotes the law
of the position Xt at time t and P01 := P{0,1} denotes the law of the endpoint position
(X0, X1). Also, for all 0≤ r ≤ s ≤ 1, X[r,s]= (Xt)r≤t≤s and P[r,s]= (X[r,s])#P.
We are mainly interested in bridges. In a general setting, one must be careful because the bridge Pxy is only defined P01-almost everywhere. But in the present case where the
state spaceX is countable, the kernel (x, y) 7→ Pxy is defined everywhere onsupp P 01, no
almost-everywhere-precaution is needed when talking about bridges. In particular, any random walk P disintegrates as
P = X
(x,y)∈supp P01
P01(x, y)Pxy ∈ P(Ω).
In the whole paper, the letters x and y are devoted respectively to the initial and final states of random walks. Current states are usually denoted by z, z0 ∈ X .
Markov walk. The Markov property of a path measure is defined in accordance with the usual notion related to processes.
Definition 1.3 (Markov walk). A random walk P ∈ P(Ω) is said to be a Markov walk if for any 0≤ t ≤ 1, P (X[t,1]∈ · | X[0,t]) = P (X[t,1] ∈ · | Xt).
A Markov walk P ∈ P(Ω) is the law of a continuous-time Markov chain with its sample paths in Ω. In our setting, all Markov walks to be encountered will be associated with some jump intensity2k: [0, 1)×A → [0, ∞) which gives rise to the infinitesimal generator that acts on any real function u∈ RX with a finite support via the formula
Ktu(z) =
X
z0:z→z0
k(t, z → z0) (u
z0 − uz), z ∈ X , t ∈ [0, 1).
The random walk P ∈ P(Ω) is such that for any real function u ∈ RX with a finite support, the process u(Xt)−
Rt
0 Ksu(Xs) ds is a local P -martingale with respect to the
canonical filtration. The average frequency of jump from z at time t is ¯
k(t, z) := X
z0:z→z0
k(t, z → z0). (5)
Note that this is a finite number because it is assumed that(X , →) is locally finite. The sample paths of the random walk P admit a version in Ω if and only if k(t, z → z0) is
t-measurable for all z and z0 and Z
[0,1)
¯
k(t, Xt) dt <∞, P -a.s.
Indeed, this estimate means that the canonical process performs P -a.s. finitely many jumps.
In the situation where k doesn’t depend on t, the dynamics of the random walk is de-scribed as follows. Once at site z, the walker waits during a random time with exponential law with parameter ¯k(z) and then jumps onto z0 according to the probability measure
¯
k(z)−1P
z0:z→z0k(z → z0)δz0 where δz0 stands for the Dirac measure at z0, and so on; all these random events being mutually independent.
Note that k(t,·) and Ktare only defined for t in the semi-open interval[0, 1). The reason
for this is that we are going to work with mixtures of bridges and the forward intensity of a bridge is singular at t= 1.
Directed subgraphs associated with an intensity. There are two relevant graph structures that are associated with the intensity k. The subgraph of k-active arcs at time t is the subset
A→(t, k) :={(z → z0)∈ A : k(t, z → z0) > 0}
and its symmetric extension is denoted by
A↔(t, k) :={(z → z0), (z0 → z); (z → z0)∈ A→(t, k)} .
Only Markov intensities such that these structures do not depend on time will be encountered, see (6) below.
Definitions 1.4 (The directed subgraphs associated to k). In the comfortable situation where A→(t, k) does not depend on t, i.e.
A→(t, k) =A→(k), ∀t ∈ [0, 1), (6)
A→(k) is called the subgraph of k-active arcs andA↔(k) denotes its symmetric extension.
The symmetrized subgraph will be necessary for considering closed walks.
2This is maybe the case for any Markov walk with finitely many jumps, but we shall not need to
Directed subgraphs associated with a random walk. Again P ∈ P(Ω) is a Markov random walk with an intensity k that satisfies (6). Since A ' (X , →) is assumed to be locally finite, and P -almost every sample path performs finitely many jumps, the assumption (6) implies that the support of Pt ∈ P(X ) remains constant for each time
0 < t < 1. We denote this set by
X (P ) := supp P1/2 ⊂ X .
It is the set of all vertices that are visited by the random walk P . Note that the initial and final times t = 0 and t = 1 are excluded to allow for P to be a bridge. We always havesupp P0 ⊂ X (P ) and supp P1 ⊂ X (P ); these inclusions may be strict.
The directed subgraph(X (P ), A→(P )) of all P -active arcs is the subgraph of (X , A→(k))
which is defined by
A→(P ) :={(z → z0)∈ A→(k) : z, z0 ∈ X (P )} .
Its symmetric extension is
A↔(P ) :={(z → z0), (z0 → z); (z → z0)∈ A→(k), z, z0 ∈ X (P )} .
Let us provide some comment to make the relation between A→(k) and A→(P ) clearer.
If the initial marginal P0 is supported by a proper subset of X , it might happen that
A→(P ) is a proper subset of A→(k) and also that A↔(P ) is a proper subset of A↔(k).
For instance, let k be the intensity of the Poisson process on X = Z given by k(t, n → n + 1) = λ > 0 for all 0 ≤ t ≤ 1 and n ∈ Z. Let P be the Poisson random walk with intensity λ and initial state n0 ∈ Z. Then, A→(k) = {(n → n + 1); n ∈ Z} and
A→(P ) ={(n → n + 1); n ∈ Z, n ≥ n0} .
The reference intensity and the reference random walk. We introduce a random walk R∈ P(Ω) with jump intensity j. Both R and j will serve as reference path measure and intensity. We assume that A→(t, j) doesn’t depend on time:
A→(t, j) =A→(j), ∀t ∈ [0, 1). (7)
We also suppose that
A↔(j) =A and projX A = X (8)
where as a definition, for any A⊂ X2,
projX(A) :={z ∈ X : (z → z 0
)∈ A for some z0 ∈ X } .
Although (7) is a restriction, when requiring (8) one doesn’t loose generality since all random walks to be encountered later are absolutely continuous with respect to R. Note that this implies that (X , A ) is a symmetric directed graph. Similarly we denote
A→(j) =A→
to ease notation.
Assumption 1.5. The Markov jump intensity j : [0, 1] × A → [0, ∞) verifies the following requirements.
(1) The jump intensity j is such that the estimate sup
t∈[0,1],z∈X¯j(t, z) < ∞
(9) holds where, as in (5), ¯j(t, z) :=P
z0:z→z0j(t, z → z0) stands for the average frequency of jump from z at time t.
(2) There is some subset A→ of A such that all 0 ≤ t < 1, j(t, z → z0) > 0 for any
(z → z0)
∈ A→ and j(t, z → z0) = 0 for any (z → z0)∈ A \ A→. This is another way
for expressing (7).
(3) The intensity j is continuously t-differentiable, i.e. for any (z → z0)
∈ A→ the
func-tion t7→ j(t, z → z0) is continuously differentiable on [0, 1].
The Assumption 1.5(1) implies that for each x∈ X , there exists a unique solution Rx ∈ P(Ω)
to the martingale problem with initial marginal δx associated with the generator L =
(Lt)0≤t≤1 defined for all finitely supported functions u by
Ltu(z) =
X
z0:z→z0
j(t, z → z0) (uz0 − uz), t∈ [0, 1], z ∈ X . For any x ∈ X , the bridges Rxy := Rx(· | X
1 = y) of Rx are well defined for all y ∈
supp Rx
1. The reference random walk is defined by
R :=X
x∈X
R0(x) Rx ∈ P(Ω)
where the initial marginal R0 is any probability measure on X with a full support, i.e.
supp R0 =X .
Definitions 1.6 (About the j-active arcs from x to Y). Let x ∈ X be any vertex and Y be any nonempty subset of supp Rx
1.
(a) We define the subgraph
AR
→(x,Y) := ∪y∈YA→(Rxy)
of all the arcs that constitute the A→-walks from x to Y.
(b) We denote AR
↔(x,Y) its symmetric extension.
(c) We define the set
XR(x,
Y) := projX A→R(x,Y)
of all vertices visited by the A→-walks starting at x and ending in Y.
Remark 1.7. Remark that{x} and Y may be proper subsets of XR(x,Y) as the example of a bridge Rxy suggests in many situations. We also have for any P ∈ P(Ω),
({x} ∪ supp Px
1)⊂ X (P x
) = projX A→(Px)
where the inclusion may be strict.
2. Main results
Before stating the main results of the article, we still need to introduce two objects which are related to the notion of reciprocal walk, see Definition 0.1 for this notion. Reciprocal class. The reciprocal class R(j) is defined at (1). It is the main object of our study.
Proposition 2.1. The reciprocal class R(j) is a set of reciprocal walks in the sense of Definition 0.1, which are absolutely continuous with respect to R.
Proof. By its very definition, R(j) is the subset of all convex combinations of the bridges of the Markov walk R. RemarkA.3(d) tells us that any P ∈ R(j) is reciprocal. Moreover, PropositionA.1 tells us that P R because supp P01 = supp π⊂ supp R01.
Remark 2.2. Since any element of R(j) is absolutely continuous with respect to R, by Girsanov’s theory it admits a predictable intensity, see [Jac75, Thm. 4.5]. This will be used constantly in the rest of the article.
Reciprocal characteristics. We are going to give a characterization of the elements of R(j) in terms of reciprocal characteristics which we introduce right now.
Definitions 2.3 (Reciprocal characteristics of a Markov random walk). Let P ∈ P(Ω) be a Markov random walk with its jump intensity k which is assumed to satisfy (6) and to be continuously t-differentiable, i.e. for any (z → z0)
∈ A→(P ) the function t7→ k(t, z → z0)
is continuously differentiable on the semi-open time interval [0, 1). (a) We define for all t∈ [0, 1) and all (z → z0)∈ A→(P ),
χa[P ](t, z→ z0
) := ∂tlog k(t, z→ z0) + ¯k(t, z0)− ¯k(t, z)
where ¯k is defined at (5).
(b) We define for all t ∈ [0, 1) and any closed walk c = (x0 → · · · → x|c| = x0) on the
directed graph(X , A↔(P )) associated with the symmetric extensionA↔(P ) ofA→(P ),
χc[P ](t, c) := Y (xi→xi+1)∈A→(P ) k(t, xi → xi+1) Y (xi→xi+1)∈A0(P ) k(t, xi+1→ xi) where A0(P ) :=A↔(P )\ A→(P ) ={(z → z0)∈ A↔(P ) : k(t, z → z0) = 0, ∀t ∈ [0, 1)}
is the set of all k-inactive arcs. No graph structure is associated to A0(P ). See
Defi-nition B.1 for the notion of closed walk.
(c) We call χ[P ] = (χa[P ], χc[P ]) the reciprocal characteristic of P .
The term χa[P ] is the arc component and χc[P ] is the closed walk component of χ[P ].
(d) We often write
χ[R] =: χ[j] to emphasize the role of the reference intensity j.
(e) A closed walk c as in item (b) above is shortly called a closed A↔(P )-walk.
Note that no division by zero occurs and that under our regularity assumption on k, ∂t
acts on a differentiable function: χ[P ] is well defined.
The main results. They are stated at Theorems2.4,2.5and Corollary2.6. Theorem2.4
gives a characterization of the reciprocal class of j in terms of the reciprocal characteristics. Theorem2.5provides an interpretation of the reciprocal characteristics of a reciprocal walk by means of short-time asymptotic expansions of some conditional probabilities. Putting together these theorems leads us to Corollary 2.6 which states a characterization of the reciprocal class in terms of these short-time asymptotic expansions.
Theorem 2.4 (Characterization ofR(j)). We suppose that (X , →) satisfies Assumption
1.1 and j satisfies Assumption1.5.
A random walk P ∈ P(Ω) belongs to R(j) if and only if the following assertions hold for all x∈ supp P0.
(i) The conditioned random walk Px is Markov3, Px Rx and its intensity kx is
t-differentiable on [0, 1).
3But this doesn’t imply thatP is Markov. More precisely, P is Markov if and only if kx
≡ k doesn’t depend onx.
(ii) The subgraph of all Px-active arcs doesn’t depend on t and is given by X (Px) = XR(x, supp Px 1), A→(P x) = AR →(x, supp P1x).
(iii) For any t∈ [0, 1) and any (z → z0)∈ AR
→(x, supp P1x), we have
χa[Px](t, z
→ z0) = χa[j](t, z → z0). (10)
(iv) For any t∈ [0, 1) and any closed AR
↔(x, supp P1x)-walk c, we have
χc[Px](t, c) = χ
c[j](t, c). (11)
About property (ii), notice that the dynamics of the reference random walk R plays an important role. Indeed, AR
→(x, supp P1x) is the subset of A→ =A→(R) of the j-active
arcs of all bridges of R from x to supp Px 1.
In some cases where the graph enjoys regularity, the property (iv) above can be weak-ened by only considering the identity (11) on a proper subset of the closedAR
↔(x, supp P1x
)-walks. This is made precise below at Proposition 2.7.
The reciprocal characteristics come with a natural probabilistic interpretation which is expressed in terms of short-time asymptotics of the distribution of bridges. We shall show that they can be recovered as quantities related to Taylor expansions as h > 0 tends to zero of conditional probabilities of the form P(X[t,t+h] ∈ · | Xt, Xt+h). This is the content
of Theorem 2.5 below.
Let us introduce the notation needed for its statement. For any integer k≥ 1 and any 0≤ t < 1, we denote by Tt
kthe k-th instant of jump after time t. It is defined for k= 1 by
Tt
1 := inf{s ∈ (t, 1] : Xs− 6= Xs} and for any k ≥ 2 by Tt
k := infs ∈ (Tk−1t ,1] : Xs− 6= Xs with the convention inf∅ = +∞.
Theorem 2.5 (Interpretation of the characteristics). We suppose that (X , →) satisfies Assumption 1.1 and j satisfies Assumption 1.5. Let P be any random walk in R(j). (a) For any t∈ [0, 1), any (z → z0)∈ A→(Px) and any measurable subset I ⊂ [0, 1] , we
have P(Tt 1 ∈ t + hI | Xt= z,Xt+h= z0, T2t > t+ h) = Z I dr+ hχa[j](t, z → z0) Z I (1/2− r) dr + oh→0+(h). (12)
(b) For any t∈ [0, 1) and any closed A→(Px)-walk c, we have
P (Xt→ XTt 1 → · · · → XT|c|t = Xt) = c, T t |c|< t+ h < T|c|+1t | Xt = Xt+h = χc[j](t, c)h|c|/|c|! + oh→0+(h|c|). (13)
Note that in statement (b), only closed walks with respect to A→(Px) and not its
symmetrized version A↔(Px) must be taken into account.
This theorem is close to some results of [CDPR] where similar short time expansions are related to reciprocal characteristics of compound Poisson processes on Rd.
In the same spirit that a Markov walk is specified by the Markov property and its jump intensity which can be obtained as the limit in small time of a conditional expectation, we obtain the following characterization of R(j).
Corollary 2.6 (Short-time expansions characterizeR(j)). Let us suppose that in addition to Assumptions 1.1 and 1.5, the directed graph is symmetric, i.e.
then a random walk P ∈ P(Ω) belongs to R(j) if and only if the following assertions hold for all x∈ supp P0.
(i) The conditioned random walk Px is Markov, Px Rx and its intensity kx is t-differentiable on [0, 1).
(ii) The subgraph of all Px-active arcs doesn’t depend on t and is given by
X (Px) =
XR(x, supp Px
1), A→(Px) = A→R(x, supp P1x).
(iii) For any t ∈ (0, 1), any (z → z0) ∈ A→(Px) and any measurable subset I ⊂ [0, 1] ,
the identity (12) is satisfied with P = Px.
(iv) For any t ∈ (0, 1) and any closed A↔(Px)-walk c the identity (13) is satisfied with
P = Px.
The assumptionA→=A↔is needed for Corollary 2.6to hold. Without any restriction
on the structure of the graph, this result is false in general. However, it is possible to relax this restriction in some specific situations. For instance, it is the case of the non-oriented triangle at page 24.
Proof. The necessary condition is a direct consequence of Theorems 2.4 and 2.5. For the sufficient condition, all we have to show is that the properties (a) and (b) of Theorem2.5
respectively imply the properties (iii) and (iv) of Theorem 2.4.
Based on identity (25), we see that the same calculations as in Theorem 2.5’s proof at page 17shows that replacing Rx by Px and j by kx lead to the same conclusions with kx
instead of j. It remains to compare the resulting expansions to conclude that (10) and
(11) are satisfied.
It is possible to improve the statements of Theorem 2.4 and Corollary 2.6 as follows. Proposition 2.7. The conclusions of Theorem 2.4 and Corollary 2.6 remain unchanged when weakening the properties (iv) by only considering closed walks c in any generating subset of the closed A↔(Px)-walks, see Definition B.3 and Lemma B.5.
Proof. It is a direct consequence of DefinitionB.3and the fact that the proofs of Theorem
2.4 and Corollary 2.6 rely on Lemma B.2.
We could replace item (i) in Theorem 2.4 and Corollary 2.6 by P is a reciprocal walk with a predictable intensity kX0.
3. Proofs of the main results
Let P be any element ofR(j). We know with Proposition2.1that P R. Therefore P admits a predictable intensity k(t, X[0,t); z) and the related Girsanov formula (see [Jac75])
is for each x∈ supp P0,
dPx dRx = 1{τ =∞}exp X 0<t≤1:Xt−6=Xt log k j(t,X[0,t); Xt) − Z [0,1] X z:Xt−→z (k− j)(t, X[0,t); z) dt (14)
where the stopping time τ is given by τ := infnt∈ [0, 1); k(t, X[0,t); Xt) = 0 or Z [0,t] X z:Xs−→z k(s, X[0,s); z) ds =∞ o ∈ [0, 1] ∪ {∞} with the convention inf∅ = ∞.
Note that Rx-almost surely j(t, X
[0,t); Xt) = j(t, Xt− → Xt) > 0, for all t ∈ [0, 1) and R
[0,1)¯j(t, Xt−) dt <∞.
As a general result [LRZ14], it is known that for any reciprocal walk P and any x ∈ supp P0, Px is Markov. Therefore, we already know that the intensity k has the Markov
form k(t, X[0,t); z) = k(t, Xt− → z). We will recover this result without invoking this general result.
Lemma 3.1 (HJB equation). For any x ∈ X and any nonnegative function h1 : X →
[0,∞) such that ERxh1(X1) = 1, the function ψx defined by (
ψx(t, z) := log ERx[h1(X1)|Xt= z]∈ R, t ∈ (0, 1), z ∈ XR(x, supp h1)
ψx(0, x) := 0, t= 0, z = x,
is a well-defined real function which satisfies the following regularity properties: (i) for all z ∈ XR(x, supp h
1), t 7→ ψtx(z) s continuously differentiable on (0, 1),
(ii) limt→0+ψtx(x) =: ψ0x(x) = 0,
(iii) limt→1−ψtx(z) =: ψ1x(z)∈ R exists for all z ∈ supp h1, and for each 0≤ T ≤ 1, the Itô formula
ψTx(XT) := log hT(XT) = X 0<t≤T ;Xt−6=Xt [ψtx(Xt)− ψxt(Xt−)] + Z [0,T ] ∂tψtx(Xt) dt∈ R, h1(X1)Rx-a.s. (15) is meaningful.
Furthermore, ψx is a classical solution of the Hamilton-Jacobi-Bellman equation
∂tψt(z) + P z0:(z→z0)∈AR →(x,supp h1) j(t, z → z 0) [eψt(z0)−ψt(z)− 1] = 0, t∈ (0, 1), z ∈ XR(x, supp h 1)
lims→1−ψs(y) = log h1(y), t= 1, y∈ supp h1.
(16)
It is important to see that the identity (15) is only valid almost surely with respect to 1supp h1(X1) R, but not with respect to R.
Proof. The function h(t, z) := ERx[h1(X1) | Xt = z], 0 < t≤ 1, z ∈ X (Rx) is space-time harmonic. This means that it satisfies the Kolmogorov equation
(∂t+ Lt)h(t, z) = 0, 0 < t < 1, z ∈ X (Rx).
Remark that it is needed that z ∈ supp Rxt for the conditional expectation to be well defined as a finite number. But the assumption (7) implies that supp Rx
t =X (Rx) for all
0 < t≤ 1. We obtain h(t, ·) = ←−exp(R1
t Lsds) (h1) where the ordered exponential is defined
by ←− exp Z 1 t Lsds := Id + X n≥1 Z t≤s1≤···≤sn≤1 Ls1· · · Lsnds1· · · dsn
and the continuity of Lt (recall that j is continuous) ensures that its formal left t-derivative is −Lt←−exp R1 t Lsds. Furthermore, ←−exp R1 t Lsds and Lt←−exp R1 t Lsds are
absolutely summable series under the assumption (9). More precisely, for any nonnegative h1 in L1(Rx1), we know by a martingale argument that ht= ←−exp
R1
t Lsdsh1 is in L 1(Rx
t).
Hence, h(·, z) is continuous at t = 1 for all z ∈ X (Rx) and h(·, x) is continuous at t = 0. In
addition, with the assumed uniform boundedness of Lt, we see that Lt←−exp
R1
t Lsdsh1 =
Ltht is also in L1(Rxt). Therefore, h is continuously left t-differentiable on [0, 1). But
the continuity of the left derivative implies both the existence and the continuity of the derivative. Consequently, the backward differential system
((∂
t+ Lt)h(t, z) = 0, 0 < t < 1, z ∈ X (Rx),
lim
t→1−h(t, y) =: h(1, y) = h1(y), t = 1, y ∈ X (R
x), (17)
can be considered in the classical sense and lim
t→0+h(t, x) =: h(0, x) = 1 since h(0, x) = 1 is fixed by hypothesis.
On the other hand, the assumption (7) implies that for all 0 < t < 1 and z ∈ XR(x, supp h
1) ⊂ X (Rx), h(t, z) is positive. It follows that we are allowed to define
ψx
t(z) := log h(t, z) as a real number for any 0 < t < 1 and any z ∈ XR(x, supp h1). Of
course, for t= 0 one must only consider z = x and limt→0+ψtx(x) = ψx0(x) = 0. We have shown that the regularity properties (i,ii,iii) are satisfied.
The Itô formula
ψTx(XT) = ψSx(XS) + X S<t≤T ;Xt−6=Xt [ψx t(Xt)− ψxt(Xt−)] + Z [S,T ] ∂tψtx(Xt) dt∈ (−∞, ∞), h1(X1) Rx-a.s. (18)
is meaningful for all 0 < S ≤ T < 1. Indeed, under Assumption 1.5(1) there are finitely many jumps Rx-a.s. and we have already seen that ψx
t(Xt) is finite for every 0 ≤ t < 1,
h1(X1) Rx-a.s. Therefore, ψTx(XT), ψSx(XS) and PS<t≤T ;X
t−6=Xt[ψ
x
t(Xt) − ψtx(Xt−)] are finite. It follows that the integralR(S,T ]∂tψxt(Xt) dt is also well-defined h1(X1) Rx-a.s.
Letting S tend to 0 and T to 1 in (18), with the limits (ii) and (iii) we obtain (15) where the integral R[0,T )∂tψxt(Xt) dt is well defined h1(X1) Rx-a.s.
Finally, considering h = eψx
in (17) gives the HJB equation (16) and completes the
proof of the lemma.
The following result is the key lemma of the proof of Theorem2.4.
Lemma 3.2. If the random walk P ∈ P(Ω) belongs to R(j) then P R and for all x∈ supp P0, the following assertions are satisfied.
(i) The conditioned random walk Px is Markov, Px Rx and its intensity kx is t-differentiable on [0, 1).
(ii) The subgraph of all Px-active arcs doesn’t depend on t and is given by X (Px) =
XR(x, supp Px
1), A→(P
x) = AR
→(x, supp P1x).
(iii) There exists a real function ψx : (0, 1) × XR(x, supp Px
1) → R such that for all
z ∈ XR(x, supp Px
are linked by the relations log k x j (t, z → z 0 ) = ψx t(z 0 )− ψx t(z), (19) ∂tψtx(z) + (¯k x − ¯j)(t, z) = 0, (20)
for all t∈ (0, 1), z ∈ XR(x, supp Px
1) and (z → z0)∈ A→R(x, supp P1x).
In (20), the average frequency of jumps ¯kx(t, z) := P
z0:z→z0kx(t, z → z0) is finite ev-erywhere on [0, 1)× XR(x, supp Px
1).
Furthermore, one can choose ψx in (iii) such that it solves the HJB equation (16) with
hx1 = dPx 1/dRx1.
Recall that Px Rx implies that Px admits a jump intensity kx.
Proof. Let us first take some P ∈ R(j) and show that it satisfies the announced properties. As supp P01 ⊂ supp R01, we can apply Proposition A.1 which states that for every x ∈
supp P0, we have Px Rx,
Px = hx(X 1) Rx
and ERxhx(X1) = 1 where hx := dP1x/dRx1. Comparing with (14), we see that the events {τ = ∞} and {X1 ∈ supp hx = supp P1x} match, up to an Rx-negligible set. This proves
that A→(t, Px) doesn’t depend on t and that it is equal to A→R(x, supp P1x). Let us define
ψtx(z) := log ERx(hx(X1)| Xt= z).
We know that ψx shares the regularity properties (i), (ii) & (iii) of Lemma 3.1. Applying (15) with T = 1 to Px = hx(X 1) Rx leads us to dPx dRx = 1{X1∈supp P1x} exp X 0<t≤1;Xt−6=Xt [ψtx(Xt)− ψxt(Xt−)] + Z (0,1] ∂tψtx(Xt) dt . Comparing with (14), we obtain
X 0<t≤1;Xt−6=Xt log k j(t, X[0,t); Xt)− Z [0,1] dt X z:Xt−→z (k− j)(t, X[0,t); z) = X 0<t≤1;Xt−6=Xt [ψtx(Xt)− ψxt(Xt−)] + Z (0,1] ∂tψtx(Xt) dt, 1{X1∈supp Px 1}R x -a.s. Identifying the jumps, we see that
1{Xt−6=Xt} log k j(t, X[0,t); Xt) = ψ x t(Xt)− ψtx(Xt−), 1{ X1∈supp P1x}R x-a.s. (21)
Hence, for any z ∈ XR(x, supp Px
1) and (z → z 0)
∈ AR
→(x, supp P1x), we can write
1{Xt−=z}k(t, X[0,t); z 0 ) = kx(t, z → z0), 1{X 1∈supp P1x}R x-a.s.
signifying that k(t, X[0,t);·) only depends on (t, Xt−). This shows that Px is Markov. More precisely, (21) gives us (19).
By Lemma3.1 we know that ψx satisfies the HJB equation (16). With (19), this leads us to (20).
Remark that (19) also implies that kx is t-continuously differentiable on [0, 1). Note that since ¯j is finite everywhere, it follows with (20) that ¯k is also finite everywhere. This
Proof of Theorem 2.4. In order to simplify notation, for a given x∈ supp P0, we write
Z→=A→R(x, supp P1x) and Z↔ =A↔R(x, supp P1x) during this proof.
• Proof of the necessary condition. Let us show that P ∈ R(j) shares the announced properties. The first items (i) and (ii) are already proved at Lemma 3.2.
Now, we rely on Lemma 3.2(3). Differentiating (19) and plugging (20) into the resulting identity gives us χa[kx] = χa[j] on Z→ which is (iii).
Let us prove (iv). For any t ∈ (0, 1) and any (z → z0) ∈ Z→, we denote `(t, z → z0) =
logkx
j (t, z → z
0). If the reversed arc (z0 → z) is also in Z
→, we see with (19) that
`(t, z0
→ z) = −`(t, z → z0). Otherwise, we extend `(t,
·) from Z→ toZ↔by means of this
identity. Therefore, `(t, z → z0) = ψx t(z 0 )− ψx t(z), ∀(z → z 0 )∈ Z↔
and we are allowed to apply Lemma B.2 to obtain χc[kx](t, c) = χc[j](t, c) for all closed
Z↔-walks, which is the desired result.
• Proof of the sufficient condition. Take P ∈ P(Ω) such that for every x ∈ supp P0, Px is
Markov and its intensity kx satisfies the properties (i-iv) of Theorem2.4. Fix x∈ supp P 0.
We start exploiting the property (iv). Because χc[kx](t, c) = χc[j](t, c) for any t ∈ (0, 1)
and any closed Z↔-walk c, by LemmaB.2 there exists a function ϕx which satisfies
ϕx(t, z0)− ϕx(t, z) = logkx
j (t, z → z
0
), ∀t ∈ (0, 1), (z → z0)∈ Z→. (22)
On the other hand, the property (iii) implies that ∂tϕx(t, z) + (¯kx− ¯j)(t, z) = ∂tϕx(t, z0) +
(¯kx− ¯j)(t, z0) for all t
∈ (0, 1) and (z → z0)
∈ Z→. Since ϕx is defined up to some
time-dependent additive function, we can decide that ∂tϕx(t, x) + (¯kx − ¯j)(t, x) = 0, for all
t∈ (0, 1). Therefore, we obtain with the property (ii) and our assumption (10) that ∂tϕx(t, z) + (¯kx− ¯j)(t, z) = 0, ∀t ∈ (0, 1), z ∈ XR(x, supp P1x). (23)
We know with the property (i) that Px Rx. Restricting the path measures to the
sub-σ-field σ(X[0,t]) for any 0 ≤ t < 1, and plugging (22) and (23) into Girsanov’s formula
(14), we obtain dPx [0,t] dRx [0,t] = 1{τt=∞}exp X 0<s≤t:Xs6=Xs− [ϕx(s, X s)− ϕx(s, Xs−)] + Z (0,t) ∂sϕx(s, Xs) ds where τt:= inf n r∈ [0, t); kx(r, Xr− → Xr) = 0 or Z (0,r] ∂sϕx(s, Xs) ds =−∞ o ∈ [0, t] ∪ {∞} . Thanks to property (ii), kx does not vanish on [0, 1)× Z
→. Hence, τt is finite if and only
if R[0,r]∂sϕx(s, Xs) ds =−∞ for some 0 < r ≤ t. But for any 0 ≤ t < 1, we have
X 0<s≤t:Xs6=Xs− [ϕx(s, X s)− ϕx(s, Xs−)] + Z (0,t] ∂sϕx(s, Xs) ds = ϕx(t, Xt)− ϕx(0, X0).
This implies that R(0,t]∂sϕx(s, Xs) ds is finite for every 0 ≤ t < 1 and it follows that τt is
infinite Rx-a.s. for all 0≤ t < 1. Consequently,
dP[0,t]x dRx
[0,t]
= exp(ϕx(t, X
since the prefactor 1{τt=∞} does not vanish. Let us denote Z := dPx dRx and Zt := ERx(Z | X[0,t]) = dPx [0,t] dRx [0,t]
for all 0≤ t < 1. We see with (24) that Zt is Xt-measurable. This implies
that Z is X[t,1]-measurable for all 0 ≤ t < 1 and consequently that Z is X1-measurable.
We conclude with Proposition A.1that P belongs to R(j).
Proof of Theorem 2.5. Let us fix x∈ supp P0 and t ∈ (0, 1). Note that for h > 0 such
that t+ h < 1 and (z → z0) ∈ AR
→(x, supp P1x), the conditional distribution P (· | Xt =
z, Xt+h = z0) is well defined. Because of (43) and the reciprocal property of P , we have
P(Tt
1 ∈ t + hI | Xt= z, Xt+h = z0, T2t> t+ h)
= R(T1t∈ t + hI | Xt= z, Xt+h = z0, T2t> t+ h).
(25) Therefore it suffices to do the proof with R instead of P .
• Proof of (a). Recall that for a Poisson process with intensity λ(t) the density of the law of the first instant of jump is t7→ λ(t) exp(−Rt
0 λ(s) ds), t≥ 0. Therefore, R(Tt 1 ∈ t + hI, Xt+h= z0, T2t> t+ h| Xt = z) = Z hI ¯j(t + r, z) exp−Z r 0 ¯j(t + s, z) dsj(t + r, z → z0) ¯j(t + r, z) exp − Z h r ¯j(t + s, z0 ) dsdr = h Z I exp− Z hr 0 ¯j(t + s, z) dsj(t + hr, z → z0 ) exp− Z h hr ¯j(t + s, z0 ) dsdr. Using the following expansions as h tends to zero:
exp− Z hr 0 ¯j(t + s, z) ds = 1− ¯j(t, z)hr + o(h), exp− Z h hr ¯j(t + s, z0 ) ds = 1− ¯j(t, z0 )(1− r)h + o(h), j(t + hr, z → z0 ) = j(t, z → z0 ) + ∂tj(t, z→ z0)hr + o(h), we obtain R(Tt 1 ∈t + hI, Xt+h = z0, T2t> t+ h| Xt= z) = hj(t, z → z0) Z I 1 + h ∂tj(t, z → z 0) j(t, z → z0) r− ¯j(t, z)r − ¯j(t, z 0 )(1− r) dr+ o(h2) = hj(t, z → z0) Z I 1 + hχa[j](t, z → z0 )r− ¯j(t, z0) dr+ o(h2).
In particular, with I = [0, 1] this implies that R(Xt+h= z0, T2t> t+ h| Xt = z)
= hj(t, z → z0) 1 + hχ
a[j](t, z → z0)/2− ¯j(t, z0) + o(h2).
Taking the ratio of these probabilities leads us to R(Tt 1 ∈ t + hI | Xt= z,Xt+h= z0, T2t> t+ h) = Z I 1 + hχa[j](t, z → z0)r − ¯j(t, z0) + o(h) 1 + hχa[j](t, z → z0)/2− ¯j(t, z0) + o(h)dr = Z I 1 + h{χa[j](t, z → z0)(r− 1/2)} dr+ o(h). With (25) this gives (12).
• Proof of (b). Since R(Xt = Xt+h = z) = R(Xt = z)(1 + o(1)) as h → 0+, we can
write the proof with R(· | Xt = z) instead of R(· | Xt = Xt+h = z). Therefore, if
c= (z = x0 → x1· · · → x|c|= z), R(Xt→ XTt 1 → · · · → XT|c|t ) = c, T t |c|< t+ h < T|c|+1t | Xt= z = Z {t<t1<···<t|c|<t+h} |c| Y i=1 exp − Z ti ti−1 ¯j(s, xi) ds j(ti, xi → xi+1) × exp " − Z t+h t|c| ¯j(s, z) ds # dt1· · · dt|c| = χc[j](t, c)(1 + o(1)) Z {t<t1<···<t|c|<t+h} exph− c X i=1 Z ti+1 ti ¯j(s, xi) ds − Z t+h t|c| ¯j(s, z) dsidt1· · · dt|c| = χc[j](t, c)h|c|/|c|! + o(h|c|)
where we used the convention that t0 := t. This completes the proof of the theorem.
4. More results
We give some additional results about the characterization of the reciprocal classR(j). Jump intensity. Proposition4.1below expresses characterizations of the reciprocal class in terms of the jump intensities by exploiting the fact that for any P ∈ R(j) and each x, Px is an h-transform of Rx.
Proposition 4.1 (Representation of the intensity of an element of R(j)). Let P ∈ P(Ω) be a random walk. The following assertions are equivalent.
(a) P ∈ R(j).
(b) There exists h:X2 → [0, ∞) such that
Px = h(x, X1) Rx
with P
y∈X Rx1(y)h(x, y) = 1, for all x∈ supp P0.
(c) There exists g:X2 → [0, ∞) such that
(i) supp P0 ={g(x, yo) > 0 for some yo},
(ii) for all x∈ supp P0,
P
y∈X R x
1(y)g(x, y) < ∞ and
(iii) P is a random walk with intensity k(t, X[0,t), Xt− → z) = 1{X t−∈XR(x,supp g(x,·))} gX0 t (z) gX0 t (Xt−) j(t, Xt− → z), R-a.s. (26) where for any 0≤ t ≤ 1 and z ∈ X (Rx),
gtx(z) := ER[g(x, X1)| Xt= z] =
X
y∈X
r(t, z; 1, y)g(x, y), with r(t, z; 1, y) := R(X1 = y | Xt = z).
In addition, for each x∈ supp P0, gx solves the heat equation
(∂t+ Lt)gx = 0, 0≤ t < 1,
gx
1 = g(x,·), t = 1.
The link between h and g is h(x, y) = g(x, y)/gx 0(x) where g0x(x) = P y∈X R x 1(y)g(x, y) is finite.
Proof. The equivalence of (a) and (b) is proved at Proposition A.1. Let us prove the equivalence of (b) and (c).
Statement (b) tells us that Px is an h-transform of Rx. It is a general result that the extended generator of this h-transform is given for any function u with a finite support by
Atxu(Xt−) = Ltu(Xt−) + Γt(gtx, u)(Xt−)/gtx(Xt−), Px-a.s. where Γt(gt, u)(z) :=
P
z0:z→z0j(t; z → z0)[gtx(z0) − gtx(z)][u(z0) − u(z)] is the carré du champ operator. This identity characterizes the h-transformation. Note that it is only valid Px-almost surely and not Rx-almost surely.
As for any t ∈ [0, 1) and z ∈ X (Rx), gx
t(z) > 0 ⇔ z ∈ XR(x, supp g(x,·)), we see that
Axtu(z) = 1{z∈XR(x,supp g(x,·)}Pz0:z→z0j(t; z → z0)gtx(z0)/gtx(z)[u(z0) − u(z)] which gives (26). This completes the proof Proposition 4.1.
A direct proof of the equivalence of (b) and (c), which does not rely on a general result about the extended generator of an h-transform, consists of identifying dPx/dRx = h(x, X1) by means of Girsanov’s formula (14) and to apply the representation result (27)
under its HJB form (16), via the transformation g = eψ, as in the proof of Lemma3.2.
As a special case of Proposition 4.1, we recover the known fact that for each (x, y)∈ supp R01, the jump intensity kxy of the bridge Rxy is
kxy(t, z → z0) = r(t, z 0; 1, y) r(t, z; 1, y)j(t, z → z 0 ), 0≤ t < 1, (z → z0)∈ AR →(x,{y}). (28)
Characteristic equation. We see with Theorem2.4that for any given Markov intensity j, the description of the reciprocal class R(j) is linked to the solution of some equation of the form
A→(kx) = A→R(x,Yx),
χ[kx] = χ[j], x∈ S (29)
where we use notation
χ[Px] =: χ[kx]
to emphasize the role of the intensity. In (29), the given subsets S ⊂ X and Yx ⊂ X (Rx), x∈ S
are non-empty and the unknown is the collection of Markov intensities(kx; x∈ S). More
precisely, (29) is a shorthand for the following list of properties that must hold for all x∈ S.
(i) The intensity kx is t-differentiable on [0, 1).
(ii) The subgraph of all kx-active arcs doesn’t depend on t and is
A→(kx) := {(z → z0) : kx(t, z → z0) > 0} = A→R(x,Y x
). (iii) For any t∈ [0, 1) and any (z → z0)∈ AR
→(x,Yx), we have
χa[kx](t, z
→ z0) = χa[j](t, z → z0).
(iv) For any t∈ [0, 1) and any closed A↔R(x,Yx)-walk c, we have
χc[kx
](t, c) = χc[j](t, c).
Because of Theorem 2.4, we say that (29) is a characteristic equation. It is natural to ask for the solutions (kx; x∈ S) of (29) where j, S and(Yx, x∈ S) are given.
Theorem 4.2 (Solving the characteristic equation (29)).
(a) Take any nonnegative function g : X2 → [0, ∞) such that supp g ⊂ supp R 01 and
P
y∈X R x
1(y)g(x, y) <∞ for all x ∈ X .
Let us denote gx
t(z) := ER[g(x, X1) | Xt = z] > 0, for any t ∈ [0, 1) and z ∈
XR(x, supp g(x,·)). Then, kx(t, z → z0 ) := g x t(z0) gx t(z) j(t, z → z0 ), t ∈ [0, 1), (z → z0)∈ AR →(x, supp g(x,·))) (30)
solves (29) with S :={x ∈ X ; g(x, y) > 0 for some y ∈ X } and Yx = supp g(x,·).
(b) Conversely, any solution (kx; x∈ S) of (29) which verifies the additional requirement
∀x ∈ S, ∀0 ≤ t < 1, sup y∈Yx Z t 0 ¯ kx(s, y) ds <∞, (31) has the above form (30) for some function g and for any x∈ S,
Px := g(x, X1) gx
0(x)
Rx ∈ P(Ω),
defines a Markov probability measure on Ω with intensity kx given by (30).
The main point about this result is that unlike Theorem 2.4 and Proposition4.1, it is not assumed that kx given at (30) is the intensity of a random walk. It might happen a priori that such a random process “explodes” in finite time with a positive probability. The assumption (31) rules this bad behavior out.
Proof. The proof mainly consists of addressing some comments about the proof of Theo-rem 2.4.
The first statement (a) follows from a straightforward computation. The regularity issues are direct consequences of Lemma 3.1.
Statement (b) is proved by considering the proof of the sufficient condition of Theorem
2.4 at page 16. Let kx be a solution of (29). Mimicking the Girsanov formula (14), let us
define Qx := Zx 1 Rx with Ztx := 1{τ >t}exp X 0<s<t:Xs−6=Xs logk x j (s, Xs− → Xs)− Z t 0 (¯kx−¯j)(s, Xs; y) dt , 0≤ t ≤ 1 and the stopping time τ defined by
τ := infns≤ 1; kx(s, Xs− → Xs) = 0 or Z s 0 ¯ kx(r, Xr) dr =∞ o ∈ [0, 1] ∪ {∞} . Let us show that under the assumption (31), Qx is a probability measure. The process
Zx is a nonnegative local Rx-martingale. As such it is also an Rx-supermartingale. In
particular, Qx(Ω) = E
RxZ1x ≤ 1, but it might happen that Qx(Ω) < 1, in which case Qx is not a probability measure. However, the property (ii) implies that kx(t, X
t− → Xt) > 0,∀0 ≤ t < 1, Rx-a.s. and the assumption (31) implies that for all 0 ≤ t < 1,
τ > t, Rx-a.s. and (Zx
s)0≤s≤t is an Rx-martingale. In particular, ERxZtx = 1 and Qx
[0,t] is
probability measure for all 0 ≤ t < 1, which in turns implies that Qx is a probability
measure, since without loss of generality, one can modify the path space Ω by throwing away the R-negligible event of all paths that jump at t= 1.
Following almost verbatim the proof of the sufficient condition of Theorem2.4, we show that there exists a function ϕx such that, as in (24),
Ztx= exp(ϕx(t, X
t)− ϕx(0, x)), 0≤ t < 1.
It follows from the above modification ofΩ at time 1 that t7→ Zx
t admits a version which
is left-continuous at 1 and
Qx = exp(ϕx
1(X1)− ϕx(0, x)) Rx,
where the limit ϕx1(X1) := limt→1−ϕx(t, Xt) ∈ [−∞, ∞) exists Rx-almost surely. Now, we are back to Proposition 4.1 with the functions h(x, y) = exp(ϕx
1(y)− ϕx(0, x)) and
g(x, y) = exp(ϕx
1(y)) where as a convention exp(−∞) = 0.
5. Examples
In this series of examples, we illustrate Theorem 2.4 improved by Proposition 2.7. We compute the reciprocal characteristic χ[j] and sometimes we consider the characteristic equation (29).
Directed tree. Let R be the simple random walk on a directed tree (X , A→). By
“di-rected tree”, it is meant that (z → z0) ∈ A
→ implies that (z0 → z) 6∈ A→, while “simple”
means that j(t, z → z0
) = 1 for all (z → z0
) ∈ A→, 0 ≤ t < 1. The closed walks of the
corresponding undirected treeA↔are clearly generated by the setE of all edges, i.e. closed
walks of length two, which matches with A↔. Therefore, the closed walk characteristic is
trivial: χc[k](t, z ↔ z0) = 1 for all 0 ≤ t < 1, (z ↔ z0) ∈ A↔ and any intensity k such
that A→(k) =A→. In this situation, only the arc component
χa[j](t, z → z0
) = deg(z0)− deg(z), 0 ≤ t < 1, (z → z0)∈ A→,
is relevant, wheredeg(z) := #{z0
∈ X : (z → z0)
∈ A→} is the outer degree, i.e. the
num-ber of offsprings of z. The characteristic equation is
∂tlog kx(t, z → z0) + ¯kx(t, z0)− ¯kx(t, z) = out(z0)− out(z),
0≤ t < 1, (z → z0)∈ AR →(x,Y),
for some Y ⊂ X (Rx).
Intensity of a bridge. In particular, the intensity jxy of any bridge Rxy satisfiesA→[jxy] =
wxy : the only walk leading from x to y, and
∂tlog jxy(t, z → z0)+1{z06=y}jxy(t, z0 → z00)− jxy(t, z→ z0)
= 1{z06=y}out(z0)− out(z), 0≤ t < 1, (z → z0)∈ wxy, where z → z0 → z00 are consecutive vertices.
Birth and death process. The vertex set is X = N with the usual graph structure which turns it into an undirected tree. The reference walk R is governed by the time-homogeneous Markov intensity j(z → (z +1)) = λ > 0, z ≥ 0 and j(z → (z −1)) = µ > 0, z ≥ 1. Clearly, the of edges E = {(z ↔ z + 1), z ∈ N} generates C and the characteristics of the reference intensity are:
χa[j](z → z + 1) = χa[j](z + 1→ z) = 0, z ≥ 1, χa[j](0→ 1) = −χa(1→ 0) = µ, χc[j](z ↔ z + 1) = λµ, z ≥ 0.
Time-homogeneous Markov walks in R[j]. Let us search for such a random walk P ∈ P(Ω). We denote ˜λ(z) the intensity of (z → z + 1) and ˜µ(z + 1) the intensity (z + 1 → z) of the Markov walk P . By Theorem 2.4, P ∈ R[j] if and only if
˜ λ(z + 1) + ˜µ(z + 1)− ˜λ(z) − ˜µ(z) = 0, z ≥ 1 ˜ λ(1) + ˜µ(1)− ˜λ(0) = µ, ˜ λ(z)˜µ(z + 1) = λµ, z ≥ 0.
The solutions to the the above set of equations can be parametrized by choosing ˜λ(0) arbitrarily and finding ˜λ(z + 1), ˜µ(z + 1) recursively as follows
˜
µ(z + 1) = ˜λ(z)−1λµ, z ≥ 0,
˜
λ(z + 1) = µ + ˜λ(0)− ˜µ(z + 1), z ≥ 1.
With some simple computations one can see that for any large enough ˜λ(0), the above system admits a unique positive and bounded solution. Hence, the corresponding Markov walk has its sample paths in Ω and it is inR[j].
Hypercube. Let X = {0, 1}d be the d-dimensional hypercube with its usual directed graph structure and let{gi}di=1be the canonical basis. For x∈ X , we set xi := x + gi and
xik = x + g
i+ gk where we consider the addition modulo 2. Let
S :=(x → xi
→ xik
→ xk
→ x), x ∈ X , 1 ≤ i, k ≤ d be the set of all directed squares. The subset
S ∪ E (32)
generates the closed walks. One can produce a smaller generator for the closed walks by imposing i < k in the definition of S. However, no more than this can be done since E does not generate C there are no closed walks of length 3.
The bridge of a simple random walk on the discrete hypercube. Let j be the simple random walk on the hypercube. The intensity jxy(t, z → z0) of the xy-bridge can be computed
explicitly with (28) since the transition density of the random walk is known explicitly. We have jxy(t, z → zi) = ( cosh(1− t)/ sinh(1 − t), if zi 6= yi, sinh(1− t)/ cosh(1 − t), if zi = yi, (33) where zi and yi ∈ {0, 1} are the i-th coordinates of z and y ∈ X .
We provide an alternate proof based on the characteristic equation (29). First, it is immediate to see that under any bridge, all arcs of the hypercube are active at any time. From χc[jxy] = χc[j], we deduce that the arc function log(j/jxy)(t,·) is the gradient of
some potential ψt, see Lemma B.2. The equality of the arc characteristics implies that
for all t∈ (0, 1) and z ∈ X ∂tψ(t, z) + d X i=1 [exp(ψt(zi)− ψt(z))− 1] = ∂tψ(t, x) + d X i=1 [exp(ψt(xi)− ψt(x))− 1].
Since ψ is defined up the addition of a function of time, we can assume without loss of generality that for all 0 < t < 1, ∂tψt(x) +
Pd
i=1[exp(ψt(xi)− ψt(x))− 1] = 0. Hence ψ
solves the HJB equation ∂tψ(t, z) +
d
X
i=1
Going along the lines of the proof of Theorem 2.4, in particular equation (24), allows to deduce that the boundary data for ψ are
lim
t→1ψt(z) =
(
−∞, if z 6= y,
0, if z = y. (35)
One can check with a direct computation that the solution (34) & (35) is
ψ(t, z) =
d
X
i=1
log[1 + (−1)(zi−yi)e2(1−t)] (36) where the subtraction is considered modulo two. By the definition of ψ, we have
jxy(t, z → zi
) = j(t, z → zi
) exp(ψt(zi)− ψt(z)) = exp(ψt(zi)− ψt(z))
and (33) follows with a simple computation.
Two triangles. We look at two simple directed trees based on triangles.
A B C b A B C
Figure 1. Left: an oriented triangle. Right: a non-oriented triangle
Oriented triangle. Let X = {A, B, C} and A = {(A → B), (B → C), (C → A)}. The reference intensity is j∆ on each arc and we want to find the intensity of the AB-bridge:
jAB(t,·), using the characteristic equation. Imposing the equality of the closed walk characteristics implies that log(jAB/j)(t,·) is the gradient of some potential ψ
t:X → R.
The equality of the arc characteristics implies that ψ solves the HJB equation
∂tψt(A) + j∆[exp(ψt(B)− ψt(A))− 1] = 0,
∂tψt(B) + j∆[exp(ψt(C)− ψt(B))− 1] = 0,
∂tψt(C) + j∆[exp(ψt(A)− ψt(C))− 1] = 0,
limt→1ψt(A) = limt→1ψt(C) =−∞,
limt→1ψt(B) = 0,
(37)
where the boundary conditions for ψ follow from (24). Since the HJB equation is the logarithm of the Kolmogorov backward equation, we obtain the following solutions
ψt(A) = log{13 + 32exp(−32j∆(1− t)) sin[ √
3
2 j∆(1− t) − π 6]},
ψt(B) = log{13 + 32exp(−32j∆(1− t)) cos[ √
3
2 j∆(1− t)]},
ψt(C) = log{13 − 32exp(−32j∆(1− t)) sin[ √ 3 2 j∆(1− t) + π 6]}. (38)
We deduce the following identities jTAB(t, A→ B) = j∆ exp 3 2j∆(1− t) + 2 cos √ 3 2 j∆(1− t) exp 3 2j∆(1− t) + 2 sin √ 3 2 j∆(1− t) − π 6 , jTAB(t, B → C) = j∆ exp 3 2j∆(1− t) − 2 sin √ 3 2 j∆(1− t) + π 6 exp 3 2j∆(1− t) + 2 cos √ 3 2 j∆(1− t) , jTAB(t, C→ A) = j∆ exp 3 2j∆(1− t) + 2 sin √ 3 2 j∆(1− t) − π 6 exp 3 2j∆(1− t) − 2 sin √ 3 2 j∆(1− t) + π 6 . (39)
Non-oriented triangle. The triangle X = {A, B, C} is now equipped with the directed graph structure A = {(A → B), (B → C), (A → C)} where we reverted the direction of the arc on the edge AC, as shown at Figure1. The characteristic associated to the closed walk (A→ B → C → A) is
j(A→ B)j(B → C)/j(C → A).
Since it is not a closed walk of the graph (X , A ), the interpretation given at Theorem 2.5
is not available for this characteristic. However, we can reason in a similar way to obtain a probabilistic interpretation of this characteristic as well.
Let RA be the reference walk conditioned to start from A. If the walk reaches C at time h it is easy to see that as h→ 0, this has happened essentially only by using directly the arc(A→ C). Therefore, we obtain
RA(Xh = C) = j(A→ C)h + o(h).
Similarly, the probability of going from A to C using the path (A→ B → C) is RA(X0 → XT1 → XT2) = (A→ B → C), T2 ≤ h, T3 > h = j(A→ B)j(B → C)h2/2 + o(h2). Consequently, RA(X0 → XT1 → XT2) = (A→ B → C), T2 ≤ h, T3 > h| Xh = C) = j(A→ B)j(B → C) 2j(A→ C) h+ o(h).
We see that the characteristic is twice the driving factor of the expansion of this probability as h tends to zero.
Note that while the characteristic the oriented triangle is associated to a probability of order h3, in the present case it is associated to a probability of order h.
Planar graphs. Let (X , ↔) be an undirected planar graph. We fix a planar represen-tation and consider the set F of all counter-clockwise closed walks along the faces. We denote by E the set of all edges seen as closed 2-walks. Then,
F ∪ E (40)
