Sudoku Latin Square Sampling for Markov Chain Simulation

Sudoku Latin Square Sampling for Markov Chain Simulation

Rami El Haddad, Joseph El Maalouf, Christian L´ecot, and Pierre L’Ecuyer

Abstract We are interested in Monte Carlo simulations of discrete-time Markov chains on discrete and totally ordered state spaces. To improve simulation efficiency, we use a technique previously introduced in the context of quasi-Monte Carlo simula- tion of an array of N Markov chains. This method simulates the N copies of the chain simultaneously, reorders the chains at each step by increasing order of their states, and samples the next state by using N two-dimensional points in the unit square. The first coordinate of each point is used to match a chain, and the second coordinate is used to sample the next state by inversion from its cumulative distribution function conditional on the current state. We study the case where the N points are obtained at each step from Sudoku Latin square sampling, which means that (1) if the unit square is uniformly divided into N identical subsquares, exactly one point lies in each subsquare, (2) for each axis, the N projections of the points are distributed with exactly one projection in each of the N subintervals of length 1/N that partition the unit interval, and (3) in both cases, each individual point has the uniform distribution in the subsquare and interval to which it belongs. We prove that the variance of the Sudoku Latin square sampling estimator is of order O (N

). The same conver- gence rate is obtained when property (2) is removed, which gives simple stratified sampling. However, in our numerical experiments, we observe empirically a much smaller variance and better efficiency with the Sudoku Latin square sampling than with simple stratified sampling alone.

Rami El Haddad·Joseph El Maalouf

Laboratoire de Math´ematiques et Applications, U.R. Math´ematiques et mod´elisation, Facult´e des sciences, Universit´e Saint-Joseph, B.P. 7-5208, Mar Mikha¨el Beyrouth 1104 2020, Liban, e-mail:

[email protected], e-mail: [email protected] Christian L´ecot

Universit´e Grenoble Alpes, Universit´e Savoie Mont Blanc, CNRS, LAMA, 73000 Chamb´ery, France, e-mail: [email protected]

Pierre L’Ecuyer

DIRO, Universit´e de Montr´eal, C.P. 6128, Succ. Centre-Ville, Montr´eal, H3C 3J7, Canada, e-mail:

[email protected]

1 Introduction

We consider a discrete-time Markov chain {X

, n ≥ 0} over a countable state space X , where X

∈ X is the state at step n, and we are interested in estimating by simulation the expected cost at step n, E [c(X

)], for one or several cost functions c : X → [0, ∞) (we assume non-negative cost functions for simplicity). If the state space is finite with small cardinality and sometimes when the chain has a very special structure, it is possible to compute the exact distribution of X

and the exact expected cost at step n, for any n. Otherwise, one can use standard Monte Carlo (MC): simulate the chain until step n, repeat N times independently, and average the N realizations of c(X

). The main drawback of this general approach is its slow convergence: The variance of the Monte Carlo estimator of E [c(X

)] typically converges a O (N

) for any n.

A (deterministic) quasi-Monte Carlo (QMC) method for Markov chains has been proposed in [9] for the case where the chain has a totally ordered state space. The method simulates an array of N copies of the chain in parallel. At each step n, it reorders the chains by increasing order of their states, and it uses two-dimensional quasi-random points to move them ahead by one step. Convergence (in the deter- ministic sense) of the average cost to the expectation when N → ∞ was established, and the QMC approach outperformed plain MC in numerical experiments. However, QMC error bounds are typically too loose and inconvenient for practical error assess- ment. A randomized quasi-Monte Carlo (RQMC) approach named Array-RQMC, which resembles the previous QMC scheme, was proposed and analyzed in [11, 12], in the setting of a Markov chain model with general state space. The method was shown to provide an unbiased estimator of E [c(X

)] for any n, and variance bounds for this estimator were proved under certain conditions. In particular, it was proved that for a Markov chain with a one-dimensional state space, if stratified sampling as in [2, 8] is used at each step to advance the array of chains by one step, and under some technical conditions, the variance converges as O (N

), which beats Monte Carlo. In numerical experiments with Markov chains having one-dimensional and higher-dimensional states, the empirical variance was typically much smaller than the Monte Carlo variance, and was observed to decrease often at better rates than O (N

), sometimes even faster than O (N

): see [12, 13, 14], for example.

However, no proof of these faster rates is available so far, and the O (N

) rate has been proved only for ordinary stratification of the unit square in identical subsquares, for a one-dimensional state. A related convergence-rate result worth mentioning was obtained in [7], in the context of particle filters. The authors proved that if the RQMC point set used at each step is a (t, m, s)-net with a nested uniform scramble [18] and if the states are sorted using a Hilbert-curve when their dimension is larger than 1, then the variance of the Array-RQMC estimator converges as o(N

), which is faster than Monte Carlo.

The aim of this paper is to increase our theoretical understanding of the method by

expanding the class of sampling methods for which an O (N

) convergence rate

is proved. We revisit the simple stratified sampling (SSS) setting and we consider

a Sudoku Latin square sampling (SLSS) setting, which combines two-dimensional

stratified sampling with Latin hypercube sampling [15, 20]. Our theoretical results are consolidated by three numerical experiments in which we observe a significantly lower variance with SLSS than with simple stratification.

SLSS turns out to be a special case of the U-sampling method of [21] for sampling N points in the unit hypercube. The U-sampling first generates a random orthogonal array-based Latin hypercube design, which is a selection of N small cubic boxes of side size 1/N that form an orthogonal array of strength t [16, 17] and a Latin hypercube at the same time. Then it samples one point uniformly inside each selected small box, independently across the boxes. For the special case where t = 2, this type of design (the selection of the boxes) gives a Sudoku Latin square [19] for each two-dimensional projection of the points. Thus, each two-dimensional projection satisfies the properties (1) to (3) mentioned in the abstract. An example of a Sudoku Latin square is given in Fig. 1. Since our SLSS is in two dimensions, there is a single two-dimensional projection and it must form a Sudoku Latin square. Sudoku Latin squares are also studied in [22], although these authors are only considering discrete designs and space filling constructions, and not in sampling random points uniformly in the unit hypercube.

0 1

1

0 1

1

0 1

1

Fig. 1 Example of a Sudoku Latin square with 16 points.

A different sampling method that generalizes the SLSS to more than two dimen-

sions was studied in [5]. In d dimensions, that method generates N = p

points

in a way that (1) there is always one point per subcube when we partition the d-

dimensional unit cube into N identical subcubes, (2) there is one value in each

subinterval when we project all the points over a single coordinate to obtain N values

in the unit interval, and we partition the unit interval into N subintervals of length

1/N, and (3) each point taken individually has the uniform distribution in the sub-

cube to which it belongs. Variance bounds have been obtained when the integral of a

function over the unit cube is estimated by the average of the function values at the N

points, under certain assumptions on the integrand. Different types of variance results,

in terms of the ANOVA decomposition of the integrand, were proved in [21] for the

same type of integration problem with U-sampling. SLSS is the two-dimensional

special case of each of these two methods.

Our paper is the first to study the use of SLSS in the context of simulating an array of Markov chains. SLSS gives stronger constructions than simple stratified points over the unit square. Our aim is to investigate if, and how much, this strengthening has an impact on the variance of expected cost estimators.

The remainder is organized as follows. In Sect. 2, we define our setting for Monte Carlo simulation of discrete-state Markov chains and explain how we proceed with plain (standard) Monte Carlo, with SSS, and with SLSS. With SSS, the way we map the points to chain states at each step follows [6, 9] and differs from what was done in [12]. In Sect. 3, we analyze the variance of these schemes. We prove that the variance of the simulation estimator of an expected state-dependent cost at any given step n is O (N

) for both SSS and SLSS. This beats the known rate of O (N

) for standard Monte Carlo. Results of computational experiments and comparison between standard Monte Carlo, SSS, and SLSS are given in Sect. 4. The empirical convergence rates of the variance for SSS and SLSS are close to those established in Sect. 3, but the variance with SLSS is significantly smaller than with SSS. In Sect. 5, we give the technical proofs of some results and we conclude in Sect. 6.

2 Monte Carlo Simulations of Markov Chains

Let {X

, n ≥ 0} be a stationary discrete-time Markov chain over a countable and ordered state space X . Without loss of generality one can assume that X = N or Z . Let P(i, j) = P (X

= j|X

= i) denote the transition probabilities and P P P = (P(i, j) : (i, j) ∈ X

) the transition probability matrix. We denote by µ

(i) = P (X

= i) the state probabilities at step n and µ

= (µ

(i) : i ∈ X ) the probability vector for step n. We assume that the initial probability vector µ

is given (often, it is degenerate over a single state).

For i, j ∈ X we set

q

(i) := ∑

h≤j

P(i, h). (1)

We define the conditional cumulative distribution function F

( j) := P (X

≤ j|X

= i) = q

(i). If I denotes the unit interval (0, 1] we have a disjoint union I =

I

, where I

:= (q

(i), q

(i)]. So that for any i ∈ X and u ∈ I, there exists a unique j ∈ X such that u ∈ I

: we denote it by F

(u). If X

= i, then F

(u) is the next state.

Let δ

be the Dirac measure at i, defined by δ

( j) =

( 1 if j = i, 0 otherwise.

For any integer n, the distribution µ

is approximated by

b µ

:= 1 N

N k=1

∑

δ

k

,

where N is a fixed integer and i

, . . . , i

are calculated iteratively.

First, a set (i

: 1 ≤ k ≤ N) of N states is sampled from µ

: several techniques are proposed in [4]. In many applications, the initial state of the chain is fixed, and then b µ

= µ

.

We describe the transition from step n to n + 1 for three Monte Carlo methods.

We introduce e µ

:= µ b

P P P as an intermediate distribution (which is not used in effective calculations). This µ e

is an approximation of µ

, but it is generally not an equally-weighted sum of Dirac measures, like µ b

, so that an additional step is needed. We formulate this step as a quadrature: the MC methods correspond to quadrature algorithms, possibly combined with variance reduction techniques. To that end, let us consider an arbitrary sequence s = (s(i) : i ∈ X ) (a column vector);

we assume that s is non-negative, just to avoid worrying about convergence of series.

Then

e µ

s = µ b

P P Ps = 1 N

N

∑

k=1

∑

j∈X

P(i

, j)s( j).

Let 1

be the indicator function of the interval I

:= ((k −1)/N, k/N] and 1

denote the indicator function of the interval I

. If we associate to s the function C

defined by

C

(u) :=

N k=1

∑ ∑

j∈X

1

(u

)1

k,j

(u

)s( j), u = (u

, u

) ∈ I

, (2) then we have

µ e

s =

I²

C

(u)du.

We obtain µ b

by approximating the integral with Monte Carlo estimation. In the following, if m is an integer, we denote [1, m] := {1, 2, . . . , m}. The notation U ∼ U ( E ) means that U is a random variable uniformly distributed over the set E .

2.1 Standard Monte Carlo

The transition from step n to step n + 1 acts as follows: if the state of the chain is i, i.e. X

= i, then a random number U with U ∼ U (I) is generated and the new state of the chain is F

(U), i.e. X

= F

n

(U). The operation is repeated N times independently, in order to advance N copies of the chain. With our notations, this may be written as follows. Let {U

: 1 ≤ k ≤ N} be independent random variables with U

∼ U (I), then

i

= F

(U

), 1 ≤ k ≤ N.

That is, if, for any non-negative sequence s,

X b

:= 1 N

N

∑

k=1

C

k − 1

N ,U

, (3)

then

µ b

s = X b

. (4)

2.2 Simple Stratified Sampling

We suppose that N = p

, for some integer p > 0. The transition from n to n +1 has two steps: renumbering of the states and numerical integration.

(S1) The states are relabeled so that i

≤ · · · ≤ i

. The technique was used in the QMC context and ensures theoretical and numerical convergence of the scheme (see [9]) .

(S2) Consider a partition of I

into N squares: I

= H

×H

2

, where, for ` ∈ [1, p]

: H

:= ((`

−1)/p, `

/ p] and H

:= ((`

− 1)/ p, `

/p]. Let {V

: ` ∈ [1, p]

} be independent random variables, where V

= (V

,V

) ∼ U (I

).

For an arbitrary non-negative sequence s, let Y b

:= 1

N ∑

`∈[1,p]²

C

(V

), (5)

then

b µ

s = Y b

. (6) If u ∈ I, let

κ(u) := dNue, (7)

where dxe is the least integer greater than or equal to x. Hence equation (6) means that the next states are calculated as follows:

i

1−1)p+`₂

= F

(V

), ` ∈ [1, p]

(the numbering of the states i

is arbitrary). The first projection V

of V

is

used for selecting the state at step n and the second projection V

is used for

performing the transition to step n +1. Note that with this scheme, the mapping

between the N points and the N states is not necessarily one-to-one: it is possible

to pick the same state more than once and leave out some of the states. This differs

from the SSS scheme used in [12, 14].

2.3 Sudoku Latin Square Sampling

As before, we assume N = p

, and the transition from n to n + 1 has two steps:

renumbering of the states and numerical integration.

(S1) The states are relabeled so that i

≤ · · · ≤ i

.

(S2) We consider the same partition of I

as before: I

for ` ∈ [1, p]

. Let {W

: ` ∈ [1, p]

} be random variables, where W

= (W

, W

), with

W

= `

− 1

p + σ

(`

) − 1 + ξ

p

W

= `

−1

p + σ

(`

) −1 + ξ

p

.

Here σ

and σ

are random permutations of [1, p] and ξ

∼ U (I) and ξ

∼ U (I).

All these random variables being independent. The set of values of the random variable W

is included in I

and has the properties:

(P1) for any ` ∈ [1, p]

, there is a unique point of this set in each square I

, (P2) for any k ∈ [1,N], there is a unique point of this set in each rectangle I ×I

or I

×I.

In addition W

∼ U (I

). For an arbitrary non-negative sequence s, let Z b

:= 1

N ∑

`∈[1,p]²

C

(W

). (8)

Then

b µ

s = Z b

. (9) Due to property (P2), the mapping (see (7))

` := (`

, `

) ∈ [1, p]

→ κ (W

) ∈ [1, N]

is one-to-one: each state of step n is considered exactly once for a transition (this is not the case with SSS). Equation (9) means that the next states are calculated as follows:

i

1−1)p+`₂

= F

(W

), ` ∈ [1, p]

(as before, the numbering of the states i

is arbitrary). The first projection W

of W

is used for selecting the state at step n and the second projection W

is used for performing the transition to step n + 1.

3 Convergence Analysis

In this section we prove, for each method, that the estimator of the expected cost

at each step is unbiased and we establish that the variance of the estimator used is

O (N

) for standard MC and O (N

) for SSS and SLSS, where N is the number of simulation paths. In the following, λ is the Lebesgue measure and λ

the two- dimensional Lebesgue measure; we put | E | for the number of elements of a set E . We use the sequence s

, for h ∈ X :

s

(i) :=

( 1 if i ≤ h, 0 otherwise.

The total variation of a sequence s = (s(i) : i ∈ X ) is defined by TV(s) := ∑

i∈X

|s(i + 1)− s(i)|. (10)

We use below the total variation of q

, for h ∈ X . We recall that we have from (1):

q

(i) = P (X

≤ h|X

= i), and from (10):

TV (q

) = ∑

i∈X

| P (X

≤ h|X

= i + 1)− P (X

≤ h|X

= i)|.

In the following, we assume that M := sup

h∈X

TV (q

) < +∞.

There are situations for which q

is monotone and situations for which M < 1 (or both), but this is not always true. See [13, 14] for examples and further discussion.

Our M corresponds to Λ

in [12].

3.1 Standard Monte Carlo

Lemma 1. Let s be a non-negative sequence. The standard Monte Carlo estimator of e µ

s:

X b

:= 1 N

N

∑

k=1

C

k − 1

N ,U

has the following properties.

1. X b

is unbiased.

2. If s = s

, for h ∈ X , then

Var(b X

h

) ≤ 1

4N . Proof.

1. We have

E

C

k − 1

N ,U

= ∑

j∈X

P(i

, j)s( j),

so that E [ X b

] = µ e

s.

2. The variable

C

h

k −1 N ,U

= ∑

j∈X,j≤h

1

k,j

(U

)

is a Bernoulli random variable, with variance ≤ 1/4. Hence the result.

u t We then obtain an error bound by using the same techniques as in [12]. We assume that, for any non-negative sequence s, the standard Monte Carlo estimator µ b

s of µ

s is unbiased and that, for any h ∈ X ,

Var( b µ

s

) ≤ x

N ,

for some x

≥ 0 (as noticed before, in many applications, b µ

= µ

).

Proposition 1. For the standard Monte Carlo method, it holds:

1. for any non-negative sequence s

E [ b µ

s] = µ

s, 2. for any h ∈ X ,

Var( µ b

s

) ≤ x

N , where x

= M

x

+ 1/4 (n ≥ 0).

Proof.

1. We have

µ

s − µ b

s = µ

s− µ e

s + µ e

s− µ b

s = µ

P P Ps− b µ

P P Ps + µ e

s− X b

, so, by using Lemma 1, the result follows by induction.

2. The variables µ

s

− µ e

s

and µ e

s

− b µ

s

are uncorrelated and e µ

s

− µ b

s

= µ e

s

− X b

h

is a centered variable, consequently Var( b µ

s

) = E

(µ

s

− µ e

s

)

+ E

( µ e

s

− b µ

s

)

. (11)

For any i ∈ X , we have P P Ps

(i) = F

(h), hence µ

s

− µ e

s

=µ

P P Ps

− µ b

P P Ps

= ∑

i∈X

µ

(i)F

(h)− ∑

i∈X

b µ

(i)F

(h)

= − ∑

i∈X

µ

s

(F

(h) −F

(h)) + ∑

i∈X

µ b

s

(F

(h) − F

(h))

= ∑

i∈X

( b µ

s

− µ

s

)(F

(h) − F

(h)).

On the one hand, we write

E

(µ

s

− µ e

s

)

= E



 ∑

i∈X

( µ b

s

− µ

s

)(q

(i +1) −q

(i))

!





= ∑

(i,j)∈X²

E [( µ b

s

− µ

s

)(q

(i +1) −q

(i))( µ b

s

− µ

s

)(q

( j + 1) − q

( j))]

≤ ∑

(i,j)∈X²

q

Var( µ b

s

)Var( µ b

s

)|q

(i +1) −q

(i)| × |q

( j + 1) − q

( j)|.

On the other hand, Lemma 1 gives E

( µ e

s

− b µ

s

)

= E h

( µ e

s

− X b

h

)

i

≤ 1 4N . So, by using (11), the result follows by induction.

u t The bounds for Var(b Y

h

) (SSS) and Var(b Z

h

) (SLSS) are not so easily obtained, and the proofs of Lemma 2 and 3 are given in Sect. 5.

3.2 Simple Stratified Sampling

Lemma 2. Let s be a non-negative sequence. The SSS estimator of µ e

s:

Y b

:= 1

N ∑

`∈[1,p]²

C

(V

)

has the following properties.

1. Y b

is unbiased.

2. If s = s

, for h ∈ X , then Var(b Y

h

) ≤ 1

4N

(TV (q

) + 2).

A similar result (with the same N

order) was established in [12], but the SSS method studied there differs from the one used here, so we provide a different proof (see Sect. 5). Intermediate results from this proof (eqs. (12) and (13)) will be re-used afterwards for the analysis of SLSS.

The proof of the next result is similar to the proof of Proposition 1. We assume that, for any non-negative sequence s, the SSS estimator b µ

s of µ

s is unbiased and that, for any h ∈ X ,

Var( µ b

s

) ≤ y

N

for some y

≥ 0.

Proposition 2. For the SSS method, it holds:

1. for any non-negative sequence s

E [ b µ

s] = µ

s, 2. for any h ∈ X ,

Var( b µ

s

) ≤ y

N

, where y

= M

y

+ (M + 2)/4 (n ≥ 0).

3.3 Sudoku Latin Square Sampling

Lemma 3. Let s be a non-negative sequence. The SLSS estimator of e µ

s:

Z b

:= 1

N ∑

`∈[1,p]²

C

(W

)

has the following properties.

1. Z b

is unbiased.

2. If s = s

, for h ∈ X , and if q

is a piecewise monotonic sequence, with r pieces, then

Var(b Z

h

) ≤ 1

N

13

4 + r

(TV (q

) + 2) + 2(TV (q

) + 2)

.

The constant involved in the O (N

) bound of Var(b Z

h

) (SLSS) is larger than the corresponding constant for Var(b Y

h

) (SSS), since r ≥ 1 in Lemma 3; this would suggest degraded performance. But in the examples of Sect. 4 we see that it is not necessarily the case.

The proof of the next result is similar to the proof of Proposition 1. We assume that, for any non-negative sequence s, the SLSS estimator µ b

s of µ

s is unbiased and that, for any h ∈ X ,

Var( b µ

s

) ≤ z

N

, for some z

≥ 0.

Proposition 3. For the SLSS method, it holds:

1. for any non-negative sequence s

E [ b µ

s] = µ

s, 2. for any h ∈ X ,

Var( b µ

s

) ≤ z

N

,

where z

= M

z

+ (13/4 + r)(M +2) + 2(M + 2)

n ≥ 0.

Remark 1. The bounds in Propositions 1, 2, and 3 increase exponentially with n when M > 1, and remains bounded if M < 1, which is not uncommon; see [14].

Remark 2. The variance of each estimator is bounded for a test sequence of the form s

, for h ∈ X . We obtain a bound for a nonnegative cost function c by the same reasoning as in Proposition 1 (see [12]):

µ

c − b µ

c = ∑

i∈X

( b µ

s

− µ

s

)(c(i + 1) − c(i)), hence

E

(µ

c − b µ

c)

≤ ∑

(i,j)∈X²

q

Var( µ b

s

)Var( b µ

s

)|c(i +1)−c(i)|×|c( j +1)−c( j)|, and then

Var( µ b

c) ≤ TV(c)

× sup

h∈X

Var( µ b

s

).

4 Numerical Examples

In this section, we compare standard Monte Carlo to the variance reduction strategies analyzed previously: SSS and SLSS, for three examples. For each example, each strategy, and each N considered, we compute the unbiased estimator ˆ µ

s of µ

s for the selected n, replicate this 100 times, and compute the empirical variance Var of the 100 realizations of ˆ µ

s. We then plot log

Var as a function of log

N. Assuming that Var ≈ KN

for some positive constants K and α, we estimate the variance rate α by linear regression. We also compute the (empirical) efficiency of each simulation estimator, defined as the inverse of the product of Var by the CPU time [10], and we plot log

efficiency as a function of log

N. Note that for standard Monte Carlo, the efficiency does not depend on N. For SSS and SLSS, it takes into account the additional work to compute the estimators.

4.1 A Geo/Geo/1 Queue

We consider a discrete-time Geo/Geo/1 queue (see [1]): the queue is empty at the initial time. During each unit of time, the customer in service (if there is one) completes it with probability 0.5, and one new customer arrives with probability 0.6.

We estimate the mean number of customers in the queue at time n = 12. Figure 2

(top) shows log

Var as a function of log

N on the left and log

efficiency as a

function of log

N on the right, for N = 10

, 50

, 100

, . . . , 1 000

. We find from the

plots that SSS and SLSS give not only smaller variances than standard MC (for the same N), but also better efficiencies, and that SLSS outperforms SSS. The regression estimates of α are given in the first row of Table 1. They match the upper bounds of O (N

) established in Sect. 3.

4.2 A Gambler in a Casino

A gambler is going to a casino for four hours. He plans to play the same game every ten seconds (so he will play 1 440 times). At this game, for each Euro that he bids, he gets 0 with probability 0.9 and m ∈ {1, 2, . . . , 10} with probability 0.01 each. His policy is the following: if he has more than 100 Euros, he plays 2 Euros, but if he has 100 Euros or less, he plays only 1. To make sure that he can play during the four hours, he brings 2 780 Euros with him. The model is a Markov chain on state space E = [0, 28 700]. We estimate the probability that the gambler has more than 1 500 Euros at the end of the game. Here we use N = 10

, 20

, 30

, . . . , 200

. The results are reported in the middle rows of Fig. 2 and Table 1. We find that SSS and SLSS produce both smaller variances and better efficiencies than standard MC for large enough N, and that SLSS outperforms SSS. The regression estimate of α for SLSS corresponds to the bound established in Sect. 3, but for SSS it is better. However, the variance itself is smaller for SLSS than for SSS, and the better rate of SSS might not hold beyond the observed range (it is likely caused by a few poor values of Var for the smallest values of N).

4.3 Diffusion

The 1-D diffusion equation

∂ c

∂ t (x,t) = D ∂

c

∂ t

(x, t), x ∈ R , t > 0 and c(x, 0) = c

(x), x ∈ R

(where c

≥ 0 and

c

(x)dx = 1) may be discretized with a time step ∆ t and a spatial step ∆ x and the solution is approximated using a random walk: P(i,i −1) = P(i, i + 1) = D∆ t/∆x

, P(i,i) = 1 −2D∆t/∆ x

(we refer to [3] in a QMC context). Here we specify D = 1 and c

is the indicator function of the interval [−1/2, 1/2]; we want to approximate

c(x, T )dx. We choose T = 1, with ∆t = 6.25 10

and ∆ x = 5.0 10

. Here we take N = 11

, 19

, 31

, . . . , 199

. In a previous version of the paper, we had N = 10

, 20

, 30

, . . . , 200

, but this gave oscillations in SLSS outcomes, with better results when p = √

N was a multiple of 20, because of interactions with other

discretization parameters. To avoid this, we changed our choices of p. So we use

prime numbers near the previous ones for p. The bottom part of Figure 2 and the

last row of Table 1 give the results, which are very similar to those of the previous example. Again, SLSS outperforms the other methods.

Table 1 Calculation of orderαof the sample variance: comparison of standard Monte Carlo (MC), SSS, and SLSS for three examples and estimators

Calculation MC SSS SLSS

Geo/Geo/1 queue E[X12] 0.99 1.50 1.51

Gambler in a casino P(X_{1 440}>1 500) 1.02 1.76 1.53

Diffusion ^R_−1/2^1/2 c(x,T)dx 0.98 1.61 1.49

5 The Proofs

5.1 Proof of Lemma 2

Proof.

1. We have

E [C

(V

)] = N

N

∑

k=1

∑

j∈X

1

(u

)1

k,j

(u

)du

s( j).

Hence

E [b Y

] = 1 N

N

∑

k=1

∑

j∈X

P(i

, j)s( j) = µ e

s.

2. The function C

h

is the indicator function of the set J

:=

N [

k=1

I

×

j∈X,j≤h

I

k,j

!

=

N [

k=1

I

× (0, q

(i

)]. (12) The variable C

h

(V

) is a Bernoulli random variable, with expectation f

= Nλ

(J

∩ I

). Here f

= 1 if I

⊂ J

and f

= 0 if I

∩ J

= /0. Consequently, Var(C

h

(V

)) = f

(1− f

) ≤ 1/4 and Var(C

h

(V

)) = 0 if I

⊂ J

or if I

∩J

= /0, so that

Var(b Y

h

) ≤ 1

4N

{` ∈ [1, p]

: I

6⊂ J

and I

∩ J

6= /0}

.

a. If I

6⊂ J

, then there exists (u

,u

) which belongs to I

and not to J

; since

this u

is in some I

, we have:

1. Geo/Geo/1 queue: sample variance (left) and efficiency (right) of 100 copies of the calculation ofE[X12]as a function ofN(N=10²,50²,100², . . . ,1 000²) (log-log scale)

2. Gambler in a casino: sample variance (left) and efficiency (right) of 100 copies of the calculation ofP(X_{1 440}>1 500)as a function ofN(N=10²,20²,30², . . . ,200²) (log-log scale)

3. Diffusion: sample variance (left) and efficiency (right) of 100 copies of the calculation of R1/2

−1/2c(x,T)dxas a function ofN(N=11²,19²,31², . . . ,199²) (log-log scale)

Fig. 2 Comparison of standard Monte Carlo (MC) to SSS and SLSS (Sudoku) for three examples

∃k ∈ [1, N], ∃(u

, u

) ∈ I

: u

∈ I

⊂

`

− 1 p , `

p

and u

∈ / (0, q

(i

)], so that

∃k ∈ { p(`

−1) + 1, p(`

− 1) + 2, . . . , p`

}, ∃u

∈

`

− 1 p , `

p

: u

> q

(i

), consequently

`

> p min

p(`<sub>1</sub>−1)<k≤p`₁

q

(i

).

b. Analogously, if I

∩ J

6= /0, then there exists (u

,u

) which belongs to I

and also to J

; and we eventually obtain:

`

< p max

p(`<sub>1</sub>−1)<k≤p`₁

q

(i

) + 1.

We then have the following bounds

{` ∈ [1, p]

: I

6⊂ J

and I

∩ J

6= /0}

≤ p

p

∑

`₁=1

max

p(`<sub>1</sub>−1)<k≤p`₁

q

(i

)− min

p(`<sub>1</sub>−1)<k≤p`₁

q

(i

)

+ 2

!

≤ N

∑

i∈X

|q

(i + 1) − q

(i)| + 2

! ,

because the states are relabeled so that i

≤ · · · ≤ i

. Consequently, {` ∈ [1, p]

: I

6⊂ J

and I

∩ J

6= /0}

≤ N

(TV (q

) + 2), (13) and the result follows.

u t

5.2 Proof of Lemma 3

Proof.

1. Since W

∼ U (I

), the demonstration is the same as in Lemma 2.

2. In the following, we have many summations with `, `

, m, m

∈ [1, p]

. In order to lighten the notations, we omit this set. We have

Var(b Z

h

) = V

(b Z

h

) + 1

N

∑

(`,`⁰):`6=`⁰

Cov C

h

(W

),C

h

(W

) ,

where

V

(b Z

h

) := 1

N

∑

`

Var C

h

(W

) . a. The function C

h

is the indicator function of the set J

defined by (12). Since W

∼ U (I

), we have, as in Lemma 2:

V

(b Z

h

) ≤ 1

4N

{` ∈ [1, p]

; I

6⊂ J

and I

∩ J

6= /0}

.

From the bound (13), we deduce V

(b Z

h

) ≤ 1

4N

(TV (q

) + 2).

b. We split Var(b Z

h

) = V

(b Z

h

) + V

(b Z

h

) + V

(b Z

h

) + V

(b Z

h

), with V

(b Z

h

) := 1

N

∑

(`,`⁰):`<sub>1</sub>6=`⁰₁,`<sub>2</sub>=`⁰₂

Cov C

h

(W

),C

h

(W

) ,

V

(b Z

h

) := 1

N

∑

(`,`⁰):`<sub>1</sub>=`⁰₁,`<sub>2</sub>6=`⁰

2

Cov C

h

(W

),C

h

(W

) ,

V

(b Z

h

) := 1

N

∑

(`,`⁰):`<sub>1</sub>6=`⁰₁,`<sub>2</sub>6=`⁰₂

Cov C

h

(W

),C

h

(W

) .

We introduce the N

squares I

= H

× H

, where, for (`, m) ∈ [1, p]

: H

:=((`

− 1)/ p + (m

− 1)/N, (`

− 1)/ p +m

/N],

H

:=((`

− 1)/ p + (m

− 1)/N, (`

− 1)/ p +m

/N].

We have

V

(b Z

h

) = ∑

(`,`⁰):`<sub>1</sub>6=`⁰

1,`<sub>2</sub>=`⁰₂

N

p − 1 ∑

(m,m⁰):m₁=m⁰₁,m₂6=m⁰

2

λ

(I

∩ J

)λ

(I

∩ J

)

− λ

(I

∩ J

)λ

(I

∩ J

)

! ,

V

(b Z

h

) = ∑

(`,`⁰):`<sub>1</sub>=`⁰₁,`<sub>2</sub>6=`⁰₂

N

p − 1 ∑

(m,m⁰):m₁6=m⁰₁,m₂=m⁰₂

λ

(I

∩ J

)λ

(I

∩ J

)

− λ

(I

∩ J

)λ

(I

∩ J

)

! ,

V

(b Z

h

) = ∑

(`,`⁰):`<sub>1</sub>6=`⁰₁,`<sub>2</sub>6=`⁰₂

N

(p −1)

∑

(m,m⁰):m₁6=m⁰₁,m₂6=m⁰₂

λ

(I

∩ J

)λ

(I

∩ J

)

− λ

(I

∩ J

)λ

(I

∩ J

)

! .

i. We have

V

(b Z

h

) = ∑

`:I<sub>`</sub>6⊂Jⁿ

h,I_`∩Jⁿ

h6=/0

∑

`<sup>0</sup>:`⁰₁6=`<sub>1</sub>,`⁰₂=`₂

V

(`, `

), where

V

(`, `

) := N

p −1 ∑

(m,m⁰):m₁=m⁰₁,m₂6=m⁰

2

λ

(I

∩ J

)λ

(I

∩ J

)

− λ

(I

∩ J

)λ

(I

∩ J

).

We split V

(`, `

) = V ˆ

(`, `

) + V ˇ

(`, `

), with V ˆ

(`, `

) := N

p − 1 ∑

(m,m⁰):m1=m⁰₁,m26=m⁰₂

λ

(I

∩ J

)λ

(I

∩ J

)

− N

p ∑

(m,m⁰):m1=m⁰₁

λ

(I

∩ J

)λ

(I

∩ J

), V ˇ

(`, `

) :=p

× ∑

(m,m⁰):m₁=m⁰₁

λ

(I

∩ J

)λ

(I

∩ J

) − λ

(I

∩ J

)λ

(I

∩ J

).

On one side

V ˆ

(`, `

) =N ∑

m

λ

(I

∩ J

)

×



 1

p(p − 1) ∑

m⁰:m⁰₁=m1,m⁰₂6=m₂

λ

(I

∩ J

) − 1

p λ

(I

∩ J

)



.

Since both terms inside the parentheses are bounded by 1/(pN

), we have

| V ˆ

(`, `

)| ≤ 1/(pN

) and so

∑

`<sup>0</sup>:`⁰₁6=`<sub>1</sub>,`⁰₂=`2

V ˆ

(`, `

)

≤ p − 1 pN

.

On the other side V ˇ

(`, `

) = ∑

m₁∈[1,p]

λ

((H

× H

)∩ J

) ∑

m⁰₁∈[1,p]

V ˇ

(`, `

,m

, m

),

where ˇ V

(`, `

, m

, m

) := λ

((H

1,m₁

×H

)∩ J

)−λ

((H

1,m⁰₁

×H

) ∩J

).

We have

V ˇ

(`, `

, m

, m

) = 1 N

λ

H

∩ (0,q

(i

1−1)+m₁

)]

−λ

H

∩ (0,q

(i

1−1)+m⁰₁

)]

.

As we have

| V ˇ

(`, `

, m

,m

)| ≤ 1 N

×λ H

∩

"

min

p(`<sup>0</sup><sub>1</sub>−1)<k≤p`⁰₁

q

(i

), max

p(`<sup>0</sup><sub>1</sub>−1)<k≤p`⁰₁

q

(i

)

#!

,

we deduce

| V ˇ

(`, `

)| ≤ 1

N p λ H

∩

"

min

p(`<sup>0</sup><sub>1</sub>−1)<k≤p`⁰₁

q

(i

), max

p(`<sup>0</sup><sub>1</sub>−1)<k≤p`⁰₁

q

(i

)

#!

.

Consequently

∑

`<sup>0</sup>:`⁰₁6=`<sub>1</sub>,`⁰₂=`₂

V ˇ

(`, `

)

≤ 1 N p

× ∑

`<sup>0</sup><sub>1</sub>∈[1,p]:`⁰

16=`₁

λ H

∩

"

min

p(`<sup>0</sup><sub>1</sub>−1)<k≤p`⁰₁

q

(i

), max

p(`<sup>0</sup><sub>1</sub>−1)<k≤p`⁰₁

q

(i

)

#!

.

Since q

is a piecewise monotonic sequence, and because the states are relabeled so that i

≤ · · · ≤ i

the intervals

min

p(`<sup>0</sup><sub>1</sub>−1)<k≤p`⁰₁

q

(i

), max

p(`<sup>0</sup><sub>1</sub>−1)<k≤p`⁰₁

q

(i

)

!

, `

∈ [1, p]

are pairwise disjoint on each of the r pieces where q

is monotonic, and we obtain

∑

`<sup>0</sup>:`⁰₁6=`<sub>1</sub>,`⁰₂=`₂

V ˇ

(`, `

)

≤ r

N p λ (H

) ≤ r N

. And so, using the bound (13):

|V

(b Z

h

)| ≤ (r +1)p − 1

pN

(TV (q

) + 2).

ii. We have

V

(b Z

h

) = ∑

`:I<sub>`</sub>6⊂Jⁿ

h,I_`∩Jⁿ

h6=/0

∑

`<sup>0</sup>:`⁰₁=`<sub>1</sub>,`⁰₂6=`₂

V

(`, `

), where

V

(`, `

) := N p − 1

× ∑

(m,m⁰):m₁6=m⁰₁,m₂=m⁰₂

λ

(I

∩ J

)λ

(I

∩ J

)− λ

(I

∩ J

)λ

(I

∩ J

).

We split V

(`, `

) = V ˆ

(`, `

) + V ˇ

(`, `

), with V ˆ

(`, `

) := N

p − 1 ∑

(m,m⁰):m₁6=m⁰₁,m₂=m⁰₂

λ

(I

∩ J

)λ

(I

∩ J

)

− N

p ∑

(m,m⁰):m₂=m⁰₂

λ

(I

∩ J

)λ

(I

∩ J

), V ˇ

(`, `

) :=p ∑

(m,m⁰):m₂=m⁰₂

λ

(I

∩ J

)λ

(I

∩ J

)− λ

(I

∩ J

)λ

(I

∩ J

).

On one side V ˆ

(`, `

) =N ∑

m

λ

(I

∩ J

)

×



 1

p(p − 1) ∑

m⁰:m⁰₁6=m₁,m⁰₂=m₂

λ

(I

∩ J

) − 1

p λ

(I

∩ J

)



.

Since both terms inside the parentheses are bounded by 1/(pN

), we have

| V ˆ

(`, `

)| ≤ 1/(pN

) and so

∑

`<sup>0</sup>:`⁰₁=`1,`⁰₂6=`₂

V ˆ

(`, `

)

≤ p − 1 pN

.

On the other side V ˇ

(`, `

) = ∑

m₂∈[1,p]

λ

((H

×H

)∩ J

) ∑

m⁰₂∈[1,p]

V ˇ

(`, `

,m

, m

),

where ˇ V

(`, `

, m

, m

) := λ

((H

×H

2,m2

) ∩J

)−λ

((H

×H

2,m⁰₂

)∩J

);

we have

V ˇ

(`, `

, m

, m

) = 1

N ∑

m⁰₁∈[1,p]

λ

H

2,m₂

∩ (0, q

(i

1−1)+m⁰₁

)]

−λ H

2,m⁰₂

∩ (0, q

(i

1−1)+m⁰₁

)]

. Note that the difference in the parentheses is equal to 0 if `

6= `

(`

, m

) :=

dnq

(i

1−1)+m⁰₁

)e +1. Consequently

∑

`<sup>0</sup>:`⁰₁=`<sub>1</sub>,`⁰₂6=`₂

V ˇ

(`, `

)

≤ 1

N ∑

m₂∈[1,p]

λ

((H

× H

) ∩ J

)

× ∑

m⁰

λ

H

2(`₁,m⁰₁),m₂

∩ (0, q

(i

1−1)+m⁰₁

)]

−λ H

2(`₁,m⁰₁),m⁰₂

∩ (0,q

(i

1−1)+m⁰₁

)]

≤ 1

N ∑

m₂∈[1,p]

λ

((H

× H

) ∩ J

) ≤ 1

N λ

(I

) = 1 N

.

And so, using the bound (13):

|V

(b Z

h

)| ≤ 2 p −1

pN

(TV (q

) + 2).

iii. We have

V

(b Z

h

) = ∑

`:I<sub>`</sub>6⊂Jⁿ

h,I_`∩Jⁿ

h6=/0

∑

`<sup>0</sup>:`⁰₁6=`<sub>1</sub>,`⁰₂6=`₂

I`0 6⊂Jn h,I

`0 ∩Jn

h6=/0

V

(`, `

),

where