A Continuous Time Approach for the Asymptotic Value in Two-Person Zero-Sum Repeated Games

HAL Id: inria-00585695

https://hal.inria.fr/inria-00585695

Submitted on 14 Apr 2011

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Sylvain Sorin, Pierre Cardaliaguet, Rida Laraki

To cite this version:

Sylvain Sorin, Pierre Cardaliaguet, Rida Laraki. A Continuous Time Approach for the Asymptotic Value in Two-Person Zero-Sum Repeated Games. SADCO Kick off, Mar 2011, Paris, France. �inria- 00585695�

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

A Continuous Time Approach for the Asymptotic Value in Two-Person Zero-Sum Repeated Games

Sylvain Sorin

UPMC-Paris 6 and Ecole Polytechnique

Joint work with Pierre Cardaliaguet and Rida Laraki

Contents

1 Introduction: Shapley

2 Extensions of the Shapley operator : general repeated games

3 Extensions of the Shapley operator : general evaluation

4 Asymptotic analysis: the main results

5 Asymptotic analysis - the discounted case: games with incomplete information

6 Asymptotic analysis - the continuous approach: games with incomplete information

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

A stochastic game is a repeated game in discrete time where the state changes from stage to stage according to a transition depending on the current state and the moves of the players.

A stochastic game is a repeated game in discrete time where the state changes from stage to stage according to a transition depending on the current state and the moves of the players.

The game is specified by a state spaceΩ, move sets I andJ, a transition probabilityQ fromI ×J×Ω→Ωand a payoff function g fromI ×J×Ω→R

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

A stochastic game is a repeated game in discrete time where the state changes from stage to stage according to a transition depending on the current state and the moves of the players.

The game is specified by a state spaceΩ, move sets I andJ, a transition probabilityQ fromI ×J×Ω→Ωand a payoff function g fromI ×J×Ω→R

All sets under consideration are finite.

Inductively, at staget =1, ..., knowing the past history

h_t = (ω1,i1,j1, ....,i_t−1,j_t−1, ωt), player I choosesi_t ∈I, player J choosesj_t ∈J.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Inductively, at staget =1, ..., knowing the past history

h_t = (ω1,i1,j1, ....,i_t−1,j_t−1, ωt), player I choosesi_t ∈I, player J choosesj_t ∈J.

The new stateω_t+1∈Ωis drawn according to the probability distributionQ(it,j_t, ωt)(·).

Inductively, at staget =1, ..., knowing the past history

h_t = (ω1,i1,j1, ....,i_t−1,j_t−1, ωt), player I choosesi_t ∈I, player J choosesj_t ∈J.

The new stateω_t+1∈Ωis drawn according to the probability distributionQ(it,j_t, ωt)(·).

The triplet(it,j_t, ω_t+1)is publicly announced and the situation is repeated.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Inductively, at staget =1, ..., knowing the past history

h_t = (ω1,i1,j1, ....,i_t−1,j_t−1, ωt), player I choosesi_t ∈I, player J choosesj_t ∈J.

The new stateω_t+1∈Ωis drawn according to the probability distributionQ(it,j_t, ωt)(·).

The triplet(it,j_t, ω_t+1)is publicly announced and the situation is repeated.

The payoff at staget isg_t =g(it,j_t, ωt) and the total payoff is the discounted sumP

tλ(1−λ)^t−1g_t.

Inductively, at staget =1, ..., knowing the past history

h_t = (ω1,i1,j1, ....,i_t−1,j_t−1, ωt), player I choosesi_t ∈I, player J choosesj_t ∈J.

The new stateω_t+1∈Ωis drawn according to the probability distributionQ(it,j_t, ωt)(·).

The triplet(it,j_t, ω_t+1)is publicly announced and the situation is repeated.

The payoff at staget isg_t =g(it,j_t, ωt) and the total payoff is the discounted sumP

tλ(1−λ)^t−1g_t. This discounted game has a valuev_λ.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The Shapley Operator

TheShapley operatorT(λ,·) associates to a functionf inR^Ω the function:

T(λ,f)(ω) = max

x∈∆(I) min

y∈∆(J)[λg(x,y, ω) + (1−λ)X

˜ ω

Q(x,y, ω)(˜ω)f(˜ω)]

The Shapley Operator

TheShapley operatorT(λ,·) associates to a functionf inR^Ω the function:

T(λ,f)(ω) = max

x∈∆(I) min

y∈∆(J)[λg(x,y, ω) + (1−λ)X

˜ ω

Q(x,y, ω)(˜ω)f(˜ω)]

Lemma

The Shapley operator T(λ,·) is well defined fromR^Ω to itself. Its unique fixed point is v_λ.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Contents

1 Introduction: Shapley

2 Extensions of the Shapley operator : general repeated games

3 Extensions of the Shapley operator : general evaluation

4 Asymptotic analysis: the main results

5 Asymptotic analysis - the discounted case: games with incomplete information

6 Asymptotic analysis - the continuous approach: games with incomplete information

A recursive structure leading to an expression similar to the previous one holds in general for stationnary repeated games.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

A recursive structure leading to an expression similar to the previous one holds in general for stationnary repeated games.

Let us describe this representation for games with incomplete information.

A recursive structure leading to an expression similar to the previous one holds in general for stationnary repeated games.

Let us describe this representation for games with incomplete information.

M is a product space K×L,π is a product probability p⊗q with p∈∆(K),q ∈∆(L) and in additiona1 =k andb1=ℓ.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

A recursive structure leading to an expression similar to the previous one holds in general for stationnary repeated games.

Let us describe this representation for games with incomplete information.

M is a product space K×L,π is a product probability p⊗q with p∈∆(K),q ∈∆(L) and in additiona1 =k andb1=ℓ.

Given the parameterm= (k, ℓ), each player knows his own component and holds a prior on the other player’s component.

A recursive structure leading to an expression similar to the previous one holds in general for stationnary repeated games.

Let us describe this representation for games with incomplete information.

M is a product space K×L,π is a product probability p⊗q with p∈∆(K),q ∈∆(L) and in additiona1 =k andb1=ℓ.

Given the parameterm= (k, ℓ), each player knows his own component and holds a prior on the other player’s component.

From stage 1 on, the parameter is fixed and the information of the players after stagen is a_n+1 =b_n+1 ={in,j_n}.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

An auxiliary stochastic gameΓ^′ is as follows:

An auxiliary stochastic gameΓ^′ is as follows:

the “state space"M^′ is∆(K)×∆(L)and is interpreted as the space of beliefs on the true parameter.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

An auxiliary stochastic gameΓ^′ is as follows:

the “state space"M^′ is∆(K)×∆(L)and is interpreted as the space of beliefs on the true parameter.

X= ∆(I)^K andY= ∆(J)^L are the type-dependent mixed action sets of the players;g is extended onX×Y×M^′ by

g(p,q,x,y) =P

k,ℓ p^kq^ℓg(k, ℓ,x^k,y^ℓ).

An auxiliary stochastic gameΓ^′ is as follows:

the “state space"M^′ is∆(K)×∆(L)and is interpreted as the space of beliefs on the true parameter.

X= ∆(I)^K andY= ∆(J)^L are the type-dependent mixed action sets of the players;g is extended onX×Y×M^′ by

g(p,q,x,y) =P

k,ℓ p^kq^ℓg(k, ℓ,x^k,y^ℓ).

Given(p,q,x,y), let x(i) =P

kx_i^kp^k be the (total) probability of actioni andp(i) be the conditional probability onK given the actioni, explicitlyp^k(i) = ^p

kx_i^k

x(i) (and similarly fory andq).

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The resulting form of the Shapley operator is:

T(λ,f)(p,q) = sup

x∈X

y∈infY{λX

k,ℓ

p^kq^ℓg(k, ℓ,x^k,y^ℓ) +(1−λ)X

i,j

x(i)y(j)f(p(i),q(j))} (1) These equations are due to Aumann and Maschler (1966) and Mertens and Zamir (1971).

Contents

1 Introduction: Shapley

2 Extensions of the Shapley operator : general repeated games

3 Extensions of the Shapley operator : general evaluation

4 Asymptotic analysis: the main results

5 Asymptotic analysis - the discounted case: games with incomplete information

6 Asymptotic analysis - the continuous approach: games with incomplete information

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The classical fixed point formula for the discounted value is

v_λ =T[λ,v_λ] (2)

The classical fixed point formula for the discounted value is

v_λ =T[λ,v_λ] (2)

and the recursive formula for then stage value is obtained similarly v_n=T[1

n,v_n−1] (3)

with obviouslyv0=0.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Consider now an arbitrarly evaluation probabilityµon IN^⋆. The total payoff isP

tµ_mg_m.

Consider now an arbitrarly evaluation probabilityµon IN^⋆. The total payoff isP

tµ_mg_m.

Thenµinduces a partition Πof [0,1]with t0 =0,t_k =Pk m=1µm.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Consider now an arbitrarly evaluation probabilityµon IN^⋆. The total payoff isP

tµ_mg_m.

Thenµinduces a partition Πof [0,1]with t0 =0,t_k =Pk m=1µm. The repeated game is naturally represented as a game played between times 0 and 1, where the actions are constant on each subinterval(t_k−1,t_k): its lengthµ_k is the weight of stagek in the original game. Letv_Π be its value.

Consider now an arbitrarly evaluation probabilityµon IN^⋆. The total payoff isP

tµ_mg_m.

Thenµinduces a partition Πof [0,1]with t0 =0,t_k =Pk m=1µm. The repeated game is naturally represented as a game played between times 0 and 1, where the actions are constant on each subinterval(t_k−1,t_k): its lengthµ_k is the weight of stagek in the original game. Letv_Π be its value. The recursive equation is

v_Π =val{t1g1+ (1−t1)Ev_Πt1}

whereΠt1 is the normalization on[0,1]of the trace of the partition Πon the interval[t1,1].

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Define nowV_Π(tk) as the value of the game starting at timet_k with evaluationP

mµ_m+kg_m. One obtains the alternative recursive formula

V_Π(t_k) =val{(t_k+1−t_k)g_k+1+EV_Π(t_k+1)} (4)

Define nowV_Π(tk) as the value of the game starting at timet_k with evaluationP

mµ_m+kg_m. One obtains the alternative recursive formula

V_Π(t_k) =val{(t_k+1−t_k)g_k+1+EV_Π(t_k+1)} (4)

By taking the linear extension we define this way for every finite partitionΠ, a functionV_Π(t) on [0,1].

Lemma

Assumeµ(n) decreasing. Then V_Π is C -Lipschitz in t, where C is a bound on the payoffs.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Contents

1 Introduction: Shapley

2 Extensions of the Shapley operator : general repeated games

3 Extensions of the Shapley operator : general evaluation

4 Asymptotic analysis: the main results

5 Asymptotic analysis - the discounted case: games with incomplete information

6 Asymptotic analysis - the continuous approach: games with incomplete information

We consider now the asymptotic behavior ofv_n asn goes to∞, or ofv_λ as λgoes to 0, or more generally of V_Π as the mesh µ(1) goes to 0.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

We consider now the asymptotic behavior ofv_n asn goes to∞, or ofv_λ as λgoes to 0, or more generally of V_Π as the mesh µ(1) goes to 0.

1) Concerning games with incomplete information on one side, the first results proving the existence of limn→∞v_n and lim_λ→0v_λ are due to Aumann and Maschler (1966), including in addition an identification of the limit asCav_∆(K)u.

We consider now the asymptotic behavior ofv_n asn goes to∞, or ofv_λ as λgoes to 0, or more generally of V_Π as the mesh µ(1) goes to 0.

1) Concerning games with incomplete information on one side, the first results proving the existence of limn→∞v_n and lim_λ→0v_λ are due to Aumann and Maschler (1966), including in addition an identification of the limit asCav_∆(K)u.

Hereu(p) =val

∆(I)×∆(J)

P

kp^kg(k,x,y) is the value of the one shot non revealing game, where the informed player does not use his information andCav_C is the concavification operator: givenφ, a real bounded function defined on a convex setC,Cav_C(φ)is the smallest function greater thanφand concave, onC.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Extensions of these results to games with lack of information on both sides were achieved by Mertens and Zamir (1971). In addition they identified the limit as the only solution of the system of implicit functional equations with unknownφ:

φ(p,q) =Cav_p∈

∆(K)min{φ,u}(p,q), (5) φ(p,q) =Vex_q∈

∆(L)max{φ,u}(p,q) (6)

Extensions of these results to games with lack of information on both sides were achieved by Mertens and Zamir (1971). In addition they identified the limit as the only solution of the system of implicit functional equations with unknownφ:

φ(p,q) =Cav_p∈

∆(K)min{φ,u}(p,q), (5) φ(p,q) =Vex_q∈

∆(L)max{φ,u}(p,q) (6) Here againu stands for the value of the non revealing game:

u(p,q) =val_X×Y P

k,ℓp^kq^ℓg(k, ℓ,x,y)and we will write MZfor the corresponding operator

φ=MZ(u). (7)

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

2) As for stochastic games, the existence of lim_λ→0v_λ in the finite case (Ω,I,J finite) is due to Bewley and Kohlberg (1976) using algebraic arguments:

2) As for stochastic games, the existence of lim_λ→0v_λ in the finite case (Ω,I,J finite) is due to Bewley and Kohlberg (1976) using algebraic arguments:

the Shapley equation can be written as a finite set of polynomial equalities and inequalities involving{x_λ^k,y_λ^k,v_λ(k), λ}thus it defines a semi-algebraic set in some euclidean spaceR^N, hence by projectionv_λ has an expansion in Puiseux series.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

2) As for stochastic games, the existence of lim_λ→0v_λ in the finite case (Ω,I,J finite) is due to Bewley and Kohlberg (1976) using algebraic arguments:

the Shapley equation can be written as a finite set of polynomial equalities and inequalities involving{x_λ^k,y_λ^k,v_λ(k), λ}thus it defines a semi-algebraic set in some euclidean spaceR^N, hence by projectionv_λ has an expansion in Puiseux series.

The existence of limn→∞v_n is obtained by an algebraic comparison argument, Bewley and Kohlberg (1976).

Starting with Rosenberg and Sorin (2001) several asymptotic results have been obtained, based on the Shapley operator:

continuous absorbing and recursive games, games with incomplete information on both sides, absorbing games with incomplete information on one side, Rosenberg (2000).

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Starting with Rosenberg and Sorin (2001) several asymptotic results have been obtained, based on the Shapley operator:

continuous absorbing and recursive games, games with incomplete information on both sides, absorbing games with incomplete information on one side, Rosenberg (2000).

We describe here an approach that was initially introduced by Laraki (2002) for the discounted case.

Contents

1 Introduction: Shapley

2 Extensions of the Shapley operator : general repeated games

3 Extensions of the Shapley operator : general evaluation

4 Asymptotic analysis: the main results

5 Asymptotic analysis - the discounted case: games with incomplete information

6 Asymptotic analysis - the continuous approach: games with incomplete information

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The analysis is inspired from Larki (2001).

Recall that the recursive equation is a fixed point operator:

T(λ,v_λ)(p,q) = sup

x∈X

yinf∈Y{λg(p,q,x,y) +(1−λ)X

i,j

x(i)y(j)v_λ(p(i),q(j))}

= v_λ(p,q) (8)

The analysis is inspired from Larki (2001).

Recall that the recursive equation is a fixed point operator:

T(λ,v_λ)(p,q) = sup

x∈X

yinf∈Y{λg(p,q,x,y) +(1−λ)X

i,j

x(i)y(j)v_λ(p(i),q(j))}

= v_λ(p,q) (8)

Remark that the family of functions{v_λ(p,q)}is uniformly Lipschitz, hence relatively compact. To prove convergence it is enough to show that there is only one accumulation point.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The analysis is inspired from Larki (2001).

Recall that the recursive equation is a fixed point operator:

T(λ,v_λ)(p,q) = sup

x∈X

yinf∈Y{λg(p,q,x,y) +(1−λ)X

i,j

x(i)y(j)v_λ(p(i),q(j))}

= v_λ(p,q) (8)

Remark that the family of functions{v_λ(p,q)}is uniformly Lipschitz, hence relatively compact. To prove convergence it is enough to show that there is only one accumulation point.

Note first that any accumulation pointw satisfies

Assume now thatw1 andw2,w1 ≥w2 are two different accumulation points and let(p0,q0) an extreme point of the (convex hull of) the set where the differencew1−w2 is maximal.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Assume now thatw1 andw2,w1 ≥w2 are two different accumulation points and let(p0,q0) an extreme point of the (convex hull of) the set where the differencew1−w2 is maximal.

Using (9) and the fact that all functions involved are saddle, this implies that the setX(0,w1)(p0,q0) of profile of mixed actions x ∈X optimal in T(0,w1) is included in the setNR_X(p0) of non revealing actions atp0 (meaning that for any movei having positive probabilityp(i) =p0).

Consider a sequencev_λn converging tow1 and letx_n be optimal for T(λn,v_λn)(p⁰,q0). Jensen’s inequality leads to

v_λn(p⁰,q0)≤λ_ng(p⁰,q0,x_n,y) + (1−λ_n)vλn(p⁰,q0) ∀y ∈Y

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Consider a sequencev_λn converging tow1 and letx_n be optimal for T(λn,v_λn)(p⁰,q0). Jensen’s inequality leads to

v_λn(p⁰,q0)≤λ_ng(p⁰,q0,x_n,y) + (1−λ_n)vλn(p⁰,q0) ∀y ∈Y thusv_λn(p0,q0)≤g(p0,q0,x_n,y).

Consider a sequencev_λn converging tow1 and letx_n be optimal for T(λn,v_λn)(p⁰,q0). Jensen’s inequality leads to

v_λn(p⁰,q0)≤λ_ng(p⁰,q0,x_n,y) + (1−λ_n)vλn(p⁰,q0) ∀y ∈Y thusv_λn(p0,q0)≤g(p0,q0,x_n,y).

Let¯x be an accumulation point of the sequence {xn}, one obtains asλngoes to 0:

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Consider a sequencev_λn converging tow1 and letx_n be optimal for T(λn,v_λn)(p⁰,q0). Jensen’s inequality leads to

v_λn(p⁰,q0)≤λ_ng(p⁰,q0,x_n,y) + (1−λ_n)vλn(p⁰,q0) ∀y ∈Y thusv_λn(p0,q0)≤g(p0,q0,x_n,y).

Let¯x be an accumulation point of the sequence {xn}, one obtains asλngoes to 0:

w1(p⁰,q0)≤g(p⁰,q0,x¯,y) ∀y ∈Y

which impliesw1(p0,q0)≤u(p0,q0), since ¯x ∈NR_X(p0) (by upper

Consider a sequencev_λn converging tow1 and letx_n be optimal for T(λn,v_λn)(p⁰,q0). Jensen’s inequality leads to

v_λn(p⁰,q0)≤λ_ng(p⁰,q0,x_n,y) + (1−λ_n)vλn(p⁰,q0) ∀y ∈Y thusv_λn(p0,q0)≤g(p0,q0,x_n,y).

Let¯x be an accumulation point of the sequence {xn}, one obtains asλngoes to 0:

w1(p⁰,q0)≤g(p⁰,q0,x¯,y) ∀y ∈Y

which impliesw1(p0,q0)≤u(p0,q0), since ¯x ∈NR_X(p0) (by upper semi continuity ofX(λn,v_λn)(p⁰,q0)).

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Contents

1 Introduction: Shapley

2 Extensions of the Shapley operator : general repeated games

3 Extensions of the Shapley operator : general evaluation

4 Asymptotic analysis: the main results

5 Asymptotic analysis - the discounted case: games with incomplete information

6 Asymptotic analysis - the continuous approach: games with incomplete information

For simplicity of notations, we consider the sequence of uniform partitions.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

For simplicity of notations, we consider the sequence of uniform partitions.

For each integern, let W_n(1,p,q) :=0 and form=0, ...,n−1 defineW_n(^m_n,p,q, ω) inductively as follows.

W_n(m

n,p,q) = max

x min

y [1

ng(x,y,p,q)

+X

i,j

x(i)y(j)Wn(m+1

n ,p(i),q(j))]

Moreover ifW is an accumulation point of the equi-continuous family{Wn}then for all (t,p,q):

W(t,p,q) =max

x min

y



 X

i,j

x(i)y(j)W(t,p(i),q(j))





LetX(t,p,q,W)⊆∆(I)^K be the set of strategies for player I that are optimal for the above game.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The variational Inequalities

Theorem

For any accumulation point W of the family{W_n}, all

(p,q)∈∆(K)×∆(L) and all C¹ test function φ: [0,1]→R: (P1)If, for some t ∈[0,1),X(t,p,q,W) is non-revealing and W(·,p,q)−φ(·) has a global maximum at t, then

u(p,q) +φ^′(t)≥0.

(P2)If, for some t ∈[0,1),Y(t,p,q,W)is non-revealing and W(·,p,q)−φ(·) has a global minimum at t then

proof

Let t,p andq such thatX(t,p,q,W) is non-revealing and W(·,p,q)−φ(·)admits a global maximum at t.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

proof

Let t,p andq such thatX(t,p,q,W) is non-revealing and W(·,p,q)−φ(·)admits a global maximum at t.

Adding s−(· −t)² to φ(s) if necessary, we can assume that this global maximum is strict.

proof

Let t,p andq such thatX(t,p,q,W) is non-revealing and W(·,p,q)−φ(·)admits a global maximum at t.

Adding s−(· −t)² to φ(s) if necessary, we can assume that this global maximum is strict.

Let W_ϕ(n) converge toW and defineθ(n)∈ {0, . . . , ϕ(n)−1}

such that _ϕ(n)^θ(n) is a global maximum ofW_ϕ(n)(·,p,q)−φ(·).

Then _ϕ(n)^θ(n) →t.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

proof

Let t,p andq such thatX(t,p,q,W) is non-revealing and W(·,p,q)−φ(·)admits a global maximum at t.

Adding s−(· −t)² to φ(s) if necessary, we can assume that this global maximum is strict.

Let W_ϕ(n) converge toW and defineθ(n)∈ {0, . . . , ϕ(n)−1}

such that _ϕ(n)^θ(n) is a global maximum ofW_ϕ(n)(·,p,q)−φ(·).

Then _ϕ(n)^θ(n) →t.

One has W

θ(n) ,p,q

= maxmin[ 1

g(x,y,p,q)

proof

Letx_n be optimal for the maximizer andy ∈Y be any non-revealing strategy of playerJ.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

proof

Letx_n be optimal for the maximizer andy ∈Y be any non-revealing strategy of playerJ.

W_ϕ(n) θ(n)

ϕ(n),p,q

≤ 1

ϕ(n)g(xn,y,p,q)

+X

i

x_n(i)W_ϕ(n)

θ(n) +1

ϕ(n) ,p_n(i),q

proof

Letx_n be optimal for the maximizer andy ∈Y be any non-revealing strategy of playerJ.

W_ϕ(n) θ(n)

ϕ(n),p,q

≤ 1

ϕ(n)g(xn,y,p,q)

+X

i

x_n(i)W_ϕ(n)

θ(n) +1

ϕ(n) ,p_n(i),q

By concavity ofW_ϕ(n) with respect to p Xx (i)W

θ(n) +1

,p (i),q

≤W

θ(n) +1 ,p,q

.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

proof

Hence:

0≤g(xn,y,p,q) +ϕ(n)

W_ϕ(n)

θ(n) +1 ϕ(n) ,p,q

−W_ϕ(n) θ(n)

ϕ(n),p,q

Since _ϕ(n)^θ(n) is a global maximum ofW_ϕ(n)(·,p,q)−φ(·):

φ

θ(n) +1 ϕ(n)

−φ θ(n)

ϕ(n)

≥W_ϕ(n)

θ(n) +1 ϕ(n) ,p,q

−W_ϕ(n) θ(n)

ϕ(n),p,

proof

Hence:

0≤g(xn,y,p,q) +ϕ(n)

W_ϕ(n)

θ(n) +1 ϕ(n) ,p,q

−W_ϕ(n) θ(n)

ϕ(n),p,q

Since _ϕ(n)^θ(n) is a global maximum ofW_ϕ(n)(·,p,q)−φ(·):

φ

θ(n) +1 ϕ(n)

−φ θ(n)

ϕ(n)

≥W_ϕ(n)

θ(n) +1 ϕ(n) ,p,q

−W_ϕ(n) θ(n)

ϕ(n),p, Assume{x_n}converges to some x (hence non-revealing).

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Passing to the limit:

g(x,y,p,q) +φ^′(t)≥0.

Since this inequality holds true for everyy, taking the maximum with respect tox yields:

u(p,q) +φ^′(t)≥0.

The comparison principle

Theorem

Let W1 and W2 be two fixed points of Φin G and suppose that:

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The comparison principle

Theorem

Let W1 and W2 be two fixed points of Φin G and suppose that:

W1 satisfies (P1)

The comparison principle

Theorem

Let W1 and W2 be two fixed points of Φin G and suppose that:

W1 satisfies (P1) W1 satisfies (P2)

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The comparison principle

Theorem

Let W1 and W2 be two fixed points of Φin G and suppose that:

W1 satisfies (P1) W1 satisfies (P2)

(P3)W1(1,p,q)≤W2(1,p,q) for any(p,q)∈∆(K)×∆(L).

The comparison principle

Theorem

Let W1 and W2 be two fixed points of Φin G and suppose that:

W1 satisfies (P1) W1 satisfies (P2)

(P3)W1(1,p,q)≤W2(1,p,q) for any(p,q)∈∆(K)×∆(L).

Then W1 ≤W2 on [0,1]×∆(K)×∆(L).

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The comparison principle

We argue by contradiction, assuming that

t∈[0,1max],p∈P,q∈Q[W1(t,p,q)−W2(t,p,q)] =δ >0. Then, forε >0 sufficiently small,

δ(ε) := max

t∈[0,1],s∈[0,1],p∈P,q∈Q[W¹(t,p,q)−W²(s,p,q)−(t−s)²

2ε +εs] >0. (10)

The comparison principle

We claim that there is(t_ε,s_ε,p_ε,q_ε), point of maximum above, such thatX(tε,p_ε,q_ε,W1) is non-revealing for player I and Y(sε,p_ε,q_ε,W2) is non-revealing for player J.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The comparison principle

We claim that there is(t_ε,s_ε,p_ε,q_ε), point of maximum above, such thatX(tε,p_ε,q_ε,W1) is non-revealing for player I and Y(sε,p_ε,q_ε,W2) is non-revealing for player J.

Finally we note thatt_ε<1 and s_ε<1 forε sufficiently small, becauseδ(ε)>0 andW1(1,p,q)≤W2(1,p,q) for any (p,q)by P3.

The comparison principle

Since the mapt 7→W1(t,p_ε,q_ε)−^(t−₂^s_ε^ε)2 has a global maximum att_ε and sinceX(t_ε,p_ε,q_ε,W1)is non-revealing for player I, conditionP1implies that

u(p_ε,q_ε) +t_ε−s_ε

ε ≥0. (11)

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

The comparison principle

Since the mapt 7→W1(t,p_ε,q_ε)−^(t−₂^s_ε^ε)2 has a global maximum att_ε and sinceX(t_ε,p_ε,q_ε,W1)is non-revealing for player I, conditionP1implies that

u(p_ε,q_ε) +t_ε−s_ε

ε ≥0. (11)

In the same way, since the maps 7→W2(s,p_ε,q_ε) + ^(tε−s)2ε ² −εs has a global minimum ats_ε and sinceY(sε,p_ε,q_ε,W2) is non-revealing for player J, we have by conditionP2that

The comparison principle

Since the mapt 7→W1(t,p_ε,q_ε)−^(t−₂^s_ε^ε)2 has a global maximum att_ε and sinceX(t_ε,p_ε,q_ε,W1)is non-revealing for player I, conditionP1implies that

u(p_ε,q_ε) +t_ε−s_ε

ε ≥0. (11)

In the same way, since the maps 7→W2(s,p_ε,q_ε) + ^(tε−s)2ε ² −εs has a global minimum ats_ε and sinceY(sε,p_ε,q_ε,W2) is non-revealing for player J, we have by conditionP2that

u(pε,q_ε) + t_ε−s_ε

+ε≤0.

Introduction: Shapley Extensions of the Shapley operator : general repeated games Extensions of the Shapley operator : general evaluation

Asymptotic analysis: the main results

Asymptotic analysis - the discounted case: games with incomplete information Asymptotic analysis - the continuous approach: games with incomplete information Asymptotic analysis - the continuous approach: extensions

Contents

1 Introduction: Shapley

2 Extensions of the Shapley operator : general repeated games

3 Extensions of the Shapley operator : general evaluation

4 Asymptotic analysis: the main results

5 Asymptotic analysis - the discounted case: games with incomplete information

6 Asymptotic analysis - the continuous approach: games with incomplete information

The same tols extend to the study of absorbing games and can be applied to the “splitting game”.

Sketch of the approach :

The family of value functions is relatively compact

Consider two accumulation pointsw1 andw2 and a point (t, ω) where the differencew1−w2 is maximal.

Deduce a variational inequality at(t, ω) for any majorant ofw1 and a dual property

Prove a comparison principle.