Mean-field LQ zero-sum stochastic differential games

In this section, we will look at the situation for mean-field LQ zero-sum stochastic differential games.

According to (1.6), let us simplify some notation:



Then the cost functional will be the following:

J1(x;u1(·), u2(·)) =−J2(x;u1(·), u2(·))≡J(x;u1(·), u2(·))≡J(x;u(·)) (i= 1,2) similar to (3.4)–(3.6). Let us state the following zero-sum problem.

Problem (MF-SDG)₀. For any initial statex∈Rⁿ, Player 1 wants to find a controlu^∗₁(·) to minimize the cost functional J(x;u1(·), u2(·)), and Player 2 wants to find a control u^∗₂(·) to maximize J(x;u1(·), u2(·)) respectively, subject to (1.1) (or equivalently, (3.2)) such that (u^∗₁(·), u^∗₂(·))∈ Uad(x).

Next, we introduce the following definitions.

Definition 4.1. (i) A pair (u^∗₁(·), u^∗₂(·))∈ Uad(x) is called an open-loop saddle point of Problem (MF-SDG)0 (MF-SDG)0 if and only if one of the following holds:

(i) For anyv1(·)∈L²_F(R^m¹) andv2(·)∈L²_F(R^m²), J x;Θ^∗₁, v^∗₁(·);Θ^∗₂, v₂(·)

6J x;Θ^∗, v^∗(·)

6J x;Θ^∗₁, v₁(·);Θ^∗₂, v^∗₂(·)

. (4.5)

(ii) For anyu1(·)∈L²_F(R^m¹) andu2(·)∈L²_F(R^m²), J x;Θ^∗₁, v₁^∗(·);u2(·)

6J x;Θ^∗, v^∗(·)

6J x;u1(·);Θ^∗₂, v₂^∗(·)

. (4.6)

Definition 4.2. The following maps











V⁺(x)=^∆ inf

u1(·)∈Uad¹(x)

sup

u2(·)∈Uad²(x)

J(x;u1(·), u2(·)), V⁻(x)=^∆ sup

u₂(·)∈U_ad²(x)

inf

u₁(·)∈U_ad¹(x)

J(x;u1(·), u2(·)),

are called the upper value function and thelower value function of Problem (MF-SDG)₀, respectively. In the case that

V⁺(x) =V⁻(x)≡V(x), we call the map x7→V(x) the value function of Problem (MF-SDG)0.

Now, let (Θ^∗, v^∗(·))∈R^2m×n×L²_F(R^m), and assume the open-loop saddle points of Problem (MF-SDG)0

admit the closed-loop representation. Then by (4.1), we see that (P₁,Pb₁) = (−P₂,−Pb₂)≡(P,P) satisfy theb same equations (see (3.33))

(P A+A^>P+C^>P C+Q+ P B+C^>P D+S^>

Θ^∗= 0,

PbAb+Ab^>Pb+Cb^>PCb+Qb+ PbBb+Cb^>PDb+Sb^>Θ¯^∗= 0, (4.7) and (3.34) is equivalent to

(Σ_oΘ^∗+B^>P+D^>P C+S= 0, Σ_o≡R+D^>P D,

Σ¯oΘ¯^∗+Bb^>Pb+Db^>PCb+Sb= 0, Σ¯o≡Rb+Db^>PD.b (4.8) These equations (for Θ^∗ and ¯Θ^∗) are solvable if and only if

(R B^>P+D^>P C+S

⊆R(Σo), Θ^∗=−Σ^†_o(B^>P+D^>P C+S) + I−Σ^†_oΣo

θ, R Bb^>Pb+Db^>PCb+Sb

⊆R( ¯Σo), Θ¯^∗=−Σ¯^†_o Bb^>Pb+Db^>PCb+Sb

+ (I−Σ¯^†_oΣ¯o)¯θ, (4.9) for some θ,θ¯∈R^m×n are chosen such that (Θ^∗,Θ¯^∗)∈S[A,A, C,¯ C;¯ B,B, D,¯ D]. Putting (4.9) into (4.7), it¯ yields

(P A+A^>P+C^>P C+Q− P B+C^>P D+S^>

Σ^†_o(B^>P+D^>P C+S) = 0,

PbAb+Ab^>Pb+Cb^>PCb+Qb−(PbBb+Cb^>PDb+Sb^>) ¯Σ^†_o(Bb^>Pb+Db^>PCb+S) = 0,b (4.10) which is a system of coupled AREs, and both of them are symmetric. Thus, we have P,Pb∈Sⁿ. Next, by (4.1) again, from the componentwise form of (3.29), it is easy to see that (η1(·), ζ1(·)) = (−η2(·),−ζ2(·))

≡(ηo(·), ζo(·))∈X[0,∞)×L²_F(Rⁿ) satisfies

−dηo(t) =n

A^>ηo(t)− P B+C^>P D+S^>

Σ^†_o

B^>ηo(t) +D^>[ζo(t) +P σ(t)] +ρ(t) +C^>

ζo(t) +P σ(t)

+P b(t) +q(t)o

dt−ζo(t)dW(t), t>0,

(4.11)

with constraint

B^>(ηo−E[ηo]) +D^>(ζo−E[ζo]) +D^>P(σ−E[σ]) +ρ−E[ρ]∈ R(Σo), a.e., a.s. (4.12) Likewise, by (4.1), from the componentwise form of (3.30), we see that ¯η1(·) =−¯η2(·)≡η¯o(·)∈ L²(Rⁿ) satisfy the following:

η˙¯o(t) +Ab^>η¯o(t)−(PbBb+Cb^>PDb+Sb^>) ¯Σ^†_o Bb^>η¯o(t) +E[ ¯ζo(t)] +PE[σ(t)] +E[ρ(t)]

+Cb^> E[ ¯ζo(t)] +PE[σ(t)]

+E[q(t)] +PbE[b(t)] = 0, t>0, (4.13) with constraint

Bb^>η¯o+Db^>E[ζo] +Db^>PE[σ] +E[ρ]∈R( ¯Σo), a.e. (4.14) Finally, from (3.9), it is direct that the closed-loop representation of an open-loop saddle point is given by

u^∗(·) =

−Σ^†_o(B^>P+D^>P C+S) + I−Σ^†_oΣo

θ X^∗(·)−E[X^∗(·)]

−Σ¯^†_o Bb^>Pb+Db^>PCb+Sb

+ I−Σ¯^†_oΣ¯_oθ¯ E[X^∗(·)]

−Σ^†_o

B^>(ηo(·)−E[ηo(·)]) +D^>

ζo(·)−E[ζo(·)] +P σ(·)−E[σ(·)]

+ρ(·)−E[ρ(·)]

−Σ¯^†_o

Bb^>η¯o(·) +Db^> PE[σ(·)] +E[ζo(·)]

+E[ρ(·)] + ¯ρ(·) + I−Σ^†_oΣ_o

(ν(·)−E[ν(·)]) + I−Σ¯^†_oΣ¯_o ν(·),¯

(4.15)

where ν(·),ν¯(·)∈L²(R^m), andX^∗(·)∈X[0,∞) is the solution to (3.8) corresponding tou^∗(·).

To summarize, we have the following result.

Theorem 4.3. Let(H2)hold and the initial state x∈Rⁿ be given. Then any open-loop saddle pointu^∗(·)of Problem (MF-SDG)₀ admits a closed-loop representation if and only if the following hold:

(i)The following convexity-concavity condition holds: Fori= 1,2, (−1)ⁱ⁻¹E

Z ∞ 0

QXi(t), Xi(t) + 2

SⁱXi(t), ui(t) +

Riiui(t), ui(t)

+Q¯E[Xi(t)],E[Xi(t)]

+2S¯ⁱE[Xi(t)],E[ui(t)]

+R¯iiE[ui(t)],E[ui(t)]i

dt>0, ∀u(·)≡ u₁(·)

u₂(·)

∈Uad(x),

(4.16)

where Xi(·)∈X[0,∞)is the solution to MF-SDE (3.20).

(ii)System(4.10)admits a static stabilizing solution(P,Pb)∈Sⁿ×Sⁿ, such that the solution(η_o(·), ζ_o(·))∈ X[0,∞)×L²

F(Rⁿ)to (4.11)satisfies(4.12), and the solutionη¯_o(·)∈L²(Rⁿ)to (4.13)satisfies(4.14).

In the above case, any open-loop saddle point admits the closed-loop representation(4.15).

The following result characterizes the closed-loop saddle points of Problem (MF-SDG)₀.

Theorem 4.4. Problem (MF-SDG)0admits a closed-loop saddle point(Θ^∗,Θ¯^∗, v^∗(·))if and only if the following statements hold:

(i)The following system:







P_cA+A^>P_c+C^>P_cC+Q−(P_cB+C^>P_cD+S^>)Σ^†_c(B^>P_c+D^>P_cC+S) = 0, PbcAb+Ab^>Pbc+Cb^>PcCb+Qb−(PbcBb+Cb^>PcDb+Sb^>) ¯Σ^†_c(Bb^>Pbc+Db^>PcCb+Sb) = 0, R(B^>Pc+D^>PcC+S)⊆R(Σc), R(Bb^>Pbc+Db^>PcCb+Sb)⊆R( ¯Σc),

(4.17)

admits a static stabilizing solution (Pc,Pbc)∈Sⁿ×Sⁿ such that

(R11+D^>₁PcD1>0, Rb11+Db^>₁PcDb1>0,

R₂₂+D^>₂P_cD₂60, Rb₂₂+Db^>₂P_cDb₂60. (4.18) (ii)The following BSDE on[0,∞):











−dηc(t) =

A^>ηc(t)− PcB+C^>PcD+S^>

Σ^†_c

B^>ηc(t) +D^> ζc(t) +Pcσ(t) +ρ(t) +C^>

ζc(t) +Pcσ(t)

+Pcb(t) +q(t) dt−ζc(t)dW(t), t>0, B^>

η_c(t)−E[η_c(t)]

+D^>

ζ_c(t)−E[ζ_c(t)]

+D^>P_c

σ(t)−E[σ(t)]

+ρ(t)−E[ρ(t)]∈R Σ_c , a.e. t∈[0,∞), a.s.,

(4.19)

admits a solution(η_c(·), ζ_c(·))∈X[0,∞)×L²

F(Rⁿ), and the following ODE:







˙¯

η_c(t) +Ab^>η¯_c(t)− PbBb+Cb^>P_cDb+Sb^>Σ¯^†_c

Bb^>η¯_c(t) +Db^> E[ ¯ζ_o(t)] +P_cE[σ(t)]

+E[ ¯ρ(t)]

+Cb^> E[ ¯ζo(t)] +PcE[σ(t)]

+E[q(t)] +PbcE[b(t)], t>0,

Bb^>η¯c(t) +Db^>E[ζc(t)] +Db^>PcE[σ(t)] +E[ρ(t)]∈R( ¯Σc), a.e.t∈[0,∞),

(4.20)

admits a solutionη¯c(·)∈L²(Rⁿ). In the above case, the closed-loop saddle point is given by ( Θ^∗=−Σ^†_c(B^>Pc+D^>PcC+S) + I−Σ^†_cΣc

θ, Θ¯^∗=−Σ¯^†_c

Bb^>Πc+Db^>PcCb+Sb

+ I−Σ¯^†_cΣ¯cθ,¯ (4.21) where θ,θ¯∈R^m×n are chosen such that(Θ^∗,Θ¯^∗)∈S[A,A, C,¯ C;¯ B,B, D,¯ D], and¯

v^∗(·) =−Σ^†_c

B^> η_c(·)−E[η_c(·)]

+D^> ζ_c(·)−E[ζ_c(·)]

+D^>P σ(·)−E[σ(·)]

+ρ(·)−E[ρ(·)]

−Σ¯^†_c

Bb^>η¯c(·) +Db^> PcE[σ(·)] +E[ζc(·)]

+E[ρ(·)]

+ I−Σ^†_cΣ_c

ν(·)−E[ν(·)]

+ I−Σ¯^†_cΣ¯_c ν(·),¯

(4.22)

for some ν(·)∈L²_F(R^m),ν(·)¯ ∈L²(R^m).

Proof. Let (Θ^∗, v^∗(·)) be a closed-loop saddle point of Problem (MF-SDG)₀. System (3.35) for (P₁,Pb₁) ≡ (−P2,−Pb2) = (Pc,Pbc) becomes











P_cA+A^>P_c+C^>P_cC+Q+ (Θ^∗)^>(R+D^>P_cD)Θ^∗

+(PcB+C^>PcD+S^>)Θ^∗+ (Θ^∗)^>(B^>Pc+D^>PcC+S) = 0, Pb_cAb+Ab^>Pb_c+Cb^>P_cCb+Qb+ ( ¯Θ^∗)^>(Rb+Db^>P_cD) ¯b Θ^∗

+ PbcBb+Cb^>PcDb+Sb^>Θ¯^∗+ ( ¯Θ^∗)^> Bb^>Pbc+Db^>PcCb+Sb

= 0.

(4.23)

Thus, (3.36) becomes

(B^>P_c+D^>P_cC+S+ Σ_cΘ^∗= 0, Bb^>Pbc+Db^>PcCb+Sb+ ¯ΣcΘ¯^∗= 0.

This system is solvable if and only if (R B^>Pc+D^>PcC+S

⊆R(Σc), Θ^∗=−Σ^†_c(B^>Pc+D^>PcC+S) + I−Σ^†_cΣc θ, R Bb^>Pbc+Db^>PcCb+Sb

⊆R( ¯Σc), Θ¯^∗=−Σ¯^†_c Bb^>Pbc+Db^>PcCb+Sb

+ I−Σ¯^†_cΣ¯c

θ,¯ (4.24) whereθ,θ¯∈R^m×nare chosen such that (Θ^∗,Θ¯^∗)∈S[A,A, C,¯ C;¯ B,B, D,¯ D]. This proves (4.21). Putting (4.24)¯ into (4.23) yields that (4.17) admits a static stabilizing solution (P_c,Pb_c)∈Sⁿ×Sⁿ. (4.18) can be easily obtained from (3.37) and (4.1). (4.19) and (4.20) can be similarly proved from (ii) of Theorem3.7.

Finally, from (3.44) we have v^∗(·)−E[v^∗(·)] =−Σ^†_c

B^> ηc(·)−E[ηc(·)]

+D^> ζc(·)−E[ζc(·)]

+D^>Pc σ(·)−E[σ(·)]

+ρ(·)−E[ρ(·)] + I−Σ^†_cΣc

ν(·)−E[ν(·)]

, (4.25)

for some ν(·)∈L²_F(R^m). It follows from (3.46) that we get E[v^∗(·)] =−Σ¯^†_c

Bb^>η¯_c(·) +Db^> P_cE[σ(·)] +E[ζ_c(·)]

+E[ρ(·)] + I−Σ¯^†_cΣ¯_c

ν(·),¯ (4.26) for some ¯ν(·)∈L²(R^m). Combining the above two expressions leads to (4.22). The proof is complete.

Comparing Theorems4.3and4.4, we have the following result which might not be true for general non-zero sum differential games (see [47]).

Theorem 4.5. If both the closed-loop representation of open-loop saddle points and the closed-loop saddle points of Problem (MF-SDG)0 exist, then the closed-loop representation coincides with the outcome of the closed-loop saddle points. In the above case, the value function admits the following representation:

V(x) = P x, xb

+ 2

¯ η_c(0), x +E

Z ∞ 0

P σ(t), σ(t) + 2

ηc(t), b(t)−E[b(t)]

+ 2

ζc(t), σ(t) + 2

ηc(t),E[b(t)]

− Σ^†¹₂

B^>(η_c(t)−E[η_c(t)]) +D^>

ζ_c(t)−E[ζ_c(t)] +P σ(t)−E[σ(t)]

+ρ(t)−E[ρ(t)]

− Σ¯^†¹₂

Bb^>η¯c(t) +Db^> E[ζc(t)] +PE[σ(t)]

+E[ρ(t)]

2i dt,

(4.27)

where Σ≡Σo≡Σc,R+D^>P D≡R+D^>PcD,Σ¯ ≡Σ¯o≡Σ¯c,Rb+Db^>PDb ≡Rb+Db^>PcD,b Pb≡Pbc.

Proof. Let (Θ^∗, v^∗(·)) be a closed-loop saddle point of Problem (MF-SDG)0. By (4.6), the outcome u^∗(·) = Θ^∗ X^∗(·)−E[X^∗(·)]

+ ¯Θ^∗E[X^∗(·)] +v^∗(·),

of (Θ^∗,Θ¯^∗, v^∗(·)) is an open-loop saddle point of Problem (MF-SDG)0, whereX^∗(·)∈X[0,∞) is the solution to







dX^∗(t) =

AΘ^∗X^∗+ ¯AΘ^∗E[X^∗] +Bv^∗+ ¯BE[v^∗] +b dt +

CΘ^∗X^∗+ ¯CΘ^∗E[X^∗] +Dv^∗+ ¯DE[v^∗] +σ dW(t), t>0, X^∗(0) =x.

Thus, similar to the proof of the sufficiency on Theorem3.3, noting (4.25) and (4.26), we have

J x; Θ^∗ X^∗(·)−E[X^∗(·)]

+ ¯Θ^∗E[X^∗(·)] +v^∗(·)

−

Pb_cx+ 2¯η_c(0), x

=E Z ∞

Σc v^∗−E[v^∗]

, v^∗−E[v^∗] + 2

B^>ηc+D^>(ζc+Pcσ) +ρ, v^∗−E[v^∗] +Σ¯_cE[v^∗],E[v^∗]

+ 2

Bb^>η¯_c+Db^> E[ζ_c] +P_cE[σ]

+E[ρ],E[v^∗] +

P_cσ, σ + 2

η_c, b−E[b]

+ 2 ζ_c, σ

+ 2

η_c,E[b]i dt

=E Z ∞

h Pcσ, σ

+ 2

ηc, b−E[b]

+ 2 ζc, σ

+ 2

¯ ηc,E[b]

− (Σ^†_c)¹²

B^>(η_c−E[η_c]) +D^>(ζ_c−E[ζ_c]) +D^>P_c(σ−E[σ]) +ρ−E[ρ]

− ( ¯Σ^†_c)¹²

Bb^>η¯_c+Db^> P_cE[σ] +E[ζ_c]

+E[ρ]

2i dt,

(4.28)

where (Pc,Pbc) satisfies (4.17), (ηc(·), ζc(·)) satisfies (4.19) and ¯ηc(·) satisfies (4.20). By Theorem 4.4, (Θ^∗,Θ¯^∗, v^∗(·)) is given by (4.21)–(4.22).

On the other hand, let (P,P) be the solution pair to (4.10), (ηb o(·), ζo(·)) be the solution to (4.11), ¯ηo(·) be the solution to (4.13), and choose θ,θ¯∈R^m×n, ν(·),ν¯(·)∈L²(R^m), such that the following







Θ^∗∗ =−Σ^†_o(B^>P+D^>P C+S) + I−Σ^†_oΣo

θ, Θ¯^∗∗=−Σ¯^†_o Bb^>Pb+Db^>PCb+Sb

+ I−Σ¯^†_oΣ¯o

θ,¯ v^∗∗(·) =−Σ^†_o

B^> ηo(·)−E[ηo(·)]

+D^> ζo(·)−E[ζo(·)]

+D^>P σ(·)−E[σ(·)]

+ρ(·)−E[ρ(·)]

−Σ¯^†_o

Bb^>η¯o(·)+Db^>PE[σ(·)]+E[ζo(·)]

+E[ρ(·)] + I−Σ^†_oΣo

ν(·)−E[ν(·)]

+ I−Σ¯^†_oΣ¯o

ν(·),¯

satisfies (Θ^∗∗,Θ¯^∗∗)∈S[A,A, C,¯ C;¯ B,B, D,¯ D]. For any initial state¯ x, defineu^∗∗(·)∈Uad(x) as follows:

u^∗∗(·) = Θ^∗∗

X^∗∗(·)−E[X^∗∗(·)] + ¯Θ^∗∗E[X^∗∗(·)] +v^∗∗(·),

where X^∗∗(·)∈X[0,∞) is the solution to







dX^∗∗(t) =

AΘ^∗∗X^∗∗+ ¯AΘ^∗∗E[X^∗∗] +Bv^∗∗+ ¯BE[v^∗∗] +b dt +

C_Θ^∗∗X^∗∗+ ¯C_Θ^∗∗E[X^∗∗] +Dv^∗∗+ ¯DE[v^∗∗] +σ dW(t), t>0, X^∗∗(0) =x.

By Theorem 4.3, u^∗∗(·) is an open-loop saddle point of Problem (MF-SDG)0 forx. By the same argument as (4.28), noting (4.12) and (4.14), we obtain

J x; Θ^∗∗ X^∗∗(·)−E[X^∗∗(·)]

+ ¯Θ^∗∗E[X^∗∗(·)] +v^∗∗(·)

−

P xb + 2¯η_o(0), x

=E Z ∞

Σ_o(v^∗∗−E[v^∗∗]), v^∗∗−E[v^∗∗] + 2

B^>η_o+D^>ζ_o+D^>P σ+ρ, v^∗∗−E[v^∗∗] +Σ¯oE[v^∗∗],E[v^∗∗]

+ 2

Bb^>η¯o+Db^>PE[σ] +E[ζo]) +E[ρ],E[v^∗∗] +

P σ, σ + 2

η_o, b−E[b]

+ 2 ζ_o, σ

+ 2

η_o,E[b]i dt

=E Z ∞

h P σ, σ

+ 2

ηo, b−E[b]

+ 2 ζo, σ

+ 2

¯ ηo,E[b]

− (Σ^†_o)¹²

B^>(η_o−E[η_o]) +D^>(ζ_o−E[ζ_o]) +D^>P(σ−E[σ]) +ρ−E[ρ]

− ( ¯Σ^†_o)¹²

Bb^>η¯_o+Db^> PE[σ] +E[ζ_o]

+E[ρ]

2i dt.

(4.29)

Since bothu^∗∗(·)≡(u^∗∗₁ (·), u^∗∗₂ (·)) andu^∗(·)≡(u^∗₁(·), u^∗₂(·)) are open-loop saddle points of Problem (MF-SDG)0

forx, we have

J(x;u^∗₁(·), u^∗₂(·))6J(x;u^∗∗₁ (·), u^∗₂(·))6J(x;u^∗∗₁ (·), u^∗∗₂ (·))6J(x;u^∗₁(·), u^∗∗₂ (·))6J(x;u^∗₁(·), u^∗₂(·)).

Therefore,J(x;u^∗(·)) =J(x;u^∗∗(·)) for allx, which, together with (4.28) and (4.29), yields Pc=P, Pbc=P ,b ηc(·) =ηo(·), ζc(·) =ζo(·), η¯c(·) = ¯ηo(·).

Thus, the value function is given by (4.27). The proof is complete.

Finally, we have the following corollary for Problem SLQ), since it is a special case of Problem (MF-SDG)0whenm2= 0.

Corollary 4.6. For Problem(MF-SLQ), if the open-loop optimal control admits a closed-loop representation, then each open-loop optimal control must be an outcome of a closed-loop optimal strategy.

5. Examples

In this section, we present several examples illustrating the results established in the previous sections.

We mainly concentrate the closed-loop representation of the open-loop Nash equilibria/saddle points and the closed-loop Nash equilibria/saddle points.

The following example shows that for the mean-field LQ zero-sum stochastic differential game, it may happen that the closed-loop saddle point uniquely exists, which coincides with the closed-loop representation of the open-loop saddle point. Moreover, the system of generalized AREs admits static stabilizing solutions and the system [A,A, C,¯ C;¯ B,B, D,¯ D] is MF-L¯ ²-stabilizable.

Example 5.1. Consider the following two-dimensional state equation







dX(t) =

AX(t) + ¯AE[X(t)] +Bu(t) + ¯BE[u(t)] dt +

CX(t) + ¯CE[X(t)] +Du(t) + ¯DE[u(t)] dW(t), t>0, X(0) =x,

(5.1)

with the cost functional

and from (2.15), we have



First, we look at closed-loop representation of open-loop saddle point. According to (4.10), we have the following coupled AREs:

The above solutions satisfy (4.7) and (4.8). Further, since Σ_o and ¯Σ_oare invertible, we have



According to Proposition A.5 in [23], the system [A,A, C,¯ C;¯ B,B, D,¯ D] above is MF-L¯ ²-stabilizable, and

such that the negative definiteness conditions of (5.5) hold. Then (Θ^∗,Θ¯^∗) above is an MF-L²-stabilizer of the system [A,A, C,¯ C;¯ B,B, D,¯ D]. Now, we solve corresponding systems (4.11)–(4.14) to get (η¯ o(·), ζo(·)) and ¯ηo(·).

From (4.15), the closed-loop representation of this open-loop saddle point can be given by u^∗(·) =−Σ⁻¹_o (B^>P+D^>P C+S) X^∗(·)−E[X^∗(·)]

Next, we consider the closed-loop saddle point. Note that the corresponding system of generalized AREs (4.17) reads

Also, the range condition

R(B^>Pc+D^>PcC+S)⊆R(Σc), R(Bb^>Pbc+Db^>PcCb+S)b ⊆R( ¯Σc)

holds automatically since Σ_c and ¯Σ_c are invertible. We see that











Θ^∗=−Σ⁻¹_c (B^>Pc+D^>PcC+S) = −0.6667 −0.1667 0.0263 0.1053

! , Θ¯^∗=−Σ¯⁻¹_c (Bb^>Pbc+Db^>PcCb+S) =b

−1 0 0 0.3956

(5.9)

which coincides with (5.4). Hence, by Theorem 4.4, the problem admits a closed-loop saddle point. Solving corresponding backward systems (4.19)–(4.20), we can get (ηc(·), ζc(·)) and ¯ηc(·), the closed-loop saddle point is given by (5.9) and

v^∗(·) =−Σ⁻¹_c

B^> ηc(·)−E[ηc(·)]

+D^> ζc(·)−E[ζc(·)] −Σ¯⁻¹_c

Bb^>η¯c(·) +Db^>E[ζc(·)] . (5.10) Since P =Pc and Pb=Pbc, we get Σ = Σc,Σ = ¯¯ Σc and ηo(·) =ηc(·), ζo(·) =ζc(·), ¯ηo(·) = ¯ηc(·). This implies that the closed-loop representation of the open-loop saddle point coincides with the closed-loop saddle point.

The following example shows that for the mean-field LQ non-zero sum stochastic differential game, it may happen that the closed-loop representations of open-loop Nash equilibria are different from the closed-loop Nash equilibria. However, the solutions to the system of algebraic Riccati equations for the closed-loop representation of open-loop Nash equilibrium are symmetric.

Example 5.2. Consider the following two-dimensional state equation







dX(t) =

AX(t) + ¯AE[X(t)] +Bu(t) + ¯BE[u(t)] dt +

CX(t) + ¯CE[X(t)] +Du(t) + ¯DE[u(t)] dW(t), t>0, X(0) =x,

(5.11)

with the cost functional











J₁(x;u₁(·), u₂(·)) =E Z ∞

Q₁ S₁^>

S₁ R₁

X(t) u(t)

, X(t)

u(t)

Q¯1 S¯₁^>

S¯1 R¯1

E[X(t)]

E[u(t)]

E[X(t)]

E[u(t)]

dt, J₂(x;u₁(·), u₂(·)) =E

Z ∞ 0

Q₂ S₂^>

S2 R2

X(t) u(t)

, X(t)

u(t)

Q¯2 S¯₂^>

S¯2 R¯2

E[X(t)]

E[u(t)]

E[X(t)]

E[u(t)]

dt.

(5.12)

We let and from (2.15), we have



To look at the closed-loop representation of open-loop Nash equilibria, we solve the corresponding (3.33)–(3.34) to get

are invertible, it follows that

Θ^∗∗ = −1 −0.25 By Theorem 3.3, making use of (3.27)–(3.30), the problem admits an open-loop Nash equilibriumu^∗(·) for any initial statex∈Rⁿ. And it has the closed-loop representation (2.7) with (Θ^∗∗,Θ¯^∗∗, v^∗∗(·)) given by (5.13) and (5.14).

Next, we consider the closed-loop Nash equilibria. According to Proposition A.5 in [23], system [A,A, C,¯ C;¯

Fori= 1,2, we solve the corresponding (3.35)–(3.37) to get



such that the negative definiteness conditions of (5.15) hold. Then (Θ^∗,Θ¯^∗) above is an MF-L²-stabilizer of the system [A,A, C,¯ C;¯ B,B, D,¯ D]. Hence, by Theorem¯ 3.7, the problem admits a closed-loop Nash equilibrium.

Now, since

by Remark 2.7, we see that the closed-loop representation of the open-loop Nash equilibrium and the closed-loop Nash equilibrium are intrinsically different.

From this example, we see that the solutions to system (3.33)–(3.34), which come from the open-loop Nash equilibrium, may be symmetric, but they are different from the solution to the system of AREs (3.35)–(3.37), which come from the closed-loop Nash equilibrium. It is obvious that the closed-loop representation of the open-loop Nash equilibrium is different from the closed-loop Nash equilibrium.

The following example shows that for the closed-loop representation of open-loop Nash equilibria of the mean-field LQ non-zero sum stochastic differential game, it may happen that the solutions to the system of AREs may be asymmetric when the system [A,A, C,¯ C;¯ B,B, D,¯ D] is MF-L¯ ²-stabilizable. And, the closed-loop representation of the open-loop Nash equilibrium is different from the closed-loop Nash equilibrium.

Example 5.3. Consider the two-dimensional state equation of form (5.11) with cost functionals (5.12), but we let

To look at the closed-representation of open-loop Nash equilibria, we solve the corresponding (3.33)–(3.34) to get

are invertible, it follows that

Then, similar to Example 5.2, making use of Theorem3.3, we see that the problem admits an open-loop Nash equilibrium for any initial state x∈Rⁿ. And it has the closed-loop representation of form (2.7).

Next, we consider the closed-loop Nash equilibria. According to Proposition A.5 in [23], the system [A,A, C,¯ C;¯ B,B, D,¯ D] above is MF-L¯ ²-stabilizable, and (Θ,Θ)¯ ∈S[A,A, C,¯ C;¯ B,B, D,¯ D] if and only if there exist¯ P0>0

Fori= 1,2, we solve corresponding (3.35)–(3.37) to get



such that the negative definiteness conditions of (5.18) hold. Then (Θ^∗,Θ¯^∗) above is an MF-L²-stabilizer of the system [A,A, C,¯ C;¯ B,B, D,¯ D]. Hence, by Theorem¯ 3.7, the problem admits a closed-loop Nash equilibrium.

Finally, similar to Example 5.2, we easily check that the closed-loop representation of the optimal open-loop strategy and the optimal closed-loop strategy are intrinsically different.

From this example, we see that the solutions to system (3.33)–(3.34), which come from the open-loop saddle point, may be asymmetric, but the solutions to system (3.35)–(3.37), which come from the closed-loop saddle point, are still symmetric. It is obvious that the closed-loop representation of the open-loop Nash equilibrium is different from the closed-loop Nash equilibrium.

Dans le document , Jingtao Shi (Page 25-39)