LINEAR QUADRATIC OPTIMIZATION PROBLEMS FOR SOME DISCRETE-TIME STOCHASTIC LINEAR SYSTEMS

on the occasion of his 70th birthday

LINEAR QUADRATIC OPTIMIZATION PROBLEMS FOR SOME DISCRETE-TIME STOCHASTIC

LINEAR SYSTEMS

VASILE DRAGAN and TOADER MOROZAN

We investigate two problems of optimization of quadratic cost functions along the trajectories of a discrete-time linear system affected by Markov jump perturba- tions and independent random perturbations. Depending upon the class of admis- sible controls, the corresponding optimal control is obtained either as the minimal solution or as the maximal and stabilizing solution of a system of discrete-time Riccati type equations.

AMS 2000 Subject Classification: 93C55, 93E15, 93E20, 93D15.

Key words: linear quadratic problem, discrete-time stochastic system, Markov chain, independent random perturbations, discrete-time Riccati equa- tion.

1. INTRODUCTION

The discrete-time linear control systems have been intensively considered in the control literature in both the deterministic and the stochastic frame- work. This interest is wholly motivated by the wide area of applications in- cluding engineering, economics, and biology. The state space approach for the problem of minimization of a quadratic cost functional along the trajectories of a linear controlled system has a long history. Such an optimization problem is usually known as the linear quadratic optimization problem.

In the discrete-time stochastic framework, the linear quadratic optimiza- tion problem was separately investigated for systems with independent random perturbations and systems with Markov perturbations, respectively. For the case of discrete-time systems with independent random perturbations we re- fer to [24, 22, 26] while for discrete-time systems with Markov switching we mention [1]–[8], [15]–[21], [23, 25]. In [11] and [12] the problem of the opti- mization of a quadratic cost functional along the trajectories of a discrete-time linear system subject to Markov and independent random perturbations was

MATH. REPORTS11(61),4 (2009), 307–319

investigated and was used to solve the problem of the tracking of a given reference signal.

In this paper two problems of optimization of quadratic cost functions along the trajectories of a discrete-time linear system affected by Markov jump perturbations and independent random perturbations are investigated. De- pending upon the class of admissible controls, the corresponding optimal con- trol is obtained either as the minimal solution or as the maximal and stabilizing solution of a system of discrete-time Riccati type equations.

2. PROBLEM FORMULATION

Consider the discrete-time controlled system x(t+ 1) =h

A₀(t, η_t) +

r

X

k=1

w_k(t)A_k(t, η_t)i x(t)+

(1)

+h

B₀(t, η_t) +

r

X

k=1

w_k(t)B_k(t, η_t)i u(t),

wherex(t)∈Rⁿ is the state vector,u(t)∈R^m is the vector of the control pa- rameters,ηt,t∈Z+, is a Markov chain on a given probability space{Ω,F,P}

with transition matrices P_t = [p_t(i, j)], t ≥ 0, and state space the finite set D={1,2, . . . , N} and {w(t)}_t≥0,w(t) = (w₁(t), . . . , w_r(t))^T, is a sequence of independent random vectors.

The superscript T stands for the transpose of a matrix or a vector.

We introduce theσ-algebrasF_t=σ[w(s),0≤s≤t],G_t=σ[η_s,0≤s≤ t],H_t=F_t∨ G_t,He_t=G_t∨ F_t−1 ift≥1 andHe₀ =σ[η0].

Throughout the paper we assume that F_t is independent of G_t for each t∈Z+,E[w(t)] = 0, E[w(t)w^T(t)] =Ir,t≥0.

We associate with system (1) the two cost functionals (2) J1(t0, x0, u) =

∞

X

t=t0

E

|C(t, η_t)xu(t, t0, x0)|²+|D(t, η_t)u(t)|² ,

J2(t0, x0, u) =

∞

X

t=t0

E h

x^T_u(t, t0, x0)M(t, ηt)xu(t, t0, x0)+

(3)

+2x^T_u(t, t₀, x₀)L(t, η_t)u(t) +u^T(t)R(t, η_t)u(t)i ,

where x_u(t, t₀, x₀) is the solution of (1) corresponding to the control u, with x(t₀, t₀, x₀) =x₀,t₀∈Z₊,x₀∈Rⁿ,M(t, i) =M^T(t, i),R(t, i) =R^T(t, i). Throu- ghout the paper we assume that the sequences {A_k(t, i)}t≥0, {B_k(t, i)}t≥0,

{C(t, i)}t≥0, {D(t, i)}t≥0, {M(t, i)}t≥0, {R(t, i)}t≥0, {L(t, i)}t≥0, i∈ D, 0 ≤ k ≤ r, are bounded. Also, we assume that D^T(t, i)D(t, i) 0. This means that there exists δ >0 such thatD^T(t, i)D(t, i)≥δI_m for all t∈Z₊,i∈ D.

Two classes of admissible controls will be considered in the paper, namely,

• U₁(t₀, x₀) is the set of all sequences u ={u(t)}_t≥t0 of m-dimensional random vectors u(t) such that u(t) is He_t-measurable, E|u(t)|² <∞ and the series (2) is convergent.

• U₂(t₀, x₀) is the set of all sequences u ={u(t)}_t≥t0 of m-dimensional random vectorsu(t) such thatu(t) isHe_t-measurable,E|u(t)|² <∞, the series (3) is convergent and

(4) lim

t→∞E|x_u(t, t₀, x₀)|² = 0.

Now, we are in a position to formulate the optimization problems which are solved in this paper:

OP1. Given t₀ ∈ Z₊ and x₀ ∈ Rⁿ, find ue ∈ U₁(t₀, x₀) such that J1(t0, x0,u)e ≤J1(t0, x0, u) for all u∈ U₁(t0, x0).

OP2. Given t₀ ∈ Z₊ and x₀ ∈ Rⁿ, find e

ue ∈ U₂(t₀, x₀) such that J₂(t₀, x₀,e

u)e ≤J₂(t₀, x₀, u) for all u∈ U₂(t₀, x₀).

3. THE SOLUTION OF OP1

Let S_n be the space of n×n symmetric matrices andS_n^N =S_n⊕ S_n⊕

· · · ⊕ S_n, that is a real Hilbert space with the inner product hX, Yi=

N

X

i=1

Tr[X(i)Y(i)]; X= (X(1), . . . , X(N)), Y= (Y(1), . . . , Y(N))∈ S_n^N. Given F(t) = (F(t,1), . . . , F(t, N)), F(t, i)∈R^m×n,t∈Z₊,i∈ D, we define the Lyapunov type operator L_F(t) on S_n^N, as

(L_F(t)X)(i) = (5)

=

N

X

j=1 r

X

k=0

p_t(j, i)[A_k(t, j) +B_k(t, j)F(t, j)]X(j)[A_k(t, j) +B_k(t, j)F(t, j)]^T for X ∈ S_n^N, i ∈ D. Set T_F(t, s) = L_F(t−1)· · · L_F(s) if t > s ≥ 0 and T_F(t, s) =I_SN

n ift=s, whereI_SN

n is the identity operator on S_n^N.

Definition1. We say that the system (1) is stochastic stabilizable if there exists a bounded sequence {F(t)}_t≥0 such that kT_F(t, s)k ≤βq^t−s,t≥s≥0, for some β≥1, q∈(0,1).

If the sequence{F(t)}t≥0 has the above property, then we shall say that it is a stabilizing feedback gain for system (1).

Remark1. It follows from Theorem 3.6 in [10] that ifF(t) is a stabilizing feedback gain for system (1), then there exist β≥1 andq ∈(0,1) such that

E[|x_F(t, t0, x0)|²]≤βq^t−t0|x₀|²

for all t ≥ t0 ≥ 0, x0 ∈Rⁿ, where xF(t, t0, x0) is the solution of system (1) corresponding to the control u(t) = F(t, η_t)x_F(t), t ≥ t₀. Therefore, if the system (1) is stochastic stabilizable, then the set U₂(t0, x0) is not empty for each t0 ∈Z+,x0∈Rⁿ.

ForX∈ S_n^N,t∈Z₊,i∈ D, let us consider the linear operators Π1i(t)X=

r

X

k=0

A^T_k(t, i)E_i(t, X)A_k(t, i), Π2i(t)X=

r

X

k=0

A^T_k(t, i)E_i(t, X)B_k(t, i),

Π_3i(t)X=

r

X

k=0

B_k^T(t, i)E_i(t, X)B_k(t, i), E_i(t, X) =

N

X

j=1

p_t(i, j)X(j).

With the above notation we introduce the discrete-time system X(t, i) = Π_1i(t)X(t+ 1) +C^T(t, i)C(t, i)−

(6)

−[Π_2i(t)X(t+ 1)][D^T(t, i)D(t, i) + Π_3i(t)X(t+ 1)]⁻¹[Π_2i(t)X(t+ 1)]^T of generalized Riccati equations (DTSGRE).

Theorem 1. Assume that system(1)is stochastic stabilizable. Then the optimal control of OP1 is given by eu(t) =Fe(t, η_t)ex(t), where

(7) Fe(t, i) =−[D^T(t, i)D(t, i) + Π_3i(t)X_min(t+ 1)]⁻¹[Π_2i(t)X_min(t+ 1)]^T withX_min(t)the minimal bounded solution of(6)andex(t) =x

Fe(t),ex(t₀) =x₀. The optimal value of the cost functional is

J₁(t₀, x₀,eu) =

N

X

i=1

π_t0(i)x^T₀X_min(t₀, i)x₀, π_t0(i) =P{η_t0 =i}.

Proof. Since system (1) is stochastic stabilizable, by Theorem 6.1 in [14]

the DTSGRE (6) has a positive semidefinite bounded solution Xmin(t) which is minimal in the class of positive semidefinite bounded solutions of (6). Also, it is known that X_min(t, i) = lim

τ→∞X_τ(t, i), whereX_τ(t, i), 0≤t≤τ,i∈ D, is the positive semidefinite solution of (6) with final value Xτ(τ, i) = 0.

By Lemma 3.2 in [11], forv(t, x, i) =x^TXmin(t, i)xwe have

τ

X

t=t0

E[(|C(t, η_t)ex(t)|²+|D(t, η_t)u(t)|e ²)|η_t0 =i] = (8)

=x^T₀X_min(t₀, i)x₀−E[xe^T(τ + 1)X_min(τ + 1, η_τ+1)x(τe + 1)|η_t0 =i]

for all τ ≥t₀,i∈ D_t0 where D_s={i∈ D |π_s(i)>0}for each s∈Z₊.

Since Xmin(τ, i) ≥ 0 and Xmin(t) is bounded, from (8) we have ue ∈ U₁(t₀, x₀) and

(9) J1(t0, x0,u)e ≤

N

X

i=1

πt0(i)x^T₀Xmin(t0, i)x0.

Further, by Lemma 3.2 in [11], for v(t, x, i) =x^TXτ(t, i)x we have

τ−1

X

t=t0

E[(|C(t, η_t)xu(t)|²+|D(t, η_t)u(t)|²)|η_t0 =i] =

=x^T₀X_τ(t₀, i)x₀+

τ−1

X

t=t0

E

(u(t)−F_τ(t, η_t)x_u(t))^T(D^T(t, η_t)D(t, η_t)+

+ Π3ηt(t)Xτ(t+ 1))(u(t)−F_τ(t, ηt)xu(t))|η_t0 =i ,

where F_τ(t, i) is the feedback gain associated with X_τ(t, i) constructed as in (7) with Xτ(t, i) instead of Xmin(t, i). Since Xτ(t, i)≥0, we can write (10)

τ−1

X

t=t0

E[(|C(t, η_t)xu(t)|²+|D(t, η_t)u(t)|²)|η_t0 =i]≥x^T₀X(t, i)x0. Letting τ → ∞in (10), we obtain

(11) J1(t0, x0, u)≥ X

i∈Dt0

πt0(i)x^T₀Xmin(t0, i)x0. Writing (11) for u(t) =u(t) and taking into account (9) we gete

J1(t0, x0, u)≥

N

X

i=1

πt0(i)x^T₀Xmin(t0, i)x0 =J1(t0, x0,eu).

This shows that u(t) is the optimal control which completes the proof.e

4. THE SOLUTION OF OP2

Since in (3) no assumption concerning the sign of the weighting matri- ces M(t, i), L(t, i) and R(t, i) was made, it is possible that J₂(t₀, x₀,·) be unbounded from bellow.

Let V(t0, x0) = inf

u∈U2(t0,x0)J2(t0, x0, u), (t0, x0) ∈ Z+×Rⁿ be the value function associated toOP2.

Definition 2. We say that OP2 is well possed if −∞ < V(t₀, x₀) < ∞ for all t0 ∈Z+ and x0 ∈Rⁿ.

To make clearer the statement of the next results we adopt the notation:

Q(t) = (Q(t,1), . . . , Q(t, N)),Π(t)X= ((Π(t)X)(1), . . . ,(Π(t)X)(N))∈ S_n+m^N , Q(t, i) =

M(t, i) L(t, i) L^T(t, i) R(t, i)

, (Π(t)X)(i) =

Π_1i(t)X Π_2i(t)X (Π_2i(t)X)^T Π_3i(t)X

. With the pair Σ = (Π, Q) we associate the so called dissipation operator D^Σ:`<sup>∞</sup>(Z+,S<sub>n</sub><sup>N</sup>)→`^∞(Z+,S_n+m^N ) and the subsets Γ^Σ andeΓ^Σ of`^∞(Z+,S_n^N) by

(12) (D^ΣX)(t) = ((D₁^ΣX)(t),(D₂^ΣX)(t), . . . ,(D^Σ_NX)(t)), (D^Σ_i X)(t) =

Π1i(t)X(t+ 1) +M(t, i)−X(t, i) L(t, i) + Π2i(t)X(t+ 1) (L(t, i) + Π2i(t)X(t+ 1))^T R(t, i) + Π3i(t)X(t+ 1)

for arbitrary X={X(t)}_t≥0∈`^∞(Z+,S_n^N),

Γ^Σ ={X={X(t)}_t≥0 ∈`^∞(Z+,S_n^N)|(D^ΣX)(t)≥0, (13)

R(t) + Π3(t)X(t+ 1)0, t≥0},

(14) eΓ^Σ={X={X(t)}_t≥0 ∈`^∞(Z+,S_n^N)|(D^ΣX)(t)0, t≥0}.

Let us consider the system

X(t, i) = Π1i(t)X(t+ 1) +M(t, i)−[L(t, i) + Π2i(t)X(t+ 1)][R(t, i)+

(15)

Π_3i(t)X(t+ 1)]⁻¹[L(t, i) + Π_2i(t)X(t+ 1)]^T, t∈Z₊, i∈ D of discrete-time Riccati equations.

Definition 3. We say that X_s(t) = (X_s(t,1), . . . , X_s(t, N)), t∈Z₊, is a stabilizing solution of (15) if

(16) F^Xs(t, i) =−[R(t, i) + Π_3i(t)X_s(t+ 1)]⁻¹[L(t, i) + Π_2i(t)X_s(t+ 1)]^T is a stabilizing feedback gain for system (1).

Theorem 2. Assume that

a)the system (1) is stochastic stabilizable;

b) the set Γ^Σ is not empty.

Then OP2 problem is well possed. Moreover, we have

(17) V(t₀, x₀) =

N

X

i=1

π_t0(i)x^T₀X_max(t₀, i)x₀

for all t0 ∈ Z+ and x0 ∈ Rⁿ, where {X_max(t)}t≥0 is the maximal bounded solution of DTSGRE (15) which verifies

(18) R(t, i) + Π3i(t)Xmax(t+ 1)0, t∈Z+, i∈ D.

Proof. First, we remark that under assumptions a) and b), by Theo- rem 4.2 in [14] the DTSGRE (15) has a maximal and bounded solutionXmax(t) which satisfies condition (18). Also, it follows from Remark 1 that U₂(t0, x0) is not empty, ∀t₀ ∈ Z₊, x₀ ∈ Rⁿ. By Lemma 3.2 in [11], for v(t, x, i) = x^TX_max(t, i)x, whateveru∈ U₂(t₀, x₀) we have

τ−1

X

t=t0

E

"

xu(t) u(t)

T

Q(t, ηt)

xu(t) u(t)

+E[x^T_u(τ)Xmax(τ, ητ)xu(τ)

# (19)

= X

i∈D_t0

π_t0(i)x^T₀X_max(t₀, i)x₀+

τ−1

X

t=t0

E[(u(t)−Fe(t, η_t)x_u(t))^T(R(t, η_t)+

Π_3ηt(t)X_max(t+ 1))(u(t)−Fe(t, η_t)x_u(t))].

Since the left hand side of (19) converges for τ → ∞, the right hand side is also convergent. Lettingτ → ∞in (19) and taking into account (4) we obtain

J2(t0, x0, u) = X

i∈Dt0

πt0(i)x^T₀Xmax(t0, i)x0+ (20)

+

∞

X

t=t0

Eh

u(t)−Fe(t, η_t)x_u(t)T

R(t, η_t)+

+ Π3ηt(t)Xmax(t+ 1)

u(t)−Fe(t, ηt)xu(t)i

for all u ∈ U₂(t₀, x₀),(t₀, x₀) ∈ Z₊×Rⁿ. Further, (20) together with (18) imply

(21) J2(t0, x0, u)≥ X

i∈Dt0

πt0(i)x^T₀Xmax(t0, i)x0

for all u∈ U₂(t₀, x₀),(t₀, x₀)∈Z₊×Rⁿ. Hence (22) V(t₀, x₀)≥ X

i∈D_t0

π_t0(i)x^T₀X_max(t₀, i)x₀.

Thus, we deduce that the linear quadratic optimization problem under con- sideration is well-posed. It remains to show that in (22) we have equality. To this end, we choose a decreasing sequence of positive numbers {ε_j}j≥0 such

that lim

j→∞εj = 0. We associate the cost functionals (23) J^εj(t0, x0, u) =J2(t0, x0, u) +εj

∞

X

t=t0

E

|x_u(t)|² ,

u ∈Ue₂(t0, x0), where Ue₂(t0, x0) = {u ={u(t)}t≥0 ∈ U₂(t0, x0) |xu(t, t0, x0) is such that the series (23) is convergent}.

Let V^j(t0, x0) = inf

u∈Ue₂(t0,x0)

J^εj(t0, x0, u). Since Ue₂(t0, x0) ⊂ U₂(t0, x0) andJ₂(t₀, x₀, u)≤J^εj(t₀, x₀, u), u∈Ue₂(t₀, x₀), we deduce thatV^j(t₀, x₀)≥ V(t₀, x₀) for allj ≥0.

Consider the DTSGRE

X(t, i) = Π_1i(t)X(t+ 1) +M(t, i) +ε_jI_n−[L(t, i) + Π_2i(t)X(t+ 1)]× (24)

×[R(t, i) + Π3i(t)X(t+ 1)]⁻¹[L(t, i) + Π2i(t)X(t+ 1)]^T. It is defined by the pair Σj = Π(t), Q^j(t)

, where Π(t) is as before and Q^j(t) = Q^j(t,1), . . . , Q^j(t, N)

with Q^j(t, i) =

M(t, i) +εjIn L(t, i) L^T(t, i) R(t, i)

.

For each j ≥ 0,Γe^Σj is not empty since eΓ^Σj ⊃ Γ^Σ. By Theorem 5.4 in [14]

we deduce that for each j ≥0, DTSGRE (24) has a bounded and stabilizing solution Xs^j(t) = Xs^j(t,1), . . . , Xs^j(t, N)

, t ≥0. It follows from Proposition 5.1 in [14] that Xs^j(t) coincides with the maximal solution of (24). Further, from Theorem 4.3 in [14] we deduce that Xs^j(t, i)≥Xs^j+1(t, i)≥X_max(t, i) for all j ≥ 0 and lim

j→∞Xs^j(t, i) = Xmax(t, i) for all t ≥ 0, i ∈ D. As in the first part of the proof we deduce that

J₂^εj(t0, x0, u) = X

i∈Dt0

πt0(i)x^T₀X_s^j(t0, i)x0+

∞

X

t=t0

E h

u(t)−F_s^j(t, ηt)xu(t)T

(25)

× R(t, ηt) + Π3ηtX_s^j(t+ 1)

(u(t)−F_s^j(t, ηt)xu(t)i

for all u ∈ Ue₂(t0, x0), where Fs^j(t, i) = F^Xsj(t, i) is a stabilizing feedback associated with Xs^j(t). Take the control u^js(t) = Fs^j(t, ηt)x^js(t),{x^j_s(t)}_t≥t0, the solution of system (1) with u(t) replaced by u^js(t). Since Xs^j(t) is the stabilizing solution of (24), we have u^j_s = {u^j_s(t)}_t≥t0 ∈ Ue₂(t₀, x₀). Taking u=u^js in (25) we obtain

J₂^εj t0, x0, u^j_s

= X

i∈Dt0

πt0(i)x^T₀X_s^j(t0, i)x0.

This leads to V(t0, x0)≤V^j(t0, x0)≤ P

i∈Dt0

πt0(i)x^T₀Xs^j(t0, i)x0 for allj ≥0.

Letting j→ ∞ we obtain (26) V(t0, x0)≤ X

i∈Dt0

πt0(i)x^T₀Xmax(t0, i)x0, ∀(t0, x0)∈Z+×Rⁿ. From (26) and (22) we get (17) and the proof is complete.

Definition 4. We say that a control u_opt = {u_opt(t)}_t≥t0 ∈ U₂(t₀, x₀) is called an optimal control for the linear quadratic optimization problem under consideration if V(t0, x0) = J2(t0, x0, uopt) ≤ J2(t0, x0, u) for all u ∈ U₂(t₀, x₀).

The following result provides a sufficient condition for the existence of an optimal control for OP 2.

Proposition 3. If DTSGRE(15)has a bounded and stabilizing solution {X_s(t)}_t≥0 which satisfies

(27) R(t, i) + Π_3i(t)X_s(t+ 1)0,

then the linear quadratic optimization problem under consideration has an optimal control given by uopt(t) = Fs(t, ηt)xs(t), where Fs(t, i) is defined in (16) and {x_s(t)}_t≥t0 is the solution of system(1) for u(t) =Fs(t, ηt)xs(t) and the initial condition x_s(t₀) =x₀.

Proof. Since {X_s(t)}_t≥0 is the bounded and stabilizing solution of (15), the control u_opt =F_s(t, η_t)x_s(t) is admissible. The conclusion follows imme- diately from (20) for u=uopt and taking into account (27).

Now, we prove a result which provides a necessary and sufficient condition for the existence of an optimal control.

Theorem 4. Assume that

a)the assumptions of Theorem 2 are fulfilled;

b) PN

i=1pt(i, j)>0 for allt≥0 and j∈ D;

c)π0(i) =P{η₀ =i}>0 for 1≤i≤N. Then the following assertions are equivalent:

(i)for any (t0, x0)∈Z+×Rⁿ the optimization problemOP 2admits an optimal control bu_t0,x0(t), t≥t₀, that is V(t₀, x₀) =J₂(t₀, x₀,ub_t0,x0) ;

(ii)we have

(28) lim

t→∞

T

Fe(t, t0)

ξ= 0, ∀t₀ ∈Z+,

where T_Fe(t, t0) is the linear evolution operator onS_n^N defined by the sequence of Lyapunov operators {L

Fe(t)}_t≥0,L

Fe being defined by(5)withFe(t, i)instead of F(t, i) and Fe(t, i) =F^Xmax(t, i) andk · k_ξ is the Minkovski norm (see [13]).

If (i) or (ii) are fulfilled, then the optimal control of the problem under consideration is given by u_opt(t) =Fe(t, η_t)bx(t), where bx(t) is the solution of system (31) below.

Proof. Let us assume that (i) is fulfilled. Let (t₀, x₀) ∈ Z₊×Rⁿ and ub={bu(t)}t≥t₀ ∈ U₂(t₀, x₀) be such thatV(t₀, x₀) =J₂(t₀, x₀,bu). From (20) we get

V(t₀, x₀) =X

i∈D

π_t0(i)x^T₀X_max(t₀, i)x₀+

∞

X

t=t0

E[(u(t)b −Fe(t, η_t)x(t))b ^T (29)

×(R(t, η_t) + Π3ηt(t)Xmax(t+ 1))(u(t)b −Fe(t, ηt)x(t))],b where bx=x

ub(t) is the optimal trajectory. Combining (17) and (29) yields

∞

X

t=t0

Eh

u(t)b −Fe(t, η_t)bx(t)T

(30) ×

×(R(t, η_t) + Π3ηt(t)Xmax(t+ 1)) u(t)b −Fe(t, ηt)bx(t)i

= 0.

On account of (27), the last equation leads to u(t) =b Fe(t, η_t)x(t) a.s.b t≥t₀. Substituting this equality in (1), we deduce that bx(t) is the solution of the problem

x(tb + 1) = h

A0(t, ηt) +B0(t, ηt)Fe0(t, ηt) (31)

+

r

X

k=1

w_k(t) A_k(t, η_t) +B_k(t, η_t)Fe(t, η_t)i

bx(t), x(tb ₀) =x₀,

with given initial value. It follows from assumptions b) and c) that D_t0 =D.

Since ub∈ U₂(t₀, x₀), from (4) we have

(32) lim

t→∞E

|x(t)|b ²|η_t0 =i

= 0, i∈ D.

If ΦFe(t, t0) is the fundamental matrix solution of (31), then (32) may be rewritten as

t→∞lim Eh x^T₀Φ^T

Fe(t, t₀)Φ

Fe(t, t₀)x₀|η_t0 =ii

= 0, ∀i∈ D,(t₀, x₀)∈Z₊×Rⁿ. By the representation theorem (see [10]), the last equation is equivalent to

t→∞lim x^T₀ T^∗

Fe(t, t0)J

(i)x0 = 0 for all (t0, x0) ∈ Z+×Rⁿ, i ∈ D, where J = (I_n, . . . , I_n)∈ S_n^N. Recalling that

T^∗

Fe(t, t₀) ξ =

T^∗

Fe(t, t₀)J

ξ = max

i∈D sup

|x₀|≤1

x^T₀ T^∗

Fe(t, t₀)J (i)x₀,

we deduce that

(33) lim

t→∞

T^∗

Fe(t, t₀) ξ= 0.

Finally, using the properties of the Minkovski norm (see [13]), we deduce that (33) is equivalent to (28). So, the implication (i)⇒(ii) does hold.

To prove the converse implication, we remark that by the representation theorem in [10] if (28) holds then (32) holds, too. This means that the control u(t) =b Fe(t, η_t)bx(t) is admissible. Further, from (20) and (17) we deduce that ubis an optimal control and thus the proof is complete.

Remark 2. From the definition of the stabilizing solution of a system of discrete-time Riccati equations of stochastic control (see [14]) we deduce that the maximal solution Xmax(t) of DTSGRE (15) is a stabilizing solution if and only if there exist β ≥1 andq ∈(0,1) such that

(34)

TFe(t, t₀)

ξ≤βq^t−t0 for all t≥t₀ ≥0.

From Theorem 4 we deduce that the condition verified by the maximal solution of (15), which is equivalent to the existence of an optimal control of the problem under consideration, is weaker than (34). This can explain why the result proved in Proposition 3 only provides a sufficient condition for the existence of an optimal control.

Theorem 5. Assume that

a)the coefficients of system (1)and the weights of the cost functional(3) are periodic sequences with period θ≥1;

b) the assumptions of Theorem 2 are fulfilled.

Under these assumptions the following assertions are equivalent:

(i) for any (t₀, x₀) ∈ Z₊ ×Rⁿ the optimization problem described by the controlled system (1), the cost functional (3) and the class of admissi- ble controls U₂(t0, x0) has the optimal control ubt0x0 = {u_t0x0(t)}_t≥t0, i.e., V(t₀, x₀) =J₂(t₀, x₀,ub_t0x0);

(ii)the DTSGRE(15)has a bounded stabilizing solution{X_s(t)}_t≥0which satisfies (27).

Proof. The implication (ii) ⇒ (i) follows from Proposition 3. If (i) is fulfilled, reasoning as in the proof of Theorem 4, we deduce, by Theorem 4.1 in [10], thatFeis the stabilizing feedback gain for system (1). This allows us to conclude that the maximal solution {X_max(t)} coincides with the stabilizing solution of (15). Thus, the proof is complete.

REFERENCES

[1] H. Abou-Kandil, G. Freiling and G. Jank, On the solution of discrete-time Markovian jump linear quadratic control problems.Automatica J. IFAC32(1995),5, 765–768.

[2] W.P. Blair and D.D. Sworder,Feedback control of a class of linear discrete systems with jump parameters and quadratic cost criteria. Internat. J. Control21(1975),5, 833–841.

[3] E.K. Boukas and K. Benjelloun,Robust control for linear systems with Markovian jump- ing parameters. Preprints of 13th IFAC World Congress, San Francisco, USA, 1996, 451–456.

[4] H.J. Chizeck, A.S. Willsky and D.Castanon,Discrete time Markovian jump linear qua- dratic optimal control. Internat. J. Control43(1986),1, 213–231.

[5] O.L.V. Costa,Linear minimum mean square error estimation for discrete time markov- ian jump linear systems. IEEE Trans. Automat. Control39(1994),8, 1685–1689.

[6] O.L.V. Costa and M.D. Fragoso,Discrete time LQ-optimal control problems for infinite Markov jump parameter systems. IEEE Trans. Automat. Control40(1995),12, 2076–

2088.

[7] O.L.V. Costa,Mean-square stabilizing solutions for discrete time coupled algebraic Ric- cati equations. IEEE Trans. Automat. Control41(1996),4, 593–598.

[8] O.L.V. Costa, M.D. Fragoso and R.P. Marques, Discrete-time Markov Jump Linear Systems. Springer-Verlag, London, 2005.

[9] J.L. Doob,Stochastic Processes. Wiley, New York, 1967.

[10] V. Dragan and T. Morozan,Mean square exponential stability for some stochastic linear discrete time systems. Eur. J. Control12(2006),4, 373–399.

[11] V. Dragan and T. Morozan, The linear quadratic optimization problem for a class of discrete-time stochastic linear systems. Internat. J. Innovative Computing, Information and Control4(2008),9, 2127–2137.

[12] V. Dragan and T. Morozan,Discrete-time Riccati type equations and the tracking prob- lem. ICIC Express Lett.2(2008),2, 109–116.

[13] V. Dragan T. Morozan,Discrete time linear equations defined by positive operators on ordered Hilbert spaces. Rev. Roumaine Math. Pures Appl.53(2008),2-3, 131–166.

[14] V. Dragan and T. Morozan, A class of discrete time generalized Riccati equations.

Preprint no. 1/2008, “Simion Stoilow” Institute of Mathematics of the Romanian Acad- emy. To appear in J. Difference Equations Appl.

[15] M.D. Fragoso, Discrete-time jump LQG problem. Internat. J. Systems Sci.20(1989), 12, 2539–2545.

[16] K. Furuta and M. Wongsaisuwan, Discrete time LQG dynamic controller design using plant Markov parameters. Automatica J. IFAC31(1995), 1317–1324.

[17] B.E. Griffiths and K.A. Loparo,Optimal control of jump linear gaussian systems. Inter- nat. J. Control42(1985),4, 791–819.

[18] Y. Ji, H.J. Chizeck, X. Feng and K. Loparo,Stability and control of discrete-time jump linear systems. Control Theory Adv. Tech.7(1991),2, 247–270.

[19] M. Mariton, Jump Linear Systems in Automatic Control. Marcel Dekker, New York, 1990.

[20] T. Morozan,Stability and control of some linear discrete-time systems with jump Markov disturbances. Rev. Roumaine Math. Pures Appl.26(1981),1, 101–119.

[21] T. Morozan,Optimal stationary control for dynamic systems with Markovian perturba- tions. Stoch. Anal. Appl.1(1983),3, 299–325.

[22] T. Morozan,Discrete-time Riccati equations connected with quadratic control for linear systems with independent random perturbations.Rev. Roumaine Math. Pures Appl.37 (1992),3, 233–246.

[23] T. Morozan,Stability and control for linear discrete-time systems with Markov pertur- bations. Rev. Roumaine Math. Pures Appl.40(1995),5-6, 471–494.

[24] T. Morozan, Stabilization of some stochastic discrete-time control systems. Stochastic Anal. Appl.1(1983),1, 89–116.

[25] T. Morozan,Dual linear quadratic problem for linear control systems with jump Markov perturbations. Rev. Roumaine Math. Pures Appl.41(1996),5-6, 363–377.

[26] J. Zabczyk,Stochastic control of discrete-time systems.In: Control Theory and Topics in Functional Analysis. Vol. III, pp. 187–224. IAEA, Vienna, 1976.

Received 28 April 2009 Romanian Academy

“Simion Stoilow” Institute of Mathematics P.O. Box 1-764, 014700 Bucharest, Romania

Vasile.Dragan@imar.ro Toader.Morozan@imar.ro