on the occasion of his 70th birthday
ON THE CORRECT SOLUTION
OF A TRIVIAL OPTIMAL CONTROL PROBLEM IN MATHEMATICAL ECONOMICS
S¸TEFAN MIRIC ˘A and TOUFFIK BOUREMANI
We use the Dynamic Programming approach to obtain the correct solution of the optimal control problem studied in [1] using Pontryagin’s Maximum Principle. We prove that all the optimal trajectories of the problem are constant functions, hence the “solution” in [1] is wrong and the problem is rather trivial.
AMS 2000 Subject Classification: 49L20, 49K05, 49K24, 49N35, 49N90, 90B30.
Key words: optimal control, mathematical economics, maximum principle, dy- namic programming, value function, verification theorem.
1. INTRODUCTION
This work is first intended, as a warning to the, unfortunately, increas- ing number of authors try to solve concreteoptimal control problems without enough knowledge and even basic mathematical abilities; second, our aim is to show that the Dynamic Programming approach in [6]–[7] is much more efficient in the study of this type of problems.
For the sake of completeness we present in Section 2 the (not very con- vincing) “economic problem”, the corresponding mathematical model and the
“solution” obtained in [1] using a non-existent variant of Pontryagin’s Maxi- mum Principle (PMP); we also point out the main errors made by the authors of [1].
In Section 3 we use the Dynamic Programming approach (adapted to non-autonomous problems) to obtain the correct solution of the same problem, and prove that the solution in [1] iswrongand their problem is rather trivial since all the optimal trajectories are constant.
MATH. REPORTS9(59),1 (2007), 77–86
2. THE ECONOMIC PROBLEM, THE MATHEMATICAL MODEL AND THE WRONG SOLUTION IN [1]
The aim of paper [1] is to solve the optimal control problem ofminimizing the cost functional:
(2.1) J(P(·), I(·)) :=
T
0 e−ρt[k(P(t)) +h(I(t))]dt, subject to the constraints
(2.2) I(t) =P(t)−D(t, I(t)), I(0) =I0, I(T) =IT, p(t)≥D(t, I(t)), where the functions involved have the following “economic significances”:
I(t): inventory level (state variable) at timet≥0;
P(t): the chosen production rate (control variable);
D(t, I(t)): the demand rate (minimal production rate);
h(I(t)): the holding cost rate of the inventory I(t);
k(P(t)): the cost rate of the chosen production rateP(t);
ρ≥0: the discount rate;
I(0) =I0, I(T) =IT: the initial and, respectively terminal values of the inventory (fixed, but, apparently, arbitrary).
At a first stage, the authors assume that the functionsh(·), k(·), D(·,·) have “non-negative” values, are of class C2 and k(·) is increasing.
In fact, the authors only studied the problem in more detail only in the particular case
(2.3) k(u) := ku2
2 , h(x) := hx2
2 , D(t, x) :=d1(t) +d2x, wherek, h, d2 >0 and d1(·)≥0 is of classC1.
One may note that, in the absence of a more concrete example, the economic problem does not seem very convincing since the “customers” should know the level I(t) of the inventory rate in order to choose their “minimal demand rate”D(t, I(t)) while the “patron” should know the choiceD(t, I(t)) of the customers.
Anyway, the authors of [1] invoke the classical variant of Pontryagin’s Maximum Principle (PMP) in [9] to write some differential equations and, in the particular case in (2.3), to conclude in their Theorem 1 that in this case there exist constantsa1, a2, m1, m2and a functionQ(·) such that theoptimal trajectory of problem (2.1)–(2.2) is given by
(2.4) I∗(t) =a1em1t+a2em2t+Q(t), t∈[0, T].
However, as we shall prove in the next section, in the particular case (2.3) the only optimal trajectories are the constant functions, I(t) ≡ I0,
t∈[0, T], hence the solution in [1] iswrongand problem (2.1)–(2.2) is rather trivial.
Even if, by any chance, the solution I∗(·) would be correct, the authors of [1] among several others, made the following errors:
(1) they apply the classical variant of Pontryagin’s Maximum Principle (PMP) (in which the set of control parameters, U, is constant) to problem (2.1)–(2.2) where the set of control parameters, U(t, x) := [D(t, x),∞), is variable; in this case the dynamics in (2.1) is defined by a differential inclusion of the form
(2.5) x ∈f(t, x, U(t, x)), x(0) =y,
for which the Pontryagin’s Maximum Principle (PMP) has more complicated statements as shown in [2], [4], etc;
(2) even if a correct variant of Pontryagin’s Maximum Principle (PMP) were used, the authors could not conclude the optimality of the extremalI∗(·) without additional arguments proving:
(2a) the existence of an optimal trajectory joiningI0 and IT; (2b) the uniqueness of the extremal I∗(·) in (2.4);
(3) the authors are apparently solving the elementary problem of max- imizing the real function u → α(t, p, u) in (3.7) below on the interval u ∈ [D(t, x),∞) using the “Lagrange multiplier rule”.
3. THE DYNAMIC PROGRAMMING CORRECT SOLUTION
In order to use the Dynamic Programming Approach in [5]–[7], adapted to non-autonomous problems as in [8], we reformulate problem (2.1)–(2.2) us- ing standard notation inOptimal Control Theoryand imbedding this problem in a set of problems associated with each initial point in the phase-space; thus, we obtain the “standard”Lagrange optimal control problem below.
Problem 3.1. Given T > 0, ρ ≥ 0 and smooth non-negative functions D(·,·), h(·), k(·), find
(3.1) inf
u(·)C(s, y;u(·)) ∀(s, y)∈E0:= [0, T)×R∗+ subject to
(3.2) C(s, y;u(·)) :=g(T, x(T)) + T
0 f0(t, x(t), u(t))dt, (3.3) x(t) =f(t, x(t), u(t)), u(t)∈U(t, x(t))a.e. ([0, T]), (3.4) (t, x(t))∈E0 ∀ t∈[0, T), (T, x(T))∈E1 :={T} ×R∗+,
defined by the data (3.5)
f(t, x, u) :=u−D(t, x), f0(t, x, u) := e−ρt[h(x) +k(u)], E0:= [0, T)×R∗+, E1:={T} ×R∗+, g(T, ξ)≡0 ∀(T, ξ)∈E1, U(t, x) := [D(t, x),∞) ∀(t, x)∈E:=E0∪E1= [0, T]×R∗+.
One may note here that Problem 3.1 solves problem (2.1)–(2.2) for the particular case (s, y) = (0, I0) in the case the (possible) optimal trajectory,
x(·), takes the terminal valuex(T ) =IT; however, as we shall prove in what follows, ifIT =I0, Problem 3.1 (and also problem (2.1)–(2.2)) does not have an optimal control (hence an optimal trajectory, too).
In what follows we shall solve Problem 3.1 in the case the datah(·),k(·), D(·,·) verify the assumptions
(3.6) D(t, x)>0, k(x)>0, h(x) +k(D(t, x))·∂D
∂x(t, x)>0∀t∈[0, T], k(u)>0 ∀u≥0, lim
u→∞k(u) =∞, x >0,
which are obviously satisfied by the particular data in (2.3); a more general case does not seems worth the effort in the absence of some convincing exam- ples.
Characterization of the Hamiltonian. The first step of the Dynamic Programming Approach consists in characterization of the “true Hamiltonian”
of the problem.
The “pseudo-Hamiltonian” H(t, x, p, u) := p, f(t, x, u)+f0(t, x, u) is given in our case by
(3.7) H(t, x, p, u) = e−ρth(x)−pD(t, x)+α(t, p, u), α(t, p, u) :=pu+e−ρtk(u).
The Hamiltonian and the corresponding multifunction of minimum points are given by the formulas
(3.8) H(t, x, p) = min
u∈U(t,x)H(t, x, p, u) = e−ρth(x)−pD(t, x)+ min
u∈U(t,x)α(t, p, u) U(t, x, p) :={u∈U(t, x); H(t, x, p, u) =H(t, x, p)}.
In order to obtain a more explicit description of the HamiltonianH(·,·,·) in (3.8), we introduce the following notation: for (t, x, p)∈Z := [0, T]×R∗+×R andu∈R+, we set
(3.9) α(t, x, p) := min
u∈U(t,x)α(t, p, u), β(t, x, p) :=p+ e−ρtk(D(t, x)).
Proposition3.2. If the assumptions in(3.6)hold, then for any(t, x, p)∈ Z there exists a unique real number u(t, x, p)∈[D(t, x),∞) such that:
(3.10) α(t, x, p) = α(t, p,u(t, x, p)).
Moreover,
(1)if β(t, x, p)≥0 then u(t, x, p) =D(t, x);
(2) if β(t, x, p) < 0 then there exists a unique real number u∗(t, p) ∈ (D(t, x),∞) such that
(i) ∂α∂u(t, p, u∗(t, p)) =p+ e−ρtk(u∗(t, p)) = 0;
(ii)u(t, x, p) = u∗(t, p);
(3)the functionu( ·,·,·) is given by
(3.11) u(t, x, p) :=
D(t, x) if β(t, x, p) ≥0 u∗(t, p) if β(t, x, p) <0 and is locally-Lipschitz onZ.
Proof. Using the fact thatk(u)>0∀u∈[0,∞), we deduce thatk(·) is strictly increasing, hence,u→ ∂α∂u(t, p, u) =p+ e−ρtk(u) is strictly increasing.
(1) If β(t, x, p) = ∂α∂u(t, p, D(t, x))≥0, then ∂α∂u(t, p, u)≥ ∂α∂u(t, p, D(t, x))
≥0 ∀u ≥ D(t, x), hence α(t, p, u) ≥α(t, p, D(t, x)) ∀ u ≥ D(t, x) and there- fore,u(t, x, p) = D(t, x).
(2) If β(t, x, p) < 0, since ∂α∂u(t, p,·) = p+ e−ρtk(·) is strictly increas- ing and lim
u→∞∂α
∂u(t, p, u) = lim
u→∞[e−ρtk(u)] = ∞, it follows from the Dar- boux property that there exists a unique point u∗(t, p) ∈ (D(t, x),∞) such that ∂α∂u(t, p, u∗(t, p)) = 0 and, moreover, ∂α∂u(t, p, u) < ∂α∂u(t, p, u∗(t, p)) ∀u ∈ [D(t, x), u∗(t, p)), ∂α∂u(t, p, u) > 0 ∀ u > u∗(t, p), hence, u∗(·,·) is the global minimum point of the functionα(t, p,·), therefore u(t, x, p) =u∗(t, p).
Since ∂∂u2α2(t, p, u) = e−ρtk(u) > 0 ∀ u ∈ [0,∞), it follows from the implicit functions theorem thatu∗(·,·) is given by
(3.12) u∗(t, p) := (k)−1(−peρt)
and is of classC1 on the subsetZ−:={(t, x, p)∈Z; β(t, x, p)<0}.
Since D(·,·), u(·,·,·) are locally-Lipschitz, according to a well-known
“quasi-elementary” result, it follows thatu(·,·,·) is locally-Lipschitz.
Corollary3.3. The HamiltonianH(·,·,·) in (3.8) is given by (3.13)
H(t, x, p) =
H+(t, x, p) := e−ρt[h(x)+k(D(t, x))] if (t, x, p)∈Z+, H0(t, x, p) := e−ρt[h(x)+k(D(t, x))] if (t, x, p)∈Z0, H−(t, x, p) := e−ρth(x)−p[D(t, x)−u∗(t, p)]+
+e−ρtk(u∗(t, p)) if (t, x, p)∈Z−,
whereu∗(·,·) is the function defined in(3.12), hence it isC1-stratified and the strataZ±, Z0 are defined by
Z+:={(t, x, p)∈Z;β(t, x, p)>0}, β(t, x, p) :=p+e−ρtk(D(t, x)), (3.14)
Z0:={(t, x, p)∈Z;β(t, x, p) = 0}, Z−:={(t, x, p)∈Z;β(t, x, p)<0}. Proof. The formula in (3.14) follows from Proposition 3.2 while since the functions D(·,·), k(·), k(·) are differentiable, it follows that the subsets Z± are relatively open andZ0 is a 2-dimensional differentiable manifold.
The Hamiltonian system. Since the Hamiltonian in (3.13) isC1-stratified, we choose the stratified Hamiltonian orientor fieldd#SH(·,·,·) defined by the formula
(3.15) d#SH(t, x, p) ={(x, p)∈R2n; (1, x, p)∈T(t,x,p)Z, x∈f(t, x,U(t, x, p)), x, p − p, x=DH(t, x, p)·(0, x, p)∀(0, x, p)∈T(t,x,p)Z}.
Since the manifolds Z± ⊂ Z are open subsets, the Hamiltonian orien- tor fields d#SH±(·,·,·) in (3.15) coincide with the classical Hamiltonian vec- tor fields
(3.16) d#SH±(t, x, p) := ∂H±
∂p (t, x, p),−∂H±
∂x (t, x, p)
if (t, x, p)∈Z±. The terminal transversality conditions. As specified in the Dynamic Programming Algorithm, in the case of a stratified Hamiltonian a generalized Hamiltonian flow is obtained by “backward integration” for t ≤ T of the Hamiltonian inclusion
(3.17) (x, p)∈d#SH(t, x, p), (T, x(T), p(T)) = (T, ξ, q)∈Z1∗,
where the set of terminal “transversality” points is defined in the general case by
Z1∗ :={(τ, ξ, q)∈Z; q, ξ−τ H(τ, ξ, q) =Dg(τ, ξ)·(τ , ξ)∀(τ , ξ)∈T(τ,ξ)E1}. Since g(τ, ξ) = 0 andT(T,ξ)E1={0} ×R, in our case we have
Z1∗ ={(T, ξ, q)∈Z; q, ξ= 0∀ξ∈R}={(T, ξ,0); ξ >0}.
Further, sinceβ(T, ξ,0) = e−ρtk(D(T, ξ)), it follows that in the general case the setZ1∗ above admits the stratification
Z1,+∗ :={(T, ξ,0); k(D(T, ξ))>0} ⊂Z+, (3.18)
Z1,0∗ :={(T, ξ,0); k(D(T, ξ)) = 0} ⊂Z0, Z1,−∗ :={(T, ξ,0); k(D(T, ξ))<0} ⊂Z−,
while in the case of an additional hypothesis in (3.6) according to whichk(u)>
0 ∀ u ≥0, one has Z1,0∗ =Z1,−∗ = ∅, hence all the trajectories are ending on the stratumZ+.
Construction of the generalized Hamiltonian flow. Sincek(D(T, ξ))>0, we have (T, ξ,0)∈Z+ hence on an interval (τ+(ξ), T) the trajectory X+∗(·) = (X+(·), P+(·)) of the Hamiltonian system in (3.17) will remain on the stratum Z+, hence it is a solution of the “smooth Hamiltonian system”
(3.19)
x = 0, x(T) =ξ >0
p =−e−ρt[h(x) +k(D(t, x)).∂D∂x(t, x)], p(T) = 0.
Therefore, we obtain its solution in the form of a “maximal flow”X+∗(·,·)
= (X+(·,·), P+(·,·)) whose first component is given by X+(t, ξ) ≡ ξ while P+(·, ξ), is the solution of the elementary differential equation
(3.20) p =−e−ρt
h(ξ) +k(D(t, ξ)).∂D
∂x(t, ξ)
, p(T) = 0, hence it is a primitive of the function on the right-hand side.
It follows from the Dynamic Programming algorithm in [6] that, on the stratumZ+ we must retain only the trajectories X+∗(·, ξ), ξ >0, that satisfy the conditions
(3.21) X+∗(t, ξ)∈Z+, (t, X+(t, ξ))∈E0 ∀ t∈(τ1+(ξ), T), ξ >0, on the maximal intervals (τ1+(ξ), T), ξ >0, hence the extremityτ1+(ξ)< T is defined by
τ1+(ξ) := inf{τ < T; β+(t, ξ)>0, X+(t, ξ)>0 ∀t∈[τ, T)}, (3.22)
β+(t, ξ) :=β(t, ξ, P+(t, ξ)) =P+(t, ξ) + e−ρtk(D(t, ξ)), whereP+(·, ξ), ξ >0 is the only solution of equation (3.20).
It follows from hypothesis (3.6) (in particular, hypothesis (2.3)) that h(ξ) +k(D(t, ξ)).∂D∂x(t, ξ) > 0 ∀ t ∈ [0, T], ξ > 0 hence P+(·, ξ) is strictly decreasing, P+(t, ξ) >0 ∀ t∈[0, T], therefore β+(t, ξ) >0 ∀t∈[0, T], ξ >0, which proves that the extremityτ1+(·) in (3.22) is given by
(3.23) τ1+(ξ) = 0 ∀ξ >0.
Therefore, the trajectoriesX+(·, ξ), ξ >0 “cover” the domainE0+ defined by E0+:={(t, X+(t, ξ)); t∈[0, T), ξ >0}= [0, T)×R∗+=E0, (3.24)
B+:= dom(X+(·,·)) = [0, T)×R∗+.
An essential step in using the general algorithm in [6]–[7] consists in the fact that the value of the cost functional in (3.2) is given by the functionV(·,·)
defined by
V(t, ξ) :=
t
T
P+(σ, ξ),∂X+
∂σ (σ, ξ)
−H(σ, X+∗(σ, ξ))
dσ= (3.25)
=− t
T f0(σ, X+(σ.ξ),u(σ, X +∗(σ, ξ)))dσ =
= T
t H+(σ, ξ, P+(σ, ξ))dσ, whereH+(·,·,·) is defined in (3.13).
Thus, the Hamiltonian system in (3.19) generates the smooth charac- teristic flow C+∗(·,·) := (X+∗(·,·), V(·,·)) on the open stratum Z+ and ac- cording to a well known classical results (e.g., Miric˘a ([6])) satisfies the basic differential relation:
(3.26) DV(t, ξ)·(t, ξ) = P+(t, ξ), DX+(t, ξ)·(t, ξ) ∀(t, ξ)∈T(t,ξ)B+, which may be proved directly in this case.
On the other hand, it follows from (3.25) that for any initial point, (s, y) = (s, X+(s, ξ)) = (s, ξ)∈E0, the mappingu(s,y)(·) defined by
(3.27) us,y(t) :=u(t, X +∗(t, y)) =D(t, y), t∈[s, T], is an admissible control for which the cost functional is given by (3.28) C(s, y;us,y(·)) =V(s, y) =
T
s H+(σ, y, P+(σ, y))dσ.
Therefore, we have obtained the “feasible selection” of admissible controls
(3.29) A(s, y) :={us,y(·)}, (s, y)∈E0, whosevalue functionis given by
(3.30) W+(s, y) := 0 if (s, y)∈E1:={T} ×R∗+, W0+(s, y) :=V(s, y) if (s, y)∈E0:= [0, T)×R∗+, whereV(·,·) is the function defined in (3.25).
Since the value function W+(·,·) in (3.30) is of class C1, the optimality of the controlsus,y(·) in (3.27), therefore of the corresponding trajectories (3.31) xs,y(t) =y ∀t∈[s, T], (s, y)∈E0,
follows from the so calledElementary Verification Theorem (e.g. [3], [5], [7], etc.), according to which a sufficient optimality condition for the admissible controlsus,y(·) in (3.27) is the verification of the differential inequality (3.32) DW0+(s, y)·(1, f(s, y, u)) +f0(s, y, u)≥0∀ u∈U(s, y), (s, y)∈E0.
Thus, we obtain the main result of this paper.
Theorem3.4. The selectionA(·,·)of admissible controls in (3.29) with value functionW+(·,·) in (3.30) is optimal.
Proof. First, it follows from (3.26), (3.30) that
DW0+(s, y)·(1, f(s, y, u)) =DV(s, y)·(1, f(s, y, u)) = (3.33)
=P+(s, y)DX+(s, y)·(1, f(s, y, u)).
Since X+(s, y) = y, it follows that DX+(s, y) ·(1, v) = v, hence DX+(s, y)·(1, f(s, y, u)) =f(s, y, u) and therefore, it follows from (3.33) that
DW0+(s, y)·(1, f(s, y, u)) +f0(s, y, u) =P+(s, y)f(s, y, u) +f0(s, y, u) =
=H(s, y, P+(s, y), u)> H+(s, y, P+(s, y)) =
= e−ρs[h(y) +k(D(s, y))>0 ∀u∈U(s, y),
which proves inequality (3.32), hence the optimality of the selec- tionA(·,·).
Remark 3.5. It follows, in particular, from Theorem 3.4 above that if I0, IT ∈R∗+ are such that I0 =IT (in fact, one should have IT > I0), then it does not exist an optimal trajectory joiningI0 and IT since the only optimal trajectory starting at (0, I0) is the constant function I(t) ≡ I0 (in fact, for any admissible controlu(·)=u0,I0(·) one hasC(0, I0;u(·))≥ C(s, y;u0,I0(·)) = W+(0, I0)).
REFERENCES
[1] M. Bounkhel, L. Tadj and Y. Benhadid, Optimal control of a production system with inventory-level-dependent demand. Appl. Math. E-Notes5(2005), 36–43.
[2] A. Cernea and S¸t. Miric˘a,Minimum principle for some classes of nonconvex differential inclusions. An. S¸tiint¸. Univ. “Al.I. Cuza” Ia¸si Mat.XLI(1995), 307–324.
[3] L. Cesari, Optimization-Theory and Applications. Springer-Verlag, New York–Berlin, 1983.
[4] H. Frankowska,The maximum principle for an optimal solution to a differential inclu- sion with end point constraints. SIAM J. Control Optim.25(1987), 145–157.
[5] V. Lupulescu and S¸t. Miric˘a,Verification theorems for discontinuous value functions in optimal control. Math. Reports2(52)(2000),3, 299–326.
[6] S¸t. Miric˘a,Constructive Dynamic Programming in Optimal Control Autonomous Prob- lems. Editura Academiei Romˆane, Bucharest, 2004.
[7] S¸t. Miric˘a,User’s Guide on Dynamic Programming for autonomous differential games and optimal control problems. Rev. Roumaine Math. Pures Appl. 49 (2004), 5-6, 501–529.
[8] S¸t. Miric˘a and C. Necul˘aescu,On the solution of an optimal control problem in Mathe- matical Economics. Anal. Univ. Bucure¸sti Mat. XLVII(1998), 49–57.
[9] L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze and E.F. Mishchenko,The Mathe- matical Theory of Optimal Processes. Wiley, N.Y., 1962.
Received 8 September 2006 University of Bucharest
Faculty of Mathematics and Computer Science Str. Academiei 14, 010014 Bucharest, Romania
mirica@fmi.unibuc.ro, bouremani@yahoo.com
