Stability and Suboptimality Without Stabilizing Constraints
6.6 Design of Good Running Costs
Fig. 6.1 Suboptimality regions for different optimization horizonsNdepending onCandσin (6.3) forα =0 (left) andα =0.5 (right)
6.6 Design of Good Running Costs
In this section we illustrate by means of several examples how our theoretical find-ings and in particular Theorem6.14 in conjunction with Proposition6.17can be used in order to identify and design running costssuch that the NMPC feedback lawμN exhibits stability and good performance with small optimization horizons N. To this end we first visualize Formula (6.19) for differentβ∈KL0, starting with the case of exponential controllability (6.3). Note that in this case (6.6) and thus (6.20) always holds.
Given a desired suboptimality levelα ≥0, we use Formula (6.19) in order to determine the regions in the(σ, C)-plane for which αN ≥α holds for different optimization horizonsN. Figure6.1shows the resulting regions for α =0 (i.e.,
“plain” stability) andα =0.5.
Looking at Fig.6.1 one sees that the parametersC and σ play a very differ-ent role. While for both parameters the necessary optimization horizonN becomes the smaller the smaller these parameters are, small overshootC (i.e., values ofC close to 1) have a much stronger effect than small decay ratesσ (i.e., values ofσ close to 0). Indeed, Fig.6.1(left) shows that for sufficiently smallCwe can always achieve stability forN=2 while forC≥8 even values ofσ very close to 0 will not yield stability forN≤16. For the required higher suboptimality levelα≥0.5, Fig.6.1(right) indicates a qualitatively similar behavior.
For finite time control, i.e., controllability withKL0-functions satisfying (6.4), the situation is very similar. For instance, consider functions of the formβ(r,0)= c0r,β(r,1)=c1r,c0≥c1, and β(r, n)=0 for n≥2, i.e.,n0=2. This function again satisfies (6.6), hence (6.20) holds. For thisβ, Fig. 6.2shows the analogous graphs as in Fig.6.1.
One immediately sees that the qualitative behavior depicted in Fig.6.2is very similar to the analogous graphs in Fig.6.1: again, reducing the overshootc0we can
134 6 Stability and Suboptimality Without Stabilizing Constraints
Fig. 6.2 Suboptimality regions for different optimization horizonsNdepending onc0andc1/c0 in (6.4) withn0=2 forα =0 (left) andα =0.5 (right)
always achieve stability withN=2 regardless of the ratioc1/c0while reducingc1
and keepingc0fixed, in general we needN >2 in order to guarantee stability.
Finally, in Fig.6.3we compare the effect of the overshoot c0 and the timen0 in (6.4) by usingβ(r,0)=c0r,β(r, n)=c0r/2 forn=1, . . . , n0andβ(r, n)=0 forn≥n0. Again, it turns out that the timen0needed to control the system tox∗ is less important than the overshoot: for all timesn0≥1 we can always achieve stability forc0sufficiently close to 1 while for fixedc0this can in general not be achieved even forn0=1, i.e., for controllability in one step. Note that forc0<2 this functionβ does not satisfy (6.6), thus for these values of c0 Formula (6.19) only provides a lower bound forα, cf. (6.18). Consequently, forc0<2 the regions depicted in Fig.6.3may underestimate the true regions. Still, for alln0 the lower bounds obtained from (6.18) ensure both asymptotic stability and the desired per-formance boundα≥0.5 forN=2 wheneverc0is sufficiently close to 1.
Together, these examples lead to a conclusion which is as intuitive as simple:
an NMPC controller without stabilizing terminal constraints will yield stability and good performance for small horizonsNifcan be chosen such that Assumption6.4 is satisfied with aβ∈KL0with small overshoot. Thus, the criterion “small over-shoot” can be used as a design guideline for selecting a good running cost.
For some systems, it is possible to rigorously compute β in Assumption 6.4, which leads to a precise determination of, e.g.,C andσ in (6.3). Examples where this is possible also include infinite-dimensional systems, like the linear wave equa-tion or certain classes of semilinear parabolic equaequa-tions, cf. [3]. However, more often than not precise estimates forβ cannot be obtained due to the complexity of the dynamics. Still, using heuristic arguments it may be possible to determine run-ning costsfor which the overshoot is reduced. In the remainder of this section we will illustrate this procedure for two examples.
6.6 Design of Good Running Costs 135
Fig. 6.3 Suboptimality regions for different optimization horizonsNdepending onc0andn0in (6.4) withcn=c0/2 forn=1, . . . , n0−1 forα =0 (left) andα =0.5 (right)
Example 6.26 We consider Example 2.3, i.e., f (x, u):=
x1+ x2+
=
sin(ϑ (x)+u) cos(ϑ (x)+u)/2
with
ϑ (x)=
arccos 2x2, x1≥0, 2π−arccos 2x2, x1<0
using the control valuesU= [0,0.2], i.e., the car can only move clockwise on the ellipse
X=
x∈R2
x1
2x2
=1
.
As in illustrated in Fig.6.4, we want to stabilize the system at the equilibriumx∗= (0,−1/2)starting from the initial valuex0=(0,1/2).
InterpretingXas a subset ofR2, we can try to achieve this goal by using NMPC without terminal constraints with the running cost
(x, u)= x−x∗2+u2. (6.24) As the simulations in Fig. 6.5 show, asymptotic stability of x∗ =(0,−1/2) is achieved forN=11 but not forN=10.
The reason for the closed loop not being asymptotically stable forN=10 (and, in fact, for allN≤10) is the overshoot in the running costwhen moving along the ellipse; see Fig.6.6.
The fact that this overshoot of appears along the NMPC closed-loop trajec-tory does in general not imply that the overshoot is present for all possible control sequencesu controlling the system tox∗. However, in this example a look at the geometry reveals that forfrom (6.24) the overshoot is in fact not avoidable: no
136 6 Stability and Suboptimality Without Stabilizing Constraints
Fig. 6.4 Illustration of the stabilization problem
matter how we control the system tox∗, before we can eventually reduceto 0, we need to increasewhen moving along the ellipse around the curve. Thus, loosely speaking, the loss of asymptotic stability forN≤10 is caused by the fact that the optimizer does not “see” that in the long run it is beneficial to move around the curve and thus stays at the initial valuex0for all future times.
Looking closer at the geometry of the example, one easily sees that the overshoot is entirely due to thex1-component of the solution: whilex2converges monotoni-cally to the desired positionx∗2= −0.5,x1first needs to move from 0 to 1 before we can eventually control it tox∗1=0, again. From this observation it follows that the overshoot in can be avoided by putting more weight on thex2-component.
Indeed, if we replace(x, u)= x−x∗2+u2=(x1−x∗1)2+(x2−x∗2)2+u2 from (6.24) by
(x, u)=(x1−x∗1)2+5(x2−x∗2)2+u2, (6.25) then we obtain asymptotic stability even forN=2, cf. Fig.6.7.
Figure6.8shows the running cost along the closed-loop trajectory for this ex-ample. The figure clearly shows that the overshoot has been removed completely, which explains why the NMPC closed loop is stable forN=2.
We would like to emphasize that for removing the overshoot we did not use any quantitative information, i.e., we did not attempt to estimate the functionβ in As-sumption6.4. For selecting a good cost functionit was sufficient to observe that putting a larger weight onx2will reduce the overshoot. On the basis of this obser-vation, the fact that the weight “5” used in (6.25) is sufficient to achieve asymptotic stability withN=2 was then determined by a simple try-and-error procedure using numerical simulations.
Example 6.27 As a second example we consider the infinite-dimensional PDE models introduced in Example 2.12. We first consider the system with distributed control, i.e.,
yt(t, x)=θyxx(t, x)−yx(t, x)+ρ
y(t, x)−y(t, x)3
+u(t, x) (6.26)
6.6 Design of Good Running Costs 137
Fig. 6.5 NMPC closed-loop trajectories for Example 2.3 with running cost (6.24) and optimiza-tion horizonsN=11 (left),N=10 (right)
Fig. 6.6 Running cost (6.24) along the NMPC closed-loop trajectory forN=11
with control functionu∈L∞(R×,R), domain =(0,1)and real parameters θ=0.1,ρ=10. Hereyt andyxdenote the partial derivatives with respect tot and x, respectively, andyxxdenotes the second partial derivative with respect tox.
The solutiony of (6.26) is supposed to be continuous inand to satisfy the boundary and initial conditions
y(t,0)=0, y(t,1)=0 for allt≥0 and
(6.27) y(0, x)=y0(x) for allx∈
for some given continuous functiony0:→Rwithy0(0)=y0(1)=0.
Observe that we have changed notation here in order to be consistent with the usual PDE notation: x∈is the independent space variable while the unknown functiony(t,·):→Rin (6.26) is the state now. Hence, the state is now denoted byy (instead of x) and the state space of this PDE control system is a function space, more precisely the Sobolev spaceH01(), although the specific form of this space is not crucial for the subsequent reasoning.
138 6 Stability and Suboptimality Without Stabilizing Constraints
Fig. 6.7 NMPC closed-loop trajectories for Example 2.3 with running cost (6.25) and optimization horizonN=2
Fig. 6.8 Running cost (6.25) along the NMPC closed loop forN=2
Figure6.9shows the solution of the uncontrolled system (6.26), (6.27), i.e., with u≡0. For growingt the solution approaches an asymptotically stable steady state y∗∗=0. The figure (as well as all other figures in this section) was computed nu-merically using a finite difference scheme with 50 equidistant nodes on(0,1)(finer resolutions did not yield significantly different results) and initial value y0 with y0(0)=y0(1)=0,y0|[0.02,0.3]≡ −0.1,y0|[0.32,0.98]≡0.1 and linear interpolation in between.
By symmetry of (6.26) the function−y∗∗must be an asymptotically stable steady state, too. Furthermore, from (6.26) it is obvious thaty∗≡0 is another steady state, which is, however, unstable. Our goal is now to use NMPC in order to stabilize the unstable equilibriumy∗≡0.
To this end we consider the sampled data system corresponding to (6.26) with sampling period T =0.025. In order to obtain a more intuitive notation for the solution of the sampled data system, instead of introducing the abstract variablez
6.6 Design of Good Running Costs 139
Fig. 6.9 Solutiony(t, x)of (6.26), (6.27) withu≡0
as in Example 2.12 here we denote the state of the sampled data system at thenth sampling instant, i.e., at timenT byy(n,·). For penalizing the distance of the state y(n,·)toy∗≡0 a popular choice in the literature is theL2functional
y(n,·), u(n,·)
=y(n,·)2
L2()+λu(n,·)2
L2(), (6.28) which penalizes the mean squared distance fromy(n,·)toy∗≡0 and the control effort with weighting parameterλ >0. Here we chooseλ=0.1.
Another possible choice of measuring the distance toy∗≡0 is obtained by using theH1norm
y(n,·)
H1()=y(n,·)2
L2()+yx(n,·)2
L2(). This leads us to define
y(n,·), u(n,·)
=y(n,·)2
L2()+yx(n,·)2
L2()
+λu(n,·)2
L2(), (6.29)
which in addition to theL2distance and the control effort as in (6.28) also penal-izes the mean squared distance fromyx(n,·)toy∗,x≡0. Figs.6.10and6.11show the respective NMPC closed-loop solutions with optimization horizonsN =3 and N=11.
Figure6.10indicates that forN=3 the NMPC scheme withfrom (6.28) does not stabilize the system aty∗≡0, while forfrom (6.29) it does. For (6.28) we need an optimization horizon of at leastN=11 in order to obtain a stable closed-loop solution, cf. Fig.6.11. Forfrom (6.29) the right images in Figs.6.10and6.11show that enlarging the horizon does not improve the closed-loop behavior any further.
Using our theoretical results we can explain whyfrom (6.29) performs much better for small horizonsN. For this example our controllability condition Assump-tion6.4reads
y(n,·), u(n,·)
≤Cσn∗ y(0,·)
. (6.30)
140 6 Stability and Suboptimality Without Stabilizing Constraints
Fig. 6.10 NMPC closed loop for (6.26) withN=3 andfrom (6.28) (left) and (6.29) (right)
Fig. 6.11 NMPC closed loop for (6.26) withN=11 andfrom (6.28) (left) and (6.29) (right)
Forfrom (6.28) this becomes y(n,·)2
L2()+λu(n,·)2
L2()≤Cσny(0,·)2
L2(). (6.31) Now in order to control the system toy∗≡0, in (6.26) the control needs to com-pensate foryxandρ(y(t, x)−y(t, x)3), i.e., any control steeringy(n,·)to 0 must satisfy
u(n,·)2
L2()≈yx(n,·)2
L2()+ρ
y(n,·)−y(n,·)32
L2(). (6.32) Inserting this approximate equality into (6.31) implies—regardless of the value of σ—that the overshoot boundCin (6.31) is large ifyx(n,·)2L2() y(0,·)2L2()
holds, which is the case in our example.
Forfrom (6.29) Inequality (6.30) becomes y(n,·)2
L2()+yx(n,·)2
L2()+λu(n,·)2
L2()
≤Cσny(0,·)2
L2()+yx(0,·)2
L2()
. (6.33)
6.6 Design of Good Running Costs 141
Fig. 6.12 NMPC closed loop for (6.34) withN=15 andfrom (6.28) (left) and (6.29) (right)
Due to the fact thatyx(0,·)2L2() y(0,·)2L2()holds in our example, inserting the approximate equation (6.32) into (6.33) does not imply largeC, which explains the considerable better performance forfrom (6.29).
The fact that theH1-norm penalizes the distance toy∗≡0 in a “stronger” way than theL2-norm may lead to the conjecture that the better performance for this norm is intuitive. Our second example shows that this is not the case. This example is similar to equations (6.26), (6.27), except that the distributed control is changed to Dirichlet boundary control. Thus, (6.26) becomes
yt(t, x)=θyxx(t, x)−yx(t, x)+ρ
y(t, x)−y(t, x)3
, (6.34)
again withθ=0.1 andρ=10, and (6.27) changes to
y(t,0)=u0(t ), y(t,1)=u1(t ) for allt≥0, y(0, x)=y0(x) for allx∈
withu0, u1∈L∞(R,R). The cost functions (6.28) and (6.29) change to
y(n,·), u(n,·)
=y(n,·)2
L2()+λ
u0(n)2+u1(n)2
(6.35) and
y(n,·), u(n,·)
=y(n,·)2
L2()+yx(n,·)2
L2()
+λ
u0(n)2+u1(n)2
, (6.36)
respectively, again withλ=0.1.
Due to the more limited possibilities to control the equation the problem obvi-ously becomes more difficult, hence we expect to need larger optimization horizons for stability of the NMPC closed loop. However, what is surprising at first glance is thatfrom (6.35) stabilizes the system for smaller horizons thanfrom (6.36), as the numerical results in Fig.6.12confirm.
A closer look at the dynamics reveals that we can again explain this behavior with our theoretical results. In fact, steering the chosen initial solution toy∗=0 requires u1to be such that a rather large gradient appears close tox=1. Thus, during the
142 6 Stability and Suboptimality Without Stabilizing Constraints transient phaseyx(n,·)2L2() becomes large, which in turn causesfrom (6.36) to become large and thus causes a large overshoot boundC in (6.30). Infrom (6.35), on the other hand, these large gradients are not “visible”, which is why the overshoot in (6.30) is smaller and thus allows for stabilization with smallerN.