Consistent Approximations for the Optimal Control of Constrained Switched Systems-Part 2: An Implementable Algorithm

Consistent Approximations for the Optimal Control of Constrained

Switched Systems-Part 2: An Implementable Algorithm

The MIT Faculty has made this article openly available.

Please share

how this access benefits you. Your story matters.

Citation

Vasudevan, Ramanarayan, et al. "Consistent Approximations

for the Optimal Control of Constrained Switched Systems-Part

2: An Implementable Algorithm." Siam Journal on Control and

Optimization 51 6 (2013): 4484-503.

As Published

http://dx.doi.org/10.1137/120901507

Publisher

Society for Industrial and Applied Mathematics

Version

Final published version

Citable link

http://hdl.handle.net/1721.1/86029

Terms of Use

Article is made available in accordance with the publisher's

policy and may be subject to US copyright law. Please refer to the

publisher's site for terms of use.

CONSISTENT APPROXIMATIONS FOR THE OPTIMAL CONTROL OF CONSTRAINED SWITCHED SYSTEMS—PART 2:

AN IMPLEMENTABLE ALGORITHM∗

RAMANARAYAN VASUDEVAN†, HUMBERTO GONZALEZ‡, RUZENA BAJCSY§, AND

S. SHANKAR SASTRY§

Abstract. In the first part of this two-paper series, we presented a conceptual algorithm for

the optimal control of constrained switched systems and proved that this algorithm generates a sequence of points that converge to a necessary condition for optimality. However, since our algorithm requires the exact solution of a differential equation, the numerical implementation of this algorithm is impractical. In this paper, we address this shortcoming by constructing an implementable algorithm that discretizes the differential equation, producing a finite-dimensional nonlinear program. We prove that this implementable algorithm constructs a sequence of points that asymptotically satisfy a necessary condition for optimality for the constrained switched system optimal control problem. Four simulation experiments are included to validate the theoretical developments.

Key words. optimal control, switched systems, chattering lemma, consistent approximations AMS subject classifications. Primary, 49M25, 90C11, 90C30, 93C30; Secondary, 49J21,

49J30, 49J52, 49N25

DOI. 10.1137/120901507

1. Introduction. This paper, which is an extension of [14], continues our

pre-sentation on algorithms to solve the following optimal control problem: infh₀x(1)| ˙x(t) = ft, x(t), u(t), d(t), x(0) = x₀, h_jx(t)≤ 0,

(1.1)

x(t)∈ Rn, u(t)∈ U, d(t) ∈ Q, for a.e. t ∈ [0, 1] ∀j ∈ J,

whereQ and J are finite sets, U ⊂ Rmis a compact, convex, nonempty set, x₀∈ Rn,

f : [0, 1]× Rn× U × Q → Rnis a vector field that defines a switched system,{h_j}_j∈J is a collection of functions defining a feasible set, and h₀: Rn → R is a cost function.

In this paper, we discretize (1.1) to generate a finite-dimensional nonlinear pro-gram. We present an implementable algorithm, Algorithm 3.1, that constructs a sequence of finite-dimensional optimization problems that approximate the problem in (1.1). Our algorithm does not address the complexities that arise due to limited precision computations [8, 11]. We prove in Theorem 4.20 that the sequence of points generated by Algorithm 3.1 asymptotically satisfies the optimality condition defined in [14].

1.1. Related work. Understanding the relationship between each of the

solu-tions to a sequence of optimization problems and the solution to the optimization problem that they attempt to approximate is a well-studied problem. For example, the epiconvergence of a sequence of optimization problems, as in Definition 7.1 in [13],

∗_{Received by the editors December 6, 2012; accepted for publication (in revised form) September 3,}

2013; published electronically December 11, 2013. This material is based upon work supported by the National Science Foundation under award ECCS-0931437.

http://www.siam.org/journals/sicon/51-6/90150.html

†_{CSAIL, Massachusetts Institute of Technology, Cambridge, MA 02139 (ramv@csail.mit.edu).} ‡_{ESE, Washington University in St. Louis, St. Louis, MO 63130 (hgonzale@ese.wustl.edu).} §_{EECS, University of California at Berkeley, Berkeley, CA 94720 (bajcsy@eecs.berkeley.edu,}

sastry@eecs.berkeley.edu).

4484

guarantees the convergence of the sequence of global minimizers for each optimiza-tion problem to the set of global minimizers of the limiting problem (Theorem 7.33 in [13]). Unfortunately derivative-based optimization algorithms are only capable of constructing points that satisfy a necessary condition for optimality. However, if the sequence of optimization problems consistently approximates some limiting optimiza-tion problem, as in Definioptimiza-tion 3.3.6 in [12], then every sequence of points that satisfy a necessary condition for optimality for each optimization problem converges to the set of points that satisfy a necessary condition for optimality for the limiting optimization problem (Theorem 3.3.11 in [12]).

1.2. Our contribution and organization. In this paper, we devise an

im-plementable algorithm that generates a sequence of optimization problems and cor-responding points that consistently approximates the constrained switched optimal control problem defined in (3.13) in [14]. In sections 2 and 3, we formalize the finite-dimensional optimization problem we solve at each iteration of our implementable algorithm. In section 4, we prove that the sequence of points generated by our imple-mentable algorithm asymptotically satisfies the necessary condition for optimality for the switched system optimal control problem in [14]. In section 5, we illustrate the su-perior performance of our approach on four examples when compared to a commercial mixed integer programming solver.

2. Notation. First, we summarize the notation used in this paper, defined in

the first part of this two-paper series [14].

·_p standard vector space p-norm

·_i,p matrix induced p-norm

·_Lp functional Lp-norm

Q = {1, . . . , q} set of discrete mode indices (section 1)

J = {1, . . . , Nc} set of constraint function indices (section 1)

Σq_r q-simplex inRq (equation (3.5))

Σq_p={e_i}_i∈Q corners of the q-simplex inRq (equation (3.6))

U ⊂ Rm compact, convex, nonempty set (section 1)

Dr relaxed discrete input space (section 3.2)

Dp pure discrete input space (section 3.2)

U continuous input space (section 3.2)

(X , ·_X) general optimization space and its norm (section 3.2)

Xr relaxed optimization space (section 3.2)

Xp pure optimization space (section 3.2)

d_i or [d]_i ith coordinate of d : [0, 1]→ Rq f : [0, 1]× Rn× U × Q → Rn switched system (section 1)

V : L2([0, 1],Rn)→ R total variation operator (equation (3.3))

h₀:Rn→ R terminal cost function (section 3.3)

{hj:Rn→ R}_j∈J collection of constraint functions (section 3.3) x(ξ): [0, 1]→ Rn trajectory of the system for ξ∈ X_r(equation (3.8))

φ_t:X_r→ Rn flow of the system at t∈ [0, 1] (equation (3.9))

J : X_r→ R cost function (equation (3.10))

Ψ :X_r→ R constraint function (equation (3.11))

ψ_j,t:X_r→ R component constraint functions (equation (3.12))

D derivative operator (equation (4.2))

θ :X_p→ (−∞, 0] optimality function (equation (4.3))

g : X_p→ X_r descent direction function (equation (4.3))

ζ :X_p× X_r→ R optimality function objective (equation (4.4))

½A: [0, 1]→ {0, 1} indicator function of A⊂ [0, 1],½=½[0,1] (section 4.4)

FN:Dr→ Dr Haar wavelet operator (equation (4.7))

PN: Dr→ Dp pulse-width modulation operator (equation (4.8)) ρ_N:X_r→ X_p projection operator (equation (4.9))

Next, we summarize the notation used and defined in this paper. Note that whenever the subscript τ appears, τ is assumed to belong toT_N with N ∈ N.

TN N th switching time space (equation (3.1))

Dτ,r discretized relaxed discrete input space (equation (3.3))

Dτ,p discretized pure discrete input space (equation (3.2))

Uτ discretized continuous input space (equation (3.4))

Xτ discretized general optimization space (equation (3.5))

Xτ,r discretized relaxed optimization space (section 3.1) Xτ,p discretized pure optimization space (section 3.1)

Nτ(ξ, ε) ε-ball around ξ in the τ -inducedX -norm (Definition 3.2)

|τ| last index of τ (section 3.1)

z_τ(ξ): [0, 1]→ Rn discretized trajectory given ξ ∈ X_τ,r (equation (3.7))

φ_τ,t:X_τ,r→ Rn discretized flow of the system at t∈ [0, 1] (equation (3.8))

J_τ:X_τ,r→ R discretized cost function (equation (3.9)) Ψ_τ:X_τ,r→ R discretized constraint function (equation (3.10))

ψ_τ,j,t:X_τ,r→ R discretized component constraint function (equation (3.11))

P_τ discretized relaxed optimal control problem (equation (3.12))

θ_τ: X_τ,p→ (−∞, 0] discretized optimality function (equation (3.13))

g_τ:X_τ,p→ X_τ,r discretized descent direction function (equation (3.13))

ζ_τ:X_τ,p× X_τ,r → R discretized optimality function objective (equation (3.14))

μ_τ: X_τ,p→ N discretized step size function (equation (3.15))

σ_N:X_r→ T_N induced partition function given N ∈ N (equation (3.16))

ν_τ:X_τ,p× N → N discretized frequency modulation function (equation (3.17)) Γ_τ:X_τ,p→ X_p implementable algorithm recursion function (equation (3.18))

3. Implementable algorithm. In this section, we begin by describing our

dis-cretization strategy in order to define our discretized optimization spaces. Next, we construct the discretized trajectories, cost, constraints, and optimal control problems. This allows us to define a discretized optimality function and a notion of consistent

approximation between the optimality function and its discretized counterpart. We

conclude by describing our numerically implementable optimal control algorithm for constrained switched systems.

3.1. Discretized optimization space. For each N ∈ N we define the Nth

switching time space as

(3.1) T_N =  {ti}k_i=0⊂ [0, 1] | k ∈ N, 0 = t0≤ · · · ≤ tk = 1, |t_i− t_i−1| ≤ 1 2N ∀i  . TN is the collection of finite partitions of [0, 1] whose samples have a maximum distance of 2−N. Given τ ∈ T_N, we abuse notation and let|τ| denote the last index of τ, i.e.,

τ = {t_k}|τ|_k=0. We use this convention since it simplifies our indexing in this paper. Notice that for each N ∈ N, T_N+1⊂ T_N.

Given N ∈ N and τ = {t_i}|τ|_i=0 ∈ T_N, let the discretized relaxed discrete input space be (3.2) D_τ,r=  d∈ D_r| d = |τ|−1 k=0 ¯ d_k½[t k,tk+1), _¯ d_k|τ|−1 k=0 ⊂ Σqr .

Similarly, let the discretized pure discrete input space be

(3.3) D_τ,p=  d∈ D_p| d = |τ|−1 k=0 ¯ d_k½[t k,tk+1), _¯ d_k|τ|−1_k=0 ⊂ Σq_p .

Finally, let the discretized continuous input space be

(3.4) U_τ =  u∈ U | u = |τ|−1 k=0 ¯ u_k½[t k,tk+1), {¯uk} |τ|−1 k=0 ⊂ U .

Now, we can define the discretized pure optimization space asX_τ,p =U_τ× D_τ,p and the discretized relaxed optimization space asX_τ,r =U_τ× D_τ,r. Recall that the general optimization space,X , is defined as X = L2([0, 1],Rm)×L2([0, 1],Rq) and is endowed with the normξ_X =u_L2+d_L2 for each ξ = (u, d)∈ X . We define a subspace

ofX , the discretized general optimization space: (3.5) Xτ =  (u, d)∈ X | (u, d) = |τ|−1 k=0  ¯ u_k, ¯d_k½[t k,tk+1),  ¯ u_k, ¯d_k|τ|−1_k=0 ⊂ Rm× Rq .

We endow the discretized pure and relaxed optimization spaces with the X -norm as well.

The following lemma, which follows since every measurable function can be ap-proximated in the L2-norm by sums of indicator functions (Theorem 2.10 in [5]), proves that the spaces X_τ,r and X_τ,p can be used to build approximations of points inX_p andX_r, respectively.

Lemma 3.1. Let {τ_k}

k∈N such that τk ∈ Tk for each k∈ N.

(1) For each ξ∈ X_p there exists{ξ_k}_k∈N, with ξ_k ∈ X_τk,p, such thatξ_k− ξ_X →

0 as k→ ∞.

(2) For each ξ ∈ X_r there exists{ξ_k}_k∈N, with ξ_k ∈ X_τk,r, such thatξ_k− ξ_X →

0 as k→ ∞.

3.2. Discretized trajectories, cost, constraint, and optimal control problem. Let N ∈ N, τ = {t_i}|τ|_i=0 ∈ T_N, ξ = (u, d) ∈ X_τ,r, and x₀ ∈ Rn. We define the discretized trajectory, z_τ(ξ): [0, 1]→ Rn, by performing linear interpolation over points computed via the forward Euler integration formula, i.e., z_τ(ξ)(0) = x₀, for each k∈ {0, . . . , |τ| − 1}, (3.6) z_τ(ξ)(t_k+1) = z_τ(ξ)(t_k) + (t_k+1− t_k)ft_k, z(ξ)_τ (t_k), u(t_k), d(t_k) and (3.7) z_τ(ξ)(t) = |τ|−1 k=0  z_τ(ξ)(t_k) + (t− t_k)ft_k, z(ξ)_τ (t_k), u(t_k), d(t_k)½[t k,tk+1)(t).

For notational convenience, we suppress the dependence on τ in z_τ(ξ)when it is clear in context. The discretized flow of the system, φ_τ,t:X_τ,r → Rn, for each t∈ [0, 1] is

(3.8) φ_τ,t(ξ) = z(ξ)_τ (t). The discretized cost function, J_τ:X_τ,r→ R, is (3.9) J_τ(ξ) = h₀z_τ(ξ)(1).

Similarly, the discretized constraint function, Ψ_τ:X_τ,r → R, is (3.10) Ψ_τ(ξ) = maxh_jz(ξ)_τ (t_k)| j ∈ J , k ∈ {0, . . . , |τ|}.

For any positive integer N and τ ∈ T_N, the discretized component constraint function,

ψ_τ,j,t:X_τ,r→ R, for each t ∈ [0, 1] and each j ∈ J is (3.11) ψ_τ,j,t(ξ) = h_jz(ξ)_τ (t).

3.3. Local minimizers and a discretized optimality condition. Let N ∈ N

and τ ∈ T_N. Proceeding as we did in section 4.1 in [14], we define the discretized

relaxed optimal control problem as

inf{J_τ(ξ)| Ψ_τ(ξ)≤ 0, ξ ∈ X_τ,r} . (P_τ)

(3.12)

The local minimizers of this problem are then defined as follows.

Definition 3.2. Let N ∈ N and τ ∈ T_N. Let an ε-ball in the τ -inducedX -norm around ξ be defined as N_τ(ξ, ε) =ξ¯∈ X_τ,r | ξ− ¯ξ

X < ε 

. A point ξ ∈ X_τ,r is a

local minimizer of P_τ if Ψ_τ(ξ)≤ 0 and there exists ε > 0 such that J_τ( ˆξ)≥ J_τ(ξ) for

each ˆξ∈ N_τ(ξ, ε)∩ξ¯∈ X_τ,r| Ψ_τ( ¯ξ)≤ 0.

Let the discretized optimality function, θ_τ:X_τ,p → (−∞, 0], and the discretized

descent direction, g_τ:X_τ,p→ X_τ,r, be defined by

θ_τ(ξ) = min ξ_∈Xτ,rζτ(ξ, ξ _{) + V(ξ}_{− ξ), g} τ(ξ) = arg min ξ_∈Xτ,r ζτ(ξ, ξ _{) + V(ξ}_{− ξ),} (3.13) where (3.14) ζ_τ(ξ, ξ) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ max  max (j,k)∈J ×{0,...,|τ|}Dψτ,j,tk(ξ; ξ− ξ) + γΨτ(ξ), DJ_τ(ξ; ξ− ξ)  +ξ− ξ_X if Ψ_τ(ξ)≤ 0, max  max (j,k)∈J ×{0,...,|τ|}Dψτ,j,tk(ξ; ξ− ξ), DJ_τ(ξ; ξ− ξ) − Ψ_τ(ξ)  +ξ− ξ_X if Ψ_τ(ξ) > 0, where γ > 0 is the same design parameter used for θ in [14]. Recall that any element of the discretized relaxed optimization space by definition can be naturally identified with an element of the relaxed optimization space. The total variation is applied to each component of the discretized relaxed optimization space once this natural identification has taken place. Note that θ_τ(ξ) ≤ 0 for each ξ ∈ X_τ,p as shown in Theorem 4.10. Also note that the directional derivatives in (3.14) consider directions

ξ− ξ in order to ensure that all constructed descent directions belong to X_τ,r. As we

did in the infinite-dimensional case, we prove in Theorem 4.10 that if θ_τ(ξ) < 0 for some ξ∈ X_τ,p, then ξ is not a local minimizer of P_τ, or in other words, ξ satisfies the

discretized optimality condition if θ_τ(ξ) = 0. Note that the total variation of terms in

Xτ can be computed using a finite sum by (3.3) in [14].

Theorem 5.10 in [14], which is an extension of the chattering lemma, describes a constructive method, with a known error bound, to find pure discrete-valued ap-proximations of points in X_r in the weak topology. The result of that theorem is fundamental in this paper, since we use it not only to construct suitable approxima-tions but also to determine when to increase the discretization precision by choosing a finer switching time τ . The following definition and theorem lay the foundation for the technique we use to prove that our implementable algorithm converges to the same points as Algorithm 4.1 in [14] does by recursive application.

Definition 3.3 (Definition 3.3.6 in [12]). Let {τ_i}

i∈N be such that τi ∈ Ti for each i∈ N. We say that {P_τi}_i∈N is a consistent approximation of P_p if given{ξ_i}_i∈N such that ξ_i∈ X_τi,p for each i∈ N, then lim inf_i→∞θ_τi(ξ_i)≤ lim inf_i→∞θ(ξ_i).

Theorem 3.4. Let {τ_i}

i∈N and {ξi}i∈N be such that τi ∈ Ti and ξi ∈ Xτi,p

for each i ∈ N, and suppose that {P_τi}_i∈N is a consistent approximation of P_p. If

lim_i→∞θ_τi(ξ_i) = 0, then lim_i→∞θ(ξ_i) = 0.

We omit the proof of Theorem 3.4 since it follows trivially from Definition 3.3. Note that Theorem 3.4 implies that our discretized optimality condition is asymptot-ically equivalent to the optimality condition in [14]. In Theorem 4.12 we show that

{Pτ}_i∈N is indeed a consistent approximation of P_p.

3.4. Choosing a discretized step size and projecting the discretized relaxed discrete input. We employ a line search equivalent to the one formulated

in (4.5) in [14], which is inspired by the traditional Armijo algorithm [1], in order to choose a step size. Letting N ∈ N, τ ∈ T_N, and α, β ∈ (0, 1), a step size for a point ξ is chosen as βμτ(ξ)_{, where the discretized step size function, μτ:}X_τ,p→ N, is

defined as (3.15) μ_τ(ξ) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ mink∈ N | J_τξ + βk(g_τ(ξ)− ξ)− J_τ(ξ)≤ αβkθ_τ(ξ), Ψ_τξ + βk(g_τ(ξ)− ξ)≤ αβkθ_τ(ξ) if Ψ_τ(ξ)≤ 0, mink∈ N | Ψ_τξ + βk(g_τ(ξ)− ξ)− Ψ_τ(ξ)≤ αβkθ_τ(ξ) if Ψ_τ(ξ) > 0. Lemma 4.13 proves that if θ_τ(ξ) < 0 for some ξ∈ X_τ,p, then μ_τ(ξ) <∞, which means that we can construct a pointξ + βμτ(ξ)_(g

τ(ξ)− ξ)∈ Xτ,rthat produces a reduction in the cost (if ξ is feasible) or a reduction in the infeasibility (if ξ is infeasible).

Continuing as we did in section 4.4 in [14], notice that the Haar wavelet operator

FN induces a partition inTN according to the times at which the constructed relaxed discrete input switched. That is, let the induced partition function, σ_N:X_r→ T_N, be defined as (3.16) σ_N(u, d) ={0} ∪  k 2N + 1 2N i j=1 [F_N(d)]_j  k 2N  k∈{0,...,2N₋₁} i∈{1,...,q} .

Thus, ρ_N(ξ)∈ X_σN(ξ),p for each ξ∈ X_r. Let N ∈ N, τ ∈ T_N, ¯α∈ (0, ∞), ¯β ∈ (√1

2, 1), ω ∈ (0, 1), and kM ∈ N. Then, similar to our definition of ν in [14], we modulate a point ξ with frequency 2−ντ(ξ,kM)_,

where the discretized frequency modulation function, ν_τ: X_τ,p× N → N, is defined as

(3.17) ν_τ(ξ, k_M) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ mink∈ N |k ≤ k_M, ξ= ξ + βμτ(ξ)_gτ(ξ)− ξ_, J_σk(ξ₎ρ_k(ξ)− J_τ(ξ)≤αβμτ(ξ)− ¯α ¯βk_θτ(ξ), Ψ_σk(ξ₎ρ_k(ξ)≤αβμτ(ξ)− ¯α ¯βk_θτ(ξ), ¯ α ¯βk ≤ (1 − ω)αβμτ(ξ) _{if Ψ} τ(ξ)≤ 0, mink∈ N |k ≤ k_M, ξ= ξ + βμτ(ξ)_g τ(ξ)− ξ, Ψ_σ k(ξ)  ρ_k(ξ)− Ψ_τ(ξ)≤αβμτ(ξ)− ¯α ¯βk_θ τ(ξ), ¯ α ¯βk ≤ (1 − ω)αβμτ(ξ) _{if Ψτ(ξ) > 0.}

There is no guarantee that the optimization problem in ν_τis feasible. Without loss of generality, we let ν_τ(ξ, k_M) =∞ for each ξ ∈ X_τ,p when there is no feasible solution. Importantly, in Lemma 4.16 we prove that given N₀ ∈ N, τ ∈ T_N0, and ξ ∈ X_τ,p, if

θ(ξ) < 0, then for each η∈ N there exists N ≥ N₀ such that ν_σN(ξ)(ξ, N + η) is finite, i.e., that using (3.17) we can compute a new point that moves closer to the optimality condition with a known switching time sequence.

3.5. An implementable switched system optimal control algorithm.

Algorithm 3.1 describes our numerical method to solve P_p, as defined in [14]. Note that at each step of this algorithm, ξ_j ∈ X_τj,p. Also note that the condition in line 4 checks if θ_τ gets too close to zero or if ν_τ equals infinity, in which case the discretiza-tion precision is increased. To understand why the first condidiscretiza-tion in line 4 is required, recall that we look for points ξ such that θ(ξ) = 0 and that θ_τj(ξ_j) = 0 does not imply that θ(ξ_j) = 0, and thus if θ_τj(ξ_j) is too close to zero the algorithm may terminate without reaching our objective. Finally, notice that due to the definitions of DJ_τand Dψ_τ,j,tk for each j∈ J and k ∈ {0, . . . , |τ|} which appear in Corollary 4.5, θ_τ can be solved via a quadratic program.

Algorithm 3.1Numerically tractable optimization algorithm for P_p.

Require: N₀∈ N, τ₀∈ T_N0, ξ₀∈ X_τ0,p, α, β, ω∈ (0, 1), ¯α, γ, Λ ∈ (0, ∞), ¯β ∈√1 2, 1  , η∈ N, χ ∈0,1₂. 1: Set j = 0. 2: loop 3: Compute θ_τj(ξ_j), g_τj(ξ_j), μ_τj(ξ_j), and ν_τj(ξ_j, N_j+ η). 4: if θ_τj(ξ_j) >−Λ2−χNj _{or ντ} j(ξj, Nj+ η) =∞ then 5: Set ξ_j+1= ξ_j. 6: Set N_j+1= N_j+ 1. 7: else 8: Set ξ_j+1= ρ_ν τj(ξj,Nj+η)  ξ_j+ βμτj(ξj)_(g τj(ξj)− ξj)  . 9: Set N_j+1= maxN_j, ν_τj(ξ_j, N_j+ η). 10: end if 11: Set τ_j+1= σ_Nj+1(ξ_j+1). 12: Replace j by j + 1. 13: end loop

For analysis purposes, we define the implementable algorithm recursion function, Γ_τ: {ξ ∈ X_τ,p| ν_τ(ξ, k_M) <∞} → X_p, as

(3.18) Γ_τ(ξ) = ρ_ντ(ξ,kM)ξ + βμτ(ξ)_(g

τ(ξ)− ξ).

We prove several important properties about the sequence generated by Algorithm 3.1. First, we prove in Lemma 4.17 that as the iteration index i increases lim_i→∞N_i=∞ which implies that the discretization mesh size tends to zero. Second, letting{N_i}_i∈N,

{τi}i∈N, and {ξi}i∈N be the sequences produced by Algorithm 3.1, then, as we prove in Lemma 4.18, there exists i₀ ∈ N such that if Ψ_τi0(ξ_i0) ≤ 0, then Ψ(ξ_i) ≤ 0 and Ψ_τi(ξ_i)≤ 0 for each i ≥ i₀. Finally, as we prove in Theorem 4.20, lim_j→∞θ(ξ_j) = 0, i.e., Algorithm 3.1 produces sequences that asymptotically satisfy the optimality condition.

4. Implementable algorithm analysis. In this section, we derive the various

components of Algorithm 3.1 and prove that it produces sequences that asymptotically satisfy our optimality condition. Our argument proceeds as follows: first, we prove the continuity and convergence of the discretized state, cost, and constraints to their infinite-dimensional analogues; second, we construct the components of the optimality function and prove the convergence of these discretized components to their infinite-dimensional analogues; finally, we prove the convergence of our algorithm.

4.1. Continuity and convergence of the discretized components. In this

subsection, we note the continuity and convergence of the discretized state, cost, and constraint. The following two lemmas follow from the discrete Bellman–Gronwall inequality and Jensen’s inequality.

Lemma 4.1. _{There exists C > 0 such that for each N} ∈ N, τ ∈ T_{N, ξ} ∈ X_τ,r,

and t∈ [0, 1], z_τ(ξ)(t) ₂≤ C.

Lemma 4.2. There exists L > 0 such that for each N∈ N, τ ∈ T_N, ξ₁, ξ₂∈ X_τ,r,

and t∈ [0, 1], φ_τ,t(ξ₁)− φ_τ,t(ξ₂)₂≤ L ξ₁− ξ₂_X.

We now find the rate of convergence of our discretization scheme.

Lemma 4.3. _{There exists C > 0 such that for each N} ∈ N, τ ∈ T_{N, ξ} ∈ X_τ,r,

and t∈ [0, 1],

(1) z_τ(ξ)(t)− x(ξ)(t)

2≤2CN,

(2) |J_τ(ξ)− J(ξ)| ≤ ₂CN, and

(3) |Ψ_τ(ξ)− Ψ(ξ)| ≤ ₂CN.

Proof. The first result follows directly from Picard’s lemma (Lemma 5.6.3 in [12]).

The second result follows from the Lipschitz continuity of h₀and Lemma 4.3. To prove the third result, first note that for each τ ∈ T_N, k ∈ {0, . . . , |τ| − 1}, and t∈ [t_k, t_k+1], h_jx(ξ)(t)− h_jx(ξ)(t_k) ≤ ₂C_N, which follows from Theorem 3.3 and Assumption 3.4 in [14]. Thus, by the triangle inequality there exists C > 0 such that for each t∈ [t_k, t_k+1],h_jx(ξ)(t)− h_jz_τ(ξ)(t_k) ≤ ₂C_N.

Let t ∈ arg max_t∈[0,1]h_jx(ξ)(t) and κ(t) ∈ {0, . . . , |τ| − 1} such that t ∈ [t_κ(t₎, t_κ(t₎₊₁]. Then, (4.1) max t∈[0,1]hj  x(ξ)(t)− max k∈{0,...,|τ|}hj  z_τ(ξ)(t_k)≤ h_jx(ξ)(t)− h_jz_τ(ξ)(t_κ(t₎).

Similarly if k∈ arg max_{k∈{0,...,|τ|}}h_jz_τ(ξ)(t_k), then

(4.2) max k∈{0,...,|τ|}hj  z_τ(ξ)(t_k)− max t∈[0,1]hj  x(ξ)(t)≤ h_jz_τ(ξ)(t_k)− h_jx(ξ)(t_k).

The result follows directly from these inequalities.

4.2. Derivation of the implementable algorithm terms. Next, we formally

derive the components of θ_τ, prove its well-posedness, and bound the error between

θ_τ and θ.

Lemma 4.4. Let N ∈ N, τ ∈ T_N, ξ = (u, d)∈ X_τ,r, ξ= (u, d)∈ X_τ. Then, for

each k∈ {0, . . . , |τ|}, the directional derivative of φ_τ,tk is

Dφ_τ,tk(ξ; ξ) = k−1 j=0 (t_j+1− t_j)Φ(ξ)_τ (t_k, t_j+1)  ∂f ∂u  t_j, φ_τ,tj(ξ), u(t_j), d(t_j)u(t_j) (4.3) + ft_j, φ_τ,tj(ξ), u(t_j), d(t_j),

where Φ(ξ)_τ (t_k, t_j) is the solution of the following matrix difference equation: (4.4) Φ(ξ)_τ (t_k+1, t_j) = Φ(ξ)_τ (t_k, t_j) + (t_k+1− t_k)∂f ∂x  t_k, φ_τ,tk(ξ), u(t_k), d(t_k)Φ(ξ)_τ (t_k, t_j) ∀k ∈ {0, . . . , |τ| − 1} , Φ(ξ) τ (tj, tj) = I.

Proof. For notational convenience, let z(λ) = z(ξ+λξ), u(λ) = u + λu, d(λ) =

d + λd, Δz(λ)= z(λ)− z(ξ), and Δt_k= t_k+1− t_k. Thus, for each k∈ {0, . . . , |τ|},

Δz(λ)(t_k) = k−1 j=0 Δt_j  λ q i=1 d_i(t_j)ft_j, z(λ)(t_j), u(λ)(t_j), e_i (4.5) +∂f ∂x  t_j, z(ξ)(t_j) + υ_x,jΔz(λ)(t_j), u(λ)(t_j), d(t_j)Δz(λ)(t_j) + λ∂f ∂u 

t_j, z(ξ)(t_j), u(t_j) + υ_u,jλu(t_j), d(t_j)u(t_j) 

,

where {υ_x,j}|τ|_j=0,{υ_u,j}|τ|_j=0⊂ [0, 1]. Let {y(t_k)}|τ|_k=0be recursively defined as follows:

y(t₀) = 0, and for each k∈ {0, . . . , |τ| − 1},

y(t_k+1) = y(t_k) + Δt_k  ∂f ∂x  t_k, z(ξ)(t_k), u(t_k), d(t_k)y(t_k) (4.6) +∂f ∂u  t_k, z(ξ)(t_k), u(t_k), d(t_k)u(t_k) + ft_k, z(ξ)(t_k), u(t_k), d(t_k).

We want to show that lim_λ↓0 1_λΔz(λ)(t_k)− y(t_k) = 0. Consider ∂f∂x  t_k, z(ξ)(t_k), u(t_k), d(t_k)y(t_k) (4.7) −∂f ∂x  t_k, z(ξ)(t_k) + υ_x,kΔz(λ)(t_k), u(λ)(t_k), d(t_k) Δz (λ)_(tk) λ 2 ≤ L y(tk)−Δz (λ)_(t k) λ 2 + L Δz(λ)_(t k) ₂+ λu(tk)2  y(tk)2, which follows from Assumption 3.2 in [14] and the triangle inequality. Also,

∂f_∂ut_k, z(ξ)(t_k), u(t_k), d(t_k) (4.8) −∂f ∂u 

t_k, z(ξ)(t_k), u(t_k) + υ_u,kλu(t_k), d(t_k)u(t_k) 2

≤ Lλ u_(t k)22

and q i=1 d_i(t_k)  ft_k, z(ξ)(t_k), u(t_k), e_i− ft_k, z(λ)(t_k), u(λ)(t_k), e_i 2 (4.9) ≤ L Δz(λ)_(t k) ₂+ Lλu(tk)2.

Using the discrete Bellman–Gronwall inequality and the inequalities above, y(tk)−Δz (λ)_(t k) λ 2≤ Le Lk−1 j=0 Δt_j Δz(λ)(t_j) ₂+ λu(t_j)₂y(t_j)₂ (4.10) + λu(t_j)2₂+ Δz(λ)(t_j) 2+ λu(tj)2  ,

where we used the fact that1 + ₂LN

 L

2N ≤ eL_{. By Lemma 4.2, the right-hand side}

of (4.10) goes to zero as λ ↓ 0, thus showing that Δz(λ)(tk)

λ → y(tk). The result of the lemma is obtained after noting that Dφ_τ,tk(ξ; ξ) is equal to y(t_k) for each

k∈ {0, . . . , |τ|}.

Next, we construct a directional derivative of the discretized cost and component constraint functions, which follow by the chain rule and Lemma 4.4.

Corollary 4.5. Let N ∈ N, τ ∈ T_N, ξ ∈ X_τ,r, ξ ∈ X_τ, j ∈ J , and k ∈

{0, . . . , |τ|}. Then the directional derivatives of the discretized cost Jτ and component constraint ψ_τ,j,tk in the ξ direction are

(4.11) DJ_τ(ξ; ξ) = ∂h0 ∂x  φ_τ,1(ξ)Dφ_τ,1(ξ; ξ), Dψ_τ,j,tk(ξ; ξ) = ∂hj ∂x  φ_τ,tk(ξ)Dφ_τ,tk(ξ; ξ). We omit the proof of the following lemma since it follows from arguments similar to Lemma 5.4 in [14].

Lemma 4.6. There exists L > 0 such that for each N∈ N, τ ∈ T_N, ξ₁, ξ₂∈ X_τ,r,

ξ∈ X_τ, and k∈ {0, . . . , |τ|},

(1) Dφ_τ,tk(ξ₁; ξ)− Dφ_τ,tk(ξ₂; ξ)₂≤ L ξ₁− ξ₂_Xξ_X,

(2) |DJ_τ(ξ₁; ξ)− DJ_τ(ξ₂; ξ)| ≤ L ξ₁− ξ₂_Xξ_X,

(3) |Dψ_τ,j,tk(ξ₁; ξ)− Dψ_τ,j,tk(ξ₂; ξ)| ≤ L ξ₁− ξ₂_Xξ_X, and

(4) |ζ_τ(ξ₁, ξ)− ζ_τ(ξ₂, ξ)| ≤ L ξ₁− ξ₂_X.

Lemma 4.7. _{There exists C > 0 such that for each N} ∈ N, τ ∈ T_{N, ξ} ∈ X_τ,r,

ξ∈ X_τ and k∈ {0, . . . , |τ|},

(1) Dφ_tk(ξ; ξ)− Dφ_τ,tk(ξ; ξ)₂≤₂CN,

(2) |DJ_τ(ξ; ξ)− DJ(ξ; ξ)| ≤₂CN,

(3) |Dψ_τ,j,tk(ξ; ξ)− Dψ_j,tk(ξ; ξ)| ≤ ₂C_N, and

(4) |ζ_τ(ξ, ξ)− ζ(ξ, ξ)| ≤ ₂C_N.

Proof. By applying Picard’s lemma (Lemma 5.6.3 in [12]) together with the triangle inequality, one can show that the error between Φ(ξ)_τ and Φ(ξ) is bounded by ₂C_N, for some C > 0. Conditions (1), (2), and (3) then follow from Lemma 4.6, the boundedness and Lipschitz continuity of the vector fields and their derivatives, and the boundedness of the state transition matrix.

To prove condition (4), let κ(t)∈ {0, . . . , |τ| − 1} such that t ∈ [t_κ(t), t_κ(t)+1] for each t∈ [0, 1]. Then, applying condition (3) and the triangle inequality we get that

there exists C > 0 such that Dψ_j,t(ξ; ξ− ξ) − Dψ_τ,j,tκ(t)(ξ; ξ− ξ) ≤ ₂C_N for each

j ∈ J and t ∈ [0, 1]. The result follows using the same argument as in the proof of

Lemma 4.3.

The following result shows that g_τ is a well-defined function.

Lemma 4.8. Let N ∈ N, τ ∈ T_N, and ξ ∈ X_τ,p. Then the map ξ → ζ_τ(ξ, ξ) + V(ξ− ξ), with domain X_τ,r, is strictly convex and has a unique minimizer.

Proof. The map is strictly convex since it is the sum of the maximum of linear

functions of ξ− ξ (which is convex), the total variation of ξ− ξ (which is convex), and theX -norm of ξ− ξ (which is strictly convex). The uniqueness of the minimizer follows after noting thatX_τ,r is a compact set.

Employing these results we can prove the continuity of θ_τ. This result is not strictly required in order to prove the convergence of Algorithm 3.1. However, it is useful from an implementation point of view, since continuity gives a guarantee that our numerical computations are good approximations.

Lemma 4.9. Let N ∈ N and τ ∈ T_N. Then θ_τ is continuous.

Proof. First note that V, when restricted to elements in X_τ,r, is a continuous operator, since the supremum in its definition is always attained at the partition induced by τ .

Consider a sequence {ξ_i}_i∈N ⊂ X_τ,r converging to ξ ∈ X_τ,r. Since θ_τ(ξ_i) ≤

ζ_τξ_i, g_τ(ξ)+ V(g_τ(ξ)− ξ_i) for each i ∈ N, then by condition (4) in Lemma 4.6 and the continuity of V we have lim sup_i→∞θ_τ(ξ_i)≤ θ_τ(ξ), which proves the upper semicontinuity of θ_τ. Using a similar argument and the reverse triangle inequality, we get that

θ_τ(ξ)≤ ζ_τξ, g_τ(ξ_i)+ V(g_τ(ξ_i)− ξ) − θ_τ(ξ_i) + θ_τ(ξ_i) (4.12)

≤ L ξ − ξiX+ V(ξ− ξi) + θτ(ξi).

Taking a lim inf proves the lower semicontinuity of θ_τ and our desired result. Next, we prove that the local minimizers of P_τ are in fact zeros of θ_τ.

Theorem 4.10. Let N ∈ N, τ ∈ T_N. Then, θ_τ is nonpositive valued. Moreover,

if ξ∈ X_τ,p is a local minimizer of P_τ, then θ_τ(ξ) = 0.

Proof. Note that ζ_τ(ξ, ξ) = 0 and θ_τ(ξ)≤ ζ_τ(ξ, ξ), which proves the first part. To prove the second part, we begin by making several observations. Given ξ ∈

Xτ,r and λ∈ [0, 1], using the mean value theorem and condition (2) in Lemma 4.6, there exists L > 0 such that

(4.13) J_τξ + λ(ξ− ξ)− J_τ(ξ)≤ λ DJ_τξ; ξ− ξ+ Lλ2ξ− ξ2_X.

LetA_τ(ξ) = arg minh_jφ_τ,tk(ξ)| (j, k) ∈ J × {0, . . . , |τ|}; then there exists a pair (j, k)∈ Aξ + λ(ξ− ξ)such that using condition (3) in Lemma 4.6

(4.14) Ψ_τξ + λ(ξ− ξ)− Ψ_τ(ξ)≤ λ Dψ_τ,j,tkξ; ξ− ξ+ Lλ2ξ− ξ2_X.

We proceed by contradiction, i.e., we assume that θ_τ(ξ) < 0 and show that for each ε > 0 there exists ˆξ ∈ N_τ(ξ, ε)∩ξ¯∈ X_τ,r| Ψ_τ( ¯ξ)≤ 0 such that J_τ( ˆξ) < J_τ(ξ). Note that since θ_τ(ξ) < 0, g_τ(ξ) = ξ. For each λ > 0 by using (4.13),

J_τξ + λ(g_τ(ξ)− ξ)− J_τ(ξ)≤ θ_τ(ξ)λ + 4A2Lλ2, where A = maxu₂+ 1| u ∈ U and we used the fact thatξ− ξ2_X ≤ 4A2 and DJ_τ(ξ; g(ξ)− ξ) ≤ θ_τ(ξ). Hence for each λ∈0,−θτ(ξ)

4A2_L



, J_τξ + λ(g_τ(ξ)− ξ)− J_τ(ξ) < 0. Similarly, for each λ > 0, by using (4.14), we have

(4.15) Ψ_τξ + λ(g_τ(ξ)− ξ)≤ Ψ_τ(ξ) +θ_τ(ξ)− γΨ_τ(ξ)λ + 4A2Lλ2,

where we used the fact that Dψ_τ,j,tk(ξ; g(ξ)− ξ) ≤ θ_τ(ξ)− γΨ_τ(ξ) for each (j, k) ∈

J × {0, . . . , |τ|}. Hence for each λ ∈ 0, min−θτ(ξ)

4A2_L ,1_γ



, Ψ_τ(ξ + λ(g_τ(ξ)− ξ)) ≤ (1− γλ)Ψ_τ(ξ)≤ 0.

Summarizing, suppose ξ is a local minimizer of P_τ and θ_τ(ξ) < 0. For each ε > 0, by choosing any (4.16) λ∈  0, min  −θτ(ξ) 4A2L , 1 γ, ε gτ(ξ)− ξ_X  ,

we can construct a new point ˆξ = ξ + λg_τ(ξ)− ξ ∈ X_τ,r such that ˆξ ∈ N_τ(ξ, ε),

J_τ( ˆξ) < J_τ(ξ), and Ψ_τ( ˆξ)≤ 0. This is a contradiction and proves our result.

Lemma 4.11. Let N ∈ N, τ ∈ T_N, and S_τ: L2([0, 1],Rn)∩ BV ([0, 1], Rn) →

L2([0, 1],Rn) defined by S_τ(f ) = |τ|−1_k=0 f (t_k)½[t

k,tk+1). Then, Sτ(f )− fL2 ≤

2−N2V(f ).

Proof. Let ε > 0, ¯ε = _|τ|ε, and α_k ∈ [t_k, t_k+1] such thatf(t_k)− f(α_k)2₂+ ¯ε2 ≥

sup_s∈[t

k,tk+1]f(tk)− f(s)

2

2for each k∈ {0, . . . , |τ| − 1}. Consider (4.17) Sτ(f )− f_L2=    |τ|−1 k=0  _tk+1 tk f(tk)− f(s)2₂ds≤     1 2N |τ|−1 k=0 f(tk)− f(α_k)2₂+ ¯ε2 ≤ 2−N 2 |τ|−1 k=0  f(tk)− f(α_k)₂+ ¯ε≤ 2−N2_{V(f ) + ε}_,

where we used the fact that x₂ ≤ x₁ for each x ∈ Rn, and the last inequality follows by the definition of V ((3.3) in [14]). The result is obtained after noting that (4.17) is valid for each ε > 0.

Finally, we prove that P_τ consistently approximates P_p when the cardinality of the sequence of switching times satisfies an upper bound, which is satisfied by the sequence of switching times generated by Algorithm 3.1.

Theorem 4.12. Let{τ_i}

i∈Nand{ξi}i∈Nsuch that τi∈ Ti and ξi∈ Xτi,pfor each

i∈ N. Then lim inf_i→∞θ_τi(ξ_i)≤ lim inf_i→∞θ(ξ_i).

Proof. By the nonpositivity of θ, shown in Theorem 5.6 in [14], as well as

Theo-rem 3.3 and Lemma 5.4 in [14], V(g(ξ)− ξ) is uniformly bounded for each ξ ∈ X_p. Let

ξ_i = S_τg(ξ_i)− ξ_ifor each i∈ N, where S_τ is as defined in Lemma 4.11 and where we abuse notation by writing S_τ(ξ) =S_τ(u), S_τ(d)for each ξ = (u, d). Hence, also by Lemma 4.11, there exists C> 0 such that g(ξ_i)− ξ_i− ξ_i_X ≤ C2−N2.

Now, using condition (4) in Lemma 4.7, there exists C > 0 such that

θ_τi(ξ_i)− θ(ξ_i)≤ ζ_τi(ξ_i, ξ_i+ ξ_i) + V(ξ_i)− ζ(ξ_i, ξ_i+ ξ_i)− V(ξ_i) (4.18) + ζ(ξ_i, ξ_i+ ξ_i) + V(ξ_i)− ζξ_i, g(ξ_i)− V(g(ξ_i)− ξ_i) ≤ C 2i + ζ(ξi, ξ  i+ ξi)− ζξ_i, g(ξ_i)+ V(ξ_i)− V(g(ξ_i)− ξ_i). Using the same argument as in the proof of Lemma 5.4 in [14], there exists L > 0 such that ζ(ξ_i, ξ_i+ ξ_i)− ζξ_i, g(ξ_i) ≤ Lg(ξ_i)− ξ_i− ξ_i_X. Also notice that by the definitions of the set {ξ_i}_i∈N and V, V (ξ_i) ≤ V (g(ξ_i)− ξ_i) for each i ∈ N. Finally, using all the inequalities above we have lim inf_i→∞θ_τi(ξ_i)− θ(ξ_i) ≤ 0, as desired.

4.3. Convergence of the implementable algorithm. In this subsection, we

prove that the sequence of points generated by Algorithm 3.1 asymptotically satisfies our optimality condition. We begin by proving that μ_τ is bounded whenever θ_τ is negative.

Lemma 4.13. Let α, β∈ (0, 1). For every δ > 0, there exists M_δ∗<∞ such that

if θ_τ(ξ)≤ −δ for N ∈ N, τ ∈ T_N, and ξ∈ X_τ,p, then μ_τ(ξ)≤ M_δ∗. Proof. Assume that Ψ_τ(ξ)≤ 0. Using (4.13),

(4.19) J_τξ + βk(g_τ(ξ)− ξ)− J_τ(ξ)− αβkθ_τ(ξ)≤ −(1 − α)δβk+ 4A2Lβ2k,

where A = maxu₂+ 1| u ∈ U. Hence, for each k∈ N such that βk ≤ (1−α)δ_4A2_L we

have that J_τξ + βk(g(ξ)− ξ)− J_τ(ξ)≤ αβkθ_τ(ξ). Similarly, using (4.14),

(4.20) Ψ_τξ +βk(g(ξ)−ξ)−Ψ_τ(ξ)+βkγΨ_τ(ξ)−αθ_τ(ξ)≤ −(1−α)δβk+4A2Lβ2k; thus for each k∈ N such that βk≤ min(1−α)δ_4A2_L ,_γ1



, Ψ_τξ+βk(g(ξ)−ξ)−αβkθ_τ(ξ)≤ 

1− βkγΨ_τ(ξ)≤ 0.

Now, assume Ψ_τ(ξ) > 0. Using (4.14),

(4.21) Ψ_τξ + βk(g_τ(ξ)− ξ)− Ψ_τ(ξ)− αβkθ_τ(ξ)≤ −(1 − α)δβk+ 4A2Lβ2k.

Hence, for each k ∈ N such that βk ≤ (1−α)δ_4A2_L , Ψτ



ξ + βk(g_τ(ξ)− ξ)− Ψ_τ(ξ) ≤

αβkθ_τ(ξ).

Finally, let M_δ∗= 1+maxlog_β(1−α)δ_4A2_L



, log_β_γ1, then we get that μ_τ(ξ)≤ M_δ∗ as desired.

The following corollary follows directly from the proof of Lemma 4.13.

Corollary 4.14. Let α, β ∈ (0, 1). There exists δ₀, C > 0 such that if δ∈ (0, δ₀)

and θ_τ(ξ)≤ −δ for N ∈ N, τ ∈ T_N, and ξ∈ X_τ,p, then μ_τ(ξ)≤ 1 + log_β(Cδ). Next, we show a bound of the error introduced in the trajectory after discretizing and applying the projection operator ρ_N. The proof follows by the triangle inequality, Theorem 5.10 in [14], and condition (1) in Lemma 4.3.

Lemma 4.15. There exists K > 0 such that for each N₀, N ∈ N, τ ∈ T_N

0, ξ = (u, d)∈ X_r,τ, and t∈ [0, 1], (4.22) φ_σN(ξ),tρ_N(ξ)− φ_τ,t(ξ) ₂≤ K  1 √ 2 N_ V(ξ) + 1+  1 2 N0 .

Now we show that ν_τ is eventually finite for points that do not satisfy the opti-mality condition infinitely often as the precision increases.

Lemma 4.16. Let N₀ ∈ N, τ ∈ T_N

0, and ξ ∈ Xτ,r. Let K ⊂ N be an infinite

subsequence and δ > 0 such that θ_σN(ξ) ≤ −δ for each N ∈ K. Then for each η ∈ N there exists a finite N∗∈ K, N∗≥ N₀, such that ν_σ

N ∗(ξ)(ξ, N + η) is finite.

Proof. For each N ∈ N, let ξ_N = ξ + βμσN (ξ)(ξ)g_σN(ξ)(ξ)− ξ. Note that this lemma is equivalent to showing that the feasible set in the optimization problem solved by ν_σ

N(ξ)is nonempty for some N ∈ K. Also note that by the nonpositivity of θσN(ξ),

as shown in Theorem 4.10, together with Lemmas 4.1 and 4.6, there exists C > 0 such that for each N ∈ N, Vg_σ

N(ξ)(ξ)− ξ



≤ C, and thus V(ξ

N)≤ V(ξ) + C. Using Lemma 4.15, there exists N∗∈ K such that for each N ≥ N∗, N ∈ K,

J_σ k(ξ)  ρ_k(ξ)− J_σ N(ξ)(ξ  N)≤ LK  1 √ 2 N_ V(ξ) + 2 (4.23) ≤ ¯α ¯βN1 2δ≤ −¯α ¯β N_θ σN(ξ)(ξ),

and at the same time, ¯α ¯βN ≤ (1 − ω)αβMδ∗ ≤ (1 − ω)αβμσN(ξ), where M_δ∗ is as in Lemma 4.13. Let A_τ(ξ) = arg max{ψ_τ,j,t(ξ)| (j, t) ∈ J × [0, 1]} and (j, t) ∈

AσN(ξ)(ξ

_{); then using the same argument as in (4.23),} (4.24) Ψ_σ k(ξ)  ρ_k(ξ)− Ψ_σN(ξ)(ξ)≤ ψ_σ k(ξ),j,t  ρ_k(ξ)− ψ_σN(ξ),j,t(ξ)≤ −¯α ¯βNθ_σN(ξ)(ξ), as desired. Lemma 4.17. Let {N_i}

i∈N, {τi}i∈N, and{ξi}i∈N be the sequences generated by Algorithm 3.1. Then lim_i→∞N_i=∞.

Proof. Suppose that N_i≤ N∗ for all i∈ N. Then, by definition of Algorithm 3.1, there exists i₀ ∈ N such that θ_τi(ξ_i) ≤ −Λ2−χNi ≤ −Λ2−χN∗_{, ντ}

i(ξi, Ni+ η) <∞,

and ξ_i+1 = Γ_τi(ξ_i) for each i≥ i₀. Moreover, by the definition of ν_τ, if there exists

i₁ ≥ i₀ such that Ψ_τi1(ξ_i1)≤ 0, then Ψτi(ξi)≤ 0 for each i ≥ i1. Assume that there

exists i₁≥ i₀ such that Ψ_τi1(ξ_i1)≤ 0; then, using Corollary 4.14,

(4.25) J_τi+1(ξ_i+1)− J_τi(ξ_i)≤αβμτi(ξi)− ¯α ¯βντi(ξi,Ni+η)_θ(ξ

i)≤ −ωαβMδ∗δ

for each i≥ i₁, where δ = Λ2−χN∗. But this implies that J_τi(ξ_i)→ −∞ as i → ∞, which is a contradiction since J_τi(ξ_i) is lower bounded for all i∈ N by Lemma 4.1.

The argument is identical when the sequence is perpetually infeasible.

We now prove that if Algorithm 3.1 finds a feasible point, then every point gen-erated afterward remains feasible.

Lemma 4.18. Let {N_i}

i∈N, {τi}i∈N, and{ξi}i∈N be the sequences generated by Algorithm 3.1. Then there exists i₀ ∈ N such that if Ψ_τi0(ξ_i0)≤ 0, then Ψ(ξ_i)≤ 0

and Ψ_τi(ξ_i)≤ 0 for each i ≥ i₀.

Proof. LetI ⊂ N be a subsequence defined by

(4.26) I =  i∈ N | θ_τi(ξ_i)≤ − Λ 2χNi and ντi(ξi, Ni+ η) <∞  .

Note that by definition of Algorithm 3.1, Ψ(ξ_i+1) = Ψ(ξ_i) for each i /∈ I. For each

i ∈ I such that Ψ_τi(ξ_i)≤ 0, by definition of ν_τ together with Corollary 4.14, there exists C > 0 such that

(4.27) Ψ_τi+1(ξ_i+1)≤αβμτi(ξi)− ¯α ¯βντi(ξi,Ni+η)_θ

τi(ξi)≤ −ωαβC  Λ 2χNi 2 .

By Lemma 4.3 and the fact that N_i+1≥ N_i, there exists C> 0 such that

(4.28) Ψ(ξ_i+1)≤ C  2Ni − ωαβC  Λ 2χNi ₂ ≤ 1 22χNi  C 2(1−2χ)Ni − ωαβCΛ 2_. Hence, if Ψ_τi1(ξ_i1)≤ 0 for i₁ ∈ N such that N_i1 is large enough, then Ψ(ξ_i)≤ 0 for each i≥ i₁. Moreover, by (4.28), for each N ≥ N_i and each τ∈ T_N,

(4.29) Ψ_τ(ξ_i+1)≤ 1 22χNi  C 2(1−2χ)Ni − ωαβCΛ 2₊ C 2N ≤ 1 22χNi  2C 2(1−2χ)Ni − ωαβCΛ 2_.

Thus, if Ψ_τi2(ξ_i2)≤ 0 for i₂∈ N such that N_i2 is large enough, then Ψ_τ(ξ_i2)≤ 0 for each τ ∈ T_N such that N ≥ N_i2. But note that this is exactly the case when there is an uninterrupted finite subsequence of the form i₂+ k /∈ I with k ∈ {1, . . . , n}; thus we can conclude that Ψ_τi2+k(ξ_i2+k) ≤ 0. Also note that the case of i ∈ I is trivially satisfied by the definition of ν_τ. Setting i₀= max{i₁, i₂}, we get the desired

result.

Next, we show that the points produced by our algorithm asymptotically satisfy our discretized optimality condition.

Lemma 4.19. _Let {N_i}

i∈N, {τi}i∈N, and{ξi}i∈N be the sequences generated by Algorithm 3.1. Then lim_i→∞θ_τi(ξ_i) = 0.

Proof. Let us suppose that lim_i→∞θ_τi(ξ_i) = 0. Then there exists δ > 0 such that lim inf_i→∞θ_τi(ξ_i) < −2δ. Hence, using Lemma 4.17, there exists an infinite subsequence K ⊂ N defined by K = {i ∈ N | θ_τi(ξ_i) <−2δ}. Let I ⊂ N be as in (4.26). Note that by Lemma 4.16 we get thatK ∩ I is an infinite set.

We analyze Algorithm 3.1 by considering the behavior of each step as a function of its membership to each subsequence. First, for each i /∈ I, ξ_i+1 = ξ_i, and thus

J (ξ_i+1) = J (ξ_i) and Ψ(ξ_i+1) = Ψ(ξ_i). Second, let i∈ I such that Ψ_τi(ξ_i)≤ 0; then arguing just as we did in (4.27), (4.28), and (4.29), we have that

(4.30) J (ξ_i+1)− J(ξ_i)≤ 2C  2Ni − ωαβC  Λ 2χNi 2 ≤ 1 22χNi  2C 2(1−2χ)Ni − ωαβCΛ 2_.

Since χ ∈0,1₂, we get that for N_i large enough J (ξ_i+1) ≤ J(ξ_i). Using a similar argument, for N_i large enough, Ψ(ξ_i+1) ≤ Ψ(ξ_i). Third, let i ∈ K ∩ I such that Ψ_τi(ξ_i)≤ 0. Then, by Lemma 4.13,

(4.31) J_τi+1(ξ_i+1)− J_τi(ξ_i)≤αβμτi(ξi)− ¯α ¯βντi(ξi,Ni+η)_θ

τi(ξi)≤ −2ωαβ

M∗

2δ_δ;

thus, by Lemmas 4.3 and 4.17, for N_i large enough, J (ξ_i+1)− J(ξ_i) ≤ −ωαβM2δ∗_δ.

Similarly, if Ψ_τi(ξ_i) > 0, using the same argument, for N_i large enough, Ψ(ξ_i+1)− Ψ(ξ_i)≤ −ωαβM2δ∗ δ.

Now let us assume that there exists i₀ ∈ N such that N_i0 is large enough and Ψ_τi0(ξ_i0)≤ 0. Then by Lemma 4.18 we get that Ψ_τi(ξ_i)≤ 0 for each i ≥ i₀. But as shown above, either i /∈ K ∩ I and J(ξ_i+1)≤ J(ξ_i) or i∈ K ∩ I and J(ξ_i+1)− J(ξ_i)≤

−ωαβM∗

2δδ, and since K ∩ I is an infinite set we get that J(ξ_i)→ −∞ as i → ∞,

which is a contradiction as J is lower bounded by Lemma 4.1.

The same argument is valid for the case when Ψ_τi(ξ_i) > 0 for each i ∈ N, now using the bounds for Ψ. Together, both contradictions imply that θ_τi(ξ_i) → 0 as

i→ ∞, as desired.

In conclusion, we prove that the sequence of points generated by Algorithm 3.1 asymptotically satisfies the optimality condition defined in [14], which follows since

P_τ is a consistent approximation of P_p, shown in Theorem 4.12, and Theorem 3.4. Theorem 4.20. Let{N_i}_i∈N,{τ_i}_i∈N, and{ξ_i}_i∈Nbe the sequences generated by

Algorithm 3.1; then lim_i→∞θ(ξ_i) = 0.

5. Examples. In this section, we apply Algorithm 3.1 to calculate an optimal

control for four examples. Before describing each example, we begin by describing the numerical implementation of Algorithm 3.1. First, observe that the time interval for the trajectory of the system can be generalized to any interval [t₀, t_f], instead of [0, 1] as used in the previous sections. Second, we employ the LSSOL [6] numerical solver, using TOMLAB [9] as an interface, in order to compute θ_τ at each iteration.

Third, for each example we employ a stopping criterion that terminates Algorithm 3.1 whenever θ_τ gets too close to zero. Next, for the sake of comparison, we solved every example using Algorithm 3.1 and the mixed integer program (MIP) described in [4], which combines branch and bound steps with sequential quadratic programming steps. Finally, all our experiments are performed on an Intel Xeon, 12 core, 3.47 GHz, 92 GB RAM machine.

5.1. Constrained switched linear quadratic regulator. Linear quadratic

regulator (LQR) examples have been used to illustrate the utility of many optimal control algorithms [3, 15]. We consider an LQR system with three continuous states, three discrete modes, and a single continuous input. The dynamics in each mode are as described in Table 5.1, where

(5.1) A = ⎡ ⎣_−0.01051.0979 −0.0105 0.01671.0481 0.0825 0.0167 0.0825 1.1540 ⎤ ⎦ .

The system matrix is purposefully chosen to have three unstable eigenvalues and the control matrix in each mode is only able to control along a single continuous state. Hence, while the system and control matrix in each mode are not a stabilizable pair, the system and all the control matrices taken together simultaneously are stabilizable and are expected to appropriately switch between the modes to reduce the cost. The objective of the optimization is to have the trajectory of the system at time t_f be at (1, 1, 1) while minimizing the energy of the input, as described in Table 5.2.

Algorithm 3.1 and the MIP are initialized at x₀ = (0, 0, 0), with continuous and discrete inputs as described in Table 5.3, using 16 equally spaced samples in time. Algorithm 3.1 took 11 iterations, ended with 48 time samples, and terminated when θ_τi(ξ_i) > −10−2. The result of both optimization procedures is illustrated in Figure 5.1. The computation time and final cost of both algorithms can be found in Table 5.3. Notice that Algorithm 3.1 is able to compute a lower cost than MIP and is able to do it more than 75 times faster.

5.2. Double-tank system. To illustrate the performance of Algorithm 3.1

when there is no continuous input present, we consider a double-tank example. The

Table 5.1

Vector fields of each of the modes of the examples considered in section 5.

Example Mode 1 Mode 2 Mode 3

LQR x(t) = Ax(t) +˙ ⎡ ⎢ ⎣ 0.9801 −0.1987 0 ⎤ ⎥ ⎦ u(t) x(t) = Ax(t) +˙ ⎡ ⎢ ⎣ 0.1743 0.8601 −0.4794 ⎤ ⎥ ⎦ u(t) ˙x(t) = Ax(t) + ⎡ ⎢ ⎣ 0.0952 0.4699 0.8776 ⎤ ⎥ ⎦ u(t) Tank x(t) =˙  1 − x1(t) x1(t) − x2(t) ˙ x(t) =  2 − x1(t) x1(t) − x2(t) N/A Quadrotor ¨x(t) = ⎡ ⎢ ⎢ ⎣ sin x3(t) M (u(t) + Mg) cos x3(t) M (u(t) + Mg) − g 0 ⎤ ⎥ ⎥ ⎦ ¨x(t) = ⎡ ⎢ ⎢ ⎣ g sin x3(t) g cos x3(t) − g −Lu(t) I ⎤ ⎥ ⎥ ⎦ x(t) =¨ ⎡ ⎢ ⎢ ⎣ g sin x3(t) g cos x3(t) − g Lu(t) I ⎤ ⎥ ⎥ ⎦ Needle x(t) =˙ ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ sinx5(t)u1(t) − cosx5(t)sinx4(t)u1(t) cosx4(t)cosx5(t)u1(t) κ cosx6(t)secx5(t)u1(t) κ sinx6(t)u1(t) −κ cosx6(t)tanx5(t)u1(t) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ ˙ x(t) = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ 0 0 0 0 0 u₂(t) ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ N/A