1 Introduction to optimization 1.1 Notations and definitions

1 Introduction to optimization

1.1 Notations and definitions

1. acriterion, orcost function, orobjective function: a function J defined over V with values in R, whereV is the space (normed vector space) in which the problem lies, also called the space ofcommand variables.

2. someconstraints: for example, (a) v∈K, where Kis a subset of V

(b) equality constraints: F(v) = 0, whereF :V →R^m(mrealconstraints: Fi(v) = 0)

(c) inequality constraints: G(v)≤0, whereG:V →R^p (pconstraintsG_i(v)≤0).

IfG_i(v) = 0, the constraint isactive, orsaturated.

IfG_i(v)<0, the constraint isinactive.

(d) Functional equation: the constraint is given by an ODE, or a PDE, to be satisfied optimal command of an evolutive problem.

The different types of constraints can be mixed: v∈KandF(v) = 0 andG(v)≤0.

We denote byU the set ofadmissibleelements of the problem:

U ={v∈V;v satisfies all the constraints}.

Minimization problem:

(P) Findu∈U such that J(u)≤J(v), ∀v∈U Maximization problem: idem.

uis theoptimalsolution, or the solution of the optimization problem. J(u) is the optimal value of the criterion.

Local optimum: ¯u is a local optimum if there exists a neighborhood V(¯u) such that J(¯u)≤J(v),∀v∈ V(¯u).

Note that a global optimum is a local optimum, but the converse proposition is not true (except in a convex case).

1.2 Examples

• Finite dimension: J :Rⁿ→R

• Infinite dimension: J :V →R Finite dimension:

1.2.1 Example 1: linear problem Food rationing (e.g. during wars):

ntypes of food

mfood components (proteins, vitamins, . . . ) cj: unitary price of foodj

3

v_j: quantity of foodj

a_ij quantity of componentiper unit of foodj bi: minimal (vital) quantity of componenti Food ration of minimal cost: minimize

J(v) =

n

X

j=1

cjvj,

under the constraintsv_j≥0,

n

X

j=1

a_ijv_j ≥b_i,i= 1, . . . , m.

1.2.2 Example 2: least squares problem

Av=b, where Ais am×nmatrix, of rankn < m, andv∈Rⁿ. J(v) =kAv−bk²_Rm

Infinite dimension:

1.2.3 Calculus of variations V functional space,

J(v) = Z

Ω

L(x, v(x), P v(x), . . .)dx, whereP is a differential operator.

Problem: inf

v∈VJ(v).

1.2.4 Example 3: optimal trajectory

From point (a, y0), reach (b, y1) as soon as possible.

The speed at point (x, y(x)) isc(x, y(x)).

Boundary conditions: y(a) =y0,y(b) =y1.

c(x, y(x)) =

pdx²+y⁰²dx²

dt = dx

dt

p1 +y⁰²

dt=

p1 +y⁰²dx c(x, y(x)) and then

J(y) = Z b

a

p1 +y⁰² c(x, y(x))dx

to be minimized under the constrainty(a) =y₀,y(b) =y₁, andy∈V =C¹([a, b]).

4

1.2.5 Example 4: geodesic

A geodesic is the shortest path between points on a given space (e.g. on the Earth’s surface).

x=x(u, v), y =y(u, v), z=z(u, v)⇒ds²=dx²+dy²+dz² = (xudu+xvdv)² +(yudu+ yvdv)²+(zudu+zvdv)²=e du²+ 2f dudv+g dv², wheree=x²_u+y_u²+z_u²,f =xuxv+yuyv+ zuzv,g=x²_v+y²_v+z_v².

The shortest path between points (u0, v0) and (u1, v1) is given by a functionv: [u0, u1]→R solution of

infJ(v) = Z u1

u0

pe+ 2f v⁰+gv⁰²du under the constraintsv(u0) =v0,v(u1) =v1, andv∈C¹([u0, u1]).

1.2.6 Other examples

• Energy principle (or variational principle) for a PDE:

infJ(v) =1 2

Z

Ω

|∇v|²dx− Z

Ω

f v dx

• Inverse problems: identification and estimation of parameters:

−div(K(x)∇u) =f whereK is unknown:

J=X

i

[u(xi)−ui]²

5

6