Optimization Modeling
2. An objective function, which is a function of the decision variables (f (x))
3. A set of constraints defined by functions (gi(x), hj(x)), i=1,. . .,I, j= 1,. . .,J of the general form gi(x)≤0 (inequality constraints) and hj(x)= 0 (equality constraints).
The decision variables are numerical quantities that represent the deci-sions to be made. In the portfolio example, the decision variables could be the portfolio weights (alternatively, they could be the amounts to allocate to each asset class). The objective function is a mathematical expression of the goal, and the constraints are mathematical expressions of the limitations in the business situation. In our example, the objective function could be an expression to compute the expected portfolio return, and the constraints could include an expression that computes total portfolio risk. We can then maximize the expression of the objective function subject to an upper limit on the risk we are willing to tolerate. (We will derive actual formulations in Chapters 7 through 10.)
Optimization software expects users to specify all three components of an optimization problem, although it is sometimes possible to have optimiza-tion problems with no constraints. The latter kind of optimizaoptimiza-tion problems is referred to as unconstrained optimization. Unconstrained optimization problems are typically solved with standard techniques from calculus,3and the optimal solution is selected from all possible points in the N-dimensional space of the decision variables x. When there are constraints, only some points in that space will be feasible, that is, will satisfy the constraints.
The values of the decision variables that are feasible and result in the best
value for the objective function are called the optimal solution. Optimization solvers typically return only one optimal solution. However, it is possible to have multiple optimal solutions, that is, multiple feasible solutions x that produce the same optimal value for the objective function.4
When formulating optimization problems, it is important to realize that the decision variables need to participate in the mathematical expressions for the objective function and the constraints that are passed to an optimization solver because the whole idea of optimization algorithms is that they can tweak the values of the decision variables in these expressions in a smart, computationally efficient way, in order to produce the best value for the objective function with values for the decision variables that satisfy all of the constraints. In other words, we cannot simply pass the expression
Maximize Portfolio expected return
to an optimization solver, unless the portfolio expected return is expressed as a function of the decision variables (the portfolio weights). As we will derive in Chapter 7, the expected portfolio return can be expressed in terms of the portfolio weights as wµ, where w=(w1, . . . , wN) and µ=(µ1, . . . , µN) are N-dimensional arrays containing the weights and the expected returns of the N assets in the portfolio, respectively. So, the objective function would be written as
maxw wµ
which is interpreted as “maximize the value of wµover the possible values for w.”
In addition, the input data in a classical optimization problem formu-lation need to be fixed numbers, not random variables. For example, the objective function of an optimization problem cannot be passed to a solver as
Maximize Portfolio return
where portfolio return=w˜r, and ˜r=( ˜r1, . . . ,˜rN)is the N-dimensional ar-ray with (uncertain) asset returns with some probability distributions. Some areas in optimization, such as robust optimization and stochastic program-ming, study methodologies for solving optimization problems in which the input data are subject to uncertainty and follow theoretical or empirical probability distributions. However, in the end, the methods for solving such problems reduce to specifying the coefficients in the optimization problem as fixed numbers that are representative of the underlying probability distri-butions in a particular way.
5 . 1 . 1 M i n i m i z a t i o n v s . M a x i m i z a t i o n
Most generally, optimization solvers require an optimization problem for-mulation to be of the kind
minx f (x)
subject to gi(x)≤0 i =1, . . . ,I hj(x)=0 j=1, . . . ,J
There are variations on this formulation, and some have to do with whether the optimization problem falls in a specific class. (Different cate-gories of optimization problems based on the form of their objective function and the shape of their feasible set are discussed in section 5.2.) Some op-timization software syntax, such as the MATLAB Opop-timization Toolbox syntax, allows for specifying only minimization problems, while other opti-mization software packages are more flexible, and accept both miniopti-mization and maximization problems. Standard formulation requirements, however, are not as restrictive as they appear at first sight. For example, an optimiza-tion problem that involves finding the maximum of a funcoptimiza-tion f (x) can be recast as a minimization problem by minimizing the expression−f (x), and vice versa—an optimization problem that involves finding the minimum of a function f (x) can be recast as a maximization problem by maximizing the expression−f (x). To obtain the actual value of the objective function, one then flips the sign of the optimal value. Exhibit 5.1 illustrates the situation for a quadratic function of one variable x. The optimal value for max f (x) is obtained at the optimal solution x*. The optimal value for min−f (x) is obtained at x* as well. Notice also that
maxx f (x)= −min
x −f (x)
For the previous portfolio expected return maximization example, stat-ing the objective function as
maxw wµ or
minw −wµ
will produce the same optimal values for the decision variables w. To get the actual optimal objective function value for max
w wµ, we would flip the sign of the optimal objective function value obtained after minimizing−wµ.
–f(x) –f(x*) f(x)
x*
max f(x) min –f(x)
f(x*) 0
E X H I B I T 5 . 1 Example of the optimal objective function values for a quadratic objective function f (x) of a single decision variable x.
5 . 1 . 2 L o c a l v s . G l o b a l O p t i m a
In optimization, we distinguish between two types of optimal solutions:
global and local optimal solutions. A global optimal solution is the “best”
solution for any value of the decision variables vector x in the set of all feasible solutions. A local optimal solution is the best solution in a neigh-borhood of feasible solutions. In other words, the objective function value at any point “close” to a local optimal solution is worse than the objec-tive function value at the local optimal solution. Exhibit 5.2 illustrates the global (point A) and local (point B) optimal solution for the unconstrained minimization of a function of two variables.
Most classical optimization algorithms can only find local optima. They start at a point, and go through solutions in a direction in which the objective function value improves. Their performance has an element of luck that has to do with picking a “good” starting point for the algorithm. For example, if a nonlinear optimization algorithm starts at point C in Exhibit 5.2, it may find the local minimum B first, and never get to the global minimum A. In the general case, finding the global optimal solution can be difficult and time consuming, and involves finding all local optimal solutions first, and then picking the best one among them.
The good news is that in some cases, optimization algorithms can ex-plore the special structure of the objective function and the constraints to
B C
A
x1
f(x1,x2)
x2
E X H I B I T 5 . 2 Global (point A) versus local (point B) minimum for a function of two variables x1and x2.
deliver stronger results. In addition, for some categories of optimization problems, a local optimal solution is in fact guaranteed to be the global opti-mal solution, and many optimization problems in finance have that property.
(We review the most important kinds of such “nice” optimization problems in section 5.2.) This makes recognizing the type of optimization problem in a given situation and formulating the optimization problem in a way that enables optimization algorithms to take advantage of the special problem structure even more critical.
5 . 1 . 3 M u l t i p l e O b j e c t i v e s
In practice, we often encounter situations in which we would like to opti-mize several objectives at the same time. For example, a portfolio manager may want to maximize the portfolio expected return and skew, while min-imizing the variance and the kurtosis. There is no straightforward way to pass several objectives to an optimization solver. A multiple-objective op-timization problem needs to be reformulated as an opop-timization problem with a single objective. There are a couple of commonly used methods to do this. We can assign weights to the different objectives, and optimize the weighted sum of objectives as a single-objective function. Alternatively, we can optimize the most important objective, and include the other objectives as constraints, after assigning to each of them a bound on the value we are willing to tolerate.
5 . 2 I M P O R T A N T T Y P E S O F O P T I M I Z A T I O N P R O B L E M S
Optimization problems can be categorized based on the form of their objec-tive function and constraints, and the kind of decision variables. The type of optimization problem with which we are faced a particular situation de-termines what software is appropriate, the efficiency of the algorithm for solving the problem, and the degree to which the optimal solution returned by the optimization solver is trustworthy and useful. Awareness of this fact is particularly helpful in situations in which there are multiple ways to formu-late the optimization problem. The way in which we state the formulation will determine whether the optimization solver will be able to exploit any special structure in the problem, and whether it can achieve stronger results.
5 . 2 . 1 C o n v e x P r o g r a m m i n g
As mentioned in section 5.1.2, some general optimization problems have a “nice” structure in the sense that a local optimal solution is guaranteed to be the global optimal solution. Convex optimization problems have that property. A general convex optimization problem is of the form
minx f (x)
subject to gi(x)≤0 i =1, . . . ,I Ax=b
where both f (x) and gi(x) are convex functions, and Ax =b is a system of linear equalities. A convex function of a single variable x has the shape showed in Exhibit 5.3(A). For a convex function, a line that connects any two points on the curve is always above the curve. The “opposite” of a convex function is a concave function (see Exhibit 5.3(B)), which looks like
f(x)
x (A)
f(x)
x (B) E X H I B I T 5 . 3 Examples of (A) a convex function; (B) a concave function.
a “cave.” For a concave function, a line that connects any two points on the curve is always below the curve.
Convex programming problems encompass several classes of problems with special structure, including linear programming (LP), some quadratic programming (QP), second-order cone programming (SOCP), and semidefi-nite programming (SDP). Algorithms for solving convex optimization lems are more efficient than algorithms for solving general nonlinear prob-lems, but it is important to keep in mind that even within the class of convex problems, some convex problems are computationally more chal-lenging than others. LP problems are best studied and easiest to solve with commercial solvers, followed by convex QP problems, SOCP problems and SDP problems.
We introduce LP, QP, and SOCP in more detail next. Many classical problems in finance involve linear and quadratic programming, including asset-liability problems, portfolio allocation problems, and some financial derivative pricing applications. SDP problems are advanced formulations that have become more widely used in financial applications with recent advances in the field of robust optimization. They are beyond the scope of this book, but we refer interested readers to Fabozzi, Kolm, Pachamanova, and Focardi (2007) for a detailed overview of robust optimization formulations.
5 . 2 . 2 L i n e a r P r o g r a m m i n g
Linear programming refers to optimization problems in which both the objective function and the constraints are linear expressions in the deci-sion variables.5The standard formulation statement for linear optimization problems is
minx cx subject to Ax=b
x≥0
All optimization problems involving linear expressions for the objective function and the decision variables can be converted to this standard form.
Section 5.3.1 will present an example.
Linear optimization problems are the easiest kind of problems to solve.
Modern specialized optimization software can handle LP formulations with hundreds of thousands of decision variables and constraints in a matter of seconds. In addition, linear optimization problems belong to the class
of convex problems for which a local optimal solution is guaranteed to be the global optimal solution. (This is discussed in more detail in section 5.4.2.) LPs arise in a number of finance applications, such as asset alloca-tion and identificaalloca-tion of arbitrage opportunities. The sample optimizaalloca-tion problem formulations in section 5.3.1 and 5.3.2 are linear optimization problems.
5 . 2 . 3 Q u a d r a t i c P r o g r a m m i n g
Quadratic programming problems have an objective function that is a quadratic expression in the decision variables, and constraints that are lin-ear expressions in the decision variables. The standard form of a quadratic optimization problem is
minx
1
2xQx+cx subject to Ax=b
x≥0
where x is an N-dimensional vector of decision vectors as before, and the other arrays are input data:
Q is an N×N matrix.
c is an N-dimensional vector.
A is a J×N matrix.
b is an J-dimensional vector.
When the matrix Q is positive semidefinite,6then the objective function is convex. (It is a sum of a convex quadratic term and a linear function, and a linear function is both convex and concave.) Since the objective function is convex and the constraints are linear expressions, we have a convex op-timization problem. The problem can be solved by efficient algorithms, and we can trust that any local optimum they find is in fact the global optimum.
When Q is not positive semidefinite, however, the quadratic problem can have several local optimal solutions and stationary points, and is therefore more difficult to solve.
The most prominent use of quadratic programming in finance is for asset allocation and trading models. We will see examples in Chapters 7 through 10.
5 . 2 . 4 S e c o n d - O r d e r C o n e P r o g r a m m i n g
Second-order cone programs (SOCPs) have the general form minx cx
subject to Ax=b
||Cix+di|| ≤cix+ei, i =1, . . . ,I
where x is an N-dimensional vector of decision vectors as before, and the other arrays are input data:
c is an N-dimensional vector.
A is a J×N matrix.
b is a J-dimensional vector.
Ciare an Ii×N matrix.
diare Ii-dimensional vectors.
eiare scalars.
The notation . stands for second norm, or Euclidean norm. (It is sometimes denoted.2to differentiate it from other types of norms.) The Euclidean norm of an N-dimensional vector x is defined as
||x|| =
x12+ · · · +x2N
The SOCP class of problems is more general than the classes covered in sections 5.2.2 and 5.2.3. LPs, convex QPs, and convex problems with quadratic objective function and quadratic constraints can be reformulated as SOCPs with some algebra.
It turns out that SOCP problems share many nice properties with lin-ear programs, so algorithms for their optimization are very efficient. SOCP formulations arise mostly in robust optimization applications. Such formu-lations are discussed in section 6.3.
5 . 2 . 5 I n t e g e r a n d M i x e d I n t e g e r P r o g r a m m i n g
So far, we have classified optimization problems according to the form of the objective function and the constraints. Optimization problems can be classified also according to the type of decision variables x. Namely, when the decision variables are restricted to be integer (or, more generally, discrete)
values, we refer to the corresponding optimization problem as an integer programming (IP) or a discrete problem. When some decision variables are discrete and some are continuous, we refer to the optimization problem as a mixed integer programming (MIP) problem. In special cases of integer problems in which the decision variables can only take values 0 or 1, we refer to the optimization problem as a binary optimization problem.
Integer and mixed integer optimization formulations are useful for formulating extensions to classical portfolio allocation problems. Index-tracking formulations and many constraints on portfolio structure faced by managers in practice require modeling with discrete decision variables. Ex-amples of constraints include maximum number of assets to be held in the portfolio (so-called cardinality constraints), maximum number of trades, round lot constraints (constraints on the size of the orders in which assets can be traded in the market),7 and fixed transaction costs. Simple illustra-tions of integer modeling are provided in the next section and are further discussed in Chapter 9.
5 . 3 O P T I M I Z A T I O N P R O B L E M F O R M U L A T I O N E X A M P L E S
To provide better intuition for how optimization problems are formulated, we give a few simplified examples of financial problem formulations. We will see more advanced nonlinear problems formulations in the context of portfolio applications in Chapters 7 through 9.
The first example in this section is explained in detail, so that the process of optimization problem formulation can be explicitly outlined.
The problem formulation is the crucial step—once we are able to define a business situation as one of the optimization problem types reviewed in the previous section, we can find the optimal solution with optimiza-tion software. Later in this chapter, we explain how optimizaoptimiza-tion formu-lations can be input into solvers, and how the output can be retrieved and interpreted.
5 . 3 . 1 P o r t f o l i o A l l o c a t i o n
The portfolio manager at a large university in the United States is tasked with investing a $10 million donation to the university endowment. He has decided to invest these funds only in mutual funds8and is considering the following four: an aggressive growth fund (Fund 1), an index fund (Fund 2), a corporate bond fund (Fund 3), and a money market fund (Fund 4), each with a different expected annual return and risk level.9The investment
E X H I B I T 5 . 4 Data for the portfolio manager’s problem.
Fund Type Growth Index Bond Money Market
Fund number 1 2 3 4
Expected return 20.69% 5.87% 10.52% 2.43%
Risk level 4 2 2 1
Max. investment 40% 40% 40% 40%
guidelines established by the Board of Trustees limit the percentage of the money that can be allocated to any single type of investment to 40% of the total amount. The data for the portfolio manager’s task are provided in Exhibit 5.4. In addition, in order to contain the risk of the investment to an acceptable level, the amount of money allocated to the aggressive growth and the corporate bond funds cannot exceed 60% of the portfolio, and the aggregate average risk level of the portfolio cannot exceed 2. What is the optimal portfolio allocation for achieving the maximum expected return at the end of the year, if no short selling is allowed?10
To formulate the optimization problem, the first thing we need to ask ourselves is what the objective is. In this case, the logical objective is to maximize the expected portfolio return. The second step is to think of how to define the decision variables. The decision variables need to be specified in such a way as to allow for expressing the objective as a mathematical expression of the quantities the manager can control to achieve his objec-tive. The latter point is obvious, but sometimes missed when formulating optimization problems for the first time. For example, while the market re-turn on the assets is a variable and increasing market rere-turns will increase the portfolio’s return, changing the behavior of the market is not under the manager’s control. The manager, however, can change the amounts he in-vests in different assets in order to achieve his objective.11Thus, the vector of decision variables can be defined as
x=(x1,x2,x3,x4): amounts (in$) invested in Fund 1, 2, 3, and 4, respectively
Let the vector of expected returns be µ=(20.69%,5.87%,10.52%, 2.43%). Then, the objective function can be written as
f (x)=µx=(20.69%)·x1+(5.87%)·x2+(10.52%)·x3+(2.43%)·x4.
It is always a good idea to write down the actual description and the units for the decision variables, the objective function and the constraints.
For example, the units of the objective function value in this example are dollars.
Finally, we have several constraints:
The total amount invested should be $10 million. This can be formulated as x1+x2+x3+x4=10,000,000.
The total amount invested in Fund 1 and Fund 3 cannot be more than 60% of the total investment ($6 million). This can be written as x1+ x3≤6,000,000.
The average risk level of the portfolio cannot be more than 2. This con-straint can be expressed as 4*(proportion of investment with risk level 4)+2*(proportion of investment with risk level 2)+1*(proportion of investment with risk level 1)≤2 or, mathematically,
4·x1+2·x2+2·x3+1·x4
x1+x2+x3+x4 ≤2.
Note that this is not a linear constraint. (We are dividing decision vari-ables by decision varivari-ables.) Based on the discussion in section 5.2.2, from a computational perspective, it is better to have linear constraints
Note that this is not a linear constraint. (We are dividing decision vari-ables by decision varivari-ables.) Based on the discussion in section 5.2.2, from a computational perspective, it is better to have linear constraints