D Now suppose that we don't know the initial state of the Markov process, but we

do know the probabilities for each state; the probability that the initial state S(0)

SECTION 4.2: MARKOV CHAINS 7 1

is equal to S^k is given by Pfc(O), k G [l,n]. Then we can use the total probability theorem [Mitzenmacher and Upfal, 2005] to obtain

Pr(S(l) = St) = Pr(5(0) = S¹)pli + Pr(5(0) = S²)p2i + ■■■ + Pr(S(0) = Sn)Pni

= £ P r ( S ( 0 ) = Sfc)Pki

n

= ^2PkiPk(0).

^(4.16) Generalizing the above equation, we obtain

Pr(S(l)=S0

n T

Pr(5(l) = S„)

^{:pT(0)P (4.17)} where p(0) is the column vector comprised of pfc(0), fc G [l,n]. Generalizing this development for multiple time steps, we obtain

PT(t) =

Pr(S(t) = Si)

Pr(S(t) = S

ⁿ

)

^{= pr(0)P*. (4.18)}

EXAMPLE 4.3

Today's weather forecast in Oz is 80% sun and 20% snow. What is the weather forecast for two days from now?

Prom Equation (4.18), p^T(2) = pT(0)P2, where P is given in Example 4.1 andp(0) - [ 0.0 0.8 0.2 ]^T. This gives p(2) = [ 0.3750 0.2375 0.3875 ]^T. That is, two days from now, there is a 37.5% chance of rain, a 23.75% chance of sun, and a 38.75% chance of snow.

EXAMPLE 4.4

Consider a simple hill-climbing EA comprised of a single individual [Reeves and Rowe, 2003, page 112]. The goal of the EA is to minimize /(#). We use Xi to denote the candidate solution at the i-th generation. Each generation we randomly mutate Xi to obtain x\. If /(#$) < f(xi), then we set a^+i = x'^{. If f(Xi) > f{xi), then we use the following logic to determine 2^+1. If we had set Xk+i = x'k at the previous generation k at which f(x'^k) > f(xk), then we set x;+i = x\ with a 10% probability, and x^+i = Xi with a 90%

probability. If, however, we had set a^+i = x^k at the previous generation k at which f(x^fk) > f(xk), tn e n w e s e t x%+\ = x[ with a 50% probability, and we set Xi_|_i = X{ with a 50% probability. This EA is greedy in that it always accepts a beneficial mutation. However, it also includes some exploration in

that it sometimes accepts a detrimental mutation. The probability of accept-ing a detrimental mutation varies dependaccept-ing on whether or not the previous detrimental mutation was accepted. The algorithm for this hill-climbing EA is shown in Figure 4.1.

Initialize x\ to a random candidate solution Intialize AcceptFlag to false

For i = 1,2,···

Mutate Xi to get x\

If/(*5) </(*<)

Xi+l <- X'i

else

If AcceptFlag

P r ^ + i <— x'j) = 0.1, and Pr(xi +i <— Xi) — 0.9 else

Pr(xj+i ^— χ[) = 0.5, and Pr(xj+i <— xi) = 0.5 end if

AcceptFlag «— (x^+i = χ£) end if

Next i

Figure 4.1 The above pseudo-code outlines the single-individual hill-climbing EA of Example 4.4. AcceptFlag indicates if the previous detrimental mutation replaced the candidate solution.

We can analyze this EA by considering what happens if x\ is worse than Xi.

We use Z^k to denote the state the fc-th time that /(#£) > f(xi)· We define Y\

as the "accept" state; that is, x^+i <— x\. We define Y^ as the "reject" state;

that is, Xi+i ^— Xi. Then, by examining the algorithm of Figure 4.1, we can write

Pi(Zk = Y1\i Zk-i=Yi) = 0.1 Pr(zfc = y2|Zfc_i = yi) - 0.9 Pr(Zk = Y1\Zk.l = Y2) = 0.5 Pi(Zk = Y2\Zk-1=Y2) = 0.5 This equation shows that the transition matrix is

P = ' 0.1 0.9 "

0.5 0.5

(4.19)

(4.20) Notice that the rows of P sum to 1. We also see that all the elements of P* are nonzero for some t (actually, for all t in this case) so P is a regular transition matrix. Theorem 4.2 assures us that: (1) P^l converges as t —> oo;

(2) All the rows of P°° are identical; (3) Each element of P°° is positive; (4) The probability that the Markov process is in state Y{ after an infinite number of transitions is equal to the z-th element of each row of P°°; and (5) Each

SECTION 4.3: MARKOV MODEL NOTATION FOR EVOLUTIONARY ALGORITHMS 7 3

row of P°° is equal to the transpose of the eigenvector corresponding to the eigenvalue 1 of P^T.

We use a numerical calculation to find poo _ _j_

14 (4.21)

We also find that the eigenvalues of PT are equal to —0.4 and 1, and the eigenvector corresponding to the 1 eigenvalue is [ 5/14 9/14 ]T. These re-sults tell us that in the long run, the ratio of acceptances to rejections of detrimental mutations is 5/9.

4.3 MARKOV MODEL NOTATION FOR EVOLUTIONARY ALGORITHMS In this section we define the notation that we will use later to derive a Markov model and dynamic system model for EAs. Markov models can be valuable tools for analyzing EAs because they give us exact results. We can run simulations to in-vestigate the performance of EAs, but simulations can be misleading. For instance, a set of Monte Carlo simulations might happen to give misleading results due to the particular sequence of random numbers generated during the simulation. Also, the random number generator using in the EA simulation may be incorrect, which happens more often than we would like to think, and which would give misleading results [Savicky and Robnik-Sikonja, 2008]. Finally, the number of Monte Carlo simulations to estimate highly improbable outcomes might be so high as to not be attainable in a reasonable amount of computational time. The Markov model re-sults that we derive avoid all of these pitfalls and give exact rere-sults. The drawback of the Markov models is the high amount of computational effort that is required for their implementation.

We will focus on EAs with a population size N operating in a discrete search space of cardinality n. We will assume that the search space consists of all q-bit binary strings, so that n = 2^q. We use Xi to denote the z'-th bit string in the search space. We use v to denote the population vector; that is, Vi is the number of xi individuals in the population. We see that

Y^Vi = N. (4.22)

This equation simply means that the total number of individuals in the population is equal to JV. We use y^ to denote the k-th individual in the population. The population Y of the EA can be represented as

Y = {VI,---,VN} (4-23)

= \X\ -, X\ -> * ' ' i X\i X2i ^2i i %2i ' ' ' %ηι %ηι i %nj

vi copies V2 copies vn copies

where the y^s have been ordered to group identical individuals. We use T to denote the total number of possible populations Y. That is, T is the number of n x 1 integer vectors v such that Σ7=ι ^vi — N and Vi G [0, N].

EXAMPLE 4.5

Suppose that N = 2 and n — 4; that is, the search space consists of the bit strings {00,01,10,11}, and there are two individuals in the EA. The search space individuals are How many possible EA populations exist for a population size of N in a search space of cardinality n? It can be shown [Nix and Vose, 1992] that T is given by the following binomial coefficient, also called the choose function:

τ = ( ^{η +} £ ^{- 1} ) · ( ⁴ - ²⁶ )

We can also use the multinomial theorem [Chuan-Chong and Khee-Meng, 1992], [Simon et al., 2011a] to find T. The multinomial theorem can be stated in several ways, including the following: given K classes of objects, the number of different ways that N objects can be selected, independent of order, while choosing from each class no more than M times, is the coefficient ÇN in the polynomial

q(x) = (1 + χ + χ2 + ··· + χΜ)κ

- 1 + q^lX + q²x2 + ■ · · + q^NxN + · ■ · + x^MK'. (4.27) Our EA population vector v is an n-element vector where each element is an integer

between 0 and N inclusive, and whose elements sum to N. T is the number of unique population vectors v. So T is the number of ways that iV objects can be selected, independent of order, from n classes of objects while choosing from each class no more than N times. Applying the multinomial theorem (4.27) to this problem gives

T = qN

where q(x) = (1 + x + x² + · · · + x^N)n

= 1 + q^lX + q²x2 + · · · + q^NxN + · · · + x^Nn. (4.28) We can also use a different form of the multinomial theorem to find T [Chuan-Chong and Khee-Meng, 1992], [Simon et al., 2011a]. The multinomial theorem can be stated as follows:

Dans le document EVOLUTIONARY OPTIMIZATION ALGORITHMS (Page 104-108)