LISP: THE LANGUAGE OF GENETIC PROGRAMMING

Genetic programming is often implemented in the Lisp computer language be-cause the structure of Lisp ties in so nicely with computer program crossover and mutation. This section provides an overview of Lisp, and provides a conceptual description of how Lisp programs can be combined to create new programs.

The evolution of computer programs is challenging because programs are not usually represented in a way that makes mutation and crossover feasible. This is the primary obstacle that we need to overcome to evolve computer programs. For example, consider two standard MATLAB programs:

Program 1

How could we perform crossover on these programs? A crossover operation that does not carefully consider syntax will result in an illegal program (that is, a pro-gram that does not run, or even compile). For example, if we replace the first two

lines in the left program above with the first two lines from the right program, we

The child program above is clearly not a legal program. Most of the crossover operations and mutation operations that we perform on MATLAB programs will result in illegal programs. The same can be said for programs written in most other popular languages (C, Java, Fortran, Basic, Perl, Python, and so on). All of these languages have similar structures, and so none of them can be easily mutated or crossed over with other programs.

However, there is one language that is suitable for crossover and mutation. That language is called Lisp [Winston and Horn, 1989]. Lisp, invented in 1958, is prob-ably the second oldest programming language, only one year newer than Fortran.

Lisp is an acronym for "list processing." Linked lists are one of the major struc-tures in Lisp, and this made it a likely choice for artificial intelligence applications (that is, expert systems and their chains of inference rules) in its early days. Lisp is not particularly popular any more because it is different from other languages.

However, it has become popular among GP researchers and practitioners because of its suitability for crossover and mutation.

Lisp program code is written with parentheses, with the function name followed by its arguments. For example, the following code adds x and 3:

(+ x 3) .

This is an example of prefix notation because the mathematical operator precedes its inputs. A parenthetical expression in Lisp is also called an s-expression, which is short for symbolic expression. All s-expressions can be viewed as functions that return the value that they compute. S-expressions that compute multiple values return the last value that they compute. Not only does (+ x 3) add x and 3, but it returns x + 3 to the next higher level of function execution.

We present a few more examples. The following code computes the cosine of (a?+ 3):

(cos (+ x 3)) .

The following code computes the minimum of cos(x + 3) and zj 14:

(min (cos (+ x 3)) (/ z 14)) . The following code copies the value of y to x if z > 4:

(if (> z 4) (setf x y)) .

Note that expressions are like sets in that expressions can contain other s-expressions. In the above s-expression, (> z 4) and (setf a: y) are both s-expressions that are part of the higher-level s-expression (if (> z 4) (setf x y)).

SECTION 7.1: LISP: THE LANGUAGE OF GENETIC PROGRAMMING 1 4 5

The reason Lisp code is evolvable is that s-expressions directly correspond to tree structures, also called syntax trees. For example, a Lisp s-expression for the calculation of xy -f \z\ can be written as follows:

(+ (* x y) (abs z)) .

This s-expression can be represented with the syntax tree shown in Figure 7.1.

We interpret a syntax tree like the one in Figure 7.1 by working our way up from the bottom. Figure 7.1 shows that x and y are at the bottom of the tree, and are connected to each other with a multiplication operator. This gives us the expression xy, or the s-expression (* x y). We see that this sub-s-expression corresponds to a subtree in Figure 7.1. The symbols that appear at the bottom of a syntax tree (for example, x, y, and z in Figure 7.1) are called leaves.

Figure 7.1 Syntax tree for the function xy+ \z\, which is represented with the s-expression (+ (* x y) (abs z)). The "A" node represents the absolute value operator.

Figure 7.1 also shows that z is at the bottom of the tree, and is operated on by the absolute value function. This gives us the expression \z\, or the s-expression (abs z). We see again that the sub-s-expression corresponds to a subtree in Figure 7.1.

Finally, Figure 7.1 shows that xy and \z\ meet at the addition node at the top of the tree. This gives us the expression xy + |z|, or the s-expression (+ (* x y) (abs z)). We see that this high-level s-expression corresponds to the entire tree structure in Figure 7.1.

For another example, consider a function that returns (x + y) if t > 5, and (x + 2 + z) otherwise:

If* > 5

return (x + y) else

return (x + 2 + z) End

This function can be written in Lisp notation as:

(if (> i 5 ) ( + x | / ) ( + x 2 z)) .

Figure 7.2 shows the syntax tree for this function. Note that many Lisp functions, like the addition function in the above example, can take a variable number of arguments.

Figure 7.2 Syntax tree for the function, "If t > 5 then return (x + y), else return (x + 2 + 2:)."

Crossover with Lisp Programs

The correspondence between s-expressions and subtrees makes it conceptually straight-forward to perform operations like crossover and mutation on Lisp computer pro-grams. For example, consider the following functions:

Parent 1: xy + \z\ => (+ (* x y)) abs z)

Parent 2: (x + z)x — (z + y/x) => (— (* (+ x z) x) (+ z (/ y x))).

(7.1) These two parent functions are shown in Figure 7.3. We can create two child functions by randomly choosing a crossover point in each parent, and swapping the subtrees that lie below those points. For example, suppose that we choose the multiplication node in Parent 1, and the second addition node in Parent 2, as crossover points. Figure 7.3 shows how the subtrees in the parents that lie below those points can be swapped to create child functions. Child functions that are created in this way are always valid syntax trees.

Now consider crossover between the s-expressions of Equation (7.1). The fol-lowing equation emphasizes the parenthetical pairs that correspond to the subtrees of Figure 7.3, and shows how swapping parenthetical pairs in the two original s-expressions (parents) creates new s-s-expressions (children):

(4- [ * x y ]) abs z) 1 = > ί (+ { + z ( / y x ) } ) abs z) ( - (* (+ x z) x) { + z ( / y x ) } ) J \ ( - (* (+ x z) x) [ * x y ] )

(7.2) where the crossed-over s-expressions are shown in bold font. This is called tree-based crossover. Any expression in a syntax tree can be replaced by any other s-expression, and the syntax tree will remain valid. This is what makes Lisp a perfect language for GP. If we want to perform crossover between two Lisp programs, we simply find a random left parenthesis in Parent 1, then find the matching right parenthesis; the text between those two parentheses forms a valid s-expression.

Similarly, we find a random left parenthesis in Parent 2, then find the matching right parenthesis, to obtain another s-expression. After we swap the two s-expressions, we have two children. We can perform mutation in a similar way by replacing a randomly selected s-expression with a randomly-generated s-expression, which is called tree-based mutation.

SECTION 7.1: LISP: THE LANGUAGE OF GENETIC PROGRAMMING 1 4 7 Parent 1 Parent 2

Child 1 Child 2

Figure 7.3 Two syntax trees (parents) cross over to produce two new syntax trees (children). Crossover performed in this way always results in valid child syntax trees.

Figure 7.4 shows an additional crossover example. We have the same two parents as in Figure 7.3, but we randomly select the z node as the crossover point in Parent 1, and the division node as the crossover point in Parent 2. The crossover operation of Figure 7.4 is represented in s-expression notation as follows (where, as before, the crossed-over s-expressions are shown in bold font):

(+ ( * x y )) abs z) \ [ (+ ( * x y )) abs [ / y x ] ) {-{*(+xz)x)( + z[/yx])) J \ ( - ( * ( + x z) x) ( + z z ) ) .

(7.3)

Parent 1 Parent 2

Child 1 Child 2

Figure 7.4 Two syntax trees (parents) cross over to produces two new syntax trees (children). The parents shown here are the same as those in Figure 7.3, but the crossover points are chosen differently.