Parallelization via Concurrent Meta-interpretation

As the Linda extension interpreter indicates, the underlying parallelism in GDC may be used through interpreters to provide parallelism in a form which is not directly

provided in GDC itself. Huntbach, [1991] gives an interpreter which models in GDC the explicit message-passing parallelism of Occam, the transputer language based on Dijkstra’s CSP [Dijkstra, 1975]. But the implicit parallelism in GDC may be used to realize parallelism which exists implicitly in another language, through the use of a GDC interpreter for that language.

Landin’s usage of lambda calculus as a metalanguage for describing the semantics of Algol and the more direct basis of the functional languages on lambda calculus was noted in Section 8.1. Landin proposed the abstract SECD machine to evaluate lambda calculus expressions. The SECD machine reifies much of the control aspects of lambda calculus evaluation by using explicit stacks, which can be seen as a sacrifice of generality for the sake of efficiency. Because the control of the SECD machine is explicit, it does not parallelize without modification. McCarthy’s Eval/Apply interpreter [McCarthy, 1960] is more general. As a Lisp-like interpreter for Lisp, it can be seen as another part of his interest in meta-interpreters discussed with respect to Algol, in this case mapping the recursion of the functional language implicitly onto the recursion of the interpreter. A GDC version of the Eval/Apply interpreter will automatically parallelize lambda calculus evaluation, since the control is minimally specified. Incorporation of such an interpreter for those problems where the power of functional programming, particularly higher order functions, is useful, may be seen as an alternative to developing a new language which explicitly combines logic and functional programming [Belia and Levy, 1986].

The lambda calculus interpreter given below is based on one given by Field and Harrison [1988]. Variables are stored in an explicit environment, similar to the environments used to build an interpreter for a backtracking logic language in Section 8.7. The beta-reduction mechanism is implemented by adding the bindings for the bound variable to the environment rather than actual textual substitution. Correct binding of variables is achieved by the standard method of constructing a closure in which an abstraction is linked with an environment giving values for any free variables within it.

The interpreter works for expressions in lambda calculus, with λx.E, where E is any expression, represented by lambda(x,E), the expression E1 E2, that is E1 applied to E2 represented by apply(E1,E2) and the variable x represented by vbl(x). Arithmetic and other built-in operators are represented in their curried form by op(p) where p is the operator, or op1(p,E) where E is an expression for the partially applied form:

eval(apply(E1,E2),Env,R)

:- eval(E1,Env,R1), eval(E2,Env,R2), apply(R1,R2,R).

eval(lambda(X,Exp),Env,R) :- R=closure(X,Exp,Env).

eval(vbl(X),Env,R) :- lookup(X,Env,R).

eval(Exp,Env,R) :- otherwise | R=Exp.

apply(closure(X,Exp1,Env),Exp2,R) :- eval(Exp1,[X/Exp2|Env],R).

apply(op(P),Exp,R) :- R=op1(P,Exp).

apply(op1(P,Exp1),Exp2,R) :- dobuiltin(P,Exp1,Exp2,R).

dobuiltin(plus,Exp1,Exp2,R) :- R:=Exp1+Exp2.

// plus code for other built-in operations

If actors were executed sequentially, this interpreter would give us eager evaluation, since both function and argument expressions in an application are evaluated before applying the function. However, as we have unrestricted parallelism, initiation of the evaluation of the function, evaluation of its argument and the function application itself is concurrent. The effect is that if the function evaluates to a closure, the function application may take place even though computation of its argument is still in progress. With an actor network

:- eval(Exp2,Env,R2), apply(closure(X,Exp,Env1),R2,R).

applying the closure gives:

:- eval(Exp2,Env,R2), eval(Exp,[X/R2|Env2],R).

Although the value of R2 is still being computed we may proceed with the evaluation of Exp. The channel R2 plays a role similar to MultiLisp’s “future” construct [Halstead, 1985]: a place-holder which may be manipulated as a first-class object while its final value is being computed. Application of strict operators however will be suspended until their arguments are evaluated, for example, an arithmetic operation will reduce to a call to GDC’s built-in arithmetic and suspend until its arguments are ground.

If a curried operator implements conditional expressions, we will end up by computing both branches of the conditional even though only one is needed since computation of both branches will commence in parallel. To inhibit this, as in Field and Harrison’s eager interpreter, we can treat conditionals as a special case by including an additional constructor in the definition of expressions to accommodate them. So “if E1 then E2 else E3” is parsed to cond(E1,E2,E3) rather than apply(apply(apply(op(cond),E1),E2),E3). We then need to add additional behaviors:

eval(cond(E1,E2,E3),Env,R)

:- eval(E1,Env,TruthVal), branch(TruthVal,E2,E3,Env,R).

branch(true,E2,E3,Env,R) :- eval(E2,Env,R).

branch(false,E2,E3,Env,R) :- eval(E3,Env,R).

The dependency in branch means that evaluation of the branches of the conditional will not take place until the condition has been fully evaluated to a Boolean constant.

Full lazy evaluation, as used in modern functional languages, may be obtained by passing the argument in an application in the form of a suspension containing the unevaluated expression and its environment to ensure that if it is eventually evaluated its channels are correctly bound. This gives us the following rule to replace the rule for evaluating applications:

eval(apply(E1,E2),Env,R)

:- eval(E1,EnvR1), apply(E1,susp(E2,Env),R).

We also need a rule to evaluate suspensions when necessary:

eval(susp(E,Env1),Env2,R) :- eval(E,Env1,R).

Since the environment may contain suspensions, when we look up an identifier we may need to evaluate it further, so we alter the rule for variable lookup to:

:- eval(vbl(X),Env,R) :- lookup(X,Env,V), eval(V,Env,R).

Since some primitives such as the arithmetic functions require their arguments to be fully evaluated before the operation can be completed (that is, the primitives are strict), we must add this to the code for executing the primitive, for example:

dobuiltin(plus,Exp1,Exp2,R)

:- eval(Exp1,[],R1), eval(Exp2,[],R2), R:=R1+R2.

Parallelism is limited in the lazy interpreter but not completely excised. We no longer evaluate the function application and the argument simultaneously since evaluation of the argument is suspended. However evaluation of the arguments to strict primitives, such as plus above, does take place in parallel. The effect is to give conservative parallelism: we only do those computations in parallel whose results we know are definitely needed. A more sophisticated combination of lazy functional programming could be obtained by using operators such as those proposed by Trinder et al [1998].

The interpreters given here for lambda calculus are not the most efficient that could be achieved and are given mainly for illustration of the technique of using interpreters to embed one language in another. One major issue we have not dealt with in lazy evaluation is that in practice call-by-need is essential to ensure that suspensions are evaluated once with the evaluation shared by all references to them, rather than re-evaluated every time they are referenced. An efficient way of dealing with recursion, used by Field and Harrison [1988], is to build circular environments rather than rely on the fact that the fixpoint operator can be represented directly in lambda calculus [Barendregt, 1984]. (These issues can be dealt with in GDC, but there is not space for further detail here.) Circular environments can be represented directly if we take up Colmerauer’s proposal [Colmeraurer, 1982] to recognize the lack of the occur check as a language feature which can be interpreted as the logic of circular or infinite structures. A more efficient way however would be to dispense with environments altogether and use a combinator [Turner, 1979] or super-combinator [Hughes, 1982]

based evaluator.

8.10 Conclusion

The use of meta-interpreters may be seen as both a way of structuring programs and a way of avoiding cluttering a programming language with a variety of features. It has been recognized that programs are clearer if the logic of the program is separated from the control. This was one of the guiding principles in the development of logic programming languages. For efficiency reasons a detailed user-defined control mechanism may be necessary, but this should not be mixed in with the specification of the logic of the program. A meta-interpreter may be regarded as the third element of a program that combines the logic and the control. It may be an implicit part of the language or the programmers may themselves provide it. It is often the case that program clarity is aided by writing the program in a simple problem-oriented

language and implementing that language in a language closer to the underlying machine. Recursively, the implementation language may itself be similarly implemented. This is a technique already familiar under the name structured programming [Dahl et al., 1972]. But metaprogramming provides a clear division between the levels of structure and the potential separation of logic and control at each layer means that the top-level layer is not constrained to inherit directly the control of the machine level, or to be cluttered with explicit control structures to subvert it.

On language features, meta-interpretation provides a facility to add features or change aspects of that language as required through the use of an interpreter specifically designed to add or change a particular feature. This compares with complex single-level languages where every feature a programmer may at any time have to use must be added as part of the language definition. This creates an unwieldy language that is difficult to learn, use safely and debug. It should be recalled that every new feature added to a language must not only be considered in its own terms but also in terms of its impact on other features.

One important use of interpreters (not considered in detail) here is their use to assist in the problem of mapping the abstract parallelism of a language like GDC onto a real parallel architecture. Such an interpreter would deal with problems like load-balancing and deciding when the increased communication costs involved in moving some computation to another processor are balanced by an increased utilization of parallelism.

The biggest barrier against the use of meta-interpreters as a program or language-definition structuring device is the overhead. A program, which must work through several layers of interpreter before getting to the machine level, will not be as efficient as one that directly controls the machine level. To some extent this can be overcome, as we have shown, through the use of interpreters in which much of the lower level is inherited implicitly by the upper level rather than explicitly reimplemented. However, even the “vanilla meta-interpreter” of logic programming which inherits almost everything from the underlying level has been shown in practice to increase execution time by an order of magnitude. A solution to the problem is to use partial evaluation to flatten out the layers of interpretation into a single-level program. This technique will be explored in detail in the next chapter.

M.M. Huntbach, G.A. Ringwood: Agent-Oriented Programming, LNAI 1630, pp. 213–246, 1999.

© Springer-Verlag Berlin Heidelberg 1999