Examples of Discovered Rules

A selection of the 568 new rules discovered by the genetic algorithm in the series of seven runs with 5 schemata appears in Table 10.1. Rows 1 through 5 show a single rule that was discovered in five of the seven runs. Note that the rule has dif-ferent strengths in the runs. This useful rule allows an identifier to be appended to the head of a noun phrase. This accepts constructions such as “Register X” and

“Pin AG4/2”, which often occur in this domain. Rule 6 occurred in two runs and allows a modal (can, may, should) to prefix an active verb. It is surprising that this rule was not discovered in all runs. That may be due to other rules that prefixed predicate structures with modals, such as Rule 7, which was discovered in two runs. The discovered rule with greatest strength is shown in row 8 of the table.

This rule forms a sentence from a simple sentence followed by a nominal and a pe-riod. Its strength is twice the next-strongest rule and tenfold most other strong new rules, and yet it was found in only one run. The rule in row 9 was found in two runs and forms a predicate with an active verb structure and two nominals. This accommodates sentences with both an indirect and direct object (a construct not

covered by the initial grammar). Note that Rule 8 serves a similar function pro-vided the simple sentence contains no adverbials.

Rule 10 allows noun phrases to be strung together, which is useful in parsing long nominal compounds, such as “the computer multiplexer chip strobe pin of the processor.” Rule 11 forms a coordinated noun phrase from three noun phrases and a coordinating conjunction, and allows for the grammatical error of omitting a comma. A similar rule was found in another run. Rule 13 constructs a nominal from an adjective preceding a noun phrase. This would appear to be a useful rule, but most word sequences accepted by this rule are also recognized by a combina-tion of three rules in the initial grammar. So, while this rule may accommodate a few cases, it will also introduce additional ambiguity in the grammar by allowing multiple parses of some strings. Other rules that can introduce ambiguity were found among the discovered rules.

Table 10.1 Examples of Discovered Rules

# Runs Strength Rule

1 0.9 head --> head id 2 20.3 head --> head id 3 1.8 head --> head id 4 0.5 head --> head id 5 8.8 head --> head id 6 2 - avs --> mod verb 7 2 - pred --> mod pvs d 8 1 220.1 s --> ss n . 9 2 - pred --> avs n n 10 1 4.4 n --> np np of n 11 1 1.6 nc --> np np conj np 12 2 - avs --> conj verb 13 4 - n --> adjs np 14 4 - np --> det det head 15 6 - n --> prep np

The rules mentioned so far are consistent and desirable to the phrase-structured parsing goals. Not all discovered rules were desirable. Some rules introduce structures that are not attractive candidates for semantic analysis. For example rule 12 considers a conjunction followed by a verb to be an active verb structure. This rule was probably discovered to accommodate coordinated verbs, such as in “the device clears and loads the register”, which were not accepted by the initial gram-mar. Since a wildcard matches only one symbol, there is no mechanism or intro-ducing conjunctions in a single step. Mechanisms for accommodating conjunc-tions are being considered. The rule in row 14 was discovered in four runs and accepts a noun phrase with two determiners (a, an, the, this, that) and was probably generated by word sequences such as “that the register” or possibly by typographi-cal errors. The last rule in Table 10.1 is particularly disturbing because it

con-structs a nominal from a preposition followed by a noun phrase. Such concon-structs are indeed phrases but should be classified as prepositional phrases (that modify nouns) or adverbials (phrases that directly modify verbs). Similar rules were dis-covered that form noun phrases from prepositions and nouns. This rule may have been discovered to work with higher-order rules such as Rule 8 above. Presently, a simple sentence may have only up to three nominals and adverbials after the verb.

Rule 8 with Rule15 allows one additional adverbial.

Although the competence of the grammar evolves vigorously, and most new rules appear reasonable, some of the discovered rules do not appear desirable from the standpoint of conventional English grammar, or because they introduce ambi-guity in the grammar. The “ungrammatical” or undesirable rules may be useful, however, in that they will enable semantic analysis of ungrammatical sentences.

Ambiguity in the grammar is inconvenient because it increases the computation time for both grammatical and semantic analysis. Ambiguity is introduced in the grammar when the result nonterminal they construct can be synthesized by one or more existing rules. In any case, the manner in which these undesirable rules are introduced needs to be investigated so that they might be avoided.

10.6 Conclusions

A genetic algorithm has been presented for evolving context-free phrase-structured grammars from an initial, small, handcrafted grammar. The algorithm uses distinct discovery and evaluation steps, as well as a culling step to reduce the ambiguity of the grammar being evolved. The algorithm performs surprisingly well on devel-oping grammars for very complex sentences in a restricted domain of English. The algorithm was generally run for a small number of generations to observe the ini-tial growth of the grammar, but even then, it reached 90% competence in some cases.

Much remains to be investigated in this area. First, a method is needed to cull or avoid rules that form undesirable phrase structures from the semantic analysis perspective. This might be done by interaction with a user who affirms desirable parsings from time to time, as with the occasional environmental rewards in classi-fier systems. This interactive feedback could have the effects of negative training strings. Alternatively, a semantic analyzer can be run on some of the parse trees and have the rules that participated in parse supporting a successful semantic analysis rewarded enthusiastically. A mechanism needs to be added that will allow conjunction to be introduced in the form X conj X, rather than in two separate rules.

The competence of the evolving grammars seems to encounter ceilings below 100% competence. This barrier must be overcome. It would also be interesting to increase the number of schemata introduced in a generation as the grammar evolves. While this would tend to sustain rapid growth during the later genera-tions, it would also increase the evaluation time substantially, since that time de-pends heavily on the size of the grammar.

Acknowledgments

The research reported here was supported in part by the National Science Founda-tion, Grant MIP-9707317.

References

1. W. A. Cook, (1989) Case Grammar Theory, Georgetown University Press, Washing-ton, DC.

2. C. Manning and H. Schütze, (1999) Foundations of Statistical Natural Language Proc-essing, MIT Press, Cambridge, MA.

3. E. Charniak, (1996) Tree-bank Grammars, Proceedings of the Thirteenth National Conference on Artificial Intelligence, AAAI Press/MIT Press, Menlo Park, pp.1031-1036.

4. G. Carroll and E. Charniak, (1992) Two Experiments on Learning Probabilistic De-pendency Grammars from Corpora, Workshop Notes, Statistically-Based NLP Tech-niques, 1-13, AAAI Press, Menlo Park, CA,.

5. P. Wyard, (1991) Context free grammar induction using genetic algorithms, Proceed-ings of the 4th International Conference on Genetic Algorithms, 514–518.

6. S. Lucas, (1994) Structuring Chromosomes for Context-Free Grammar Evolution, First IEEE Int’l Conf. on Evolutionary Computation, 130-135.

7. T. Smith and I. Witten, (1995) A Genetic Algorithm for the Induction of Natural Lan-guage Grammars Proc. IJCAI-95 Workshop on New Approaches to Learning for Natural Language Processing,17-24, Montreal, Canada.

8. L. Booker, D. Goldberg, J. Holland, (1989) Classifier Systems and Genetic Algo-rithms,” Artificial Intelligence, 40, 235-282.

9. K-S. Fu, T. Booth, (1986) Grammatical Inference: Introduction and Survey, IEEE Trans. Pattern Analysis and Machine Intelligence, 343-375, May

10. M. Gold, (1967) Language Identification in the Limit, Information and Control, 10, 447-474.

11. W. R. Cyre (1995) A Requirements Sublanguage for Automated Analysis, Interna-tional Journal of Intelligent Systems, 10 (7), 665-689.

12. W. R. Cyre, (1997) Capture, Integration, and Analysis of Digital System Require-ments with Conceptual Graphs, IEEE Trans. Knowledge and Data Engineering, 9(1), 8-23.

1

2 3

Summary.

11.1 Introduction

Evaluating Protein Structure Prediction