... both a development from Impressionism and a reaction against it. Post-Impressionism was based on scientific principles and re-sulted in highly formalized compositions.
[Chilvers and Osborne, 1988]
Minsky was one of the foremost critics of the use of logic for representing common-sense knowledge. A widely disseminated preprint [Minsky, 1975] had an appendix entitled Criticism of the Logistic Approach (the appendix did not appear in the pub-lished version):
Because logicians are not concerned with systems that will later be enlarged, they can design axioms that permit only the conclusions they want. In the development of intelligence, the situation is
differ-ent. One has to learn which features of situations are important, and which kinds of deductions are not to be regarded seriously.
Some of the confusion with semantic nets was dispelled with frames [Minsky, 1975].
Frames were intended for the large-scale organization of knowledge and were origi-nally introduced for scene representation in computer vision. They adopted a rather more structured approach to collecting facts about a particular object and event types and arranging the types into a taxonomic hierarchy analogous to a biological hierar-chy. In frames, intransitive links were encapsulated in nodes. The fields (slots) of the frame are filled with the values of various default attributes associated with the ob-ject. Contrary to some opinion, frames are related to the frame problem of situation calculus. In the frame representation, only certain attributes need change their value in response to actions. Procedural knowledge on how the attributes are (or are not) updated as a result of actions are included in the frame. The restriction of links to is-a relations relates a class to a more general one. This produces a partial order on classes that organizes them into a hierarchy of specialization. Properties associated with general types can be inherited by more specialized ones. By adding a second (non-transitive) instance relation, the frame representation can be extended to allow the distinction to be made between general and specific. The instance relations allow default values to be inherited from generic class frames and the determination of all instances of a given class.
Minsky supervised a series of Ph.D. projects, known as microworlds, which used frames to represent limited domains of knowledge. Slagle’s [1963] SAINT system solved closed form integration problems typical of college calculus courses. Evans’
[1968] ANALOGY system solved geometric analogy problems that appear in IQ tests. Raphael’s [1968] SIR (Semantic Information Retrieval) was able to accept statements in a restricted subset of English and answer questions on them. Bobrow’s [1967] STUDENT system solved algebra story problems. The most famous mi-croworld was the Blocks World. It consisted of a set of children’s building blocks stacked on a tabletop. A task in this world is to rearrange the stack in a certain way using a robot arm than can only pick up one block at a time.
The Blocks World was the setting for many applications: Huffman’s [1971] vision system; Waltz’s constraint propagation vision system; the learning system of Winston [1975]; the natural language understanding program of Winograd and the planner of Fahlman [1974]. The use of constraint solving in Waltz’s [1972] extension of Huff-man’s [1971] and Clowes [1971] computer vision systems was claimed to demon-strate that combinatorial explosion can be controlled. The system attempts to recog-nize objects in scenes from contours. Waltz’s [1972] extension incorporated shadows to the scene analysis program. This contributed such an enormous increase in com-plexity to the problem that simple backtracking search became intractable.
Widely criticized as a trivial combination of semantic nets and object-oriented pro-gramming [Dahl et al., 1970], Minksy’s frames paper served to place knowledge representation as a central issue for AI. Briggs [1985] suggests that knowledge repre-sentation research began with ancient Indian analysis of Shastric Sanskrit in the first millennium BC. Shastric Sanskrit grammatical theory proposed not only a formal syntax and vocabulary but also analysis of its semantics using semantic nets. In
con-trast, the linguist Schank [Schank and Abelson, 1977] claimed: There is no such thing as syntax. Schank and his students built a series of natural language understanding programs [Schank and Abelson, 1977; Schank and Riesbeck, 1981; Dyer, 1983]
which represented stereotypical situations [Cullingford, 1981] describing human memory [Rieger, 1976; Kolodner, 1983], plans and goals [Wilensky, 1983]. LUNAR [Woods, 1972] allowed geologists to ask English language questions about rock sam-ples brought back from the Apollo Moon Mission.
Although the original frame representation provided only single inheritance, later extensions allowed more than one superclass (this is called multiple and mixed in-heritance). While multiple inheritance allows the user to gain further expressiveness, it brings a new range of problems. The inheritance network effectively becomes an arbitrary directed graph. Retrieving a value from a slot then involves search. Frames do not incorporate any distinction between ‘essential’ properties (those an individual must possess to be considered an instance of a class) and accidental properties (those that all instances of the class just happen to possess). The psychological intuition behind this is that conceptual encoding in the human brain is not concerned with defining strictly exhaustive properties of exemplars of some category. Categorization is concerned with the salient properties that are typical of the class. Brachmann [1985] pointed out that this makes it impossible to express universal truths, or even construct composite ideas out of simpler conceptual units in any reliable way.
A long-term research effort that attempted to build a system with encyclopedic knowledge using the frame representation is Cyc (from encyclopedia) [Lenat and Guha, 1990]. Cyc was a privately funded project at MCC that was part of the US response to the Japanese FGCS. Despite ten years effort and hundreds of millions of dollars in funding, Cyc failed to find large-scale application. The failure to choose a sufficiently expressive common representation language was admitted to be an over-sight near the end of the project [Lenat, 1995]:
Another point is that a standard sort of frame-and-slot language proved to be awkward in various contexts: ... Such experiences caused us to move toward a more expressive language, namely first-order predicate calculus with a series of second order exten-sions ...
Two years after Minsky’s defining paper, Hayes [1977] gave a formal interpretation of what frames were about. Hayes argues that a representational language must have a semantic theory. For the most part, he found that frame representation is just a new syntax for a subset of first-order logic. While subsequently hotly disputed, Hayes’
conclusion is that, except for reflexive reasoning, frames had not achieved much.
Reflexive reasoning is one in which a reasoning agent can reason about its own rea-soning process.
While generally not as expressive as first-order predicate calculus, semantic nets and frames do carry extra indexing information that makes many common types of infer-ence more efficient. In defense of logic, Stickel [1982, 1986] and Walther [1984] give examples of how similar indexing can be done in implementations of systems that carry out inferences on predicate calculus expressions. One benefit of the frame
or-ganization of knowledge is economy of storage. Hierarchical oror-ganization gives im-provement in system understanding and ease of maintenance. A feature of frames not shared by logic is object-identity, the ability to distinguish between two instances of a class with the same properties.
On the general endeavor of knowledge representation, Dreyfus [1988], an MIT phi-losopher, notes:
Indeed, philosophers from Socrates through Leibniz to early Witt-genstein carried on serious epistemological research in this area for two thousand years without notable success.
With a slight exaggeration, Plato’s theory of forms can be identified with frames:
forms represent ideal perfect classes of object; earthly instances of forms are imper-fect copies.
1.10 Precisionism
... in which urban and industrial subjects were depicted with a smooth precise technique.
[Chilvers and Osborne, 1988]
In the Precisionist Movement there was pressure from AI’s principal funding agency, DARPA (the United States Defense Advanced Research Projects Agency), to make research pay off. DARPA’s lead was followed by other governments’ funding bodies that implicitly and explicitly directed AI to tackle real world, engineering problems instead of toy or mathematical problems. Feigenbaum and others at Stanford began the Heuristic Programming Project (HPP) to investigate the extent to which mi-croworld technology could be applied to real world problems.
The first expert system, Dendral, was initiated in 1965 at Stanford University and grew in power throughout the 1970s [Lindsay et al., 1980]. Given data from mass spectroscopy, the system attempted to determine the structural formula for chemical molecules. The improvement in performance was brought about by replacing first level structural rules by second level (larger grain and possibly incomplete) rules elicited from experts. Dendral was followed by other successful expert systems in the 1970s that epitomized the ‘Precisionist Movement’. Mycin [Shortliffe et al., 1973]
gives advice on treating blood infections. Rules were acquired from extensive inter-viewing of experts who acquired their knowledge from cases. The rules had to reflect the uncertainty associated with the medical knowledge. Another probabilistic rea-soning system Prospector [Duda et al., 1979] was a consultation system for helping geologists in mineral exploration. It generated enormous publicity by recommending exploratory drilling at a geological site that proved to contain a large molybdenum deposit. The first commercial expert system Xcon (originally called R1) [McDermott, 1981], is an advice system for configuring the DEC’s VAX range of computers. By 1986, it was estimated to be saving the company $40 million a year.
The difficulty in building expert systems led to proprietary pluralist, high-level pro-gramming environments such as Loops, Kee, and Art. These environments provide the user with a myriad of tools that had been found useful in building expert systems.
Such systems are criticized because they provide no guidance on how to compose the tools. This was claimed to encourage ad-hoc programming styles in which little at-tention is paid to structure. Clancey [1983] analyzed Mycin’s rules and found it use-ful to separate base-level medical knowledge from metalevel diagnostic strategy.
MetaX can be understood as “X about X.” (This understanding does not apply to metastable states.) So for instance, metaknowledge is knowledge about knowledge, metareasoning is reasoning about reasoning. This separation followed a previous trend in program design. Wirth [1976], coined the adage
program = algorithm + datastructure
in the title of a book on programming. Kowalski [1979] continued the reductionist view with
algorithm = logic + control
in particular regard to the programming language Prolog. Using the separation of knowledge, Meta-Dendral [Buchanan and Mitchell, 1978] was able to learn rules that explain mass spectroscopy data used by the expert system Dendral [Buchanan et al., 1971].
A metareasoning facility allows reflection on the reasoning process that Hayes claimed was the only distinctive attribute of frames. The universal deductive system noted earlier is a meta-interpreter for a deductive system. More specifically, there are two theories: one, called the object theory, and another, the metatheory that concerns the object theory. Metaknowledge can be strategic and tactical, knowledge about how to use the base knowledge. An example is the metarule, always try this rule before any other. The base level knowledge could be from a conventional database or a set of production rules so that a distinction of degree arises. The term expert system is generally reserved for systems with many more rules than facts. Deductive databases have many more facts than rules. The term knowledge-based system was coined to encompass both extremes and dissociate the movement from the hyperbole that had become associated with AI. The distinction between a knowledgebase and a database is that in the former not all knowledge is represented explicitly. The emotive term knowledge engineering typifies the Movement. Feigenbaum [1980] defined it as the reduction of a large body of experience to a precise body of rules and facts.
The main difference between McCarthy’s Advice Taker [1958] and Newell et al.’s Logic Theorist [1956] was the way in which heuristics were embodied. McCarthy wanted to describe the process of reasoning with sentences in a formally defined metalanguage, predicate calculus. For McCarthy the metalanguage and object lan-guage coincide. Hayes [1973] introduced a metalanlan-guage for coding rules of infer-ence and for expressing constraints on the application of those rules. He showed that by slightly varying the constraints it was possible to describe markedly different reasoning methods. He proposed a system called Golux based on this language, but it was never implemented.
According to Jackson [1986], successful expert systems are generally those with restricted domains of expertise. Here, there is a substantial body of empirical knowl-edge connecting situations to actions (which naturally lend themselves to production systems) and where deeper representations of knowledge such as spatial, temporal or causal can be neglected. With such systems it is known in advance exactly what ternal parameters are required to solve the problem (i.e., the items the system is ex-pected to configure). The expertise of such a system is in making local decisions that do not violate global constraints and global decisions that allow local solutions (step-wise refinement).
By 1988, DEC’s AI group had deployed 40 expert systems with more on the way.
DuPont had 100 in use and 500 in development, saving an estimated $10 million a year. Nearly every major US corporation had its own knowledge engineering group and was either using or investigating expert system technology. A high point of the
‘Precisionist Movement’ came in 1981 with Japanese Fifth Generation National Ini-tiative to build machines that give hardware support to knowledge-based systems [Moto-oka, 1982]. So as not to be left behind, other governments initiated national and multinational (EC) research programs of their own. In the US, the Microelec-tronics and Computer Technology Corporation (MCC) was formed as a research consortium. In the UK, the Alvey Report reinstated some knowledge-based systems funding that had been cut as a result of the Lighthill report. In Europe, Esprit sup-ported industrial and academic collaborations. Industry sales of knowledge-based systems related components went from a $3 million in 1980 to $2 billion in 1988.
Sales included software tools to build expert systems, specialized AI workstations based on Lisp and industrial robotic vision systems.
One of the difficulties of expert systems is eliciting knowledge from experts. Domain experts were visibly able to perform complex diagnostic tasks but found it difficult to explain how they had done so. With early systems, Dendral and Mycin, knowledge engineers conducted long interviews with domain experts to extract rules but the results often contained omissions and inconsistencies that had to be laboriously de-bugged. Quinlan [1982] observed:
While the typical rate of knowledge elucidation by this method is a few rules per man-day, an expert system for a complex task may re-quire hundreds or even thousands of rules. It is obvious that the interview approach cannot keep pace with the burgeoning demand for expert systems.
Feigenbaum [1977] had written:
… the acquisition of domain knowledge is the bottleneck problem in the building of applications-oriented intelligent agents.
The problem has since become known as the Feigenbaum bottleneck.