HAL Id: hal-00190371
https://telearn.archives-ouvertes.fr/hal-00190371
Submitted on 23 Nov 2007
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Virtual mental space: interacting with the characters of works of literature
Boris Galitsky
To cite this version:
Boris Galitsky. Virtual mental space: interacting with the characters of works of literature. 4th International Conference on Cognitive Technology: Instruments of Mind, CT 2001, Warwick, UK, August 6-9, 2001. Lecture Notes in Computer Science 2117 Springer 2001, 2001, Warwick, United Kingdom. pp.282-288. �hal-00190371�
Virtual mental space: interacting with the characters of works of literature
Boris Galitsky iAskWeb, Inc.
261 Main Str #1B Waltham MA 02451 USA boris@iaskweb.com, http://www.iaskweb.com
Abstract. We build the software implementation of virtual mental space: its inhabitants are interacting with the literature characters in natural language.
The players are encouraged to ask questions about intentions and desires, knowledge and believes, pretending and deceiving of the literature characters.
Works of literature are identified based on the patterns of mental interaction between the characters. Reasoning about mental states and natural language processing with the semantic focus are the techniques required to implement the literature search component of the virtual mental world.
1 Introduction
Interactive forms of entertainment and education are based on such modalities as video, sound, speech, etc. Combination of various modalities in computer entertainment and educational systems is important to achieve the overall impression of being close to the “real world” and being “smart”. We believe that the latter component should involve, in particular, interactive mental environment. The phenomena of “mental virtual world” its cognitive features and sociological consequences have been subject to rather limited exploration in comparison with the
“physical virtual reality”, approached from the viewpoint of simulation, visualization, etc.
Software implementation of mental world is an important component of cognitive technology, accompanying perception of physical reality. Stories are one of the many ways in which a person is presented to others and herself [. Training of mental reasoning is important way of developing emotional and intellectual capabilities of children and adults with various mental disorders.
In recent years, there was a lot of attention to the formal background of reasoning about mental states and actions. However, reasoning about mental states
of automatic and human agents just has started to find a variety of applications.
Planning and scheduling, multiagent management and prediction of investors’
mental states are some examples of implementation of reasoning about beliefs [1,2].
Customers of these systems are usually aware that there is some kind of implementation of reasoning about customers’ beliefs and intentions.
There is a variety of expectations for such systems: some customers prefer to have the system capability of decision-making based on mental modeling reduced;
therefore the system autonomy decreases. At the same time, other customers enjoy the features of advanced cooperation, assuming they understand the system’s capability of mental modeling sufficiently to trust its autonomous decisions. Our overall experience with the customers’ satisfaction with the system, modeling their mental states for advising or performing certain functions shows that it is dangerous to involve the virtual mental space
We focus on the design and implementation of interactive entertainment environment based on the mental states of the characters of works of literature (WOL). The game players are involved in the literature world by means of asking question about the mental states of their favorite characters. The system is capable of understanding these natural language (NL) questions and providing the introduction to the adequate WOL scenario. In addition, the system is capable of synthesis the scenario, involving the found characters.
We present the InPlainWords system, which implements the search of the data with the logical structure, which is very specific on one hand and is hardly subject to formalization on the other hand; modeling of mental states of literature characters is used to encode a WOL plot. In contrast to the behavior simulation systems (see, for example, [6,7]), we focus on just a single interaction modality: question-answering.
The Internet entertainment tool is developed, which allows to “intelligently”
access books without explicit mentioning of its title or its author’s name. Our system introduces to customers a new way to exchange ideas about the literature plots, using the belief states of characters. Extension of the literature library is implemented as well as its querying: a player can input a brief description and an essential mental state (conflict) of a WOL such that the other players can search through it.
Mental states of literature characters
What is the role of mental states of literature characters in the classification and schematization of the works of literature? The analysis we performed with our WOL database, which includes mental states of WOL characters, allows us to make the following conclusions (with the confidence below the acceptance level for the computer science audience). Note that a mental state is represented by a set of mental formulas, consisting from the mental predicates know, want, inform, pretend, etc. The mental formulas are expressed in a second-order predicate language, extended by the metapredicates [4] of mental states and actions.
1. As a rule, the main plot of a WOL deals with the development of human emotions. The mental states, to a certain degree, reflect and code the very
essence of a particular WOL. It is mainly true for the small forms (a verse, a story, a sketch, etc.). When we touch upon a novel, a poem, a drama, etc.
which have a complex and highly structured nature, then we may talk about the set of individual plots. Each of them is depicting its own structure of mental states, which is not necessarily unique. Taken all together, they have the highly complex forms, appropriate to identify the WOL.
2. Extraction of the mental states from a WOL allows us to clarify psychological, social and philosophical problems, encoded by this work. The mental components, in contrast to the “physical” ones are expressed implicitly and always have a number of meanings.
3. The same mental formula may be a part of different WOLs of literature, which are written by the distinguishing authors (the so called “repeated plots”).
Therefore, it is impossible to identify a certain WOL or author when we take into consideration just a single mental formula. However, the frequency of repetition of certain mental formulas shows us the importance of the problem raised by a WOL.
4. The sets of mental formulas (mental states) of WOLs are sufficient for their identification. The possibility to identify a certain author according to a collection of mental states of his or her WOLs is beyond our current considerations.
Scheme of the literature search system
The following problems have to be resolved to implement the literature search based on the mental state of characteres:
1) NL understanding of a query or statement (see [3] for details).
2) Domain representation in the form of semantic headers, where mental formulas are assigned to the textual representation (abstract) of a story.
3) Synthesis of all well-written mental formulas in the given vocabulary.
4) Matching of the translated NL query against the semantic headers. We use the approximate match in case of failure of the direct match.
5) Synthesis of canonical NL sentence based on mental formula.
6) Keyword search initiation in case of failure of mental formula search.
Fig.1 presents the interaction between the components of the WOL search system. The flexible system architecture allows functioning in two modes: WOL search and WOL database extension. Similar NL processing 1) and mental formula analysis 3)-5) features are employed in both modes, but the component interaction is different. See the “Domain Extension Wizard” Section for explanation.
Component 3) is required, because the traditional axioms for knowledge and belief (see, for example, [1,2] ) are insufficient to handle the totality of all realizable mental formulas. We developed the algorithm to extract the realizable mental formula from the totality of all well-written mental formulas, represented via metapredicates. In addition, introduction of the classes of equality of mental formulas are required for the approximate match of mental formulas 4), which is also inconsistent with traditional axiomatic of reasoning about knowledge and belief.
NL synthesis of mental expression is necessary for the verification of the system’s deduction. A customer needs it to verify that he/she was understood by the system correctly before this customer starts to evaluate the answer. NL synthesis in such strictly limited domain as mental expression is straightforward and does not require special considerations.
In the Fig.1 below, we describe the peculiarities of semantic processing of InPlainWords system, specific to the mental states domain. Ideally, the semantic component should be domain-independent. However, semantic rules for the analysis of mental formulas require specific machinery for complex embedded expressions and metapredicate substitutions.
Sy n t a x r e p r e s e n t a t i o n a d a p t e r :
• S u b j e c t - Re l a t i o n - Ob j e c t
• Ob j e c t - r e l a t e d - t o S u b j e c t
• Sy n o n y ms
• No r m a l f o r ms o f wo r d s ( i n f i n it i v e s f o r v e r b s , e t c . )
Ca t e g o r y o f t h e i n p u t s e n t e n c e
• Qu e r y
• Ne w f a c t
• Ne w d e f i n i t i o n
No u n a n d v e r b f o r m n o r ma l i z a t i o n
Su b s t i t u t i o n o f mu l t i wo r d s Obt a i ni n g k e y e n t i t i e s ( p r e d i c a t e s ) Pr o b l e m Do ma i n p r e - r e c o g n it i o n
Obt a i ni n g s u bj e c t s a n d o bj e c t s ( a r g ume n t s o f pr e di c a t e s ) Obj e c t c o ns t a n t s ub s t i t ut i o n i n t o f i r s t - o r de r p r e d i c a t e s Li n k i n g a n t isy mm e t r i c
p r e d i c a t e s At t e n u a t i o n o f t r a n sl a t i o n b y me a n s o f r e l e a s e o f c o n s t r a i nt s f o r a r g ume n t s
Me r g e o f t r a ns l a t i o n c o mp o ne n t s wi t h s u bs t i t u t e d a nd n o n - s u bs t i t ut e d a r g u me nt s
Su b s t i t u t i o n o f me t a p r e d i c a t e s
Pr o c e s s i n g o f lo gic a l c o n n e c t i v e s
•n o t
•o r
•a n d
•o n l y
•e x c e p t
•f o r a n y I n t e r a c t i o n mo d e
• Fir s t q u e s t i o n
• Ad d i t i o n a l q u e s t i o n
• Co mme n t
Re o r d e r i n g t e r ms i n t h e t r a n sla t i o n f o r mu l a i n a c c o r d a n c e t o s e a r c h I n s e r t i o n ma x / mi n c o n d i t i o n s
Ex t r a c t i o n o f a l l p o s s i b l e a n s we r s . Ge n e r a l i t y c on t r o l [ 3 ] Sa t i s f i a b i l i t y v e r i f i c a t i o n o f t h e c o n s t r uc t e d t r a ns l a t i o n f o r mul a
Tr i g g e r wo r d s a n d d o ma i n - s p e c i f i c s y n o my ms / mu l t i wo r d s
Pr e d i c a t e p a t t e r n s : s e ma n t ic t y p e s o f a r t g u me n t s
Se ma n t i c t y p e s f o r me t a - p r e d i c a t e s u b s t i t u t i o n
Sp e c i f i c s e ma n t ic t y p e s f o r t h e o u t p u t v a l u e s Do ma i n k no wl e d g e
NL query understanding
Domain knowledge representation
Pre-designed mental formula :- Associated text (abstract), which serves as an answer
Enumeration of all well- formed mental formulas Approximate formula
match Satisfied?
Synthesis of simplified mental formula for the control of query understanding
Keyword search Satisfied?
Answer Reasoning about mental states Formal query
representation
No Yes
No Yes
Input Query
Fig.1: The scheme of the literature search system. In the search mode, NL query, containing the mental states and action of WOL characters, is converted into mental formula. If there is no semantic header in the domain knowledge component, which satisfies the resultant translation formula for a query, the approximate match is initiated. Using the enumeration of all well-formed mental formulas, the system builds the approximation and deductive link
between the translated query and existing semantic header. If even the approximate match fails, the system generates the list of keywords to search through the WOL abstracts.
The same components are employed when a customer inputs a new WOL. It is possible to explicitly input the corresponding semantic header via NL or initiate the system to generate it independently given the WOL abstract.
The Literature Search demo page
Please demonstrate your knowledge of classical literature, from medieval to modern, asking questions about the mental states of the characteres and compare the system results with your own imagination.
How do you typically look for a book (on the Internet)? Using a search engine, you type the title of this book or the author. Sometimes, knowledge on a general topic will help. Have you ever dreamed about searching for a book, which matches your current spiritual state, mentioning the mental interactions between the persons involved in a work of literature? Try to ask the system to give you advice on a narrative with characters who have intentions, knowledge and beliefs about each other, capable of understanding, asking, pretending, deceiving, forgiving, etc.
When does a person want another person not to pretend that he does not know anything?
What happens when a person pretends that he/she wants to know something?
Why does a person pretend that he does not understand that the other person does not want something?
When would a person believe that another person does not pretend that somebody wants something?
In what circumstances would a person believe that another person pretends about some intention?
She pretends that she believes that he does not pretend that he is not a murderer of her husband?
She pretends that she believes that he pretended that he was a prince . A wife wants not to confess to husband about being not faithful .
A lover believes that the husband does not want the lover to be in love with his wife .
Table 1: Introductory page to the InPlainWords virtual mental world system with the sample questions. Note that the subjects of search are identified using the mental
want(person, not pretend(other_person,_, not know(other_person, Smth))):- outputAsAnAnswer(“
The Heaven and the Hell by P. Merimie
An inquisitor wants the girl to tell him about her lover's heretical ideas. The girl gets scared and tries to shield her lover from the Inquisition. She assures the inquisitor that he did not participate in composing the libel on the Pope.
The inquisitor gets surprised at hearing about the libel as he never mentioned it during the conversation. The girl gets even more frightened and does her best to assure the inquisitor that it was he who introduced the subject and mentioned it. She says that she has never heard of anything of that sort. The inquisitor interrupts her and tells her to stop pretending. But the girl keeps insisting on him being innocent and the inquisitor lets her go.
Later the inquisitor meets the girl once again. This time he produces her lover's love letter to another woman. Poor girl turns pale with offence and indignation and betrays her lover. She says that he is the person who composed the libel on the Pope.
“
Table 2. Formal representation for the first query in Table 1 above and the resultant answer.
The special question-answering technique for the weakly structured domains has been developed to link the formal representation of a question with the formal expression of the essential idea of an answer (Tables, 1,2). These expressions, enumerating the key mental states and actions of the WOL characters, are called the semantic headers of answers [3]. The mode of automatic annotation, when a customer introduces an abstract of a plot and the system prepares it as an answer for the other customers, takes advantage of the flexibility properties of the semantic header technique.
Based on the same technique, the kid’s version has been developed, when the question – answering is oriented to the participants of a scene (Fig.2).
Mike Peter
Nick
Fred
Fig. 2: Interacting with the virtual mental world of the scene characters. NL system answers the questions about mental states of Mike, Peter, Fred, Nick and the dog.
Does Mike see that the dog is eating the sausages?
Does Peter see what is happening with Mike and the dog?
Does Nick know what is happening with Mike and the dog?
Which way does Nick express his emotions?
Does Fred know whether Peter knows what is happening with sausages?
Does Nick want to keep the dog from eating the sausages?
What would Fred do if he wants to let Peter know what is happening?
Table 3: Examples of questions the children may ask the system about, while watching the scene. Involving more and more complex mental states helps the playing children to develop creativity and imagination of thinking, as well as the communication skills of understanding other’s mental states.
Conclusions
Interaction with the virtual world of the literature characters is demonstrated to be a novel entertainment area, appealing to the adults as well as to the kids, verbally interacting with the characters of the scenes. Since the players are suggested to both ask questions and share the literature knowledge, the system encourages the cooperation among the members of the players’ community. In the demo we built, the system will only recognize the questions and statements, involving the terms mental states and actions. This way we encourage the customer to stay within a
“pure” mental world and to increase the complexity of queries and statements we expect the system to handle properly. In terms of natural language, our domain is extremely narrow, however, we have already demonstrated commercially the power of our NL technique in such linguistically and logically complex domain as tax.
References
1. Umaschi, M. and Cassell, J. (1997) Storytelling systems: constructing the Innerface of the Interface, in Second Intl. Cognitive Technology Conf. ’97, Aizu Wakamatsu City, Japan.
2. Hayes-Roth, B., van Gent, R. Story-Making with Improvisational Puppets, in International Conference on Autonomous Agents'97(1997).
3. Fagin, R., Halpern, J.Y., Moses, Y., Vardi, M.Y. (1995) Reasoning about knowledge. MIT Press, Cambridge, MA, London, England.
4. Konolige, K.(1986). A deduction model of belief. Morgan Kaufmann Publ.
5. Galitsky, B. (2000) Technique of semantic headers: a manual for knowledge engineers" DIMACS Tech. Report #2000-29, June 1999.
6. Galitsky, B. (1998) The formal scenario and metalanguage support means to reason with it. DIMACS Tech. Report #98-27, Rutgers University.
7. Galitsky, B. (1999) Automatic generation of the multiagent story for the Internet advertisement. SSS-99 Workshop on the Intelligent Agents, Stanford CA.
8. Decker,K., Pannu, A., Sycara,K. and Williamson, M. Designing Behaviors for Information Agents , in International Conference on Autonomous Agents'97 (1997).
