MIT LIBRARIES lllllllltlllllllllllllllllllllllllllllllllllllllllll 3
9080
00728
8423
HD28
.M414
no.3144-90
/A
M
1"(
••y-in
mr^i
95
!?^AYlSt991
"ITECHNICAL
REPORT
Answer
Garden
A
Tool
for
Growing
Organizational
Memory
Mark
S.Ackerman
Thomas W.
Malone
CCSTR#108
SSWP#3144
CENTER
FOR
COORDINATION
SCIENCE
Massachusetts
Institute ofTechnology
Sloan School
ofManagement
Answer
Garden
A
Tool
for
Growing
Organizational
Memory
Mark
S.Ackennan
Thomas W.
Malone
M.I.T
"^'^MAY
15
1991
Answer
Garden:
A
Tool
for
Growing
Organizational
Memory
To
be
presented at theConference
on
OfficeInformation
Systems,
April, 1990Mark
S.Ackerman^
and Thomas
W.Malone^
CenterforCoordination Science MassachusettsInstituteofTechnology
Answer
Gardenallows organizationstodevelop databases ofcommonly
askedquestionsthatgrow
"organically" asnew
questions arise and are answered. It is designed to help insituations(such as field service organizations and customer "hot lines") where there isa continuingstream ofquestions,
many
ofwhichoccuroverandover, butsome
ofwhichthe organization has never seenbefore.The
systemincludesa branching networkof diagnostic questions that helps users find the answers they want. Ifthe answer is not present, thesystem automaticallysendsthequestiontotheappropriateexpert,andtheanswerisreturned to the user as well as inserted into the branching network. Experts can also modify this
networkinresponseto users'problems.
Our
initialAnswer Garden
database contains questionsand answersabouthow
tousetheX
Window
System.1.
Introduction
Considerthe following
common
problems inorganizations: Customersgetincomplete orinconsistentanswers to theirquestions and complaints.Employees
get stuck or waste effortwhen
theycan't find the specific mformaiion they need. Meanwhile, thepeoplewho
couldanswerthesequestionsmay, insome
cases,beunawareoftheproblemspeopleare having,or, inothercases,theymay
beoverwhelmed
by thesheermassofsimple questions theyhavetoanswer.The
Answer Garden
system helps an organization solve these problems by providing a database ofanswers tocommonly
asked questions thatgrows
"organically" asnew
questionsarise andare answered. Itis designed to help insituations where there is acontinumg stream of questions,
many
ofwhich occur over and over, butsome
ofwhich the organization hasnever seenbefore.Such
situations are pervasive inmany
kinds of organizations: field service technicians repairing a company'sproducts, telephone "hot lines" for customer problems, customer contact personnel (such as airline ticket agents) respondingtocustomercomplaints, teachingassistantshelping studentsinlarge classes,coworkershelping each other with "lore"about
how
tousecomplex
computer systems, and even primarycarephysicians treating theirpatients in medicalclinics.
In this paper,
we
willdescribea cooperativework
txx)l,called theAnswer
Garden, thatwe
have designed to helpimprove an organization's
"memory"
in situations likethese.Even
though there is litUeconsensus in the organizationalliterature about
how
to define organizationalmemory
(e.g., Orr86, Levitt88, Walsh89), the concept of organizationalmemory
is an evocativeand
attractive metaphor. Forourpurposes here,we
will define organizationalmemory
as an organization'sability tobenefitfromitspastexperience inrespondingmore
effectively(e.g., fasterormore
accurately) inthe present.
We
will beparticularly concerned with situations where thereisa body of"authoritative"knowledge
abouthow
todo things(such asusingdevices or followingorganizationalprocedures).ackerman@alhena.mit.edu or ackerman@xv.mit.edu
We
beginbydescribingihekey ideas intheAnswer
Garden system andihe inceniivespeoplewould
have forusing a system like this.Then
we
present amore
detailed "tour" ofthe system and descnbe its implementation. Finally,we
presenta fewobservations from our firstuse ofthesystem(withadatabase of questions and answersabouthow
to usetheX Window
System),andwe
discussa varietyofissuesthat willarise in large scale use ofsystems like these (such ashow
torepresent conflicting pointsofviewabout aquestion).
2.
Overview
of
the
system
Even
ifsomeone
in an organization alreadyknows
the answerto a question, unless that answer can be easily located, otherpeoplemay
have torecreate it,overand overagain.The
Answer Garden
helpsan organization 'improveitsmemory"
by captunng answers to questions thatwould
otherwise be lostand then providingtools to retrieve theanswerslater. Thereare threekey ideas inthesystem:
(1)
A
branching network ofdiagnostic questions helps users find the answers they want.The
primary tool tohelp users find answers is a branching network of diagnostic questions like those an expert might ask to
diagnose auser's problem (or like thequestionspeople playing the
Twenty
Questionsgame
havetoask). For instance, the firstquestion in a database forhardware diagnosis might be"Does
thepower
lightcome
on?"
withdifferentsubsequent questionsdepending on whethertheanswertothatquestion is"yes" or"no". Atthe
endof each path throughthebranching networkof diagnostic questionsisasetof questionspeoplehaveasked
inthatsituation andtheanswerstothesequestionsprovidedbyexperts.
(2)
New
questionsfrom
usersareautomatically routedto appropriate expertsand
then inserted(along withtheiranswers)into ihenetwork.
Whenever
userscomes
totheendof a path in thebranchingnetwork anddon't find theanswerto theirquestion(orwhen
they have troubleansweringa diagnostic question along theway), theycan enter a
new
question. This question will be sentautomatically toan expertwho
is knowledgeable about questions inthatpartofthenetwork. Then,theexpert'sanswerwillbe returneddirectlytothe userwho
askedthequestion, and it will also be entered into the network (along with the question) at the point where the questionoriginated. Thus,the
Answer
Garden growsitsinformation databaseovertime.(3) Expertscan modifythediagnosticbranching networkinresponse to users'problems.
When
expertsconclude from a particular user's question (or from usage statistics for the branching network as a whole) that thebranchingnetwork ismisleading orinefficient forusers, theycanaddto orchange thediagnostic questionsin
the branching network. This
means
that heavily used and well-understood parts of the network willpresumably have well-honed branchingnetworks, while thebranchingnetworks in lightly usedor "frontier" areaswillbe
more
adhoc.Only
a smallnumber
ofsimplecomponents
areneeded technicallyto implementthis system. Yel,sucha systemserves asanexcellent testbcd fora
number
of organizationalandsocialquestions.3.
Incentives
for
using
the
system
Why
would
peoplewantto use asystem likethis?As
Grudin (Grudin88) andothershavepointedout, thesuccess ofcooperativework
systemsmay
dependin verysubtleways
ontheincentives that differentpeople haveforusing them.3.1. Incentives for questioners
For
most
questioners, the primary alternatives to using a system likeAnswer
Garden would
probably be (a)finding the answer in
some
otherwnttcnmedium,
or(b)finding a personwho
couldanswerthequestion. Insome
cases, these alternativesmay
be preferable. For in.stance, ifthe questionerknows
exactly where to find the answer ina printed manual, thatmightbefasterandeasiertolocateandread.In other cases, questioners
may
prefertoask people theyknow,
or peoplewhose
job it is toanswer questions. Dealing with live people has the advantage thatthe expert can bemore
responsive in helpingto formulate thequestion.Forinstance, thequestionercan usually justdescribe aproblem inEnglishratherthan havingtoansweraseriesof multiple choicequestions. In
some
cases,dealing with alivepersonmay
simply provide a pleasurablesocial interaction.Inanimportant
number
ofsituations,however,we
believetheseadvantages willbe outweighed bytheadvantages of using asystem likeAnswer
Garden. WithAnswer
Garden, for instance, users can quickly findanswers to previouslyasked questionsat any time, even
when
they can't find anyhuman
experts. Moreover,since the answers in theAnswer
Garden have presumably beenprovidedbyauthoritative experts, userscanusually relyon them
much
more
confidently thanon answers froma
random
colleaguedown
thehall.Even
when
theanswertoa question isnotyetinthedatabase,systemslikeAnswer Garden
provide a veryefficientmeans
oflocatingappropriate expertssincenew
questions willbe automatically routedtothem.3.2. Incentives for experts
Forexperts.
Answer
Garden eliminatestheneedtoanswermany
simplequestions.The
expertsnolongerhavetoanswerthe
same
questionsoverand over again,and can insteadconcentrateonmore
interestingproblems. For example, oneexpertinourpilotstudy estimatedthat less than10%
of questionsaskedofhimrequired himtofindnew
information.3.3. Incentives for organizations
For an organization.
Answer
Garden
provides better, faster, easier to get answers.Thus
the efficiency, consistency,andqualityoftheanswers givento internalorexternal customersis higher than before. Italso reducestheneedto havelargeamounts ofexperts' timedevoted torepeatedlyanswering the
same
questions. Finally, as the costsof information technology decline, itbecomes
feasible tocodifyandstore electronicallymore
oftheknowledge
thatisnow
storedonly informallyinthemindsof people.
4.
A
Tour
of the
Answer Garden
The
userinvokesAnswer Garden
from theprompt lineor a suitable menu. Figure 1shows
theinitial pane, the control panel. Because the Unix environment in whichthe system operates isa multi-tasking, multi-windowed system, otherprograms(like thoseshown
in thebackground of Figure 1)canalsobesimultaneously displayed.From
thecontrol panel, theusercanchoosebetween the two majorways
oflocating answers: (1) multiple choice questions,and (2)a U'eegraph.
4.1.
Helping
users locateanswers
The
primaryway
users locate answers inAnswer Garden
isby answeringa branchingseries of multiple choice questions, startingatthe control panel.Each
question ispresented inastructuredtextnode(likethoseshown
inFigure2).When
theuser selectsoneoftheanswers by clickingon theappropriatebutton,thenextstructured textnodeappears, andtheuser continuesanswering multiplechoice branching questionsuntil shefindsthequestion and answerforwhichsheis looking.Therearea
number
of automatic placementoptionsforthenodes.In
many
cases,we
expectthis branching networkto beaseries of diagnostic questionslike thosean expert(oran expertsystem)mightask todiagnoseauser'sproblem. It isimportantthatthesequestions be phrasedin away
thatusers can understand them. For example, the questions shouldusually refertosymptoms
the usercanobserve(e.g., "Does thepower
lightcome
on?"), notlo underlyingcauses orsubtle distinctionsonly an expertcan recognize(e.g., "Doesthe diskwhine soundnormal?").
Some
users,especially experts,may
alreadyknow
roughly wheretheirquestion is answeredandwill not wantto"click" their
way down
a long branchingchain to find it. In these cases, userscan selectthe"View
tree" option in the conu-ol panelandsee atree (like theoneshown
inFigure 3)ofthenodesinthe branching network.A
usercanthenjump
immediatelytoany nodein thenetwork.
The
grapherpanemay
includelinkstoothergrapher panes;in thecurrentversion, these aremarked
by usingthe"..." conventionloimplyadditional items.4.2.
Growing
theinformation
database
The scheme
justdescribed workswellwhen
theAnswer
Garden knowledgebase containscorrectanswerstoallthe questions people might ask and a well-structured branching network to help people quickly locate the answer to theirquestion. But what happens
when
theknowledge
base ofanswers in the system is incomplete or out of date?What
ifa user doesn'tunderstand aquestion?What
iftheanswerstocommon
questionsare "buried" attheend of a long branching chain?Answer
Garden
provides two basic tools for dealing with these problems: (1)new
questions, and (2) usagefeedback.
New
questions areanswered
byexperts.Whenever
usersare notsatisfied,they can clickon the 'I'm unhappy"button inany node.
They
mightclick thisbutton, for instance, iftheanswerto their questionisn't presentat a "leaf" node, iftheydon'tunderstandthe branchingquestion theyarebeing asked, orifthey are frustrated withthepathdown
whichthe branchingnetworkistakingthem.When
usersclick the "I'm unhappy" button, they gettodescribe theirproblemorasktheirquestion directly toahuman
expertwho
is knowledgeable aboutthenodeatwhich
the problem occurred. In thecurrent version ofthe system, userscommunicate
with the experts via electronic mail; future versions could allow synchronouscommunication
with expertsforpans
ofthedatabase. Figure4shows
anexample
ofthe kind ofnode throughwhich
users can ask theirnew
questions.
These
mail messagesare mailed notonly to theappropriate experts but optionally to an additional notificationlist.
The
list of experts and notification list can bechanged on
anode-by-node
basis, allowing organizational customization. Thus,some
expertsmay
be within the boundaries of the group or organization, while othersmay
be outsideofitWhen
theexpertanswers, theanswerisautomaticallyinsertedintheinformation database.Expertsuseusage feedbacktorestructure thebranchingnetwork. Therearetwokinds offeedbacktoexpertsabout problems users arehavingwiththe
knowledge
base: (1)new
questions,and(2)usagestatistics(i.e.,which nodes andpaths are accessed most frequently).New
questions and their answers are incorporated into the network immediately, thus correctingincomplete orinaccurate things as soon asa userdiscoversthem. Notethat thisensuresthatexperts will spendtheir time on questions that are actually useful to people, noton esoteric things that no one ever wants to
know
about.New
questionscan alsogivefeedback about where the branchingnetworkis ambiguous,confusing, or frustrating.Usage
statisticsprovide another kind ofindicationabouthow
thesystem isbeingused and where the problemsare. Forinstance,ifcertain very long branching paths are traveled very often, this is an indication that the branching network should be
revisedtoplacethequestions higherin thetree.
4.3.
Answers
Informationinthe
Answer Garden
canincludesimpletext, structuredtext,pictures,andactiveprocesses.As
the database grows, mail messages can be included as simple text at first. Later, an expertcan separateand
structure the information,presenting itinastructuredtextnode.A
structured textnode caninclude proportionaltext layout,avariety of fonts styles,and active buttons. Active nodes can trigger actions at run-time such asqueryinga remote database, andthereforebuild
dynamic
information.Thus
Answer Garden
canserve asthe "front-end" foranentire setofseparate, user-definedprograms.5.
Implementation
and sample
database
The
currentimplementation forAnswer Garden
is written inC
forUnix
(BerkeleyBSD
4.3) platforms. Ituses theX Window
System extensively. SinceAnswer Garden
is tobeavailable as widelyaspossible,allwork
hasbeendonewith the Xt toolkit and the
Athena
widgets, which are publicly available(McCormack89,
Swick88).Most
of the functionality isinnew
widgetscompatiblewith theAthenaset.The Answer
Gardenusesthe standardBSD
Unixmailandfilesystems.Answer Garden
nodesarekeptasUnix Oatnodes
becomes
large,butithas thesharpadvantage ofsimplicity.Usage
ofthemailsystemis largely invisibleto theenduser;
Answer Garden
invokes the mail system directly. Site-settableand user-settable options(resources) allowAnswer
GardentobetailoredtothevariousUni.x mailhandlers.
We
are testing theinitialAnswer
Garden system usingan information databaseon theX
Window
System itself.Since
X
isanewly developed andwidely usedstandard,many
organizationsdonothavetherange ofexpertiseandsupport necessary forlarge-scaledevelopmentefforts. Based onour discussions with potential users,we
believethisdatabaseaboutX
willbequite valuabletoa verylargenumber
of peopleatmany
sites.We
are gathering questions and answers from the most heavily usedX
mailing lists and interviews with staffmembers
of theX
Consortium (the organization that develops and maintains theX Window
System standards andsoftware).
We
have askedtheX
Consortium staff toprovide the most popular questions thatthey answercontinuously,andtheyhaveagreedtoanswerthe
new
questionsthatareaskedusingthesystem.As
thenumber
of usersgrows andthey asknew
questions, theX
databasewillgrow. Thus,we
are starting witha coreofquestionsandanswers,andexpectingthe userstoexpandour coverage.We
envisionthesystemashavingthree levelsof information: personal,group, andgeneral.The
groupmay
bea particular site in an inter-organizational system.Most
of the information willcome
from themembers
of theX
Consortium, and will be generally true for all
X
users. For example,X
Consortium staffmay
add nodes aboutnew
applications thatrunonspecifichardwareplatforms,andallsiteswill getthis information.
Group
orsiteinformationmay
includeinformationtrueonlyata givenlocation,suchas
who
thesiteadministratorsforX
are.6.
Usage
experience
and
issues
Inthefirstphaseofthis study, the
Answer
Garden systemhasbeen usedirregularlyby about8peoplein4 groupsat
MIT
fora period of 4months. In thenextphase ofthestudywe
will includeseveralgroupsinexternal organizationsas well.Our
earlyexperience hasakeadyhighlightedseveral issues thatwe
expecttobeimportant astheusageandsizeofthe databaseincreases.Authenticationofinformation.
One
oftheproblemsofbulletin board systemsisthatitisdifficult todetermine-letalone control- thequalityofthe information.
Answer
Gardenprovidessome
technicalmechanisms
foraccesscontrol, butmuch
ofthecontrol will besocial. In thesampleX
database, forexample,only local expertsandX
Consortium staffmembers
can changethe information database.The
X
Consortium staffmembers
are,de facto,theultimateexperts,andthereforeauthenticationofinformationisnotanissue.
However,
this lockingmechanism
can be bypassed eitherat theAnswer Garden
level or at theUnix
file level.Local siteadministrators could,of course,adulterate thedatabase. Moreover,
we
have nocontrolover site-specificnodes; these willbe underthe controloflocalexperts.We
surmisethataneditingfunction,perhapswithineachsite,may
become
necessarytocontrolthe qualityoftheinformationandmaintainthedatabase.
A
futureenhancement
almost universally requested is toallow users toannotate the entries.We
also think itusefultoallowusers to
fwm
theirown
accesstreesforquickerretrievalof frequentlyusedinformation.Scaling.
We
expect theX
information databaseto ultimatelygrow
tothousands of nodes.As
thenumber
ofnodes grows,ease of accessforusers will
become more
critical. Currently, only thebranchingquestions andtree graphsexist; however.
Answer
Garden
has been designed to allow a multiplicity ofretrieval engines. For example,we
are currently investigatingsemi-structuredretrievalsanddynamic
trees toprovidegreaterflexibilityacross ranges ofexpertise,machinecharacteristics,andtechnical interestsofusers.
Inaddition, potentiallythousands ofusers, software engineersandothers, willbe usingthis database. Theseusers are spread across diverse organizationsandgeographicareas,andaccessto a centralizedsiteis notpractical.
Many
ofthe information subtreeswillbeorganizationand sitespecific. For example, the availability andlocation ofX
binary files isoccur continuously,each dayor
more
often.To
do this willrequire a steady flow of mail traffic to eachsite,bearing thechangestothe informationdatabases.
Representing multiple points ofview. In the scenario
shown
in the figures, anexpen
has arranged the branching questionsin suchaway
thattheanswersarespecific andsmall. Inothercases,a topicmay
be too recent or general forthe experttodeterminehow
the informauon database should be structured.Or
theremay
bediffering opmions, evenamong
experts, on the correct answer. In thiscase, a "moderator" might help organize thebranching network, but not provide authoritativeanswers.For example, a user might want to
know
about the advantages ofusingX
terminalsover workstations. Afteranswenng
theappropriate questions -whether theproblem isa question ofX
usage,of platforms,andofhardware-he canbrowsethrough alarge
amount
of mail trafficon the various types of terminals andtheir vendors.From
time totime, themoderator might select the
most
interesting messages or modify the branching network to reflect the differences inicrmmals andvendors.
7.
Related
Work
Answer Garden
has points ofcommonality
with anumber
of previous systems. For example, videotextsystemsprovidea structuredbranchingnetworkto helpusers findinformation.
They
donotusually,however,aUow
users toeasily asknew
questionsandhave theanswersautomaticallyaddedintothedatabase via electronic mail.Electronic mailing lists, computerconferences, and bulletin boards allow ad hoc sharing of information
among
widely distributed groups in response to
new
questions, but they usually provide only the most rudimentary kinds of structure. For instance, it iscommon
forcomputer
conferences (e.g., HiltzSl) to include a few levels of topics andsubtopics,but
we
arenotawareofany system that treats thisstructure asa diagnosticbranchingnetwork.At theoppositeextremeofstructure, expertsystems often provide elaborate diagnostic branching networks, but they usually require significant efforttoconstructandthey arenotordinarily thought ofasbeing automatically extensible
inresponseto
new
userquestionsand answersfromexpertsviaelectronic mail.The
kindsof questionsand answers usedinAnswer Garden
couldcertainly be providedina multi-user hypertextsystem (Conklin87,Aksyn87,Trigg88). In fact,one can thinkof
Answer Garden
asanexample
of a multi-user hypertext system.We
are not aware, however, ofany other hypertext applications that exploit this kindof diagnostic branching networkandareautomatically extensible throughelectronicmail.Perhaps theclosestprecursors to
Answer Garden
aresystems designed explicitly toaid in providing immediate answers from technical support personnel to questionsabout hardware and software (Coppeto89, Dyson89).With
the exceptionofOLC,
thesesystemsappeartobe designedforuse onlybyskilled technical intermediaries, notbytheend userswho
actually have thequestions. Furthermore, thesesystemsdo not appear toincludeadiagnostic branching structure tohelpusers localeanswers.
8.
Conclusion
The Answer Garden
isnot aradicallynew
kind of system.We
believeitshows, however,how
a relativelysimplecombination of
well-known
concepts can provide a surprisingly powerful platform foranew
kind of cooperativework
application.
As
information technologybecomes
more
pervasive,we
believe that tools like this-for capturing andexploiting organizational
knowledge
thatwas
previously storedonly informally or not at all-willbecome
increasingly important.Acknowedgements
This
work
hasbeen funded inpartbyresearch grantsfrom theX
Consortium, DigitalEquipment
Corporation,andthe National Science Foundation.
Wc
would
like to thankStuartMadnick,Bob
Goldberg, Brian Pentland, Ralph Swick,References
1. Akscyn, Robert;
McCracken,
Donald; and Elise Yoder."KMS:
A
DistributedHypermedia
System forManaging
Knowledge
inOrganizations." Proceedingsof Hypertext '87,pp. 1-20.2 Conklin,Jeff. "Hypenext:
An
IntroductionandSurvey." Cowpurer, September 1987,pp. 17-41.3. Coppeto,
Thomas
J.; Anderson, Beth L.; Geer, Daniel E., Jr.; and G. Winfield Treese."OLC:
An
On-Line
Consulting System for
UNIX."
Proceedings oftheUsenixSummer
1989 Conference, 1989,pp. 83-94. 4. Dyson,Esther. "Computers,Customers and Hand-Holding." Fc>r/?e5,August 7, 1989,p. 128.5. Grudin, Jonathan.
"Why
CSCW
Applications Fail: Problems in the Design and Evaluation ofOrganizational Interfaces." Proceedings oftheConferenceon Computer-Supported Cooperative Work, 1988,pp.85-93.6. Hiltz, Starr
Roxanne,
andMurray
Turoff. "The Evolution of User Behavior in aComputerized
Conferencing System."Communications
oftheACM,
November,
1981, pp.739-762.7. Levitt,Barbara,and
James
G. March. "Organizational Learning."Annual
Review
ofSociology, 1988,pp.319-40.8.
McCormack,
Joel; Asente, Paul; and Ralph R. Swick.X
Toolkit Intrinsics -C
Language
Interface. Cambridge,MA:
Massachusetts InstituteofTechnology, 1989.9. Orr, Julian E. "Narratives at
Work:
Story TellingAs
Cooperative Diagnostic Activity." Proceedings of theConference on
Computer
Supported Cooperative Work, 1986,pp.62-72.10. Swick,RalphR.,and
Mark
S.Ackerman. "TheX
Toolkit:More
BricksforBuilding UserInterfaces,orWidgetsforHire." Proceedings ofthe Usenix Winter1988Conference, 1988,pp. 221-228.
11. Trigg, Randall H. "Guided Tours and Tabletops: Tools for
Communicating
in a Hypertext Environment." Proceedings oftheConference on Computer-Supported Cooperative Work,1988,pp.216-226.12. Walsh,