D&i^
{
JUL
26
1990 ^
-T-i-^
WORKING
PAPER
ALFRED
P.SLOAN
SCHOOL
OF
MANAGEMENT
COVERSTORY
-AUTOMATED NEWS
FINDING
INMARKETING
John D. SchmitzInformation Resources, 200 Fifth Avenue, Waltham,
MA
02154Gordon
D. ArmstrongOcean
Spray Cranberries, Lakeville-Middleboro,MA
02349John
D.C
Utile M.I.T., Cambridge,MA
02139MASSACHUSETTS
INSTITUTE
OF
TECHNOLOGY
50
MEMORIAL
DRIVE
CAMBRIDGE,
MASSACHUSETTS
02139
COVERSTORY
-AUTOMATED NEWS
FINDING
INMARKETING
John D. SchmitzInformation Resources, 200 Fifth Avenue, Waltham,
MA
02154Gordon
D. ArmstrongOcean
Spray Cranberries, Lakeville-Middleboro,MA
02349 John D.C
Little M.I.T., Cambridge,MA
02139May
1990WP#:
3176-90-MS
MCWP//:90-5
Paper presented atDSS-90
The
Tenth International Conference of Decision Support SystemsCambridge, Massachusetts
May
21-23, 1990Sponsored by the College on Information Systems
The
Institute ofManagement
SciencesAbstract
Ocean
Spray Cranberries, a $1 billion fruit processing cooperative, tracks sales and assesses the effectiveness ofits marketingprograms
with large databases collectedthrough bar-code scanners insupermarkets.
The
level ofdetail of these databases is sogreat that it can easily obscure importantfacts and trends in the market.
Ocean
Spray, likemany
grocery companies, has a very lean staffforanalyzingthis dataand so
saw
a need to automate asmuch
ofthe analysis as possible.Working
withOcean
Spray, Information Resourceshas developed an expert system, CoverStory,to uncoverthe important facts and trends in a scanner database. Using a series ofsub-analyses, CoverStory
reveals brands and markets which are currently
newsworthy
and identifies marketing factors thatmay
be causing changes in these brands and markets. CoverStory results are published in anattractive
memorandum
format.The
whole
system is organized verymuch
inthespirit of decision support.The
user can adjust weights and importance criteria and can edit the finalmemo.
CoverStory is part ofan overallDSS
atOcean
Spray.The
scope and comprehensiveness of thesystem have
made
it possiblefor a single marketing professional tomanage
the process of alerting allOcean
Spray marketing and sales managers to key problems and opportunities and providingthem
with daily problem-solving information.1. Introduction
Machine-readable bar codes
on
products in supermarkets have changed forevertheway
thepacicaged-goods industrytracics itssalesand understands
how
its markets work. Althoughthe codeswere
originally introduced and justified to save labor at check-out, the spin-off data produced bythem
providesmarvelous opportunities forretailers and manufacturers to measure the effectivenessof their marketing programs and create greater efficiencies in their merchandising and promotion.
We
shall describehow
one manufacturer.Ocean
Spray Cranberries, Inc., has responded to these opportunities with an innovative decision support system designed to serve marketing and salesmanagement.
Ocean
Spray.Ocean
Spray Cranberries, Inc. is agrower-owned
agricultural cooperativeheadquartered in Lakeville-Middleboro, Massachusetts with about nine hundred
members.
It produces and distributes a line of high quality juices and juice drinks with heavy emphasison
cranberry drinks butalsowith strong lines in grapefruitand tropical drinks.The
company
alsohasa significant business in cranberry sauces and fresh cranberries.
About
80%
ofOcean
Sprayproducts sellthrough supermarkets and otherretail storeswith lesser
amounts
flowingthrough foodservice and ingredient product channels.
Ocean
Spray is a Fortune500
company
with salesapproaching $ 1 billionper year.
Until the
mid
1980's,Ocean
Spray, like mostgrocery manufacturers, tracked the sales andshare of its products with syndicated warehouse withdrawal and retail store data provided
by
companies suchas
SAMI
and A.C.Nielsen. This datasupplemented thecompanies'own
shipmentsdata
by
providing informationon
competitive products and the total market.Such
databases haveformed
thecornerstones ofuseful andeffectivedecisionsupportsystems inmany
companies(Little,1979;
McCaim,
1988). Forsome
time, however, it has been apparent that a radicallynew
generation is
on
the way.By
themid
eighties, the penetration of scanners in supermarkets hadreached a level such thatdatasuppliers could put together valid national samples ofscanning stores
and provide
much
more
detailed and comprehensive sales tracking services than previously. In1987,
Ocean
Spraycontractedfor InfoScandataforthejuice categoryfirom Information Resources,Inc. (IRI) of Chicago.
InfoScan. IRI's InfoScan is a national and local market tracking service for the
consumer
packaged-goods industry. InfoScan follows
consumer
purchases of products at the individual itemlevel as identified bythe industry'sUniversal Product
Code
(UPC). IRI buys datafrom
a nationallyrepresentative sample ofover
2500
scanner-equipped stores covering major metropolitan markets andmany
smaller cities.These
provide basic volume, market share, distribution and price information.Added
to thisaremeasures ofmerchandisingand promotioncollected inthe storesandmarkets.
These
includeretaileradvertisinginnewspapers andflyers, in-storedisplays, and coupons.Most
ofthe measures contain several levels of coding; for example, newspaper ads are coded A,B, or C, accordingto theirprominence. In addition, the InfoScan serviceprovides accessto IRI's individual household purchase data collected
from
approximately 70,000 households across 27 market areas.Data Explosion.
The
amount
of data is almostoverwhelming. IRI adds about 2 gigabytes perweek
to its master database in Chicago.Compared
to the oldtrackingdata, acompany
buyingthe InfoScan service receives increased detail by a factor of 4 to
6
because of dealing withindividual
weeks
insteadofmulti-week totals, 3 to 5 becauseofUPC's
instead ofaggregate brands, 4 to5 becauseof50 individual markets insteadofbroad geographicregions, 2 to 3 because ofmore
tracking measures, and 1 to 3 because of breakouts to individual chains within a market. Multiplying out the factors reveals that 100 to 1000 times asmuch
data is being handled aspreviously.
Most
packaged-goods manufacturers did not initially understand the implications oftwo
to threeorders of magnitudemore
data.And,
in fact, this kind ofchange is difficult to comprehend. In terms ofamanagement
report, itmeans
that, ifa report tookan hour to look through before, thecorrespondingreport with all the possible
new
breakoutswould
take 100 hours to look through. Inother words, the
new
detail will not be looked at.The
remarkable advancesthat havetaken place incomputing have helped concealthis issue.Today's technology certainly
makes
it feasible to store and retrieve all thenew
data and, althoughthe hardware and software to
do
this are not cheap, they represent a small fraction of the sales dollars involved, so that, ifusing the data can lead tomore
effective marketing, a full scaleDSS
with on-line access to the database is certainly warranted. Indeed, it
was
clear in advance and evenmore
clear after the fact that the detailed data containmuch
information of competitive value in running the businesses.DSS
strategy. Packaged goods companies today have lean staffs.Many
have beenrestructured and lostpeople. This is in theface ofthe
huge
data increasesjust described. AlthoughOcean
Spray has not been restructured, its roots as an agricultural cooperative have always givenit an internal culture oflean self-sufficiency. It has a small IS department for the organization as a whole.
Thissituation led
Ocean
Spraynaturally to a strategyofa small marketingDSS
organizationrunning a decentralized system
where
the usersdo
most of theirown
retrieval and analysis.The
marketing
DSS
forsyndicated data currentlyconsistsofone marketingprofessional plusthedatabase administrators.The
goal is to have a largely centralized database with workstations for sales and marketing inthebusinessunits. User-interfacesmust beeasily masteredby
busy peoplewhose main
jobs are in the functional areas.The
role oftheDSS
organization is to acquire and develop toolswith whichthe end-userscando their
own
analyses.DSS
consultswith userstodevelopappropriatepre-programmed
reports to be delivered as hard copy and/or on-line.An
important characteristic of the system must be growth potential.Not
only shouldretrieval of specific numbers, tables, and graphs be easy
now,
but the system architecture and computingpower
should be there for future calculations and analyses that are likely to bemuch
more
computationally intensivethan simple retrieval.Ocean
Spray's InfoScan database.Ocean
Spray's syndicated database for juices isimpressive, almost imposing, consideringthe change
from
the pastand the level ofhuman
resources put against it. It contains about400
millionnumbers
covering up to 100 data measures, 10.000products, 125 weeks, and 50 geographic markets. It
grows
by 10 millionnew
numbers
every fourweeks. Finding the important
news amid
this detail and getting it to the right people in a timely fashion is a big task for a department of one.processing
on
anIBM
9370
mainframe with ten gigabytes of disk storage and puts user-interface taskson
eleven 386-level workstations located in the marketing and sales areas.The
basicDSS
software is IRI's DataServer, which
manages
data and mainframe computation in the fourthgeneration language
EXPRESS
and the user interface inpcEXPRESS.
This providesmenu
driven access to a family offlexible,pre-programmed
reports availableon
the workstations.Unlike
some
other solutions used by packaged-goods manufacturers, this architectureprovides easy access to
mainframe
computingpower from
the workstations as is needed, forexample, to run applications likethe CoverStory software to bediscussed.
2. Basic Retrieval
and
ReportingThe
basic retrieval, reportingandanalytic capabilities ofOcean
Spray'sDSS
are extensive.Any
particular factfrom
the database can be pulled out in a few steps with the help of pull-downmenus
and picklists.Much
ofthe usecomes from
standard reports: acompany
top line report, andfour business area reports (cranberry drinks, grapefruit, aseptic packages, and tropical drinks)
showing
status and trends including changes in share in aggregate and in detail, and changes in merchandising and distribution against a year ago or fourweeks
ago. Derived measures such asBDI
(brand development indices) andCDI
(category development indices) are available. Productmanagers
canget a quick update ofwhat
is goingon
with theirproducts. Standardized graphs can be called up and it is relatively easy to constructnew
ones. Similarly, users can readily constructmeasures thatareratios,differences, andothercombinationsofones alreadyinthedatabase.
Usage
has beengrowing
steadily since DataServer and the InfoScandatabasewere
been installed.Nevertheless, the introduction of the system has required as
much
learning for theDSS
department as theendusers.
Some
people, especiallyinsales,made
littleuseofthe system. Within marketing a few individualstook to thesystem quickly and did considerable analysisbut dierewas
also afeeling that
you would
notwant
to have a reputation for spending toomuch
time pushingnumbers
around. In fact, withinsales, the characteristic attitudehas been: "Using the computer isnot
my
job.Give
me
something that is already analyzed.Give
me
materials that are ready to useand will help
me
do
my
job."In responseto thisthe
DSS
department has developed (and continuesto develop) tools andanalysesthat willhelp solvespecificuserproblems. Thereare a
number
ofapproaches; CoverStoryto be discussed
below
isone
ofthe key directions. In additiona variety ofreportsoriented aboutselected issueshave been developed. For example, reports that rank products and point out
Ocean
Spray strengths and identify markets
where
some
Ocean
Spray product is underdistributed relative to its inherent selling power.The
intention is to let sales and marketing people identify marketopportunities and productselling points.
3. Finding the
News: CoverStory
CoverStory is an expert system developed by IRI to tackle the
problem
of toomuch
data;Ocean
Spray has been a development partner and firstclient. CoverStory automates the creationof
summary memoranda
for reports extractedfrom
large scanner databases.The
goal istoprovidereflected in the database - especially in its newest numbers.
The
project began as a teaching exercise in marketing science -"How
would you summarize what
is important in this data?"(Stoyiannidis, 1987; Little, 1988) - and has developed into a practical tool.
CoverStoryis undergoingcontinuingdevelopment as
we
gainexperience withits use innew
situations.
We
describe the following aspects of the system as itnow now
being used: (1) the roleof marketing models, (2) the basic decomposition steps
embodied
in the search strategy, (3) the linearization and ranking processes used to decide what facts aremost
worth mentioning, and (4) methods for generating and publishing the output.Marketing
Models
. CoverStory is rooted in the modeling tradition.Howevr, by
design,it does not directly present
model
results at this stage of development, but rather reports onlydatabase facts, such as share, volume, price, distribution, and measures of merchandising.
The
reason is to have the output and underlying processes as transparent and easy to understand as possible.The
program
assesses the relative importance of these facts, and selectsthem
forpresentation by using weights and thresholds
which
come
from
marketing models.However,
theuser is able to inspectand change these values.
Futhermore, inchoosingmeasures of marketing effortforCoverStorytoconsider,
we
select a set of marketing variablesfrom
the scanner database that model-building experience hasshown
to be important for driving sales and share. Measures
commonly
used include:Displays - percentof stores (weighted
by
size) that displayed a brand or item.Features - percentof stores (weighted
by
size) that ran a feature adon
a brand.Distribution - size-weighted percent ofstores that sold a brand.
Price cuts - percentofstores thatsold a brand at aprice reduced
by
more
than a threshold,such as
10%, from
the regular price.Price - Price can be represented by
many
data measures. CoverStory sometimes uses the overall average price paid at the registerbut oftendraws
from
a finer set ofpricemeasures that
may
include regular price, average merchandised price, and depth ofdiscount.
The
regular price is the price of an item not undergoing specialpromotion; average merchandised price isthe priceofan item in stores
where
it isbeing
promoted
with feature ads or displays.Depth
of discount is the differencebetween these two. In an InfoScan database,
we
can get even finer measures of price by breaking out average merchandised price and depth of price discount bytype of merchandising.
Through
a marketing model,we
quantifythe impact of each ofthese marketing leverson
share oron
salesvolume
and find their relative importance. For grocery items,among
the measuresdescribed above,
we
usually(but notalways) find that distributionismost
important then price thendisplays then features then price cuts.
Flow
of Analysis and Decomposition. Figure 1shows
the general flow of analysis inaggregate market
by
a series of decompositions or disaggregations.An
aggregate product is aproductwhich includes
more
thanoneUPC.
The
UPC
(barcodeon
thepackage) isthelowestlevelof product detail available in a scanner database.
An
example of an aggregate product isOcean
Spray Cranberry Juice and Blends. It consists ofmany
different sizes, package types, and flavorsand blends.
An
example
of an aggregate market is the total U.S.which
can be disaggregated intoregions or individual cities or even grocery chains within markets.
Indoingdecomposition,CoverStoryfollowsa stylewhich
we
have observedinthe analyticalmarketing reports used in
many
companies. Analysis proceeds by answering the following seriesofquestions. (1)
What
is goingon
overall in the aggregate product for the aggregate market? (2)What
changes does this reflect inthe components ofthe aggregate product? (3)What
changes does this reflect in the components of the aggregate market? (4)What
is happening to competitive products? In CoverStory,we
go
through each of these in turn. Within each of these sections ofthe analysis, the
program
follows a standard series ofsteps:Rank
the components (markets or products ormarket/product combinations) by
some
criteria.Select the
most
noteworthy for mention and for further analysis.Calculate causal factor changes for these top
few
markets, products, or combinations. Causal factorchanges are distribution, price, and merchandising
changes.
Rank
these causal factorchangesthen selectthetopfew causal changes to include in the report.
The
need to select "the top few" itemsfrom
different lists is dictated by the size of the scanner database.The
number
of events that can be mentioned is enormous. Without strictlylimiting theamount
of information in a report,we
found thatthenews
drowned
in the detail.Ranking
theproducts ormarkets.We
nearlyalways rankcomponent
products orcomponent
markets by share or
volume
change.When
we
are looking at size groups within an aggregate product, for example, andwe
are analyzing sharechanges, sizegroup ranks will be basedon
share changes.We
have found this to be generally effective with one exception. Ifthere has been afundamental restructuringofthe
way
acategory ismarketed, share changesmay
notbemeaningful.This
showed up
when
we
looked at coffee sales. In late 1987 and early 1988, coffee packaging switchedfrom
packages thatwere
multiples of apound
(1, 2, and 3pound
cans) to packages thatwere
multiplesof13 ounces(13, 26, and 39 ounces). Thisgave the appearance that a largeamount
of
volume
was
switchinginto"new
products" and, for a year,volume
and sharechangecalculationsrequired special treatment.
Selecting the top few products or markets.
The
top few are the few that are the mostnoteworthy.
We
generally calculate whichcomponent
products or markets are furthestaway from
average and retain these extremes for mention. Normally, this leads CoverStory to pick winners andlosers. In
some
cases, however,when
mostoftheproducts and markets are behaving insimilarchange depending
on
the factor. Factor weightis different for each ofthe marketing factors suchas distribution, price, displays, featuring, and price cuts.
These
factor weights are intended, informally speaking, tomake
different marketing changes have thesame
score iftheir impacton
sales is the same.
We
initialize factor weights basedon
analysisdone
outsideofCoverStory basedon
logitmodels ofthe type described inGuadagni
and Little(1983). Market weightis aterm whichmakes
itmore
likely that an event in a large market will be mentioned than an event in a small market.We
originallyused market sizebut found thatthiswas
too strong.Only
events fromNew
York, Chicago, and
Los
Angeleswould
be mentioned andsowe
havesoftenedthe impact ofmarket size.One
approach thathas proveneffective is touse the square rootofmarket size asthe marketweight.
Inall, thisscoring
method
yields arankedlistofcausal market changeswhere
suchachangecan be described in terms of:
What
happened? (e.g. pricewent up
by20%)
Where
did it happen? (e.g. inthe Southeastern Region)What
productdid it happen to? (e.g. the 32 ouncebottle)The
events that CoverStory describes are the ones tiiat rank highest usingthis scoring mechanism.Presenting the Results.
We
have experimented with severalmethods
for presenting theseresults.
Our
present style is to produce an English language report in distribution-qualityformat. This has been an important piece of the overall effort and has had a dramatic effecton
the acceptability of CoverStory reports to end users.The
language generation is usually straightforward; it is basedon
sentence templates (Barr, 1981).We
have considered but not yetimplemented context and
memory
(Schank and Riesbeck, 1981) inour text generation.The
useofsome
randomizationofdetailedwording
throughtheuse ofa thesauruskeeps theCoverStorymemo
from
soundingtoo mechanical.The
memo
is relatively shortand structured sothissimplelanguagegenerationhas notbeen a limitation
on
CoverStory.The
CoverStory results arepublished through ahigh-qualitydesktop publishingpackageoraword-processorwith desktop publishingcapabilities. Variationintype-face, use of graphic boxes,
and sidebars are all intendedtogivethe
memo
visual appeal andhighlight themarketing factswhich
are contained in it.
CoverStory is very
much
a decisionsupport system rather than a decisionmaking
system.The
user can adjust all major system parameters such aswho
competes withwhom,
what
weightsto use for themarketing factors, and
how
much
information is to be reported.The
finalmemo
ispublishedthrougha standard
word
processingpackage so itcanbe edited by the user, althoughthis seldom happens. Becausethememo
isautomated and easilysetup (and then leftalone)to meet theneeds of specific managers, the appropriate "news" can quickly be distributed throughout the
organizations
when new
data arrives.A
CoverStorymemorandum
attached as Figure 2 illustrates the output. In this coded example,we
presenthighlightsaboutabrand called SizzleintheTotal United States.The
recipient for thismemorandum
is the Sizzle BrandManager
and the brandmanagement
team.The
series of decompositions in this report is:To:
SizzleBrand
Manager
From:
CoverStory
Date:
07/05/89
Subject: Sizzle
Brand
Summary
forTwelve
Weeks
Ending
May
21,1989
Sizzle's share of rype in
Total
United
States
was
71.3 in theCScB
Juice/Drink
category
for thetwelve
weelcsending
5/21/89.
This
isan
increase of 1.2 pointsfrom
a
year
earlierbut
down
.5from
lastpenod.
This
reflectsvolume
sales of 10.6million
192oz
eqmv.
Category
volume
(currently 99.9 million
l92oz
equiv)declined
1.3% from
a year
earlier.Display
aaivityand
unsupported
pricecuts rose
over
the pastyear
-unsupponed
price cuts
from 38
points to 46.Featunng
and
priceremained
atabout
thesame
level asa year
earlier.Components
of SizzleShare
.•\mong
components
of Sizzle, thepnncipal gainer
is:Sizzle
64oz:
up
2.2 pointsfrom
lastyear
to 23.7and
losers:Sizzle
48oz
-0.6 to34.9
Sizzle
32oz
-0.1 to 6.7Sizzle's
share of type
is71
J
-up
12
from
the
same
period
last year.Sizzle
64oz's
share
oftype
increase
ispartly
due
to 11.3 pts rise in%
ACV
with
Display
vs yr ago.3h&r9
aad
M«
r chand
1 9 i ::g //yyyyy,
—
7a1 amtSbtr
•ZShtr*
at M«r 9hiad
i a 1Q(Competitor
Summary
Among
Sizzle'smajor
competitors,
thepnncipal
gainers are:Shakey:
up
2.5 pointsfrom
lastyear
to 2.6Private
Label
+.5
to 19.9 (butdown
.3 since lastperiod)
and
loserGeneric
Seltzer -.7 to 3.5Shakeys
share
of typeinaease
isBreaking
down
total Sizzlevolume
into sales by size groups.Looking
at Sizzle's major competitors.Looking
at submarkets oftheUS
- cities in thisdatabase.Looking
at competitiveactivity in these submarkets.The
analysis isbasedon
share change.A
sampleof a causal changeshown
by
CoverStory is theincreaseofdisplay activity to support
64oz
bottles of Sizzle.4. Benefits
Ocean
Spray'sDSS
designstrategyhas successfully solvedseveral problems.The
decisiontoput users inchargeoftheir
own
basic retrieval andanalysishas generallyworked
well and,where
ithas runinto problems, the
DSS
organizationhas respondedby
providing increasinglycustomized tools.The
DataServer interfacehas beeneasyto learn.Usage
on
the386-level workstationslocatedin the marketing area is
many
hours perweek
and rising.The
strategy casts theDSS
organization inthe roleof acquiring and buildingtools tomake
the usersmore
effective. Consultation with users has led to a set of hard copy reports that are circulated regularly to marketing, sales and topmanagement
and to customized reports that canbecalled up
on
line and printed locallyon
laser printers, ifneeded.CoverStory is a particularly desirable development because, with very little effort, it
provides users with top linesummaries and analyses across a widevarietyofsituations. Previously
this required time-consuming intervention by a skilled analyst. Furthermore the technology is an
extensible platform
on
which to build increasingly sophisticateddecentralized analysis for theusercommunity.
The
informationcoming
outofOcean
Spray's marketingDSS
isused everyday inplanning,fire-fighting, and updating people's mental models of
what
is goingon
in thecompany's
markets. Typical applications include such actions as taking a price increase and monitoring its effect;discovering sales softness in a particularmarket, diagnosingits causes, and applying remedies; and
following a
new
product introductionto alert the salesdepartment in case ofweak
results in certainmarkets
compared
to others.The
DSS
is totally integrated into business operations and it is nolonger seems possible to consider life without it.
Perhaps the easiest
way
to express the success of the system is that, with the help ofmarketing science and expert systems technology, the
DSS
hasmade
it possible for a singlemarketing professional to
manage
the process of alerting allOcean
Spray marketing and salesmanagers to key problems and opportunities and of providing
them
with daily problem-solvinginformationand guidance. This isbeing
done
across four businessunits handling scores ofcompany
products indozens of markets representing hundreds of millions ofdollars of sales.REFERENCES
Barr,
Avron
&
Edward
A.Feigenbaum
eds.,Handbook
ofArtificial Intelligence.Los
Altos,CA:
William
Kaufman
Inc. 1981.Guadagni, Peter
M.
&
JohnD.C.
Little, "A
LogitModel
ofBrand
Choice Calibratedon
ScannerData," Marketing Science. 2
(Summer
1983), 203-38.Little,John
D.C,
"Decision SupportSystems for Marketing Managers," Journal of Marketing. 43(Summer
1979) 9-26.Little, John
D.C,
"CoverStory:An
ExpertSystem
to Find theNews
in Scanner Data," internalworking
paper, Sloan School ofManagement,
M.I.T. (September 1988).McCann,
John
M.,The
MarketingWorkbench
.Dow
Jones-Irwin,Homewood,
IL, 1986.Schank,
Roger
C
&
Christopher K. Riesbeck eds.. InsideComputer
Understanding. Hillsdale, NJ:Lawrence Erlbaum
Assoc. 1981.Stoyiannidis,Demosthenes,
"A
Marketing Research Expert System," Sloan School Master'sTTiesis,M.I.T.,
Cambridge
MA
02139, June 1987.10
Date
Due
SEP^^
\%^\
NOV
IS
1991
MIT LIBRARIES