Incremental Response
MEASURING THE INCREMENTAL RESPONSE
A traditional approach to accessing the incremental response is using a differencing technique from two predictive models. Take the likelihood to purchase from the predictive model built for the treatment group:
P yt( =1| )x Figure 7.1 Control versus Treatment
Control Treatment Purchase = Yes
Purchase = No
Purchase = No Purchase = Yes Incremental
Response
144 ▸ B I G D A T A , D A T A M I N I N G , A N D M A C H I N E L E A R N I N G
The likelihood to purchase for the control group is similar:
P yc( =1 x| )
The incremental response likelihood can be calculated as the difference PD=P yt( =1| )x P y− t( =1| )x
Then sort the resulting
P
Dfrom largest to smallest, and the top deciles are the incremental responders. Any predictive model can be employed in the differencing technique, such as the regression‐based differencing model and the tree‐based differencing model.An improved method is to look only at the control group, the peo-ple who did not get the coupon, and classify each person as an outlier or not. Several techniques can be used for classifying outliers when you have only one group. A method that has good results classifying outliers is one‐class support vector machines (SVMs).
Recently a new method was suggested that uses an outlier detection technique, particularly the one‐class SVM. The suggested method uses the control group data to train the model and uses the treatment group as a validation set. The detected outliers are considered as incremental responses. This new method shows much better results than the dif-ferencing technique. The technique is illustrated with plots below, but more details can be found in the paper by Lee listed in the references.
In Figure 7.2 , we see a graphical representation of the points from the control group that have been identifi ed as outliers. The dots closer to the origin than the dashed line are classifi ed as part of the negative class, and other dots up and to the right of the dashed line are classi-fi ed to the positive class. The points in the negative class are considered outliers (those between the origin and the dashed line). They receive this designation because there are particular features, or a combination of many features, that identify them as different from the overall group.
Figure 7.2 Outlier Identifi cation of Control Group Origin
I N C R E M E N T A L R E S P O N S E M O D E L I N G ◂ 145
One reason to identify some of the points in the control group as outliers is to narrow the region that the control group identifi es so that we can better determine which observations are incremental responders when we apply the model to the treatment group. To apply this to our example, we would look at the people who purchased detergent ABC and then, using a one‐class SVM model, identify some of those as outliers. This would leave us a set of ranges for each attribute of our customers that we can use to identify them as people who would purchase detergent ABC without a coupon, as shown in Figure 7.3 .
The region of the responders for the control group as shown by the larger circle in Figure 7.3 this region can then be projected to the treatment group. Applying this region to the treatment group will identify the incremental responders, those individuals who purchased detergent ABC because of the coupon. This is illustrated graphically in Figure 7.4 . This is only a representation in two dimen-sions. In practice, this would be in dozens, hundreds, or even thou-sands of dimensions.
Figure 7.4 shows the projection of the control group, those who bought detergent ABC without a coupon after the outliers were removed using a one‐class SVM model and projecting the region to the treatment group, those who bought detergent ABC after being sent a coupon. To interpret this plot, examine the different parts.
The fi rst group to identify is the treatment group that falls inside the oval; these are responders who were unaffected by the coupon. This means that for those people inside the upper oval, the coupon did not infl uence their purchasing choice (buy detergent ABC regard-less of the promotion either to buy detergent ABC or to not buy).
The data points outside of the oval in the treatment response group
Figure 7.3 Separation of Responders and Nonresponders in the Control Group Origin
146 ▸ B I G D A T A , D A T A M I N I N G , A N D M A C H I N E L E A R N I N G
are the potential incremental responders. Those are the people who purchased because of the treatment; in this specifi c example, the coupon for detergent ABC. I used the word “potential” above be-cause there is no defi nitive way in real life to objectively measure those people who responded only as a result of the treatment. This can be tested empirically using simulation, and that work has illus-trated the effectiveness of this method. Figure 7.5 is an example of a simulation study.
Figure 7.5 shows 1,300 responders to the treatment group. This includes 300 true incremental responders. The method described above identifi ed 296 observations as incremental responders, and 280 of
Figure 7.4 Projection of Control Group to Treatment Group Treatment
Control
Figure 7.5 Simulation of 1,300 Responders to Coupon Offer
−6
−5 0 5
Respondents Nonresponders
−4 −2 0
x1
x2
2 4 6
I N C R E M E N T A L R E S P O N S E M O D E L I N G ◂ 147
Table 7.1 Classifi cation of Incremental Responders
Correctly Classifi ed Falsely Classifi ed
Responders 280 20
Nonresponders 986 16
those identifi ed were true positives. This is all the more impressive because, as you can see, there is no simple way to use straight lines and separate the gray true incremental responders from the black nonincremental responders. This leaves 20 true responders who were not identifi ed and 16 who were incorrectly identifi ed. See Table 7.1 for a tabular view.
This simulation yields a 5.4% error rate, which is a signifi cant im-provement over the differencing method explained at the beginning of the chapter. Incremental response modeling holds much promise in the areas of targeted advertising and microsegmentation. The abil-ity to select only those people who will respond only when they re-ceive the treatment is very powerful and can contribute signifi cantly to increased revenue. Consider the typical coupon sells the goods or service at 90% of the regular price (a 10% discount). Every correctly identifi ed true incremental responder will raise revenue 0.9 and every correctly identifi ed nonincremental responder (those who are not in-fl uenced by the treatment either to purchase or not) will raise revenue by 0.1 because those items will not be sold at a discount needlessly.
Then add in the nominal cost of the treatment—ad campaign, postage, printing costs, channel management. We have the following revenue adjustments:
Incremental Revenue=0 9. r+0 1. n campaign costs− where
r= incremental responders who will purchase the product if they r
receive the coupon but otherwise will not
n = nonresponders who will not buy the product even if they received the coupon
By taking the simulation example but increasing the error rate to nearly double at 10%, you can see the advantage of using incremental response:
148 ▸ B I G D A T A , D A T A M I N I N G , A N D M A C H I N E L E A R N I N G
Incremental response revenue=333units=.9 270
( )
+.1 900( )
−fixed coosts compared to:Control only=100units=.9 0
( )
+.1 1000( )
−campaign costs Treatment only=270units=.9 300( )
+.1 0( )
−campaign costscosts of each scenario, the treatment will have the largest effect be-cause a coupon is being sent to each of the 1,300 people. This will be followed by the incremental response group, where coupons are sent only to those predicted to respond; and fi nally the control group, where there are no campaign costs because of the lack of campaign.
Treatment campaign costs Incremental reponse campaign costs Con
> >
ttrol campaign costs=0
When the campaign costs are added to the calculations, the in-cremental response is an even better option to either the treatment or the control group. This increasing amount of information that is made available will over time reduces the error rates, yielding even larger revenues for those organizations that leverage this powerful technique.
149 C H A P T E R