Response Surf ace Designs Designs for contin uous var iab les Frédér ic Ber trand1 1IRMA,UniversitédeStrasbourg Strasbourg,France ENSAI 3eAnnée 2017-2018

(1)

First-OrderResponseSurfacesSecond-OrderResponseSurfacesMixtureExperimentsAppendix

Response Surf ace Designs Designs for contin uous var iab les Frédér ic Ber trand

1 1IRMA,UniversitédeStrasbourg Strasbourg,France

ENSAI 3

e

Année 2017-2018

IntroductionFirst-OrderResponseSurfacesSecond-OrderResponseSurfacesMixtureExperimentsAppendix

Ref erences This course in mainly based on: 1. the book of Gar y W .Oehler t, A Fir st Cour se in Design and Anal ysis of Experiments ,2010. Freely av ailab le at http://users.stat.umn.edu/~gary/Book.html . 2. the book of Douglas C .Montgomer y, Design and Anal ysis of Experiments ,7

th

Edition, Wile y, 2009. 3. the book of Sam uel D .Silv ey , Optimal Design ,Chapman and Hall, 1980.

Contents Introduction Setting Visualizing the Response First-Order Response Surf aces First-Order Models First-Order Designs First-Order Analysis Second-Order Response Surf aces Second-Order Models Second-Order Designs Second-Order Analysis Mixture Exper iments Models for Mixtures Designs for mixtures Models for mixture designs

(2)

Introduction Setting Man y exper iments ha ve the goals of describing ho w the response varies as a function of the treatments and deter mining treatments that giv e optimal responses ,perhaps maxima or minima . Factorial-treatment structures can be used for these kinds of exper iments ,b ut when treatment factors can be var ied across a contin uous rang e of values , other treatment designs ma y be more efficient . Response surface methods are designs and models for w or king Response with contin uous treatments when finding optima or describing the response surf ace methods is the goal.

Contents Introduction Setting Visualizing the Response First-Order Response Surf aces First-Order Models First-Order Designs First-Order Analysis Second-Order Response Surf aces Second-Order Models Second-Order Designs Second-Order Analysis Mixture Exper iments Models for Mixtures Designs for mixtures Models for mixture designs

Introduction Visualizing the Response In some exper iments ,the treatment factors can var y contin uously . When w e bak e a cak e, w e bak e for a cer tain time x

1

at a cer tain temperature x

2

;time and temper ature can var y contin uousl y . W e could, in pr inciple ,bak e cak es for an y time and temperature combination . Assuming that all the cak e batters are the same ,the quality of the cak es y will depend on the time and temper ature of baking.

Introduction Visualizing the Response Response is a function of contin uous design variab les . W e express this as y

ij

= f ( x

1,i

, x

2,i

) + ε

ij

, meaning that the response y is some function f of the design variab les x

1

and x

2

,plus experimental err or . Here j inde xes the replication at the i

th

unique set of design var iab les .

(3)

Introduction Visualizing the Response One common goal when w or king with response surf ace data is to find the settings for the design var iab les that optimiz e (maximiz e or minimiz e) the response . Often there are complications . 1) For example ,there ma y be se veral responses ,and w e m ust seek some kind of compr omise optim um that mak es all responses good but does not exactly optimiz e an y single response .

Introduction Visualizing the Response 2) Alter nativ ely ,there ma y be constraints on the design variab les ,so that the goal is to optimiz e a response ,subject to the design var iab les meeting some constr aints . A second goal for response surf aces is to understand “the lie of the land” . Where are the hills ,v alle ys ,r idge lines ,and so on that mak e up the topograph y of the response surface ? At an y giv e design point, ho w will the response chang e if w e alter the design var iab les in a given direction ?

Introduction Visualizing the Response W e can visualiz e the function f as a surface of heights ov er the x

1

, x

2

plane ,lik e a relief map sho wing mountains and valle ys . 1) A per spective plot sho ws the surface when vie w ed fr om the side ;Figure 1 is a perspectiv e plot of a fair ly complicated surf ace that is wiggly for lo w values of x

2

,and flat for higher values of x

2

. 2) A contour plot sho ws the contour s of the surface ,that is , cur ves of x

1

, x

2

pairs that ha ve the same response value . Figure 2 is a contour plot for the same surf ace as Figure 1.

Introduction Visualizing the Response

5 1 0 R es p o n se S u rf a ce D es ig n s

0

-1.0

-0.5

0.0

0.5

y

1 2

x 2

1

0.5

1.0

1.5 210 3

7654321

10987

x 1 F ig u re 1 9 .1 : S am p le p er sp ec ti v e p lo t, u si n g M in it ab . th e re sp o n se . O ft en th er e ar e co m p li ca ti o n s. F o r ex am p le , th er e m ay b e se v er al re sp o n se s, an d w e m u st se ek so m e k in d o f co m p ro m is e o p ti m u m th at m ak es al l re sp o n se s g o o d b u t d o es n o t ex ac tl y o p ti m iz e an y si n g le re sp o n se . C o m p ro m is e o r c o n s tr a in e d o p ti m u m A lt er n at iv el y, th er e m ay b e co n st ra in ts o n th e d es ig n v ar ia b le s, so th at th e g o al is to o p ti m iz e a re sp o n se , su b je ct to th e d es ig n v ar ia b le s m ee ti n g so m e co n st ra in ts . A se co n d g o al fo r re sp o n se su rf ac es is to u n d er st an d “t h e li e o f th e la n d .” W h er e ar e th e h il ls , v al le y s, ri d g e li n es , an d so o n th at m ak e u p th e to p o g ra - D e s c ri b e th e s h a p e o f th e re s p o n s e p h y o f th e re sp o n se su rf ac e? A t an y g iv e d es ig n p o in t, h o w w il l th e re sp o n se ch an g e if w e al te r th e d es ig n v ar ia b le s in a g iv en d ir ec ti o n ? W e ca n v is u al iz e th e fu n ct io n f as a su rf ac e o f h ei g h ts o v er th e x

1

,x

2

p la n e, li k e a re li ef m ap sh o w in g m o u n ta in s an d v al le y s. A p er sp ec ti v e p lo t sh o w s th e su rf ac e w h en v ie w ed fr o m th e si d e; F ig u re 1 9 .1 is a p er sp ec ti v e p lo t o f a fa ir ly co m p li ca te d su rf ac e th at is w ig g ly fo r lo w v al u es o f x

2

, an d P e rs p e c ti v e p lo ts a n d c o n to u r p lo ts fl at fo r h ig h er v al u es o f x

2

. A co n to u r p lo t sh o w s th e co n to u rs o f th e su rf ac e, th at is , cu rv es o f x

1

,x

2

p ai rs th at h av e th e sa m e re sp o n se v al u e. F ig u re 1 9 .2 is a co n to u r p lo t fo r th e sa m e su rf ac e as F ig u re 1 9 .1 . G ra p h ic s an d v is u al iz at io n te ch n iq u es ar e so m e o f o u r b es t to o ls fo r u n - d er st an d in g re sp o n se su rf ac es . U n fo rt u n at el y, re sp o n se su rf ac es ar e d if fi cu lt U s e m o d e ls fo r f to v is u al iz e w h en th er e ar e th re e d es ig n v ar ia b le s, an d b ec o m e al m o st im - p o ss ib le fo r m o re th an th re e. W e th u s w o rk w it h m o d el s fo r th e re sp o n se

Figure 1: Sample perspectiv e plot, using Minitab .

(4)

Introduction Visualizing the Response

9 .2 F ir st -O rd er M o d el s 5 1 1

-0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 9876543210

3 2 1 0

x1

x2

Contour Plot of y F ig u re 1 9 .2 : S am p le co n to u r p lo t, u si n g M in it ab . n ct io n f . 9 .2 F ir st -O rd er M o d el s A ll m o d el s ar e w ro n g ; so m e m o d el s ar e u se fu l. G eo rg e B o x e o ft en d o n ’t k n o w an y th in g ab o u t th e sh ap e o r fo rm o f th e fu n ct io n f , so y m at h em at ic al m o d el th at w e as su m e fo r f is su re ly w ro n g . O n th e o th er an d , ex p er ie n ce h as sh o w n th at si m p le m o d el s u si n g lo w -o rd er p o ly n o m ia l rm s in th e d es ig n v ar ia b le s ar e g en er al ly su ffi ci en t to d es cr ib e se ct io n s o f P o ly n o m ia ls a re o ft e n a d e q u a te m o d e ls re sp o n se su rf ac e. In o th er w o rd s, w e k n o w th at th e p o ly n o m ia l m o d el s es cr ib ed b el o w ar e al m o st su re ly in co rr ec t, in th e se n se th at th e re sp o n se rf ac e f is u n li k el y to b e a tr u e p o ly n o m ia l; b u t in a “s m al l” re g io n , p o ly n o - ia l m o d el s ar e u su al ly a cl o se en o u g h ap p ro x im at io n to th e re sp o n se su rf ac e at w e ca n m ak e u se fu l in fe re n ce s u si n g p o ly n o m ia l m o d el s. W e w il l co n si d er fi rs t- o rd er m o d el s an d se co n d -o rd er m o d el s fo r re sp o n se rf ac es . A fi rs t- o rd er m o d el w it h q v ar ia b le s ta k es th e fo rm F ir s t- o rd e r m o d e l h a s lin e a r te rm s y

ij

= β

0

+ β

1

x

1i

+ β

2

x

2i

+ ·· · + β

q

x

qi

+ ǫ

ij

Figure 2: Sample contour plot, using Minitab .

Introduction Visualizing the Response Graphics and visualization techniques are some of our best tools for understanding response surfaces . Unf or tunately ,response surf aces are difficult to visualiz e when there are three design var iab les ,and become almost impossib le for more than three . W e thus w or k with models for the response function f .

Contents Introduction Setting Visualizing the Response First-Order Response Surf aces First-Order Models First-Order Designs First-Order Analysis Second-Order Response Surf aces Second-Order Models Second-Order Designs Second-Order Analysis Mixture Exper iments Models for Mixtures Designs for mixtures Models for mixture designs

First-Order Models Introduction All models are wrong; some models are useful. George Bo x. W e often don’t kno w an ything about the shape or for m of the function f ,so an y mathematical model that w e assume for f is surely wr ong . On the other hand, exper ience has sho wn that simple models using lo w-or der pol ynomial ter ms in the design var iab les are generall y sufficient to describe sections of a response surface .

(5)

First-Order Models Introduction In other w ords ,w e kno w that the pol ynomial models descr ibed belo w are almost surely incorrect ,in the sense that the response surf ace f is unlikel y to be a true pol ynomial . But in a “small” region, pol ynomial models are usually a close enough appr oximation to the response surf ace that w e can mak e useful inf erences using polynomial models .

First-Order Models Introduction W e will consider fir st-or der models and second-or der models for response surf aces . A fir st-or der mode lwith q var iab les tak es the for m y

ij

= β

0

+ β

1

x

1i

+ β

2

x

2i

+ + β

q

x

qi

+ ε

ij

= β

0

+

q

X

k=1

β

k

x

ki

+ ε

ij

= β

0

+ x

0 i

β + ε

ij

, where x

i

= ( x

1i

, x

2i

,. .. , x

qi

)

0

and β = ( β

1

,β

2

,. .. ,β

q

)

0

. The first-order model is an or dinar y m ultiple-regression model , with design var iab les as predictors and β

k

’s as reg ression coefficients .

First-Order Models Introduction Fir st-or der models descr ibe inclined planes: flat surfaces , possib ly tilted. These models are appr opriate for descr ibing por tions of a response surf ace that are separated from maxima, minima, ridge lines ,and other str ongl y cur ved regions . For example , the side slopes of a hill might be reasonab ly appro ximated as inclined planes .

First-Order Models Local appro ximation These appr oximations are local ,in the sense that you need diff erent inclined planes to descr ibe diff erent par ts of the mountain. First-order models can appro ximate f reasonab ly w ell as long as the region of appr oximation is not too big and f is not too cur ved in that region. A first-order model w ould be a reasonab le appro ximation for the par tof the surf ace in Figures 1 or 2 where x

2

is large; a first-order model w ould w or k poor ly where x

2

is small.

(6)

First-Order Models Steepest ascent Bear ing in mind that these models are only appro ximations to the tr ue response , what can these models tell us about the surface ? Fir st-or der models can tell us whic h wa y is up (or do wn ). Suppose that w e are at the design var iab les x ,and w e w ant to kno w in whic h direction to mo ve to increase the response the most . This is the direction of steepest ascent .

First-Order Models Steepest ascent and descent It tur ns out that w e should tak e a step pr opor tional to β ,so that our ne w design variab les are x + r β ,f or some r > 0. If w e w ant the direction of steepest descent ,then w e mo ve to x − r β ,f or some r > 0. Note that this direction of steepest ascent is onl y appr oximatel y correct ,e ven in the region where w e ha ve fit the first-order model. As w e mo ve outside that region, the surface ma y chang e and a ne w direction ma y be needed .

First-Order Models Introduction Contour s or le vel cur ves are sets of design var iab les that ha ve the same expected response . For a fir st-or der surface ,design points x and x + δ are on the same contour if P β

k

δ

k

= 0. Fir st-or der model contour s are str aight lines for q = 2, planes for q = 3, and so on. Note that directions of steepest ascent are perpendicular to contour s .

Contents Introduction Setting Visualizing the Response First-Order Response Surf aces First-Order Models First-Order Designs First-Order Analysis Second-Order Response Surf aces Second-Order Models Second-Order Designs Second-Order Analysis Mixture Exper iments Models for Mixtures Designs for mixtures Models for mixture designs

(7)

First-Order Designs Three basic needs W e ha ve three basic needs from a response surf ace design. 1) W e m ust be ab le to estimate the parameter s of the model . 2) W e m ust be ab le to estimate pure err or and lac k of fit . As descr ibed belo w ,pure error and lac k of fit are our tools for determining if the fir st-or der model is an adequate appr oximation to the true mean structure of the data. 3) W e need the design to be efficient , both from a variance of estimation point of vie w and a use of resour ces point of vie w .

First-Order Designs Pure error and lac k of fit The concept of pure err or needs a little explanation . Data might not fit a model because of random err or (the ε

ij

sor tof error); this is pure err or . Data also might not fit a model because the model is misspecified and does not tr uly descr ibe the mean str ucture; this is lac k of fit .

First-Order Designs Need to detect lac k of fit Our models are appr oximations ,so w e need to kno w when the lac k of fit becomes lar ge relative to pure err or . This is par ticularl y true for fir st-or der models ,which w e will then replace with second-or der models . It is also true for second-or der models ,though w e are more likel y to reduce our region of modeling rather than mo ve to higher orders .

First-Order Designs When can there be LoF? W e do not ha ve lac k of fit for factor ial models when the full factorial model is fit. In that situation, w e ha ve fit a deg ree of freedom for ev er y factor-le vel combination—in eff ect, a mean for each combination. There can be no lac k of fit in that case because all means ha ve been fit exactl y . W e can get lac k of fit when our models contain fe wer degrees of freedom than the number of distinct design points used; in par ticular , fir st- and second-or der models ma y not fit the data .

(8)

First-Order Designs Coding the var iab les Response surf ace designs are usually giv en in ter ms of coded variab les . Coding simply means that the design variab les are rescaled so that 0 is in the center of the design, and ± 1 are reasonab le steps up and do wn from the center . For example ,if cak e baking time should be about 35 min utes , giv e or tak e a couple of min utes ,w e might rescale time by ( x

1

− 35 ) / 2 ,so that 33 min utes is a − 1 , 35 min utes is a 0 ,and 37 min utes is a 1 .

First-Order Designs Standard first order designs Fir st-or der designs collect data to fit fir st-or der models . The standar d fir st-or der design is a 2

q

factor ial with center points . The (coded) lo w and high values for each variab le are ± 1 ;the center points are m obser vations tak en with all var iab les at 0 . This design has 2

q

+ m points . W e ma y also use an y 2

q−k

fraction with resolution III or greater .

First-Order Designs One stone tw o birds The replicated center points ser ve tw o uses . 1) The variation among the responses at the center point pro vides an estimate of pure err or . 2) The contrast betw een the mean of the center points and the mean of the factorial points pro vides a test for lac k of fit .

First-Order Designs Test for lac k of fit The contrast betw een the mean of the center points and the mean of the factorial points has: 1) an expected value zer o ,when the data follo w a fir st-or der model , 2) an expectation that depends on the pure quadratic terms , when the data follo w a second-or der model .

(9)

First-Order Designs Example 1: Cak e baking Our cak e mix recommends 35 min utes at 350 °F ,b ut w e are going to tr y to find a time and temper ature that suit our palate better . W e begin with a fir st-or der design in baking time and temper ature ,so w e use a 2

2

factor ial with three center points . W e use the coded values: 1) − 1, 0, 1 for 33, 35 ,and 37 min utes for time ,and 2) − 1, 0, 1 for 340, 350 ,and 360 deg rees for temper ature .

First-Order Designs Example 1: Cak e baking W e will thus ha ve 1) three cakes bak ed at the pac ka ge-recommended time and temper ature (our center point ), 2) and four cakes with time and temper ature spread ar ound the center .

First-Order Designs Example 1: Cak e baking Our response is an av er age palatability score ,with higher values being desir ab le: x

1

x

2

y − 1 − 1 3 . 89 1 − 1 6 . 36 − 1 1 7 . 65 1 1 6 . 79 0 0 8 . 36 0 0 7 . 63 0 0 8 . 12

Contents Introduction Setting Visualizing the Response First-Order Response Surf aces First-Order Models First-Order Designs First-Order Analysis Second-Order Response Surf aces Second-Order Models Second-Order Designs Second-Order Analysis Mixture Exper iments Models for Mixtures Designs for mixtures Models for mixture designs

(10)

First-Order Analysis Possib le Goals for a First-Order Design analysis Here are three possib le goals when analyzing data fr om a fir st-or der design : • Deter mine whic h design variab les aff ect the response . • Deter mine whether there is lac k of fit . • Deter mine the direction of steepest ascent . Some exper imental situations can in volv e a sequence of designs and all these goals .

First-Order Analysis Fitting First-Order Models In all cases , model fitting for response surfaces is done using m ultiple linear regression . The model variab les ( x

1

through x

q

for the first-order model) are the “independent” or “predictor” var iab les of the regression . The estimated reg ression coefficients are estimates of the model par ameters β

k

.

First-Order Analysis Fitting First-Order Models For fir st-or der models using data from 2

q

factor ials with or without center points ,the estimated regression slopes using coded var iab les are equal to the ordinar y main eff ects for the factorial model . Let b = ˆ β be the vector of estimated coefficients for first-order ter ms (an estimate of β ).

First-Order Analysis Model Testing Model testing is done with F -tests on mean squares from the ANO VA of the reg ression; each ter m has its own line in the ANO VA tab le . Predictor var iab les are or thogonal to each other in man y designs and models ,b ut not in all cases ,and cer tainly not when there is missing data; so it seems easiest just to treat all testing situations as if the model variab les w ere nonor thogonal .

(11)

First-Order Analysis Impro vement sum of squares To test the null hypothesis that the coefficients for a set of model terms are all zer o ,get: 1) the err or sum of squares for the full model and 2) the err or sum of squares for the reduced model that does not contain the model ter ms being tested. The diff erence in these err or sums of squares is the impr ovement sum of squares for the model ter ms under test.

First-Order Analysis Test’ s statistic The impr ovement mean square is the impr ovement sum of squares divided by its degrees of freedom (the number of model ter ms in the m ultiple reg ression being tested). This impr ovement mean square is divided by the err or mean square from the full model to obtain an F -test of the null hypothesis .

First-Order Analysis Sequential ANO VA. t-tests The sum of squares for impr ovement can also be computed from a sequential (T ype I) ANO VA for the model, pro vided that the terms being tested are the last terms entered into the model. The F -test of β

k

= 0 (with one numer ator deg ree of freedom) is equiv alent to the t -test for β

k

that is pr inted by most reg ression softw are .

First-Order Analysis Noise var iab les In man y response surf ace exper iments , all variab les are impor tant ,as there has been preliminar y screening to find impor tant var iab les prior to exploring the surface . Ho w ev er ,inclusion of noise variab les into models can alter subsequent anal ysis . It is w or th noting that variab les can look iner t in some par ts of a response surf ace ,and active in other par ts .

(12)

First-Order Analysis Steepest ascent and iner tv ar iab les The direction of steepest ascent in a first-order model is pr opor tional to the coefficients β . Our estimated direction of steepest ascent is then propor tional to b . Inclusion of iner tv ariab les in the computation of this direction increases the err or in the direction of the activ e var iab les . This eff ect is w orst when the activ e var iab les ha ve relativ ely small eff ects . The net eff ect is that our response will not increase as quic kly as possib le per unit change in the design var iab les ,because the direction could ha ve a nonnegligib le component on the iner t ax es .

Residual var iation’ s decomposition Possib le Goals for a First-Order Design analysis Residual variation can be divided into tw o par ts : pure err or and lac k of fit . 1) Pure err or is var iation among responses that ha ve the same explanator y var iab les (and are in the same bloc ks ,if there is bloc king). W e use replicated points ,usually center points ,to get an estimate of pure err or . 2) All the rest of residual variation that is not pure error is lac k of fit .

Residual var iation’ s decomposition Possib le Goals for a First-Order Design analysis Thus w e can mak e the decompositions : SS

Tot

= SS

Model

+ SS

LoF

+ SS

PE

n − 1 = df

Model

+ df

LoF

+ df

PE

.

First-Order Analysis Testing lac k of fit The mean square for pure err or estimates σ

2

,the variance of ε . If the model w e ha ve fit has: 1) the correct mean structure ,then the mean square for lac k of fit also estimates σ

2

,and the F -ratio MS

LoF

/ MS

PE

will ha ve an F -distrib ution with df

LoF

and df

PE

deg rees of freedom. 2) the wr ong mean structure -for example ,if w e fit a first-order model and a second-order model is correct- then the expected value of MS

LoF

is lar ger than σ

2

.

(13)

First-Order Analysis Testing for lac k of fit Thus w e can test for lac k of fit by compar ing the F -r atio MS

LoF

/ MS

PE

to an F -distrib ution with df

LoF

and df

PE

deg rees of freedom. Example For a 2

q

factor ial design with m center points ,there are 2

q

+ m − 1 deg rees of freedom, with q for the model, m − 1 for pure error ,and all the rest for lac k of fit.

First-Order Analysis Cannot use model if significant lac k of fit Quantities in the analysis of a first-order model are not (v er y) reliab le when there is significant lac k of fit . Because the model is not trac king the actual mean structure of the data ,the impor tance of a var iab le in the first-order model ma y not relate to the variab le’ s impor tance in the mean structure of the data. Like wise ,the direction of steepest ascent from a first-order model ma y be meaningless if the the model is not descr ibing the tr ue mean str ucture .

First-Order Analysis Example 2: Cak e baking, contin ued Example 1 w as a 2

2

design with three center points . Our fir st-or der model includes a constant and linear terms for time and temper ature . With se ven data points ,there will be 4 residual degrees of freedom . The only replication in the design is at the three center points ,so w e ha ve 2 degrees of freedom for pure err or . The remaining 2 residual degrees of freedom are lac k of fit .

First-Order Analysis Example 2: Cak e baking, contin ued Estimated Regression Coefficients for y Term Coef StDev T P Constant 6.9714 0.5671 12.292 0.000 x1 0.4025 0.7503 0.536 0.620 (A) x2 1.0475 0.7503 1.396 0.235 (A) S = 1.501 R-Sq = 35.9% R-Sq(adj) = 3.8% Listing 1: Minitab output for first-order model of cak e baking data.

(14)

First-Order Analysis Example 2: Cak e baking, contin ued Analysis of Variance for y Source DF Seq SS Adj SS Adj MS F P Regression 2 5.0370 5.0370 2.5185 1.12 0.411 Linear 2 5.0370 5.0370 2.5185 1.12 0.411 Residual Error 4 9.0064 9.0064 2.2516 Lack-of-Fit 2 8.7296 8.7296 4.3648 31.53 0.031 (B) Pure Error 2 0.2769 0.2769 0.1384 Total 6 14.0435 Listing 1 : Minitab output for first-order model of cak e baking data.

First-Order Analysis Example 2: Cak e baking, contin ued Listing 1 sho ws results for this analysis . Using the 4-deg ree-of-freedom residual mean square , neither time nor temperature has an F -ratio m uc h big ger than one , so neither appear s to aff ect the response ,see (A). Ho w ev er ,look at the test for lac k of fit ,see (B). This test has an F -ratio of 31 . 5

1

,indicating that the fir st-or der model is missing some of the mean structure . 1. and p -v alue of . 03. Yet that p -v alue cannot be used since the Gaussian linear model assumptions cannot be chec ked with such a lo w sample siz e of n = 7.

First-Order Analysis Example 2: Cak e baking, contin ued The 2 degrees of freedom for lac k of fit are the interaction in the factorial points and the contrast betw een the factorial points and the center points . The sums of squares for these contr asts are 2 . 77 and 5 . 96, so most of the lac k of fit is due to the center points not lying on the plane fit from the factorial points . In fact, the center points are about 1 . 86 higher on av er age than what the fir st-or der model predicts .

First-Order Analysis Example 2: Cak e baking, contin ued The direction of steepest ascent in this model is propor tional to ( . 40 , 1 . 05 ) ,the estimated β

1

and β

2

. That is ,the model sa ys that a maximal increase in response can be obtained by increasing x

1

by . 38 (coded) units for ever y increase of 1 (coded) unit in x

2

. Ho w ev er ,w e ha ve already seen that there is significant lac k of fit using the fir st-or der model with these data, so this direction of steepest ascent is not reliab le .

(15)

Contents Introduction Setting Visualizing the Response First-Order Response Surf aces First-Order Models First-Order Designs First-Order Analysis Second-Order Response Surf aces Second-Order Models Second-Order Designs Second-Order Analysis Mixture Exper iments Models for Mixtures Designs for mixtures Models for mixture designs

Second-Order Models Definition W e use second-or der models when the por tion of the response surface that w e are descr ibing has cur vature . A second-or der model contains : 1) all the ter ms in the fir st-or der model ,plus 2) all quadratic terms lik e β

11

x

^{2 1}i

and 3) all cr oss pr oduct terms lik e β

12

x

1i

x

2i

.

Second-Order Models Definition Specifically ,it tak es the for m y

ij

= β

0

+ β

1

x

1i

+ β

2

x

2i

+ ·· · + β

q

x

qi

+ β

11

x

^{2 1}i

+ β

22

x

^{2 2}i

+ ·· · + β

qq

x

2 qi

+ β

12

x

1i

x

2i

+ β

13

x

1i

x

3i

+ ·· · + β

1q

x

1i

x

qi

+ β

23

x

2i

x

3i

+ β

24

x

2i

x

4i

+ ·· · + β

2q

x

2i

x

qi

+ ·· · + β

(q−1)q

x

(q−1)i

x

qi

+ ε

ij

= β

0

+

q

X

k=1

β

k

x

ki

+

q

X

k=1

β

kk

x

2 ki

+

q−1

X

k=1

q

X

l=k+1

β

kl

x

ki

x

li

+ ε

ij

.

Second-Order Models Definition It can also tak e the matr ix for m y

ij

= β

0

+ x

i0

β + x

i0

B x

i

+ ε

ij

, where x

i

= ( x

1i

, x

2i

,. .. , x

qi

)

0

, β = ( β

1

,β

2

,. .. ,β

q

)

0

,and B is a q × q matr ix with B

kk

= β

kk

and B

kl

= B

lk

= β

kl

/ 2 for k < l . Note that the model only includes the kl cross product for k < l ; the matr ix for m with B includes both kl and lk ,so the coefficients are halv ed to tak e this into account.

(16)

Second-Order Models Shapes of quadr atic surf aces Second-or der models descr ibe quadratic surfaces ,and quadr atic surf aces can tak e se veral shapes . Figure 3 sho ws four of the shapes that a quadratic surface can tak e: First, w e ha ve a simple minim um (a) and maxim um (b). Then w e ha ve a ridg e (c); the surf ace is cur ved (here a maxim um) in one direction, but is fair ly constant in another direction. Finally , w e see a sad dle point (d); the surf ace cur ves up in one direction and cur ves do wn in another .

Second-Order Models Shapes of quadr atic surf aces

518ResponseSurfaceDesigns (a) -1

0.0

0.5 y1 0x1

1.0 -11

0

1 x2

(b) -1

0.0

0.5 y2 0x1

1.0 -11

0

1 x2 (c) -1

0.0

0.5 y3 0x1

1.0 -11

0

1 x2

(d) -1

0.0

0.5 y4 0x1

1.0 -11

0

1 x2 Figure19.3:Samplesecond-ordersurfaces:(a)minimum,(b)maximum,(c)ridge, and(d)saddle,usingMinitab. whereonceagainxi=(x1i,x2i,...,xqi)′,β=(β1,β2,...,βq)′,andBis aq×qmatrixwithBkk=βkkandBkl=Blk=βkl/2fork<l.Note thatthemodelonlyincludestheklcrossproductfork<l;thematrixform withBincludesbothklandlk,sothecoefficientsarehalvedtotakethisinto account. Second-ordermodelsdescribequadraticsurfaces,andquadraticsurfaces cantakeseveralshapes.Figure19.3showsfouroftheshapesthataquadratic surfacecantake.First,wehaveasimpleminimumandmaximum.ThenQuadratic surfacestake manyshapeswehavearidge;thesurfaceiscurved(hereamaximum)inonedirection, butisfairlyconstantinanotherdirection.Finally,weseeasaddlepoint;the surfacecurvesupinonedirectionandcurvesdowninanother. Second-ordermodelsareeasiertounderstandifwechangefromtheorig- inaldesignvariablesx1andx2tocanonicalvariablesv1andv2.Canonical variableswillbedefinedshortly,butfornowconsiderthattheyshifttheori- gin(thezeropoint)androtatethecoordinateaxestomatchthesecond-order

Figure 3: Sample second-order surf aces: (a) minim um, (b) maxim um, (c) ridge ,and (d) saddle ,using Minitab .

Second-Order Models Canonical var iab les Second-order models are easier to understand if w e change from the or iginal design var iab les x

1

and x

2

to canonical variab les v

1

and v

2

. Canonical var iab les will be defined shor tly ,b ut for no w consider that the y shift the origin (the zero point) and rotate the coor dinate ax es to matc h the second-order surf ace .

Second-Order Models Canonical var iab les The second-order model is ver y simple when expressed in canonical var iab les: f

v

( v ) = f

v

( 0 ) +

q

X

k=1

λ

k

v

2 k

. where v = ( v

1

, v

2

,. .. , v

q

)

0

is the design var iab les expressed in canonical coordinates; f

v

( v ) is the response as a function of the canonical var iab les; and λ

k

’s are numbers computed from the B matr ix.

(17)

Second-Order Models Stationar y point The x value that maps to 0 in the canonical var iab les is called the stationar y point and is denoted by x

0

;thus f

v

( 0 ) = f ( x

0

) . The ke y to understanding canonical var iab les is the stationar y point of the second-order surf ace . The stationar y point is that combination of design variab les where the surf ace is at either a maxim um or a minim um in all directions .

Second-Order Models Stationar y point 1) If the stationar y point is a maxim um in all directions ,then the stationar y point is the maxim um response on the whole modeled surface . 2) If the stationar y point is a minim um in all directions ,then it is the minim um response on the whole modeled surface .

Second-Order Models Stationar y point 3) If the stationar y point is a maxim um in some directions and a minim um in other directions ,then the stationar y point is a sad dle point ,and the modeled surface has no overall maxim um or minim um . 4) If a ridg e surface is absolutely le vel in some direction, then it does not ha ve a unique stationar y point; this rarel y happens in pr actice .

Second-Order Models First canonical axis The stationar y point will be the or igin (0 point) for our canonical var iab les . No w imagine yourself situated at the stationar y point of a second-order surf ace . The fir st canonical axis is the direction in which you w ould mo ve so that a step of unit length yields a response as lar ge as possib le (either increase the response as m uch as possib le or decrease it as little as possib le).

(18)

Second-Order Models Second canonical axis The second canonical axis is the direction, among all those directions perpendicular to the fir st canonical axis ,that yields a response as lar ge as possib le . There are as man y canonical ax es as there are design variab les . Eac h ad ditional canonical axis that w e find m ust be perpendicular to all those w e ha ve already found. Figure 4 sho ws contours ,stationar y points ,and canonical ax es for the four sample second-order surf aces .

Second-Order Models

520ResponseSurfaceDesigns (a) x1

x2

-1.0-0.50.00.51.0

-1.0 -0.5 0.0 0.5 1.0

0.02

0.05

0.1 x0 V1V2

(b) x1

x2

-1.0-0.50.00.51.0

-1.0 -0.5 0.0 0.5 1.0

0.98

0.95

0.9 x0 V2V1 (c) x1

x2

-1.0-0.50.00.51.0

-1.0 -0.5 0.0 0.5 1.0

0.950.80.60.6

V1 (d) x1

x2

-1.0-0.50.00.51.0

-1.0 -0.5 0.0 0.5 1.0

0.3

0.3 0.4

0.4 0.5 0.50.7

x0V1 V2 Figure19.4:Contours,stationarypoints,andcanonicalaxesforsamplesecond-order surfaces:(a)minimum,(b)maximum,(c)ridge,and(d)saddle,usingS-Plus. Thisdescriptionofsecond-ordersurfaceshasbeengeometric;pictures areaneasywaytounderstandthesesurfaces.Itisdifficulttocalculatewith pictures,though,sowealsohaveanalgebraicdescriptionofthesecond-order surface.Recallthatthematrixformoftheresponsesurfaceiswritten f(x)=β0+x′β+x′Bx.

Figure 4: Contours ,stationar y points ,and canonical ax es for Figure 3.

Second-Order Models Contours As sho wn in this figure (a) and (b), contour s for surf aces with maxima or minima are ellipses . The stationar y point x

0

is the center of these ellipses ,and the canonical ax es are the major and minor ax es of the elliptical contours . For the ridg e system (c), w e still ha ve elliptical contour s ,b ut the y are ver y long and skinn y, and the stationar y point is outside the region where w e ha ve fit the model. If the is absolutel y flat ,then the contours are parallel lines . For the sad dle point (d), contours are hyperbolic instead of elliptical. The stationar y point is in the center of the hyperbolas , and the canonical ax es are the ax es of the hyperbolas .

Second-Order Models Algebr aic descr iption This description of second-order surf aces has been geometric ;pictures are an easy wa y to under stand these surf aces . It is difficult to calculate with pictures ,though, so w e also ha ve an alg ebraic description of the second-order surf ace . Recall that the matr ix for m of the response surf ace is wr itten f ( x ) = β

0

+ x

0

β + x

0