Assessment of Adaptive One-Factor-at-a-Time
Methods vs. Fractional Factorial Methods
Using Reconfigurable Paper Aircraft
by
Jeffrey B. Persons, Jr.
Submitted to the Department of
Mechanical Engineering in Partial
Fulfillment of the Requirements for the
Degree of
Bachelor of Science
at the
Massachusetts Institute of Technology
MASSACHUSETTS INSTITUTE
June 2006 OF TECHNOLOGY
AUG 2 2006
© 2006 Jeffrey B. Persons, Jr.
All rights reserved
LIBRARIES
The author hereby grants to MIT permission to reproduce and to
Distribute publicly paper and electronic copies of this thesis document in whole or
in part in any medium now know
r Iereafter created.
Signature of Author ...
'? Depardntqf Mechanical
Engineering
May 12, 2006
Certified by ..
...
· ·
/
Daniel D. Frey
Asst Proeechanica
l Engineering & Engineering Systems
Thesis Supervisor
Accepted by.
...
...
John H. Lienhard
Professor of Mechanical Engineering
Chairman, Undergraduate Thesis Committee
Assessment of Adaptive One-Factor-at-a-Time Methods vs.
Fractional Factorial Methods
Using Reconfigurable Paper Aircraft
by
Jeffrey B. Persons. Jr.
Submitted to the Department of Mechanical Engineering
On May 12, 2006 in Partial Fulfillment of the
Requirements for the Degree of Bachelor of Science
in Engineering as recommended by the
Department of Mechanical Engineering
ABSTRACT
Recent research has suggested that under certain conditions, adaptive one-factor-at-a-time (aOFAT) methods outperform more commonly used fractional factorial methods. This study sought to corroborate these claims by analyzing a case study of a real-life
experiment. A full factorial experiment was conducted to collect data for simulations of fractional factorial and adaptive one-factor-at-a-time experiments. The experiment used a reconfigurable paper aircraft template with four three-level control factors.
Results indicated that the exploitation of control factor interactions by adaptive one-factor-at-a-time occurred at similar rates as predicted by Frey and Wang (2006). AOFAT experiments proved particularly effective at avoiding factor levels that led to poor
performance. with rates of avoidance approaching 100% for the worst levels. When bias in the full factorial experiment was eliminated, aOFAT methods even returned a higher (weighted average) leading quality indicator value than full factorial methods.
Thesis Supervisor: Daniel D. Frey
MOTIVATION
This paper serves as a case study for the purposes of evaluating fractional factorial and adaptive one-factor-at-a-time methods using a real-life example. While
many other case studies consist of computer-simulated experiments, this paper discusses
an experiment in which performance was observed in person and in which experimental error is real rather than simulated. In comparing the two robust design methods, this paper attempts to validate claims by Frey of conditional improvements provided by adaptive one-factor-at-a-time plans. In particular, it seeks to address exploitation of control factor interactions and performance optimization.
BACKGROUND
ROBUST DESIGN
The purpose of Robust Design, as stated by Phadke (1989), is to improve the
quality of a product by minimi ing the effect of the causes of variaction iwithout
eliminacting the causes. In designing robustly, we usually refer to minimizing variations
in production and operation, though the technique can also be applied to maximizing (or
minimizing) performance characteristics of a product or process.
In any Robust Design operation, we must first identify our leading quality indiccator. This is the performance characteristic that we wish to
maximize/minimize/control the variance of. To ensure the usefulness of results, the
leading quality indicator must be easily observed and quantified. The next step is to select control factors. Control factors are the variables that we adjust to optimize the leading quality indicator. To arrive at the best solution, we adjust the control factors, observe the results, and keep the best value of the group. Without a method for choosing factors and factor levels, however, this procedure can be painfully inefficient. To
facilitate the Robust Design process, therefore, different robust design algorithms have
Robust Design: fractional factorial experimentation. and adaptive one-factor-at-a-time
experimentation.
Fractional Factorial Experimentation
The use of statistical experimentation for practical design benefit has been around since at least the 1920's, when Sir Ronald Fisher applied it to agricultural processes in order to determine ideal levels of control factors for crop growth. It was not until the 1950's. however, that Dr. Genichi Taguchi adapted the use of fractional factorial experiments to robust product design and developing Robust Design into its own field. Taguchi's methods are now well-known, having become the industry standard for robust design algorithms.
Fractional factorial experimentation uses orthogonal arrays to isolate a favorable combination of factor levels. In effect, it allows us to run a relative handful of experiments and based on the data collected to estimate a good selection of factor levels. The exact number of' experiments required for this method depends on the number of factors involved and the number of possible levels per factor. For example, the orthogonal array for an experiment testing four factors at three levels each would look like this: Factor: A B C D Levels (1,2, or 3): 1 1 1 1 1 2 2 2 1 3 3 31 I -3 2 3 1 2 3 1 2 3 2 1 3
Figure 1. The standard L9 orthogonal array.
For the matrix to be orthogonal, each level appears an equal number of times in each column, and in any two columns every combination of factor levels appears. This feature ensures that each factor and its various levels are equally weighted during analysis.
The process for applying the fractional factorial method is as follows:
1. Establish an orthogonal array consisting of the required experiments
2. To ensure a more robust result, cross the control factor array with an outer' noise factor array. These noise factors are predetermined by the experimenter.
3. Run the experiments set out in the crossed array.
4. Determine the signal-to-noise ratio, ir , for each data set. For a larger-the-better leading quality indicator, we use the following equation:
q, =-10log1
Y
l
Where yj is the observed leading quality indicator, and qi is the signal-to-noise
ratio for set i.
5. Using averages of these signal-to-noise ratios, determine ratios for each factor and
its level. To do this, refer to the orthogonal array created earlier. For example, if factor A was set to level 1 for data sets 1-3, then one would use the following equation to determine the S/N ratio for Al:
1
m./ - -
(71 + 7'2
+73
)Repeat for all factors and levels (for an L9 orthogonal array, this would involve a total of 12 m values.
6. By comparing these factor level signal-to-noise ratios, select the highest value for each factor. The factor levels corresponding to these values will be the best estimates suggested by the fractional factorial method
Adaptive One Factor At-a-Time (AOFAT) Experimentation
One-factor-at-a-time (OFAT) experimentation is one of the oldest and most
archaic of robust design methods. OFAT involves conducting a full factorial experiment,
running through every possible combination of factor levels, changing, as the name
suggests, one factor at a time. This method is thorough but quite cumbersome. and in most areas of industry limited resources prevent its use.
Recently, however, researchers have begun using a modified version of OFAT
that uses fewer experiments than standard OFAT methods and has the potential to
outperform fractional factorial experimentation. Dubbed adaptive one-factor-at-a-time
(aOFAT) experimentation, this procedure requires only as many experiments as the
ftractional factorial method, and according to recent work by Frey and Li (2004), aOFAT can provide better results, particularly in cases with high levels of control factor interaction and low experimental error. aOFAT is more effective in these cases because it can exploit the control factor interactions to which fractional factorial methods are
insensitive.
The process behind aOFAT is quite simple.
First, pick a random starting configuration. To use an example from a four-factor, three-level experiment as before:
A B C D
Next, select a random order in which to vary factor levels. We'll say D, A, B, C. First change the level of D to 2.
A B C D
1 2 3 2
If the signal-to-noise ratio for this configuration is higher than the first, then we keep this
new configuration. It is important that we try all levels for each factor, so we must also try D=3. Once we select the optimal level of D, we go on to A, then B, then C. Once we finish adjusting a factor, we do not come back to it. This is an important difference
between aOFAT and OFAT, and it is this feature that limits the required number of
experiments. The final configuration at which we arrive will be the best configuration as
suggested by aOFAT. For further examples of the adaptive OFAT process, refer to
Appendix D.
Another advantage of aOFAT over fractional factorial experiments is the ability
to streamline the process based on available information. If little is known about the
system, then it makes sense to pick a random starting configuration and a random order
for varying the factor levels; however, an expert's knowledge of likely interactions may
make it possible to identify a better starting point or adjust the most influential factors
first. This ability to tailor experimentation as desired provides the added benefit of
getting closer to the best solution faster. If for some reason resources are limited or
experimentation has to be ended early, one is more likely to have good results with
aOFAT.
METHOD
In order to simulate experimentation using different algorithms, we needed a full data set to draw from. To obtain this data, we conducted a full factorial experiment using the aircraft template provided by Eppinger (1995) (APPENDIX A). This template can be
modified by varying four main factors: weight placement (indicated by "Weight" on the
template and labeled factor A), winglet configuration (Stabliz., B), nose length (Nose, C) and wing angle (Wing, D).
To ensure a reliable and robust data set, the full factorial array was crossed with a
noise array consisting of four combinations of three two-level noise factors: throwing
right- or left-handed, throwing with or without a glove, and throwing while standing on one leg or two. By crossing the control array with a noise array, we ensure that the
solution at which we eventually arrive will be insensitive to anticipated noise in
operation. The noise conditions were set up as follows:
Aside from this crossed array used in the experiment, there were other sources of noise as well that could account for variation in performance. These sources include minor variations in folding the airplanes, errors in placing the weights on the airplanes, drafts or turbulent air in the room where the experiments took place, and the strength of the
throws.
When only one foot was used, the researcher stood on his right foot. When two feet were used, the researcher stood with his feet slightly offset, right foot forward. The throwing style used was that of throwing a dart - roughly level and not forced. Distances
were measured from the location of the nose of the aircraft when the aircraft touched the
Throwing hand
Glove?
# of Legs
N1I Left Yes 2
N2
Left
No
1
N3 Right Yes I
Because time allowed it, we conducted two full factorial experiments at different dates, conducted in different ways. These two data sets are referred to in this document as Data Trial I and Data Trial II.
Data Trial I was a full factorial experiment conducted sequentially by Variable B
(winglet orientation), beginning with Set I. To save time, all three configurations of
Variable A (weight placement) were done on a single aircraft, so in all only 27 different aircraft were used. Noise factors were also taken sequential, N 1 -N4 in each set.
Data Trial II was a full factorial experiment conducted randomly by set and run
through all 81 sets. Noise factors were taken N1,N2,N3,N4 for the first set, then
N2,N3,N4,NI for the next set, and so on, with the cycle repeating every four sets. The
random ordering of sets and cycling of noise factors was expected to reduce bias in the
results and to allow better observation of control factor interactions.
Analysis
Following the conduct of the two trials, the data was subjected to simulations
using two robust design algorithms: the fractional factorial method, and the adaptive
one-factor-at-a-time (aOFAT) method.
For fractional factorial simulations, a standard L
9(34)Orthogonal Array was used,
and the process of analysis was carried out as prescribed by Phadke (CITE).
To simulate the aOFAT procedure, every permutation of starting point and factor
order was assessed, for 81 x 24 = 1944 possible routes per data trial. It is from these
While previous experiments have applied simulated error to their data to estimate the effect of greater variance on their results, the author determined that the real error
contained in the experiments was sufficient for the purposes of this study.
RESULTS
Data Trial I
Fractional Factorial
For Data Trial I. the fractional factorial method returned a maximum value of 189.5 in. corresponding to the configuration A2 B3 C2 D2. However, one of the sets tested was A2 B3 C1 D2, which delivers an average distance of 209.5 in. Neither of these values approaches the recorded optimum. which is 224.5 in, found at A3 B1 C1 D1.
By subtracting the value n from each of the signal-to-noise ratios associated with the factor levels recommended by the fractional factorial method, it is possible to estimate expected contributions to the S/N ratio above the mean. Based on the cumulative effects of the configuration recommended above, one would expect a S/N ratio of 46.39 db. The observed S/N ratio associated with the configuration A2 B3 C1 D2 is 46.42 db. much in line with expectations. In contrast, the S/N ratio of the optimum configuration has an expected value of 45.15 db, while the observed value is 47.02 db, much higher than anticipated. What this means is that, unlike the phenomenon we will observe with the adaptive OFAT method, the configuration recommended by the fractional factorial method does not take advantage of control factor interactions.
Plot of factor effects
40.00000
mAl mA2 mA3
Weight placement
mB1 mB2 mB3 mC1 mC2 mC3
Winglet orientaion Nose length
mD1 mD2 mD3
Wing angle
Figure 2. Factor effects from fractional factorial experiment, Data Trial I.
aOFAT
For Data Trial I, Adaptive One Factor At-a-Time experimentation led to a
weighted average distance of 198.95 in. By eliminating all starting points with B2, this
value was increased to 204.27 in. Using the modified starting points, AOFAT met or
exceeded the value returned by fractional factorial methods 58% of the time. If we also
eliminate starting points with C3, intuitively the least stable configuration of factor C, we
further increase the weighted average to 205.25 in, and aOFAT equals or betters fiactional factorial results 62% of the time. In all three starting point groups, the optimal solution (A3 B I C 1 D1) was selected approximately 20% of the time.
More interesting than the absolute distances suggested by aOFAT is the efficiency
with which it exploits factor effects. In the original aOFAT simulations (no starting
points excluded), 100% of the recommended values avoided the configurations B2 and
C3, which are intuitively much less stable than other options.
Additionally, 93% of the results avoided the combination Al-Dl, and 73% avoid
B -D1. which were suggested by graphical evidence (see Figures 3 and 4) as producing a
46.00000 45.00000 44.00000 m 43. 00000 A rn(nVA 41.00000 ... . ... .... .. . ..1 . ... ... . ... .... I. . . .i * I .. .. . . . . I . I . ... . I - - - --- I i i I
----
_----L. vvvvv ---.. . - .. .. - ···---. - . .. .. .. ·- --. - --- · ...-- - ... -.. ··decidedly unfavorable interaction.
These interactions are not intuitive, and their
placement suggests that they may be misleading. Most likely, these apparent interactions
were the result of a sequential testing procedure in which performance was anticipated by
the experimenter. A-C Interactions 180 S 170 , 160 c 150 a 140 C 130 l, 120 , 110 100 Al A2 C1 ...W... C2 - C3 A3 Factor
Figure 3. Mild synergistic interaction between factors A and C.
A-D Interactions 180 S 170 · 160 c 150 a 140 a 130 '= 120 M 110 100 D1 ... I ... D 2 ... :~:....D3 Al A2 A3 Factors
Figure 4. Antisynergistic interaction between Al and DI.
Data Trial II
Fractional Factorial
"~' ~~-~~~---·-
~-.
·· ·- ·- ·--- ·- · ·· - --- -- -·--. ... ... .. .. .. :~--
· · ·--... ·.. .... ._ ... ... . .... ... ...For Data Trial II, the fractional factorial method returned the configuration A2 B C( Dl for an average distance of 211.5 in. The actual optimal configuration was A3 BI C1 D1, returning a flight distance of 233.25 in. The fractional factorial solution failed to capture the full effects of factor interactions; it had a signal-to-noise ratio of 46.5 db, only slightly higher than the expected value of 45.64 db, derived from individual factor effects above the mean. The optimal solution indicates that there may be some three-level interactions taking place. as it had the same C and D values as the fractional factorial solution, yet its S/N ratio was 47.4 db, significantly higher than the expected value of 45.1 db.
Plot of factor effects
Ar, nnnnn
44.00000
m
43.00000
rnAl mA2 mA3 mB1 mB2 mB3 mC1 mC2 mC3 mD1 mD2 mD3
Weight placement Winglet orientaion Nose length Wing angle
Figure 5. Factor effects for fractional factorial experiment, Data Trial II.
aOFAT
The results for aOFAT tests of Data Trial II were even more favorable than for Data Trial I. aOFAT returned a weighted average distance of 211.7 in. exceeding the distance for the configuration recommended by fractional factorial methods. Additionally. by eliminating starting points with the value B2, the weighted average increased to 213 in, and aOFAT returned the optimal value of 233.25 in 32% of the time.
Because Data Trial II was conducted in a random order, it can be expected to provide cleaner results more indicative of physical interactions inherent in the aircraft
--
--- --- ---
design. For example, graphical evidence suggests a moderate negative interaction in the configuration B3-DI (see Figure 6). This configuration corresponds to winglets down with the highest wing angle, an arrangement that has the winglets angled outboard and therefore channels air more tightly between the winglet and the fuselage toward the trailing edge of the wing, predictably increasing drag. Significantly, 70% of the time aOFAT recommended designs that avoided this antisynergistic configuration. The model created bv Frey and Wang (2006) predicted that for moderate interactions and experimental error, the largest interaction is exploited 74% of the time. The results of this case study are slightly less favorable, but very much in line with these predictions.
The results also indicate that aOFAT exploited the second largest interaction, that of D1-C3. In this configuration, wing surface area is minimized, so one would expect
that performance for this arrangement would be particularly poor. In simulations,
aOFAT avoided this combination of factor levels 100% of the time.
Figure 6. Antisynergistic interaction between DI and B3.
D-B Interactions 180 160 r.- 150 B1 140 -- B2 2 130 B3 .? 120 U- 110
--100 -- ·-
-
...
100 D1 D2 D3 FactorsC-D Interactions zuu a . 180 c 160 u) bi 140 .2 120 Inn ' ... +D1 --- D2 -...
-"-"--.-.-.
" . : ..--.-. D3 .__- ···-·-- ··~ C1 C2 C3 FactorsFigure 7. Mild antisynergistic interaction between factors C and D.
CONCLUSIONS
It is interesting to note the difference in factor interactions between Data Trial I and Data Trial II. The fact that the interactions that take place in Data Trial I are neither
intuitive nor easily explained suggests that the results stem not from any physical
interactions taking place, but rather from experimental bias in the conduct of the full
factorial array experiment. The sequential testing of factor configurations likely led the
experimenter to have expectations of performance for different configurations and,
consciously or unconsciously, to induce error in noise factors (i.e. throw strength). Based on this observation, more weight should be given to the data produced by Data Trial II.
Results of aOFAT simulations using Data Trial II results confirm Frey's theory of
the superiority of aOFAT plans over fractional factorial methods in cases with significant
control factor interactions and low experimental error. Noticeably, the weighted average
of performance indicators of factor configurations recommended by aOFAT plans
exceeded those suggested by fractional factorial methods. More importantly, however,
was the manner in which the aOFAT results exploited factor interactions. The strongest
factor interaction was exploited much to the same degree as predicted by Frey and Wang (2006) - 70% vs 74%. The second-strongest interaction was avoided 100% of the time. even better than anticipated.
It is important to remember that aOFAT plans are designed to exploit factor effects and improve performance. but they do not offer insight into the strength of interactions vice main effects, nor do they attempt to explain reasons for improvement. This study corroborated Frey and Jugulum's (2005) observation that fbr ce'ertain
carrangemens
of main efecls and interactions, cLdaptlive
one-/actor-at-a-timne
experiment
exploit interactioons ith high pirohahility despite the fact that these designes lack the
resolittion to estimate interactions.AOFAT provides good solutions for immediate
problems. particularly when time and resources are limited. If researchers wish to
understand the system better. though, fractional factorial methods remain the most
effective option.
Further Study
In the fractional factorial analysis portion of this study, we looked only at the standard L9 orthogonal array as prescribed by Taguchi. It would be interesting for the purpose of comparison to rearrange the order of the factors (i.e. B,C,D,A) or to achieve the same effect by altering the order of the columns of the orthogonal array. There are 4! =-- 24 different ways to arrange the columns and maintain orthogonality, and these different arrangements suggest different answers. The original labeling of the factors (A to weight. B to winglets, and so on) was preset, and a different original ordering could
have produced different results.
It has been posited that aOFAT methods have the further advantage of offering
improvements even if the process is interrupted. Further studies might analyze results suggested by aOFAT simulations that are ended prematurely.
The ability to choose starting points for aOFAT experimentation is another of its
advantages over fractional factorial methods. In this study, we analyzed the effects of
starting configurations. It would be interesting to assess the effects of further limiting the
starting points on overall performance.
ACKNOWLEDGMENTS
Thanks to Prof. Frey for his continued mentorship and guidance in a field to which I am very much a newcomer.
Thanks to Will Reichert for helping write code to analyze all the data - and saving me countless hours in the process.
I/11 ,"
~"t
-z as _1 J- .is [4 ' -XI r s; f _ . ell " . .lAPPENDIX A
AIRCRAFT
TEMPLATE
/ . f 7 -r I-t-, -= rrr ot_-s
- -~ -- r. s *r* i .; *3 r r .R ', -- It J'--- -' .1. h-i * -- · i-· ,r I L I i r-o r 1 i t t t j i i i I i I "-·· I I r-i r t r Ir I I - I "n .*-3 , 1 r ,;· · ·( · i t r i i i 1 r i i r · _..---x. -* 1_ L j , tr_ I it j;-? L"" a :I i i r t iAPPENDIX B.1
EXPERIMENT I DATA
Data Trial I
Order of testing:
Sets 1-81 sequentially
Full factorial experiment using mod template pic.pdf
Run 09JAN05-13JAN 05, Compiled 16JAN05
Configuration D 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 Distance (in 156 178 168 182 172 159 184 180 176 166 206 184 160 162 170 174 164 166 186 180 166 156 204 172 146 148 136 140 152 154 146 156 142 144 146 Set No. AVG. 171 1 173.75 2 183 3 166.5 174 174.5 142.5 152 143
AVG Set No. 140.5 10 143 128 138 133 103 119 11 12 13 14 15 16 124 17 112 18 163.5 19 180.5 20 165.5 21 C 3 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 D 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 N 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 Dist. 140 132 144 150 136 120 122 168 162 106 140 138 128 130 142 124 156 114 142 138 138 104 96 112 100 110 120 128 118 108 128 128 132 88 108 116 136 148 170 166 170 172 174 198 178 156 166 3 3 3 3 3 3
AVG Set No. 197 22 192.5 23 142 24 137 25 136.5 26 128.5 27 193 28 170.5 29 169 30 168 31 161.5 32 169 33 A 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Dist. 184 156 204 190 202 192 188 190 184 208 132 134 148 154 132 160 132 124 144 136 136 130 120 142 122 130 180 212 194 186 158 168 180 176 174 164 172 166 158 180 180 154 162 160 166 158 164 158
AVG Set No. 143 34 157.5 35 145 36 161 37 105 38 115.5 39 137.5 40 125.5 41 104 42 115 109 103 43 44 45 B 1 1 1 1 A 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Dist. 180 174 140 144 146 142 164 172 150 144 144 156 142 138 152 162 190 140 96 120 120 84 144 60 174 84 144 150 132 124 120 118 146 118 94 100 108 114 104 118 118 120 106 100 128 102 98 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
AVG Set No. 171.5 209.5 46 47 179.5 48 208 49 189.5 50 150 51 137.5 52 122.5 53 125 54 224.5 55 182.5 56 181.5 57 Dist. 104 102 170 172 178 166 206 194 220 218 144 188 228 158 216 216 206 194 172 182 210 194 156 148 166 130 120 160 148 122 112 126 130 122 114 112 154 120 228 236 222 212 180 174 204 172 174 152
AVG Set No. 180.5 58 161.5 59 157.5 60 152 162 61 62 137.5 63 152 146 64 65 108 66 130.5 67 115.5 68 108 69 A 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 B 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 C 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 D 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 N 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 Dist. 206 194 172 176 184 190 178 150 154 164 164 136 172 158 144 164 144 156 180 162 156 150 120 130 158 142 144 168 158 138 130 176 158 120 84 92 110 146 124 116 118 164 106 130 112 114 106
AVG Set No. 119.5 113 104.5 182.5 203.5 193.5 212 70 71 72 73 74 75 76 188.5 77 138.5 163.5 78 79 136.5 80 129 81 A 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 C 2 2 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 D 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 N 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 Dist. 122 120 106 110 134 128 94 114 120 124 92 106 120 100 182 172 190 186 266 128 224 196 196 206 210 162 226 190 224 208 198 194 178 184 144 122 148 140 186 186 134 148 148 134 140 124 122 160 3 3 3 3 3 3 3 3 3 3 3 3
A B C D N Dist. AVG Set No.
3 3 3 3 3 124
3 3 3 3 4 110
A-D are as dictated on the template. Noise conditions (N) are as follows:
N1: Left hand, gloved, both feet on ground N2: Left hand, barehand, one foot on ground
N3: Right hand, gloved, one foot on ground
APPENDIX B.2
EXPERIMENT II DATA
Data Trial IIOrder of Testing:
Sets 4, 1, 20, 33, 64, 28, 65, 35, 5, 34, 60, 79, 7, 73, 9, 10,42, 24, 2, 16, 41, 6, 49, 48, 29, 27, 44, 43, 45, 80, 8, 70, 13, 50, 54, 19, 36, 74, 69, 72, 77, 47. 68, 39, 59, 58, 81. 51, 15, 32, 67, 38, 18, 23, 57, 25, 12, 53, 21, 14, 62, 63. 66, 75. 71, 31, 56, 40, 22, 76, 46, 37. 11, 52, 61. 78, 3, 30, 17, 26, 55With noise factors rotated: 1, 2, 3, 4, then 2, 3, 4, 1, then 3, 4, 1. 2, then 4. 1.
2, 3. repeated.
Full factorial experiment using Template 27 APR.doc Run 27APR06, Compiled
01MAY05 Configuration A 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Distance (in) AVG. 186 190.5 183 204 189 159 167.25 159 177 174 153 178.5 180 201 180 183 177.75 165 198 165 144 138 141 139.5 5 Set No. 1 2 3 4
AVG. Set No. 43.25 6 145.5 7 138 8 124.5 9 147 120.75 103.5 142.5 131.25 108.75 10 11 12 13 14 15 132 16 133.5 17 C 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 D 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 N 2 3 4 1 2 3 4 1 2 3 4 1 Dist. 135 144 1 141 150 138 138 144 153 147 141 132 147 132 108 132 126 132 114 156 165 153 126 132 111 114 78 123 105 108 138 159 126 147 117 129 141 138 105 96 117 117 117 135 144 132 138 2 2 2 2 2 2 2 2 2 2 2 2
AVG. Set No. 102.75 18 215.25 204.75 170.25 19 20 21 138 22 153.75 23 139.5 24 139.5 25 140.25 26 132 27 211.5 28 198 29 A 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 B 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 1 C 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 1 D 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 Dist. 132 102 105 111 93 189 222 225 225 207 186 216 210 201 165 177 138 120 141 138 153 156 135 165 159 138 117 168 135 144 138 144 132 144 153 126 138 120 126 135 147 201 225 225 195 183 180 222
A B C D N Dist. AVG. Set No. 2 1 1 2 4 207 2 1 1 3 1 171 165 30 2 1 1 3 2 144 2 1 1 3 3 189 2 1 1 3 4 156 2 1 2 1 1 150 159.75 31 2 1 2 1 2 153 2 1 2 1 3 165 2 1 2 1 4 171 2 1 2 2 1 141 137.25 32 2 1 2 2 2 129 2 1 2 2 3 138 2 1 2 2 4 141 2 1 2 3 1 159 150 33 2 1 2 3 2 135 2 1 2 3 3 156 2 1 2 3 4 150 2 1 3 1 1 150 144 34 2 1 3 1 2 144 2 1 3 1 3 144 2 1 3 1 4 138 2 1 3 2 1 117 124.5 35 2 1 3 2 2 129 2 1 3 2 3 132 2 1 3 2 4 120 2 1 3 3 1 126 132 36 2 1 3 3 2 123 2 1 3 3 3 132 2 1 3 3 4 147 2 2 1 1 1 144 137.25 37 2 2 1 1 2 147 2 2 1 1 3 144 2 2 1 1 4 114 2 2 1 2 1 120 144 38 2 2 1 2 2 141 2 2 1 2 3 159 2 2 1 2 4 156 2 2 1 3 1 117 120 39 2 2 1 3 2 123 2 2 1 3 3 93 2 2 1 3 4 147 2 2 2 1 1 126 142.5 40 2 2 2 1 2 150 2 2 2 1 3 147 2 2 2 1 4 147 2 2 2 2 1 117 125.25 41
AVG. Set No. 111 42 134.25 43 122.25 44 114 45 182.25 181.5 46 47 171 48 175.5 154.5 134.25 144 132.75 49 50 51 52 53 A 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 B 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 C 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 D 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 N 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 Dist. 141 93 111 114 126 102 183 123 129 117 114 132 126 102 120 114 120 189 174 183 183 201 162 210 153 159 183 174 168 168 174 177 183 129 144 195 150 123 132 141 141 141 147 141 147 123 132 159
N Dist. 4 117 1 132 2 132 3 165 4 129 1 198 2 228 3 261 4 246 1 201 2 174 3 186 4 159 1 150 2 150 3 171 4 129 1 168 2 177 3 183 4 150 1 144 2 153 3 165 4 144 1 159 2 138 3 129 4 144 1 150 2 171 3 147 4 144 1 114 2 150 3 141 4 138 1 141 2 150 3 132 4 129 1 135 2 168 3 180 4 141 1 135
AVG. Set No.
139.5 233.25 54 55 180 56 150 169.5 151.5 142.5 153 135.75 138 156 123 57 58 59 60 61 62 63 64 65 B 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 D 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2
N Dist. 4 129 1 129 2 135 3 123 4 156 1 135 2 156 3 153 4 150 1 132 2 156 3 144 4 132 1 108 2 105 3 144 4 123 1 123 2 132 3 147 4 144 1 129 2 129 3 138 4 132 1 99 2 111 3 138 4 114 1 162 2 186 3 159 4 204 1 201 2 186 3 195 4 180 1 132 2 216 3 189 4 126 1 126 2 156 3 156 4 186 1 135 2 129 3 141
AVG. Set No.
135.75 148.5 66 67 141 68 120 136.5 132 115.5 177.75 190.5 165.75 156 133.5 69 70 71 72 73 74 75 76 77 A 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 B 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 C 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 D 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2
C 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 Dist. 129 114 144 138 147 96 117 138 156 153 150 159 138 120 135 114 129
A-D are as dictated on the template. Noise conditions (N) are as follows: N1: Left hand, gloved, both feet on ground
N2: Left hand, barehand, one foot on ground N3: Right hand, gloved, one foot on ground
N4: Right hand, barehand, both feet on
ground
AVG. Set No.
135.75 126.75 150 124.5 78 79 80 81
APPENDIX C
Fractional Factorial Analysis
Data Trial I
Orthogonal array i A B C 1 1 1 1 2 1 2 2 3 1 3 3 4 2 1 2 5 2 2 3 6 2 3 1 7 3 1 3 8 3 2 1 9 3 3 2I
in
= -9 i=, nl 44.61321 n2 42.37396 n3 42.12193 n4 44.52436 n5 41.16996 n6 46.39036 n7 44.13120 n8 40.11632 n9 46.46260 m 1m
1
1=
i · ·
3~
1:3
77
+z2 +173)
;
mAl mA2 mA3 mB1 mB2 mB3 mC1 mC2 mC3 mD1 mD2 mD3 43.54488 43.03637 44.02823 43.57004 44.42292 41.22008 44.99163 43.70663 44.45364 42.47437 44.08193 44.29851 42.25420The fractional factorial method suggests the following values:
A B C D 2 3 2 2 D 1 2 3 3 1 2 2 3 1 yl 156 114 120 164 104 206 180 84 226 y2 178 142 142 158 118 194 162 92 190 y3 168 138 122 180 118 220 156 110 224 y4 182 138 130 174 120 218 150 146 208
Which returns an average flight distance of 189.5 in (45.55 db).
However, one of the values tested was:
A B C D
2 3 1 2
Data Trial II
Orthogonal array 1 2 3 4 5 6 7 8 9 A 1 1 1 2 2 2 3 3 3 B 1 2 3 1 2 3 1 2 3 D 1 2 3 3 1 2 2 3 1 yl 186 117 120 159 102 201 114 129 126 y2 183 129 126 135 183 162 150 135 156 C 1 2 3 2 3 1 3 1 2(k
YJ
I -17=:- F7, 9 :=1nl
n2 n3 n4 n5 n6 n7 n8 n9 m3
-- 11 +q2
+ 113) 45.57561 42.29234 42.33626 43.46882 42.00370 44.94002 42.51424 42.55410 43.61321 mAl mA2 mA3 mB1 mB2 mB3 mC1 mC2 mC3 mD1 mD2 mD3 43.25537 43.40140 43.47085 42.89385 43.85289 42.28338 43.62983 44.35658 43.12479 42.28473 43.73084 43.24887 42.78639The fractional factorial method suggests the following values:
A B C D
2 1 1
Which returns an average flight distance of 211.5 in (46.5 db).
y3 204 141 135 156 123 210 141 123 156 y4 189 138 147 150 129 153 138 156 186
APPENDIX D
aOFAT Analysis (Examples)
I)ata Trial I
Order of factor variation: D. A, B, C
1 D,A,B,C A 67 3 3 3 3 Try D2 68 Try D3 69 keep 67 3 3 3 3 keep 67 3 3 3 3 Try Al keep 13 13 1 1 1 1 Try A2 keep 13 40 2 2 2 2 Try B1 31 Try B3 49 keep 31 2 2 2 2 keep 49 2 2 2 Set 208 B C D 2 2 2 2 N 2 2 2 2 1 1 1 1 1 2 3 4 Dist 124 116 118 164 AVG 130.5 115.5 2 2 2 2 2 2 2 2 2 2 2 2 1 2 3 4 106 130 112 114 2 2 2 2 2 2 2 2 3 3 3 3 1 2 3 4 108 106 84 122 120 138 137.5 130 142 124 156 144 150 132 124 1 2 3 4 1 2 3 4 1 2 3 4 168 158 180 180 154 3 3 2 2 2 1 1 1 2 208 216 216
Try C1 keep 49 46 2 2 2 2 KEEP Try C3 49 52 2 2 2 2 3 3 3 3 3 3 3 3 1 1 1 1 3 3 3 3 C 2 1 1 1 1 1 1 1 1 1 2 3 4 1 2 3 4 170 171.5 172 178 166 120 137.5 160 148 122
Recommended solution:
A 2 B 3 D 1APPENDIX E
aOFAT Results
January aOFAT Breakdown
450 An . ... .... I t4UU . .. . ...-..-... . .. 350 300 350 .. . ... .... . 250 Er 00 i ..
_
~!~.
L....
5 0 . _ ... ... .... [ Seriesl 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81 Ending setsFigure 8. aOFAT results for Data Trial I
A B C D Frequency Distance (in)
Weighted 1 1 1 1 1 17 171 average: 198.9522 3 1 1 1 3 243 183 5 1 1 2 2 32 174 6 1 1 2 3 48 174.5 20 1 3 1 2 41 180.5 22 1 3 2 1 112 197 24 1 3 2 3 24 142 28 2 1 1 1 98 193 29 2 1 1 2 15 170.5 33 2 1 2 3 6 169 47 2 3 1 2 268 209.5 49 2 3 2 1 6 208 55 3 1 1 1 391 224.5 56 3 1 1 2 94 182.5 58 3 1 2 1 5 180.5 74 3 3 1 2 159 203.5 76 3 3 2 1 385 193.5 Total: 1944 100% avoid B2 100% avoid C3 83% avoid D3
93% avoid A1-D1 (second strongest interaction) (misleading?) 73% avoid B1-D1 (strongest interaction) (misleading?)
May aOFAT Breakdown . . . ... [
. vf--- A''n''
'''''''''''''
s
-1 3 5 7 9 -1-1 -13 -15 -17 -19 2-1 23 25 27 29 3-1 33 35 37 39 4-1 43 45 47 49 5-1 Final sets o Sedesl n ii 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81Figure 9. aOFAT results for Data Trial 11
A B C D Frequency 1111 1
4 1 1
2
1
19 1 3 1 1 20 1 3 1 2 26 1 3 3 2 28 2 1 1 1 29 2 1 1 2 31 2 1 2 1 46 2 3 1 1 47 2 3 1 2 49 2 3 2 1 55 3 1 1 1 58 3 1 2 1 74 3 3 1 2 80 3 3 3 2 Total: 100% avoid B2 98% avoid C3 100% avoid D3 70% avoid B3-D1 120 12 393 132 24 159 45 3 171 9 15 714 42 93 12 1944 Distance (in) 190.5 177.5 215.25 204.75 140.25 211.5 198 159.75 182.25 181.5 175.5 233.25 169.5 190.5 150 (strongest interaction) 800 700 600 , 500 = 400 ut 300 200 100 0 .. i Weighted average: 211.729 :u._ ... . _____ _ .. _______ _ ,._ __ ... .__.____... __ ^ - -- l - -- - n --- --Breakdown January, no B2 in starting set 400 300 , 200 100 n U O Seriesl n .... ... .'. _'. ..' nl 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 Final set
Figure 10. aOFAT results for Data Trial 1, excluding starting sets with B2 A B C D Frequency Distance (in)
1 1 1 1 1 2 171 3 1 1 1 3 182 183 5 1 1 2 2 11 174 6 1 1 2 3 39 174.5 22 1 3 2 1 64 197 23 1 3 2 2 17 192.5 28 2 1 1 1 12 193 29 2 1 1 2 12 170.5 33 2 1 2 3 6 169 47 2 3 1 2 208 209.5 49 2 3 2 1 6 208 55 3 1 1 1 252 224.5 56 3 1 1 2 73 182.5 74 3 3 1 2 112 203.5 76 3 3 2 1 300 212 Total: 1296 Weighted average: 204.272 fl
Breakdown May, noB2 in starting set 450 400 . 350 -300 . ' 250
r
200
.
, 150 100 --- - ---50 : 1 4 7 10 13 16Figure I 1. aOFAT results
ABCD
1 1 1 1 19 1 3 1 20 1 3 1 28 2 1 1 29 2 1 1 46 2 3 1 47 2 3 1 55 3 1 1 74 3 3 1 T¢1.
:.._.
_..
...
.
....
6
..
._.
19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 Final setsfor Data Trial 11, excluding starting points with B2
Frequency Distance (in)
Weighted 1 97 190.5 average: 21 1 328 215.25 2 106 204.75 1 134 211.5 2 3 189 1 140 182.25 2 8 181.5 1 420 233.25 2 60 190.5 otal: 1296 I i I I I I I 11 Sedesl: i 3.0046
Breakdown Jan, no B2 or C3 in starting set 250 200 ---. 150 : 2 100 50-0 - n ... ... n ... .... . ... . --- ---1--~~~~I O Sedesl 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81 Final set
Figure 12. aOFAT results for Data Trial , excluding starting points with B2 or C3
A B C D
3 1 1
1
3
6 1 1
2
3
22 1 3 2 1 23 1 3 2 2 33 2 1 2 3 47 2 3 1 2 49 2 3 2 1 55 3 1 1 1 74 3 3 1 2 76 3 3 2 1 Total:Frequency Distance (in)
157 30 53 11 6 170 6 171 68 192 864 183 174.5 197 192.5 169 209.5 208 224.5 203.5 212 Weighted average: 205.2465 __ B-- - -- _ _ Y ___~
~
E~~~~
--- S · L II II-Breakdown May, no B2 or C3 in starting set 300 250 >, 200 , 150 LL 100 50 0 o Series1 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81 Final set
Figure 13. aOFAT results for Data Trial 1I, excluding starting points with B2 or C3
AB
C
111 1
19 1 3 1 20 1 3 1 28 2 1 1 29 2 1 1 46 2 3 1 47 2 3 1 55 3 1 1 74 3 3 1 Distance D Frequency (in) 1 1 2 1 2 1 2 1 2 Total: 87 190.5 247 215.25 67 204.75 105 211.5 3 198 87 182.25 8 181.5 257 233.25 3 190.5 864 Weighted average: 213.0608REFERENCES
E ppinger, S. D., 1995, "Taguchi Airplanes." in Games and Exercises for Operations
Management J. N. Heineke and L. C. Meile, (eds.), Prentice Hall, Englewood Cliffs, NJ, pp. 213-224. 1995.
Frey, D. D., and Jugulum, R. (2005), "The Mechanisms by which Adaptive
One-Factor-at-a-Time Experimentation Leads to Improvement," accepted for ASME Journal of
Mechanical Design.
Frey. D. D., and Li, X., 2004, "Validating Robust Parameter Design Methods,"
DETC2004-57518, ASME Design Engineering Technical Conferences, September
28 to October 2, Salt Lake City, Utah.
Frey, D.D., N. Sudarsanam, and J.B. Persons, 2006, "An Adaptive One-Factor-at-a-time
Method for Robust Parameter Design: Comparison with Crossed Arrays via Case
Studies." DETC2006-99593, ASME Design and Engineering Technical
Conferences, September 10-13, Philadelphia, PA.
Frey, D. D., and Wang, H., 2006, "Adaptive One-factor-at-a-time Experimentation and
