Driving robust jet engine design through metrics

(1)

Driving Robust Jet Engine Design through Metrics

By

Steve P. Sides

P.E. State of Florida, No. 50098 Masters in Engineering (1991)

Florida Atlantic University B.S., Mechanical Engineering (1986)

Mississippi State University

Submitted to the Systems Design and Management Program in Partial Fulfillment of the Requirements for the Degree of Master of Science in Engineering and Management

At the

Massachusetts Institute of Technology February 2000

The author hereby grants to

Signature of Author

MIT permission to reproduce and to distribute publicly and electronic copies

of this document in whole or in part. _1-7/

C-Systems Desi gram

'Y999

Certified by

rey Department of Aeronautics and Astro tics Thesis Supervisor

Accepted

by-Thomas A. Kochan LFM / SDM Co-Director orge Bunker Professopjf Management

Accepted bi

Professor of

6/Paul A. Lagace LFM / SDM Co-Director Aeronautics and Astronautics and Engineering Systems MASSACHUSETTS 19S TITUTE

(2)

Driving Robust Jet Engine Design through Metrics

By

Steve P. Sides

Submitted to the Systems Design and Management Program on 15 December 1999 in partial fulfillment of the requirements for the Degree of Master of Science in Engineering and

Management

ABSTRACT

This thesis proposes a methodology and defines specific program level robust system design metrics (RSDM's) applicable at a jet engine program level. The metric applies from part, to subsystem to system (program) level. Roll up of the metric results across subsystems to the system level is explained. The results of the metric were found to agree very well with the results from a traditional Monte Carlo analysis of a specific case analysis. The RSDM predicted that 99.97% of the time requirements would be met where the Monte Carlo analysis predicted 99.98% conformance to requirements.

The metric's format is dimensionless; however, it provides insight and its results are absolute. In other words, if the RSDM Key Response = 1 we have a three sigma capable system of producing the desired Key Response within a specified tolerance width. For a RSDM Key Response -0.5 we have a six sigma capable system of producing the desired Key Response.

The format of the metric is simple although the roll up from subsystems is non-trivial. This metric and approach provide for a top down flow of requirements with a feedback of the capability to achieve the stated requirement.

The four value streams, (1) the product's users and or buyers (a.k.a. customers), (2) the manufacturer's shareholders, (3) the employees of the company and (4) the community in which the company operates, were employed in a stakeholder assessment. This approach identified five key attributes for application of the metric: (1) Affordability, (2) Dependability, (3) Thrust response, (4) Cost and (5) Engine weight.

P-diagrams are presented for a case study and an observation about P-diagrams is that they provide an easy graphical means (typically on one page) to capture the main drivers affecting a system which should be considered in the design parameter selection process.

It is recommended that the proposed robust system design metric (RSDM) and roll up method be employed throughout all phases of a program and that P-diagrams be incorporated into standard work to help instill robust design practices.

Thesis Supervisor: Daniel D. Frey

(3)

Motivation

The motivation of this thesis comes from my work experience as an engineer in the jet engine industry. I made a career (or at least 14 years of assignments) by being called in to solve

system or subsystem problems that were too often the result of not considering component interface uncertainties (also known as variances or noises). After completing a robust systems design course at MIT, I realized that the classical robust design tools existed in industry to design and produce robust products. However, a management tool that evoked these tools and culture seemed to be missing. Too often I've seen robust design practice give way to schedule and cost pressures in a program. The intent of this thesis is to bring awareness of a cause of a re-work cycle that may result from such schedule and cost pressures and how a robust design culture might minimize this rework cycle. The clich6 that "you are what you measure" is recognized as being powerful and thus the need for a metric that allows the indication of the products

robustness up front in the product development phase when concepts are being selected and the details of the architecture are being solidified.

The Re-work Cycle and The Need for Robust Design Cultures.

I gained an awareness of the rework cycle from course work in the MIT SDM program. This awareness helped me appreciate the value and need of a robust design. To share this appreciation, analysis results from the re-work cycle will be reviewed. Here, quality is used in the analysis to loosely represent robust design quality. Clearly poor quality work (errors) can cause re-work, but the lack of a robust design leads to product failure. The analogy being used in the following analysis is poor quality of work done is indicative of insufficient use of robust design practices.

(5)

now better understand the product usage. These are valid arguments; however, one might question are they signs of poor up-front robust design practices. Let's examine the rework cycle and the influence of quality on the amount of work done. Figure 1 provides a simplified

Vensim T model of the rework cycle.

Staff LeveF Productivity

Potential Work Rate

Work Accomplishmen

Quality-)<IM

HJ>

Work ToDo .a .UndiscovWerrDen

Work ToDo Rewrk Generation ReworkLWokDn

<TIMEL STEP1>

/ Project Finished Rework Discovery

Cumulative Time To Discover Rework Rate Of Doing Work Work Done

Figure 1. Simplified Model of the Re-work Cycle (from [Lyneis])

The items in boxes are stocks where the hourglass icons are rates or flows. All other items are auxiliary items in the model. A stock can be thought of as a bucket in which we are

(6)

into and out of the bucket. In the above model, the stocks are Work To Do, Undiscovered Rework, Work Done and Cumulative Work Done. The flows are Rework Discovery, Rework Generation, Work Accomplishment and Rate of Doing Work. This is a simplified model in that the effect of schedule pressures, employee moral, employee experience level and their

interactions that affect the re-work cycle are not modeled. The goal here is to provide insight to the impact of quality of work done on the amount of work actually done.

The Vensim model of the re-work cycle given in Figure 1 was used to investigate the impact of productivity, work quality, and rework discovery time on completion date and total

work done. Seven model runs were conducted in which productivity, work quality and rework discovery time were varied. The results of the simulation runs are provided in the following Table 1.

Base Estimated Target Staff Productivity Quality Discovery Cum.

Model Completion Work To Delay Work

Date Do done

Run # (month after (Tasks) (people) (Tasks/ (Tasks/ (months) (Tasks)

start) month/person) Tasks)

1 40 100 4 1 0.75 4 132 2 31 100 4 1.5 0.75 4 132 3 70 100 4 0.5 0.75 4 132 4 25 100 4 1 1 4 100 5 63 100 4 1 0.5 4 198 6 35 100 4 1 0.75 2 132 7 46 100 4 1 0.75 6 132

Table 1. Rework Model Results for Varying Quality, Productivity and Discovery Delay

In Table 1, estimated completion date is the date in which the project is predicted to be completed per the model run given the number of task to do, a staff level with specified

productivity and quality of work. The numbers of task and staff level values were held constant

(7)

and every task that a staff member did was perfect (100% quality), then it would take 25 months for the 4 people to complete the 100 tasks and the cum work done would be a 100 tasks.

Runs 1,2 and 3 examine the effect of productivity with quality held constant at 0.75. This means that 75 % of the work done was actually correct. With perfect productivity and 75% quality it will take 40 months to complete the project and in actuality 132 tasks will be done at completion. None of this was from being smarter or learning more from the product. It was simply the impact of quality that lengthened the project and caused more work to be done. Runs 2 and 3 show that as we improve or degrade productivity we simply decrease or increase

respectively the time required to complete the project. Productivity did not change the amount of work to be done in this simple, but insightful model.

Run 5 shows that if the quality of work done is 50% then the total work done to complete the project is nearly doubled (198 tasks). This increased work has an adverse effect on project completion time (increased from 25 months to 63 months). Comparing runs 1,6 and 7, indicates

that re-work discovery time primarily affects project duration.

Per the analysis, quality of work performed has a significant impact on the amount of work that ends up being done to complete the project. Productivity and discovery delay of rework tend to affect the duration of the project. The message here is that to finish as planned we must produce quality work or expect significant delays in project completion due to rework. My experience has been that we are surprised from product failures late in the development process when staff level and funding are decreasing. After gaining this view of the re-work cycle, I felt that poor robust design practices and cost and schedule pressures were overriding the growth of robust design cultures. This lead me to believe that program level robust design metrics are needed and that simple means to capture and communicate the noise consideration for a

(8)

Introduction

As they should, managers of jet engine development programs tend to monitor program cost, schedule performance and Technical Performance Measures (TPM's). Typical TPM's are weight, production cost, operability (stall margin), Thrust Specific Fuel Consumption (TSFC), performance retention (Delta TSFC with wear out) and projected field reliability (failures per

1000 flight hours). These metrics tend to be end item characteristics and except for performance retention and reliability do not drive product robustness. These types of end item TPM's are

important and needed; however, I submit that in addition to these metrics, TPM's that indicate product robustness are needed to drive robustness into the design phase of a program.

When a program's cost or schedule performance exceed some predefined amount from plan, corrective action must be defined and implemented to bring the program performance and plan in agreement. A TPM metric usually provides both an indication of Achievement To Date (ATD) and the Current Estimate (CE) of the TPM at service release. The ATD gives confidence in the CE. As the CE diverges from goal, corrective action is identified and implemented to drive the CE and goal level in agreement. These corrective actions are usually design

modifications. However, the action might be to negotiate a change in the product's requirements to match expected performance when design modifications are not feasible within the program's constraints.

A TPM is schematically depicted below in Figure 2. From this figure we see that at time "I", the ATD was not on plan and the corresponding CE did not meet requirements. This metric would indicate to management that corrective action beyond the original plan was needed. Efforts would then be initiated to identify and implement corrective action. At point "II" in the program monitoring, the corrective action had been implemented and the ATD reflected this improvement. The corresponding CE was projected to then meet requirements and the program

(9)

attention particularly in the development and verification phase of a program. Thus, if a

company desires robustness of their products, a metric should exist that indicates the robustness of the product being designed prior to being produced.

ATD Before Corrective Action ATD l *g Actio Plan Line After Corrective n Implemented CE Before Corrective Action Implemented

/(

CE After Corrective

/

Action Implemented Max acceptable I I I II

Time or Program Milestone

Figure 2. Schematic of a Typical Technical Performance Measure

To date, typical TPM's have driven programs to focus on the end product characteristics and do not necessarily result in a robust design. Having a robust design from the components to the subsystems to the engine system level will result in a design that is more tolerant to usage variations thus providing improved customer satisfaction. A robust system design metric (RSDM) of a particular Key Response might look like that given in Figure 3.

0

(10)

1.5 -t 0 1.0 Plan Line o 0.5 Max acceptable SI I I II III IV

Time or Program Milestone

Figure 3. Schematic of a Robust Design Metric Technical Performance Measure

Typically, nominal design solutions are defined both initially and in the corrective action phase. Later during verification testing or in the operational environments, the discovery that the nominal design was not robust drives another design modification. This is the rework cycle and is significantly driven by insufficient engineering and manufacturing practices that result in non-robust designs. Given more non-robust initial designs and avoiding inevitable design modifications, the rework cycle will be lessened, development surprises minimized and a robust product fielded. Fewer reworks are obviously less costly and a robust product in the field minimizes problems and gains customer loyalty. The challenge is to drive robust design processes into the product development process at the start of the design phase. Providing designers and program

management with robust design training will help, but this in itself may not change the culture. As with the clich6 "if you don't use it you lose it", it takes commitment from the entire

organization (shared vision) to drive robust design philosophy. Thus, we need metrics and best practices to drive a robust design culture. To do this it is proposed that program level TPM's that

(11)

indicate product robustness be identified. Also, design solution documentation should capture (perhaps in P-diagrams and words) how variability rejection has been achieved or considered. This is not to say that a Taguchi design of experiments needs to be conducted at every level but that noises or variances in interface conditions (loads, temperatures, manufacturing tolerances, etc.) are considered in the design solution. To what level (judgement, experience, test or analysis) this occurs is believed to be situational and not covered in this thesis. However, P-diagrams offer a graphical means to capture the factors involved and typically can summarize a system on one page.

Robust systems are built from robust sub-systems and metrics that capture this from the bottom level to the top level are needed. If the metric is too complex for managers or non-Taguchi engineers to grasp and see its use, it will not succeed as a useful tool. Again, the

(12)

Proposed Metric Features

The desired characteristics of the metric are:

" Simple-- technically grounded program managers must be able to understand and use the metric without having to take a robust design course or be a Taguchi design of experiments expert. The idea of understanding sensitivity and expected variance of a main driver or noise factor is however expected.

" Has clear meaning-- the metric must have clear meaning or at least its decomposition must have meaning. Simply a dB representation of signal to noise ratio might not give needed insight. For a given system, optimizing signal to noise ratios is significant and must

continue. However, when managing a program, dB from one system to another has different meanings or importance depending on the units used in the system at hand. Thus, the metric should have meaning from one system to another and the manager should not have to

recalibrate his or her scale of measure. The metric must have a tie to economic impact (design and development cost, reliability, warranty cost, and customer value).

* Independent of units--The metric should be absolute. That is the value of the metric is independent of the units used in the roll up of influences.

" Applicable at all levels of the

system--" Applicable at the system or program level such that program managers gain insight to the products robustness features.

* Applicable at the subsystem level such that managers of subsystems may use the same methodology in the design decision process of subsystems.

" Applicable at the component level, especially for augmenting the Life Cycle Cost (LCC) decision process discussed in this paper.

" Can be rolled up-- Ability to roll up results from component to subsystem to system without generating rework or requiring re-formatting of various metrics. Though the meaning must be clear and the metric simple, the roll up may be non-trivial.

(13)

Proposed Metric Format

The general form of the proposed robust systems design metric (RSDM) that meets the desired features is given below in equation (1).

Proposed RSDM form: a(Key Response) * 6a(noise factor)i (1)

c (Noise factor) i Allowable Tolerance Key Response

This format comes from work [Frey] conducted in his precision machine design doctoral work. This dimensionless format provides insight and is simple and appears to meet the

characteristics of the sought after metric. The Key Response is the feature of interest such as that of fuel consumption in a jet engine. The noise factors would be the various items that

significantly effect the Key Response. In our jet engine model this might be component

efficiencies, control accuracy, leakage and cooling airflow and so forth. The 6a noise factor is just

that, the six sigma expected variance of the noise factor. The allowable tolerance of the Key Response is the maximum allowable variation in the Key Response that is allowable and the product or process still meets requirements.

The first part of the metric, the partial derivative of the Key Response to a noise factor represents the sensitivity of the Key Response to one of the noise factors. The six-sigma (6a noise

factor ) part of the equation captures the expected variation of the noise factor. Dividing by the allowable tolerance of the Key Response results in a dimensionless parameter.

(14)

If we let SF.equal the significance factor then:

SF, = a(Key Response) / a(Noise factor) *(6Cnoise factor)1 SF2 = (Key Response) / a(Noise factor)2 *(6anoise factor)2 SF3= a(Key Response) / a(Noise factor)3*(6Gnoise factor)3

/ Allowable Tolerance Key Response / Allowable Tolerance Key Response / Allowable Tolerance Key Response

SFn = a(Key Response) / a(Noise factor)n *(6Cnoise factor)n / Allowable Tolerance Key Response

Then the robust system design metric (RSDM) of the Key Response can be expressed as given in equation (2):

RSDM Key Response= { (Ci*SFi)2 + (C2*SF2)2 + (C3*SF3)2 + (C4*SF 4)2 +. .(Cn*SFn)2 11/2 (2)

This form assumes that the noise factors are statistically independent and that the

response surface is linear. With time if it is found that noise factors have a strong correlation then these factors should be added and then root summed squared with the other noise effects. For noise factors that have interactions we would use the format of equation (3) where say factor one

and two have a strong correlation. The "C" values in the equation (2) and (3) represent the inverse of the traditional capability index of the Key Response when considering only that noise factor.

RSDM Key Response= { ((C₁*SFI)+ (C2*SF2))2 + (C3*SF3)2 + (C4*SF4)2 +...(Cn*SFn) 2 112 (3)

Process capability indices are typically used to measure or indicate a system's ability to manufacture a product within a given specification. The process capability index (C,) is a

(15)

dimensionless ratio of the amount of variation that can be tolerated and the amount of variation present. It is defined as given in equation 4.

U-L

CP6a (4)

Where U and L are upper and lower specification limits on a random variable and a- is the standard deviation of the random variable. Typically, the random variable represents a toleranced dimension that is used in a product's acceptance criteria. In this case, the random variable is the variations of the noise factor due to all influences be it manufacturing tolerances or noises from the environment or uncertainties in interfaces driving variance in the noise factor.

For a C, = 1 we would have a three sigma capability of producing the desired results and for a C, = 2.0 we would have a six sigma capability of producing the desired results.

[-Tolerance Width L 3a U-L 3a+3 3 U U-L 6a

Figure 4. Graphical Depiction of the Capability Index j

(16)

Thus, we could represent the RSDM in terms of Cp as given in equation (5). However, if we let C1 oc l/Cpi oc RSDM (Key Response)1 we maintain the RSDM terminology from component to

subsystem to system and have equation (6), which is the same as equation (2). This allows us to talk in terms of the RSDM's from one subsystem's response to the next rather than talking in terms of the inverse of the capability index (1/Cp) and then the RSDM of the key response.

RSDM Key Response = { (3/Cpl*SF1)2

+ (3/Cp₂*SF2)2 + (3/Cp3*SF 3)2 + (3/CP4*SF4)2

+...(3/Cpn*SFn )2 112 (5)

RSDMKey Response { (Ci*SF1)2 + (C2*SF2)2 + (C3*SF3)2 + (C4*SF4)2 +...(Cn*SFn)

2

}1/2 (6)

By considering all the main noise factor effects we roll up the combined effect of these factors and the result is a capability index of the Key Response considering all these noise factors. Here we have denoted this rolled up capability index as RSDM Key Response. Note that the

RSDM is inversely proportional to the classical process capability index, Cp. Thus, if RSDM

Key Response-= 1 we have a three sigma capable system of producing the desired Key Response

within a specified tolerance width. For a RSDM Key Response= 0.5 we have a six sigma capable

system of producing the desired Key Response. For the format of equation (6) the challenge is identifying the main noise factors to consider and their effects and how to roll up their effects from level to level (component to subsystem to system).

(17)

Table 2. Meaning Of RSDM

From Table 2 we see that for a RSDM value of 3, we would have a one sigma capability of meeting the specified tolerance width and should expect 32 out of 100 systems will not meet the tolerance width requirement (Note: RSDM= 6a / (2*Cp*a) ). We desire the RSDM to be small (smaller the better). Achieving a RSDM of 0.5 means one is at 6a or in other words less than 2 in a billion times the system not meet the specified requirements.

RSDM Number Exceeding Process Capability

Tolerance Width 0.5 < 2 / 1,000,000,000 6a 0.6 < 0.6 / 1,000,000 5a 0.75 < 63 / 1,000,000 4a 1.0 < 3/1,000 3a 1.2 < 12/1,000 2.5a 1.5 < 46/1,000 2a 2.0 < 134/1,000 1.5a 3.0 < 32/100 la

(18)

Roll-Up Model

We will now consider the following model to discuss a method to roll up the noise factors to the RSDM. C1.1 1.11 C1.1.12.1 C1.1 2.1.1 C1.1.22.1 Fan~Ca Afo C Fn Psietio Rotor Spe A iClearance

S A/C Drag C1111 Engine

Cuu PowerCu

A/C Weight - -- + Setting

-- +A/C Bleed C1.12. Airflow

Engine Coo~ling -olinge& u

Arflow Airflows " C1.23 Leakage Airflow .1. C 1.13. ---> Fan Efficiency __Compressor Efficiency ---3 -1. Combustor Efficiency -- -SD CLLI4 C1.1. .1.6 HPT Efficiency --- Engine C..5 Efficiency LPT Efficiency ----Aug. Efficiency

---Nozzle Efficiency IC1.13.

Engine RSDMrsFC

TSFC CIA

Figure 5. Schematic of Robust System Design Metric Decomposition for TSFC.

If we now apply equation (6) to Figure 5, then the RSDM for TSFC can be expressed by equation

(7). RSDMTSFC = ( (C11..1 * SF1.1.1) 2 + (C1.1.2* SFI.1.2) 2 + (C1.1.3* SF1.1.3)2 1/2 ₍₇₎

(19)

TW power setting

TW TSFC

(8)

a

TSFC

a Power Setting

In equation (8), "TW" represents the tolerance width or the upper limit minus the lower limit of the denoted parameter. In equation (7), "CI.I., " is the RSDM for the power setting and represents the capability to achieve the power setting tolerance width (TW) requirements. If "C1.1.1 " = 1 then we have a 3a capability of meeting the specified limits (TW). "C1.1.2 " is the RSDM for meeting the Leakage and cooling airflow requirements and "C1.1.3 " is the RSDM for the engine efficiency.

Also,

TW Leakage & cooling airflow

TW TSFC

TW Engine Efficiency

TW TSFC

* t TSFC

a

Leakage & cooling airflow

* a TSFC

a

Engine Efficiency

Again referring to Figure 5, we would determine the RSDM for engine efficiency to be as follows.

RSDM Engine Efficiency = ₁.1.3 = ((CI.. 3.1 * SFI.I.3.1)2 + (C1.1.3.2* SFI.I.3.2)2 + ... +(C1.1.3.7

* SFI.I.3.7) 2 _)1/2 where, SFI.1.1= (9) (10)

(20)

where, SF 1.1.3.1 = TW Fan Efficiency TW Engine Efficiency SF 1.1.3.2= TW Comp. Efficiency TW Engine Efficiency SF 1.1.3.7= TW Nozzle Efficiency TW Engine Efficiency *

a

Engine Efficiency

a

Fan Efficiency *

a

Engine Efficiency

a

Compressor Efficiency *

a Engine Efficiency

a Nozzle Efficiency

The "CI1.3.1 ,C1 1.3.2 ,C1.1.3.3 ,C1.1.3.4 ,C1.I.3.5 ,C1.1.3.6 ,CI.1.3.7 " are the RSDM's for the Fan, Compressor, Combustor, High Pressure Turbine (HPT), Low Pressure Turbine (LPT), Augmentor and Nozzle respectively and represent the capability to achieve their allocated

tolerance width (TW) requirements at a given condition. Note that Engine Efficiency could have been drawn directly into the engine TSFC block in Figure 5. The figure was drawn in the manner provided to show the roll up method. Thus, note that the decomposition of the Key Response characteristic like most decomposition is not unique. This means the various components, subsystem and system teams should define, document and ensure consistency amongst the decomposition of the RSDM's.

The Robust System Design Metric (RSDM) for Fan efficiency is given as follows based on the Figure 5 decomposition.

RSDM Fan efficiency = C1.1.3.1 = ((C1.1.3.1.1 * SFI.I.3.1.1)2 + (C1.1.3.1.2* SF1.I.3.1. 2)2 + ... +(C1.1.3.1.6* SF1.1.3.1.6)2)1/2

(21)

A significant question to ask is "how robust is the Fan efficiency value that is being counted upon?" The RSDM Fan efficiency answers this question. If RSDM Fan efficiency = 0.5, then

we would expect a 6 a capability of meeting the desired Fan efficiency tolerance width. In other words less than 2.0 times out of a billion would Fan efficiency exceed (go out side) the TW allocated to the Fan.

Another observation of this method is that the TW can be assigned and held constant. The capability to achieve the specified TW will vary depending on the design decisions selected through the development process. The allocation to the subsystem or factors (in this case FVV position, hardware tolerances, etc) can be set and not constantly changed. For example, if +/-2

(plus and minus two) degrees variability or accuracy is allocated to the FVV position, this

allocation can be held as a constant requirement through the development process. The ability to achieve this requirement is represented in the RSDM for FVV position. Managers can then make decisions that improve the capability (RSDM) to meet this requirement. This approach is a top down flow of requirements with a feedback of the capability to achieve the stated requirement.

Let's assume that to meet TSFC requirement that a Fan efficiency requirement of 94 +/- 2 % at a given flight condition (power setting, altitude, Mach number) has been defined. This results in a TW Fan Eff. 4% (+/- 2 %). One might then allocate the following TW's on Fan

efficiency drivers:

TW FVv 4 degrees (+1- 2.0 degrees)

TW Airfoil geometry = 1 degree (+/- 0.5 degree)

TWAiflow = 2% (+1 %)

TW Fan Press Ratio = 3 % (+1- 1.5 %)

TW Rotor Speed = 500 rpm (+/- 250 rpm)

(22)

Then (equation (11)), RSDMFan efficiency

{

C ₁..3.1.1 C 1.1.3.1.2 C 1.1.3.1.3 C 1.1.3.1.4

[C

1

.

1

.

3

.

1

.

5 C1.1.3.1.6 40 * 4% 10 4% 2% 4% 3 % 4% 8Fan Eff.

aFVV

position * aFan Eff 2 Airfoil GeomJ * Fan Eff 2

a

Fan Airflow. * aFan Eff 2

a

Fan Press RatioI

500 rpm * Fan Eff 4 %

a

Rotor Speed

]

2

* 4 mils * aFan Eff 2 4 % Nip Clearance 2

+

The partials (aFan Eff. /

aFVV

position, aFan Eff / aAirfoil Geom., etc) could be stored in a table or matrix and only updated at significant configuration updates. These sensitivities could be based on historic data, computer models, or initial estimates. Using a matrix or table approach of sensitivities would eliminate the constant task of defining these sensitivities. This is a common approach used in Life Cycle Cost trades in the military jet engine world. Given these sensitivities and corresponding TW's and subsystem capabilities (C1.1.3.1.1, etc) the RSDM for

Fan efficiency could be computed or estimated per equation (11) above.

The prior discussions have provided a method for rolling up RSDM from various components to the system response of interest. We will now focus on the FVV positioning capability for further analysis and explanation of the method.

(23)

FVV Positioning- Model vs. Monte Carlo Analysis Results

A MathcadTM model of a jet engine FVV actuation system was built to calculate the RSDM Fvv Position and compute results using Monte Carlo analysis on the main drivers. The model and results are provided in appendix A.

Load N1 sensor T2 sensor Electronics error (LVDT) Linkage tolerances RSDMVFV Position FVV Position 1.1.11 (refer to figure 5.) TWfvv = 4 degrees

Figure 6. A RSDM Decomposition of FVV Positioning

Equation (12):

RSDMFVV _Position₌

CL TWload 6fvv 2

TWfvv 6load/

Drivers: Load error

2fv 2 + Cn- T- n1 TWfvv _6n1 NI error + Ct2 TWt2 Uv 2 TWfvv 6t2) T2 error TWlvdt 8fvv 2 + Celec TWfvv 6lvdt Electronics

/

TWlink 6fv\ + ClinkT -- 2 TWfvv 8linv Linkage tolerances

(24)

of the drivers as given in equation (13). The capability to achieve the stated TW was set to three sigma. Thus the "C" values were all set equal to 1 in the RSDM equation.

Equation (13):

RSDM calculation for a tolerance width requirement of 4.0 degrees on FVV positioning:

RSDM:= (i.0 2 2 2 500 0.09 4 250 100 2.98 + 1.0-- -4 100

S

4.5 1.18 + 1.0-- - . 4 4.5 0.020.275) + 1.0-- --4 0.01 RSDM = 0.82

A Monte Carlo analysis was run for 10,000 actuation systems in the model. The results are shown in Figure 7.

-6

Monte Carlo Results for FVV Positioning

-5 -4 -3 -2 -1 0 1 2 3 4 5 6

,6J

binerr

Error from Request (degrees)

1) ₂ 0.50.24 + 1.0--.---4 0.25 -3448,400 300 u FVVbin . 200 .0 100 ,.A

(25)

0-From Figure 7, the Monte Carlo result shows there is a bias in the error. This error is from the wind up in the system due to loading the linkage system. This bias could be removed by accounting for it in the control schedule in the engine electronic control (computer). The mean error was found to be -0.9 degrees and the standard deviation was 0.54 degrees.

The RSDM method indicates we have a 3.6 sigma capable system of meeting the 4 degree FVV tolerance width on FVV positioning as indicated by the 0.82 value of the RSDM (see appendix A for these calculations). Thus, we would predict that 0.027 % of the actuation systems would not meet the 4 degree (+/- 2 degree from nominal FVV) positioning requirement per the RSDM method. Presented another way, the RSDM methodology would estimate that 99.973% of the systems would meet requirements.

The Monte Carlo results of the program resulted in a standard deviation of 0.54 degrees. For a clearer comparison, we simply divided the Monte Carlo six sigma capability (6*0.54 = 3.2

degrees) by the allowable tolerance width to compute the RSDM value via the Monte Carlo method. The RSDM as computed by the Monte Carlo analysis equals 3.2 degrees divided by 4

degrees tolerance width, which equals 0.8. This compares very well with the RSDM methodology that resulted in a RSDM of 0.82.

Thus, the Monte Carlo analysis resulted in an estimated RSDM of 0.8. This equates to a 3.8 sigma capable system or an estimate that 99.982% of the systems built would meet

positioning requirements.

(26)

requirements are from a 35000 lb thrust class engine and of the 28 system built to date all have met the stated positioning requirements.

This FVV example analysis provides closure on the RSDM methodology as a viable means to define robust system design metrics (RSDM's) for program management and decisions. It validates both the metric and roll up method of the metric. The results of the proposed method compared well (within 0.01%) of traditional analysis tools (Monte Carlo).

(27)

Stakeholder Assessment - Identification of Key Jet Engine

Responses

The form and roll up methodology for a robust design metric has been provided. Now key responses to focus upon at the jet engine program level will be identified.

The four value streams [LAI Oct 1998 Plenary Workshop] often to be considered in framing ones decision about a product are (1) the product's users and or buyers (a.k.a.

customers), (2) the manufacturer's shareholders, (3) the employees of the company and (4) the community in which the company operates. In this context the term product is used loosely to also capture a service. My experience has been that most all consideration regarding the product can be mapped or aggregated to one of these four value streams. When making decisions, these value streams should be considered for the effect of the decision on them. If any of these is offset or favored too much, then problems will arise.

The shareholders have value in the product from their interest to make money and see the earnings per share and dividends continually increase. The customer of the product looks for value in the product or they will not purchase it. The employees look to the product for value capture to provide for their jobs and financial well being. And lastly, the community in which the product is made or the service provided is affected by the value the product provides. The community effect is typically the effect of the employees living and doing volunteer work in the community as well as the income the product brings to the area. Also spouses and children working and doing volunteer work in the community brings income to the area in addition to that from the product itself.

(28)

Product Buyer/Owner: Affordability or costs of ownership and product performance are the main items of interest to the buyer. Cost of ownership is driven by product price, maintenance price (parts and labor price), durability (how long will it last and how often it requires

maintenance) and fuel burn efficiency (how much fuel is used during operation of the product). Of course the buyer / owner is concerned that the product meets performance specifications such as thrust, thrust response, range, observables, environmental regulations, etc.

Product User: Assuming a fighter jet, the main user is the pilot. First and foremost in the pilot's mind is that the engines are dependable. In other words they work no matter what the pilot requests of the jet. Then performance of the engines becomes critical. Engine thrust response

and recently thrust vectoring is important for maneuvering the jet. Thrust Specific Fuel Consumption sets the range of the jet without aerial refueling.

A secondary but also significant user of the engine is the airplane manufacturer. Here engine weight and interface conditions such as power take off (mechanical and/or bleed air) electrical connections, engine mounting and loads, fuel and hydraulic interfaces and engine inlet aerodynamic interfaces become significant in addition to the parameters with which the pilot is concerned.

Manufacturer Shareholders: Major US jet engine company shareholders. The shareholders are interested in the bottom line (earning per share). They desire to increase revenues or reduce cost and to improve or maintain the bottom line. Revenues are increased by market share or market growth. Profits are increased by reducing the cost of goods sold and earnings per share increase with increased revenues and constant number of shares. Of course I've over simplified the dynamics of the bottom line.

(29)

Employees: Here the employees are charged with the product development and manufacturing. From product positioning in the market to the architecture of the product and its resultant development cost, warranty cost and performance. A product development framework that was developed during the MIT Systems Architecture Fall 1998 class is given below and provides a framework that may be used in architecting and designing the product and conveys the effort to be captured by the employees:

A Product Development Framework

Regulation

-Standards -Environmental Corporate strategy -ROI

-Make to order vs. stockin -Platforms -Affordability g Manufacturing -Capabilities -Production Plans Customer & Field Support

Concept _{-Product Delivery Plans}

-Service Contracts

Architecture -Repair Plans

m,

Market Data 10 Need - Goal - Function 4 - - User

-Benchmarking

--Expectations

-Customer Boundaries (Risk,

t

LCC, Schedule, Physical)

e SDevelopment & Validation

Market Strategy I-Testability

-Competitions strategy -Test Requirements / Plans

-Product line, price,

promotion and placement _{Product Disposal}

Technology

-Trends

-Extensibility Up stream influences Down stream influences

Need - Goal - 4 Concept - Function - Form -4 Timing 4-- Customer/User

Why the What the Overall How the where are When things Who buys it and

system is system arching means system the chunks do or need to who uses it

being built does to the end behaves occur

Concepts

Expand design space with many disjoint concepts Steve P. Sides 12 Oct 1998

Look to nature for potential concepts Reference: E. Crawley's SDM Fail 98 SA Lecture Material

Figure 8. A Product Development Framework

Community: Income and jobs brought to community, housing and educational impacts, and so forth are the items of consideration here. These are not in the main stream of this paper and

(30)

From the stakeholder assessment and awareness of the product development framework, we can identify the key jet engine parameters that should be focused upon at a program level to best position ourselves to satisfy stakeholder needs. From the buyer/owner assessment we select affordability. From the user (pilot) assessment we select dependability, thrust response and range. Range has influences from the drag of the airplane and fuel storage capability to engine fuel bum efficiency. Thus, from an engine perspective we will focus on Thrust Specific Fuel Consumption (TSFC) in regards to airplane range. However, as we will see TSFC is a subset of affordability in that it represents the cost of operation. Thus, we will fold TSFC under

affordability. From the airplane view we add engine weight to our considerations. From the shareholders we select cost which captures production, development and warranty cost. We focus on cost since market pressures will set the price of the product and the resultant profits. Therefore, by focusing on achieving cost goals, the marketers may feel confident in positioning the product from a price viewpoint. The employees get the opportunity to architect, design, develop and manufacture the product that best meets these needs while deploying the RSDM in

the concept selection phase.

In summary we have the following five key attributes to focus upon:

1. Affordability 2. Dependability 3. Thrust Response 4. Cost

(31)

Decomposition of the Key Responses:

RSDM key characteristic

Affordability (n level)

Decomposition Level Down (n+2 is two levels down) n+1 Development Price n+2 Test Requirements New Technologies Staff Size Product Price Part Assembly Maintenance Price Part MTBM LRU rate Fuel Cost Fuel price Fuel used n+ 3 Rig Test Component Test Subsystem Test Engine Tests Flight Test Materials Components Processes Analysis Required Test Required Schedule Labor Material Overhead Labor Material Overhead Labor Materials Overhead Labor Materials Overhead Labor Materials Overhead

Typically assumed constant Missions TSFC Dependability Thrust Response Cost Weight

MTBM (mean time between maintenance)

MTBF (mean time between failure)

Precision of Performance

Fan Stall Margin Compressor Stall Margin Rotor Weight

Temperature Limits

Control Loop Bandwidth Product Cost

Development Cost Warranty Cost

Maximum Engine Speeds Temperature Limits Manufacturing Tolerances Materials

(32)

Jet Engine Key Responses

High performance and low cost of ownership are desirable for jet engines both in the commercial and military markets. Today a jet engine has a relatively long life cycle (20 - 30 years). Thus it is desirable for it to be affordable to own and operate. A jet engine is not unlike a car in that we want it to be reliable, be affordable to own and last a long time, get good gas

mileage and accelerate or decelerate fast when needed.

Some of the top parameters monitored in the jet engine industry are thrust response, Thrust Specific Fuel Consumption (TSFC), dependability of the product, weight and cost of the product. From the stakeholder assessment the following five attributes were identified: (1) Affordability, (2) Dependability, (3) Thrust response, (4) Cost and (5) Engine weight.

From the decomposition of affordability (Figure 9), one might conclude that maintenance cost and fuel cost will be the long-term affordability drivers after the development program has been planned. Of interest is that MTBM affects both affordability and dependability.

The fuel price is typically not within the control of the engine manufacturer or product buyer. Therefore, from a product development view we focus on Thrust Specific Fuel

Consumption (TSFC). Throughout the planned missions the engine will spend significant amounts of time at flight idle, cruise (a part power setting), military power and if equipped with augmentors it will bum a lot of fuel when in augmentation. Thus, monitoring and managing the TSFC of the engine at these key power settings seems appropriate. Today, approximately 80% of the fuel bum occur at cruise conditions for aircraft. Therefore, it seems prudent to assess the robustness of TSFC at a cruise power setting. Perhaps TPM's based upon nominal performance should be monitored for all of these critical power settings, but as a minimum a RSDMTSFC at cruise

(33)

Thrust Specific Fuel Consumption is important in that it represents how much fuel in used for a given engine power or thrust setting. Poor TSFC will decrease the range and mission effectiveness an aircraft can achieve without aerial refueling and increase the cost of ownership in that more fuel is required to operate the vehicle. The first level noise factors that affect TSFC are given in equation (14) and are decomposed to lower levels in Figure 5 in prior sections of this report.

TSFC = f(Engine Efficiency, Secondary System Air Cooling Flow, Power Setting) (14)

Dependability captures reliability, durability and repeatability of the product. We want the engine to be reliable in that it performs the intended function when needed and durable in that these functions may be performed over and over without extensive maintenance. Also, we want it to be repeatable or have precision in its performance. One method to make a product reliable is by adding redundancy. Another method would be to make the design robust in that redundancy is not required. Typically adding redundancy adds maintenance cost in that more parts are available for failure and thus more maintenance. A robust design rejects noises and disturbances and thereby provides some precision in performance (i.e. meets required

performance within tolerance width requirements.). The first level noise factors proposed for dependability are given in equation (15).

Dependability = f (MTBF, MTBM, Repeatability) (15)

Thrust response refers to how well the engine responds to pilot input via the throttle. For large throttle input the pilot desires the engine thrust to change a significant amount in a short time. For small throttle input the pilot desires accuracy. Small throttle input is used for

(34)

maneuvers such as aerial refueling, formation flying or landing. The noise factors that effect thrust response are given in the following equation (16).

Thrust response = f(Fan Stall Margin, Compressor Stall Margin, Rotor Weight, (16) Temperature Limits, Control Loop Bandwidth)

Product cost refers to the manufacturing cost or the cost required to produce the product, development cost and warranty cost. The company wants to keep cost low for competitive and profit reason. The production cost captures the materials, overhead and standard cost to produce the product. Other drivers that increase or decrease cost are tolerances. Tighter tolerances tend

to drive manufacturing cost up as does exotic or specialty materials. Of course the number of units to be produced (economies of scale) affect the product cost. However, a RSDM could assume a fixed set of sales for management reasons. The baseline could be adjusted, as

management deemed necessary for significant changes in volume. The intent of the RSDM here would be to understand the robustness of the product cost estimated. For development cost the intent would be to capture the likelihood that the program could be developed within the planned schedule and assets. The intent of the warranty cost part of the metric would be an assessment of the capability of the product to meet performance (TSFC) and durability requirements. Thus the RSDM for TSFC and dependability would feed into the warranty RSDM. The proposed first level noise factors for product cost are given in equation (17).

Cost = f (Material Cost, Manufacturing Cost, Warranty Cost, Development Cost) (17)

The product's weight is initially scaled from existing engines. As the design of the product solidifies the weight estimate is refined. Thus, a noise factor would be maturity of the design. The less mature the more likely the weight target will be missed. The manufacturability of the design puts the weight at risk. Difficult to make parts due to either materials or geometry with large tolerances might drive weight variation. Duty cycle variations either from actual use

(35)

or from engines to engine performance variation could drive rotor speeds and component

temperatures, which eventually affect engine weight. Exotic high-risk materials for temperature, strength or weight reason carry some risk to engine weight. The proposed first level noise factors for engine weight are given in equation (18).

Weight = f (Materials, Geometry Complexity, Duty Cycle Variation) (18)

The above five Key Responses are offered for program level RSDM's. The intent of these Key responses and their first level noise factors has been provided. For TSFC, an example decomposition (see Figure 5) has been provided. The completion of the TSFC decomposition and the development of the remaining five responses is left to the user. The decomposition of the responses is not unique. Two different companies or even people for that matter are likely to decompose the proposed responses differently and both are correct. However, some

decomposition may be easier to work with and better map to analysis and how the company is organized. The decomposition of these responses is left to the user and is not considered a trivial or insignificant task.

(36)

P-diagrams as Part of Robust Design Culture

A case example will now be presented to provide an appreciation of how powerful a simple tool such as the P-diagram could be if included in standard work or practices. Standard work in this case refers to the work instructions that guide engineering in the product

development phase. It ranges from checklist of analysis to be done to modeling tools and analysis that should be conducted or used all the way to expected manufacturing capabilities. Standard work often captures typical test needed in the design and development of the product or process. Here we will be focusing on the up front design phase. Often standard work is referred to as the guide to best practices and process.

The P-diagram is powerful in that it is graphical, easy to understand and encourages one to consider the systems at hand. A generic P-diagram as presented by [Phadke] is presented below in Figure 10.

Noise Factors

Product or Process

Signal Factor Response

Control Factors

(37)

The signal factor is set by the system user or operator and is the desired or intended response of the system. Noise factors are those parameters that can not be controlled by the designer or may not be well known. This may be the case with interface conditions between components or manufacturing variability. The designers may agree upon design load interface conditions; however, there may be some variation in the exact load conditions. Another variation or noise might be the actual environmental conditions in which a product might operate (dirty vs. clean, hot vs. cold, etc.). The main noises in a system and their impact on the system need to be considered.

The control factors are the parameters that can be specified by the designer. Control factors are sometimes referred to as design parameters. An example of a control factor would be the mechanical advantage in an actuation system. The response is the resultant output of the process or product given the signal factor, noise factors and control factors.

Design errors during a jet engine's fan inlet guide vane actuation system design will now be examied as an effort to show how a P-diagram might have prevented the errors by bringing awareness to the design situation. The fan inlet guide vanes will be referred to as FVV's (Fan Variable Vanes). The FVV's are modulated via an actuation system in response to signals from the electronic engine control. The engine electronic control is basically the computer that is on board the engine that determines requested poistions for engine valves and actuators given thrust request from the airplane computers. The modulation of the FVV's results in turning of the air entering the fan. The intent of this turning is to optimize the angle of attack of the air to the fan rotor and pressure ratio across the rotor to maximize fan efficiency and fan stall margin. As the

FVV's close, the aerodynamic load significantly increases on the vanes. This load is then tranferred through vane arms to a synchronization ring to the linkage system and then to the actuator. This system is schematicaly depicted in Figure 11 with many details omitted for clarity of view.

(38)

Actuation Linkage Sync ring Vane arm FVV Airflow Actuator Rotor 1 FVV Rotation

Engine Center Line

Figure 11. Fan Variable Vane (FVV) Actuation System Schematic

A shortfall in the systems design was identified during engine testing at high load conditions. The FVV's did not adequately follow the engine control requested position (vane angle) and slew rate (degrees per second) as depicted in Figure 12.

0 C,, 0 Actual Response equest Signal Expected or Desired Response 0 Time - seconds 5 . R

(39)

The shortfall in FVV tracking resulted in a delay in the engine development program. A "tiger team" of experts was formed to investigate the issue. The root cause was found to be a failure to consider the noises or potential variations in the actuation system and interface loads during the design process. Thus, the system was underdesigned. Corrective action involved maximizing the mechanical advantage of the actuation linkage system, lowering system friction through material selection and the addition of liners for FVV trunions. This event could have resulted in the redesign of an actuator which is typically an expensive and long lead item.

Fortunately, multiple minor design modification could be identified and implemented to result in an acceptable system. A P-diagram of the originial design might look like that in Figure 13 had one been defined.

Signal Factor =

FVV angle request and slew rate request

Noise Factors were not considered

Response = poor

FVV Actuation _- _{tracking of requested}

System slew rate and position

Control Factors: * Actuator size * Mechanical advantage of linkage system * Materials * Part tolerances * Number of flaps * Size of flaps

(40)

Had a P-diagram that considered the noises or variations up front been considered, the likelihood of this design error would have been greatly minimized. The desired P-diagram is given below in Figure 14.

Signal Factor =

FVV angle request and slew rate request

Noise Factors:

" Aero load prediction errors

* Friction factor variation due to material variation and dirty vs. clean

environment

" Mission or Fan variation driving vane angle travel (aero load) requirement * Deflection load errors

I

Sensors and electror FVV Actuation System Control Factors: ic errors Response = acceptable positioning and tracking capability * Actuator size * Mechanical advantage of linkage system * Materials * Part tolerances * Number of flaps * Size of flaps

Figure 14. P-diagram for FVV Actuation System With Noises

The learning in this case was twofold. One was that a P-diagram allows a company to simply capture generic design considerations in a system, subsystem or component design. These considerations and awareness might lead to a significant reduction in program risk. Had this error not been correctable with minor design modifications, the program would have

(41)

incurred a significant schedule delay and cost impact (12+ months and $3+ million dollars in redesigned hardware) for a more significant change in actuator requirements.

The second learning is that not only are the noises or variances to be considered, but the design parameters or control factors must be selected to ensure the desired response is obtained even under the given noise factors. This is the basis of robust design methodology. We desire to select the design parameters of components or subsystems such that they are tolerant or robust at the interface points. In classical robust design, this means selecting the design parameters that result in the highest signal to noise ratio. That is given the design concept, we select design parameters to maximize the likelihood that the desired response will be achieved given noises or variances. If an acceptable response can not be achieved with the given concept, then we search for a concept or process that will produce acceptable response.

RSDM Could Augment Life Cycle Cost Trade Analysis.

In jet engines, as with other systems there are constraints that will bound the design. These constraints could be cost, size or envelope to operate within, manufacturing cost or capability, maintenance cost and performance requirements to name a few. One that is

constantly bumped against for vehicles that must fly is weight. The engineering teams work their design concepts and eventually come to several options, which would meet requirements. To select the best system, a Life Cycle Cost (LCC) analysis that considers the performance,

development cost, manufacturing cost, weight, maintenance cost and so forth is conducted. The system with the lowest Life Cycle Cost is selected. LCC is a significant decision metric for jet engines since most engines are designed to last 20+ years. Thus, affordability over the life of the product or cost of ownership is significant to the owner or operator of these products. Thrust Specific Fuel Consumption (how much fuel is used to produce a given amount of thrust), reliability and maintenance cost tends to be significant drivers in the LCC analysis. Though this

(42)

is done based on nominal conditions. A fundamental assumption is that the concept will perform as desired. Therefore, to ensure the validity of the LCC analysis tool a method to ensure the product will perform as desired is needed. A robust design methodology or culture will go a long way in ensuring this. The use of P-diagrams may help cultivate this type of thinking amongst the engineers and designers; however, P-diagrams will not drive program managers to manage in this culture. What is needed is a program metric that provides a measure of product robustness such

that decisions can be made to ensure the performance of the system will be met. Also, this metric will ensure the LCC decision tool is valid.

An example life cycle trade table is given below. Perhaps the addition of the RSDM on the impact of product robustness would make the LCC decision more valid as indicated.

Life Cycle Cost Impact, millions $

Development Part Cost TSFC Weight Maintenance Total LCC Robustness

Option Cost Impact

Base base base base base base base base

A 20 300 -100 -325 -20 -125 ?

B 15 150 -20 -200 0 -55 ?

C 32 168 -150 -100 10 -4C ?

D 18 180 -75 -175 -35 -87 ?

E 25 225 -125 0 -20 105 ?

Table 3. Sample Life Cycle Cost (LCC) Table for Concept or Option Selection

For the above Table 3, one would select option "A" since it has the lowest value in LCC. Recall that affordability or cost of ownership is a key response; therefore, we desire a low value of LCC. Note that life cycle cost is not necessarily today's cost but represents the cost incurred over the life of the product. Thus, a part cost increase of a few thousand dollars could end up as

(43)

part cost to LCC. This paper does not cover the development of such factors but recognizes different industries have such factors. Also, the values in the above table are deltas from a defined or selected base configuration or option. Apparently from Table 3, we conclude that for twenty million dollars in development cost and some increase in part cost, a significant

improvement in Thrust Specific Fuel Consumption (TSFC) and a weight reduction can be realized for option "A". This LCC method of selecting a configuration to go forward with is powerful and has proved extremely beneficial in the jet engine industry. What is missing from this method is the option's impact on robustness. Option "A" might be the best considering

nominal operation, but if variance were introduced it might fall apart driving high maintenance cost. Even worse yet would be if the variances caused the product not to work at all.

These tools are just that- tools to aid our decisions. Users of these tools are reminded not to get caught up in the exactness of the LCC or the RSDM indication at this stage. At this stage only a relative comparison amongst the concepts is needed. Once the configuration of choice has been identified then one might refine the LCC estimates and RSDM indicators for reporting and management reasons. Again, it appears that a RSDM as proposed previously would augment the design process.