HAL Id: hal-00610494
https://hal-supelec.archives-ouvertes.fr/hal-00610494
Submitted on 26 Jul 2012
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires
Using Expert Models in Human Reliability Analysis - A
Dependence Assessment Method Based on Fuzzy Logic
L. Podofillini, V.N. Dang, Enrico Zio, Piero Baraldi, Massimo Librizzi
To cite this version:
L. Podofillini, V.N. Dang, Enrico Zio, Piero Baraldi, Massimo Librizzi. Using Expert Models in Human Reliability Analysis - A Dependence Assessment Method Based on Fuzzy Logic. Risk Analysis, Wiley, 2010, 30 (8), pp.1277-1297. �10.1111/j.1539-6924.2010.01425.x�. �hal-00610494�
USING EXPERT MODELS IN HUMAN RELIABILITY ANALYSIS
– A DEPENDENCE ASSESSMENT METHOD BASED
ON FUZZY LOGIC
L. Podofillini, V.N. Dang, E. Zio, P. Baraldi, M. Librizzi
Abstract
In H u m an Reliability Analysis (H RA), d ep end ence analysis refers to assessing the influ ence of the failu re of the op erators to p erform one task on the failu re p robabilities of su bsequ ent tasks. A com m only u sed ap p roach is the Techniqu e for H u m an Error Rate Pred iction (TH ERP). The assessm ent of the d ep end ence level in TH ERP is a highly su bjective ju d gm ent based on general ru les for the influ en ce of five m ain factors. A frequ ently u sed alternative m ethod extend s the TH ERP m od el w ith d ecision trees.
Su ch trees shou ld increase the rep eatability of the assessm ents bu t they sim p lify the relationship s am ong the factors and the d ep end ence level. Moreo ver, the basis for these sim plifications and the resu lting tree is d ifficu lt to trace. The aim of this w ork is a m ethod for d ep end ence assessm ent in H RA that cap tu res the ru les u sed by exp erts to assess d ep end ence levels and incorp orates this know led ge int o an algorithm and softw are tool to be u sed by H RA analysts. A Fu zzy Exp ert System (FES) u nd erlies the m ethod . The m ethod and the associated exp ert elicitation p rocess are d em onstrated w ith a w orking m od el. The exp ert ru les are elicited system atically and converted into a traceable, exp licit, and com p u table m od el. Anchor situ ations are p rovid ed as gu id ance for the H RA analyst’s judgment of the input factors. The expert model and the FES- based d ep end ence assessm ent m ethod m ake the exp ert ru les accessible to the analyst in a u sable and rep eatable m eans, w ith an exp licit and traceable basis.
Keywords
: hu m an reliability analysis; hu m an action d ep end ence; exp ert ju d gm ent; fu zzy exp ert system ; exp ert elicitation.Acronym s / Abbreviations
DT – d ecision tree FES – fu zzy exp ert system
FL – fu zzy logic H FE – hu m an failu re events
H RA – H u m an Reliability Analysis MF – m em bership fu nction
PSA – Probabilistic Safety Assessm ent TH ERP – Techniqu e for H u m an Error Rate Pred iction
1 Introduction
Exp ert ju d gm ent is required in m any areas of risk analysis, w here the relevant factors and their relationship s are com p lex and the d ata are insu fficient as a basis for either a statistical estim ate or for constru cting an em p irical m od el w ith w hich estim ates m ay be obtained . Tw o w ays of u sing exp ert ju d gm ent can be d istingu ished . The first w ay relates to collecting and aggregating the ju d gm ent of exp erts on the variables of interest, e.g. a failu re p robability or a seism ic hazard cu rve (e.g. Cooke, 1991). Form al ap p roaches for this exist, w hich are very u sefu l to bring ou t the assu m p tions and reasoning und erlying the ju d gm ents and to d ocu m ent them so that they can be ap p raised by others (Cooke, 1991; O’H agan et al., 2006). H ow ever, som e d isad vantages are the cost and tim e they requ ire and the p otential p resence of biases in the exp ert estim ates (for a com p lete d iscu ssion see: Otw ay & von Winterfeld t, 1992). A second w ay of u sing exp ert ju d gm ent is based on collecting inform ation from exp erts to bu ild a com p u table m od el (called the exp ert m od el) w ith w hich the d esired valu e m ay be obtained . Exam p les inclu d e the com p u terized d iagnostic aid s u sed in nu clear p ow er p lants, e.g. Chang et al. (1995), and clinical d ecision su p p ort system s u sed in m ed icine, e.g. Kaw am oto et al. (2005).
The w ork rep orted here ad d resses an ap p lication of exp ert ju d gm ent in H RA, the p art of PSAs that d eals w ith hu m an p erform ance and its im p act on risk. In H RA, d ep end ence analysis refers to assessing the influ ence of the failu re of the op erators to p erform one task on the failu re p robabilities of su bsequ ent tasks (Swain & Guttman, 1983). In qualitative terms, a d epend ence is said to exist betw een tw o tasks, that is tw o H u m an Failu re Events (H FEs) if the failu re of the second H FE is m ore likely given th at the op erators have failed in their p erform ance of the first H FE than follow ing su ccess of the first H FE.
The assessm ent of d ep end ence has a significant im p act on the overall resu lts of a PSA, since the d ep end ent failu re p robability m ay be an ord er of m agnitu d e or m ore larger than the ind ep end ent one. An ap p rop riate assessm ent of d ep end ence is thu s essential to avoid u nd erestim ation of the risk and to ensure a realistic risk p rofile from the PSA resu lts. In id entifying the H FEs for w hich d ep end ence shou ld be consid ered , i.e. the scenarios in w hich m u ltip le H FEs ap p ear, a com m on p ractice is to u se large screening p robabilities for the H FEs. If the p robabilities estim ated w ithou t accou nting for d ep end ence are u sed , the relevant sequ ences (w ith m u ltiple H FEs ) m ay be tru ncated . A qu antification of the scenarios w ithou t consid eration of d ep end encies m ay m iss cand id ates for p otential d ep end encies (N UREG-1792, 2005).
The d evelop m ent of an exp ert m od el for H RA d ep end ence assessm ent and an assessm ent m ethod based on this m od el is aim ed at increasing the rep eatability of these assessm ents. The exp ert m od el can system atically and transp arently rep resent the assu m p tions and ru les u nd erlying the assessm ent ; at the sam e tim e, it can rep resent relatively com p lex assessm ent ru les that account for the interactions am ong the inp u t factors. The attractiveness of a m ethod based on an exp ert m od el is that it m akes the exp ert know led ge and ru les accessible to an H RA analyst. Since d ep end ence assessm ents are need ed w ithin each H RA (each PSA stu d y), su ch a m ethod can circu m vent the need to convene an exp ert or exp erts in a form al elicitation for each stu d y.
This w ork focu ses on d ep end encies am ong p ost-initiator H FEs. In cu rrent PSAs, the d ep end ence m od el from the Techniqu e for H u m an Error Rate Pred iction (TH ERP) H RA m ethod (Swain & Guttman, 1983) is commonly used . It has tw o parts: a qu alitative assessm ent of a d ep end ence level, ranging in d iscrete step s from zero (ind ep end ent tasks or actions) to com p lete d ep end ence, and the qu antification of the im p act of the assessed d ep end ence level on the cond itional p robability of the su bsequ ent task based on a set of form u las. The TH ERP m od el refers to five m ain factors: spatial related ness, tim e relationship , fu nctional related ness, stress, and the sim ilarities am ong the personnel p erform ing the tasks. While the TH ERP d ep end ence m od el p rovid es general gu id elines for the influ ence of these factors on the d ep end ence level (cf. Table 10-1 in Sw ain & Gu ttm an, 1983), the assignm ent of the level is essentially a d irect exp ert ju d gm ent, a highly su bjective p rocess that can be w eak in term s of transp arency and rep eatibility. The ASME Stand ard for Probabilistic Risk Assessment notes that ―the state of the art in H RA is such that the assessment of dependency is largely based on the analyst’s judgement.‖ (Note (1) to H R-G7, ASME, 2005).
To ad d ress these issu es and red u ce the su bjectivity inherent in ju d ging the d ep end ence level d irectly, the assignm ent of the d ep end ence level is frequ ently sup p orted w ith d ecision trees (DTs), for instance, in the SPAR-H (Gertman et al., 2005) method , in the EPRI H RA ―calculator‖(Julius et al., 2005), as w ell as in the recently presented method DEPEN D-H RA (Cep in, 2006, 2008a, 2008b). H ow ever, the d ecision tree rep resentation frequ ently very m u ch sim p lifies the relationship s am ong the inp u t factors and the d ep end ence level. In ad d ition, the basis for the d ecision tree is d ifficu lt to trace. It shou ld be noted that althou gh the m entioned m ethod s have been sp ecifically d evelop ed for nu clear p ow er p lants ap p lications, hu m an failu re d ep end ence assessm ent is an im p ortant p art of the H RA for any technical system (Kenned y et al., 2007).
Cu rrent p ractice has a nu m ber of w eaknesses. The absence of sp ecific guid ance m akes the u se of the TH ERP d ep end ence m ethod d ifficu lt and the resu lts m ay lack traceability and rep eatability. This also m akes the review of the assessm ent by a second p erson d ifficu lt (e.g. in p eer or regu latory review s). The u se of DTs im p roves the situ ation: the analyst has to give ju d gm ents on the inp u t factors, bu t is not requ ired to d raw conclu sions on the d ep end ence level, w hich the DT yield s. The central id ea is that the inp u t factors shou ld be less su bjective qu antities than the d ep end ence level (optimally, they should be ―measurable‖). Yet, DTs are not flexible in the sense that the analyst ju d gm ents are typ ically constrained to rigid op tions, w hich refer to extrem e situ ations (Yes/ N o, H igh/ Low ). Moving aw ay from binary op tions also increases the nu m ber of branches and the com binations of factors to evalu ate. Moreover, d ifferent im p lem entations of DTs exist, w hich m ay p rod u ce significantly d ifferent resu lts (Cep in, 2008c): since DTs are often not bu ilt from a traceable p rocess of exp ert elicitation, it is d ifficu lt to u nd erstand the reasons if tw o DTs give d ifferent resu lts.
Section 2 d iscu sses these shortcom ings in m ore d etail.
N ote that another recent su bject of research related to d ep end ence assessm ent is on how d ep end ent H FEs shou ld be incorp orated in large system fau lt tree analysis (Vau rio, 2000). This su bject relates to d ep end encies am ong p re-initiator H FEs and a m ore d etailed d iscu ssion is ou tsid e the scop e of the p resent p ap er . Also related to d ep end ence assessm ent is the id ea of the existence of hu m an p erform ance lim iting valu es (H PLV) (Kirw an, 2008). It m ay be the case that accid ent sequ ences have very low joint hu m an error p robability (e.g. 10-4 or 10-5), even after d ep end ence is evalu ated : the id ea is that H PLV shou ld be ap p lied to inclu d e for p ossibly overlooked error m echanism s or error-ind u cing cond itions.
The aim of this w ork is a m ethod for d ep end ence assessm ent in H RA that cap tu res the ru les u sed by exp erts to assess d ep end ence levels and incorp orates this know led ge into an algorithm and softw are tool to be u sed by H RA analysts. The Fu zzy Exp ert System (FES) form alism u nd erlies the m ethod . A FES collects the exp erts’ know led ge as a set of Fu zzy Logic (FL) ru les that are m athem atically m anip u lated by Fu zzy Set theory (Zad eh, 1965). Fu zzy set theory has been exp loited for H RA in a nu m ber of ap p lications (Terano et al., 1983; Onisaw a, 1988a, 1988b; Liang & Wang, 1993; Kim &
Bishu, 1996; Suresh et al., 1996; Huang et al., 1996; Richei et al., 2001; Konstandinidou et al., 2006; Marseguerra et al., 2006). In most of these, the focus is on using FL to convert hu m an error context d escrip tions into inp u ts for existing H RA m ethod s, w ith the aim of accounting for am bigu ity and su bjectivity of the d escrip tions. For exam p le, in Konstandinidou et al. (2006) and Marseguerra et al. (2006), fuzzy logic is applied to
com p u te H EPs via the CREAM m ethod , by converting the characterization of the p erform ance shap ing factors into fu zzy nu m bers. Only in Richei et al. (2001) is the p roblem of bu ild ing a FES from the exp ert know led ge also ad d ressed .
At this stage, the focus of the w ork has been to investigate the suitability and p racticality of the FES rep resentation for an H RA d ep end ence assessm ent m ethod for p ost-initiator H FEs. This p ap er p resents the basic concep ts of the p rop osed m ethod and d em onstrates the ap p roach u sing a w orking m od el of the d ep end ence relationship s. The w orking m od el is intend ed to rep resent a set of m od erately com p lex relationship s am ong the inp u t factors and the d ep end ence level, w hich cou ld be exp ected from an expert elicitation. These relationship s rep resent one p ossible interp retation of the TH ERP d ep end ence gu id elines, bu t one w ith m ore d etail. It ad m itted ly d oes not inclu d e all relevant factors bu t its com p lexity is su fficient for the p u rp ose of d em onstrating the m ethod ology. The d etails of the FES m od el are rep orted in a com p anion p ap er (Zio et al., 2009).
To illu strate its u se, the FES-based has been ap plied for d ep end ence assessm ent of a p air of op erator actions in resp onse to an accid ent scenario in a Boiling Water Reactor.
The FES-based m ethod for assessing d ep end ence has the ad vantage of being able to represent fully the experts’ rules (in this case, the rules of the w orking model), inclu d ing ru les for the interaction of the d ep end ence (inp u t) factors. With anchor situations provided as guidance for the H RA analyst’s judgment of the input factors, the m ethod yield s the d ep end ence level based on the exp ert ru les. An exp ert elicitation to obtain a com p rehensive set of ru les to rep lace the w orking m od el is p lanned for fu tu re w ork.
The p ap er is organized as follow s. Section 2 gives an overview of the p roblem of d ep end ence assessm ent, of the cu rrent p ractice and lim itations. Section 3 p resents the featu res of the p rop osed d ep end ence assessm ent m ethod . The ap p roach for bu ild ing the u nd erlying FES-based m od el is p resented in Section 4. Section 5 p resents an ap p lication of the m ethod . Section 6 d iscu sses traceability, rep eatability, verification and valid ity issu es.
2 Dependence assessment in HRA: practice and
limitations
2.1 The dependence assessment method in THERP
The d ep end ence assessm ent m ethod in the TH ERP H RA m ethod (Swain & Guttman, 1983) is one of the most w id ely used in the PSA practice. Referred to as the ―TH ERP
method‖ in this paper, this dependence assessment method has the follow ing main com p onents:
Use of cond itional hu man error p robabilities (H EPs) to m od el the effect of d ep end ence: the TH ERP ap p roach am ou nts to evalu ate the p robability of failu re of one task, w hen it is know n that the p reviou s task has failed .
Discretization of the cond itional H EP into five ranges rep resenting d ifferent levels of d ep end ence: zero, low , m od erate, high, com p lete.
A form u la for com p u ting the d ep end ent, cond itional p robability for each d ep end ence level. For a low level of d ep end ence, the form u la p rod u ces for low valu es of the ind ep end ent H EP (i.e. <0.01) a nom inal cond itional p robability valu e of 0.05 w ith low er and u p p er bou nd s of 0.015 and 0.15, resp ectively.
Gu id elines for assessing the level of d ep end ence (su m m arized in Table 10-1 of Swain & Guttman (1983)).
The u ser of the m ethod m u st analyze the p air of su ccessive tasks and assess the level of d ep end ence. To su p p ort the analysis of the tasks d ep end ence, the TH ERP gu id elines su ggest the factors that shou ld be consid ered (TH ERP Table 10-1 of Swain & Guttman (1983)): closeness in time and space, functional related ness (e.g. tasks related to the sam e su bsystem ), stress, sim ilarity of the p erform ers (statu s, training, resp onsibility, and „many social and psychological factors―).
For exam p le, for the factor closeness in tim e and sp ace, the gu id eline read s (item 3 of Table 10-1 of Swain & Guttman (1983)):
―Evaluate the spatial and time relationship among all events. Dependence betw een any tw o events increases as the events occu r closer in sp ace and tim e. For exam p le, d isplays or controls that are p hysically close to each other or that m u st be m anip u lated at abou t the sam e tim e have a higher level of d ep end ence than item s that are w id ely sep arated either sp atially or as to the time of their manipulation.‖
These gu id elines cannot be u sed system atically and consistently as a basis for assessing the d ep end ence level becau se a lot of room is left to interp retation. This m akes the assessm ent a rather d ifficu lt task, requ iring a consid erable am ou nt of exp ert ju d gm ent, w hich m ay lack transp arency and traceability and lead s to low rep eatability of the resu lts. Another p roblem w ith the d irect elicitation of p robability is the p resence of biases, of m any typ es (Cooke, 1991; Otw ay, H . & von Winterfeld t, D., 1992).
2.2 Supporting the THERP model with decision trees
The exp ert ju d gm ent assessm ent of the level of d ep end ence is in p ractice often su p p orted w ith a d ecision tree (DT). In these cases, the qu antitative im p act of the assessed d ep end ence level is still m od eled w ith the TH ERP d ep end ence assessm ent m ethod .
Rep eatability shou ld im p rove w hen exp ert ju d gm ent is stru ctu red and su p p orted by a DT. The analyst has to give ju d gm ents on the inp u t factors, bu t is not requ ired to d raw conclu sions on the d epend ence level, w hich com es from the m od el. An exam p le is show n in Figu re 1, w hich rep orts the SPAR-H DT for p ost-initiator H FEs (Gertman et al., 2005). The input factors of the model are (Figure 1):
Crew s (Sam e/ Different)
Tim e (Close in tim e/ N ot close in time)
Location (Sam e/ Different)
Cu es (Ad d itional cu es/ N o ad d itional cu es).
Different im p lem entations of DTs exist. Cep in (2008b) show s that this can lead to significant d ifferences in the resu lt of the H RA and in the evalu ation of the risk contribu tors. Since a traceable p rocess of exp ert elicitation is often m issing , it is d ifficu lt to u nd erstand the reasons if tw o DTs give d ifferent resu lts.
Another lim itation is that DTs are not flexible and m ay not allow assessm ents to be rep resentative of the context u nd er analysis. The need to avoid an excessive branching of the tree u su ally lim its the nu m ber of branches p er factor to tw o to three valu es (labels). These labels rep resent extrem e Yes/ N o cond itions (see Figu re 1 for exam p le), w hich m ay be d ifficu lt to m atch to a p articu lar context. Ind eed , there m ay be consensus on considering a separation by 5 minutes as ―close in time‖ and one of hours as ―not close in time‖. H ow ever, a separation of 20 minutes may be difficult to match to any of the tw o op tions, thu s requ iring a m ore interm ed iate ju d gm ent that w ou ld better rep resent the context.
INSERT Figure 1. The SPAR-H dependence decision tree
INSERT Table 1. The EPRI HRA Calculator ® dependence
decision tree (Grobbelaar et al., 2005).
3 Basic concepts of the dependence assessment
method
Figu re 2 show s a high -level overview of the d ep end ence assessm ent m ethod . The u nd erlying FES m od el and the exp ert elicitation p rocess for its constru ction are p resented in the next Section 4. The m athem atical d etails of the FES can be fou nd in a com p anion p ap er (Zio et al., 2009). The d ifferent com p onents of the m ethod are d escribed next.
INSERT Figure 2. Overview of the dependence model. Different
models are needed for pre-initiators, and different types of post-
initiators.
3.1 Input factors
Sim ilarly to the DTs app roach, each of the inp u t factors is rep resented by a lingu istic variable, qu alified in term s of lingu istic labels (Figu re 3). For exam p le: inp u t factor
―similarity of performers‖ may be qualified in terms of a linguistic variable w ith lingu istic labels: N one - Low – Med iu m … - Very high. Unlike w ith DTs, the nu m ber of lingu istic labels for each inp u t factor is higher than tw o, thu s giving m ore flexibility to the inp u t ju d gm ents.
At the sam e tim e, the u se of a higher nu m ber of lingu istic labels m ay becom e a sou rce of variability in the inp u ts, so that concrete gu id ance is need ed for the analyst ju d gm ents. As show n in Figu re 3, this is p rovid ed throu gh anchor p oints that rep resent p rototyp e cond itions of the inp u t factors for a typ ology of tasks . Different d ep end ence m od els and thu s d ifferent sets of anchors m ay be u sed for p re-initiators, d ifferen t typ es of p ost initiators, etc. The selection of the p rop er anchors and their characterization in term s of the lingu istic labels is one of the ou tp u ts of the exp ert elicitation p rocess.
N ote that nu m erou s stud ies on exp ert ju d gm ent elicitation have show n that the u se of anchors m ay be cou nterp rod u ctive and bias the ju d gm ents if the u nd erling scale is continu ou s (see Brew er & Chap m an, 2002 for an exam p le, am ong m any others, of a p ap er d iscu ssing the anchor effect, and Stevens, 1946, for the d efinitions of the m easu rem ent scales). H ow ever, in ou r case the anchors are essential becau se the scale on which they are placed is very abstract for the analyst (w hat does high ―similarity of cues‖ mean? or w hat does a similarity of performers of 0.33 mean?). Anchors therefore are necessary to p rovid e reference situ ations that can orient an analyst.
With resp ect to the context characterization, the analyst can p rovid e ju d gm ents on the inp u t factors in d ifferent w ays, for exam p le on a scale (Figu re 3). Fou r anchors are show n in the figu re: the analyst m ay p rovid e a p oint inp u t on the scale (inp u t 1), or the range w here his/ her belief belongs (inp u t 2).
A featu re of FES is that they allow overlap p ing of the lingu istic labels (overlap p ing horizontal bars in Figu re 3) to rep resent the fact that in the com m on p ercep tion , the transition betw een the lingu istic concep ts associated to the labels (e.g. betw een
―medium‖ and ―high‖) is not crisp, but often uncertain and ambiguous. This can be form ally accou nted for by introd u cing overlap ping fu zzy sets to rep resent the inp ut variables. Figu re 4 show s a p ossible association of fu z zy sets and trap ezoid al m em bership fu nctions. N ote that the association is not show n to the analyst w ho interfaces only w ith anchor p oints and lingu istic labels.
Ind eed , the natu ral scale for the inp u t factors is continu ou s and the d iscretization introd u ced by the overlap p ing labels is a sim p lification. Yet, this d oes not seem to be a lim itation. The five-level scale is actu ally attractive becau se exp erts and analysts are alread y very fam iliar w ith it from the five TH ERP levels. Fu rtherm ore, as it w ill be p resented in the next Section 3.2, the basis of the fu zzy logic m od el is a set o f ru les.
This is a concep t w ith w hich exp erts and analysts shou ld be alread y fam iliar w ith, since ru les (althou gh m uch coarser) are at the basis of DTs as w ell.
Insert Figure 3. Analyst elicitation on anchored scale; two analyst
input types are shown: as a point value (arrow) and as an interval
(horizontal bar).
Insert Figure 4. Possible association of fuzzy sets and membership
functions (trapezoids) to the input qualitative variable.
3.2 The underlying model
The inference m od el rep resents the relations betw een the inp u t factors and the d ep end ence level. This rep resents the exp ert know led ge, w hich in FES is m od eled into a set of ru les cap tu ring the relationship s betw een the d ifferent valu es of the inp u t and ou tp u t variables. An exam p le ru le read s as follow s:
IF
―Factor 1‖ is ―Low ‖ AND ―Factor 2‖ is ―Medium‖ AND … ―Factor N‖ is
―H igh‖
TH EN
Dependence is ―High‖
The next Section p resents the exp ert elicitation p rocess to convert the exp ert know led ge into fu zzy ru les.
The inp u t ju d gm ents of the analysts are converted into fu zzy num bers, w hich rep resent the d egree to w hich the ju d gm ents m atch each of the qu alitative labels. The fact that a ju d gm ent can m atch, w ith d iffer ent d egrees, m u ltip le labels allow s FES to hand le u ncertain and am bigu ou s statem ents. Mu ltip le ru les are then activated , w ith a d egree that follow s from the d egree to w hich each inp u t statem ent m atches the labels involved in the ru le. The d egree of activation of the ru les is then the basis for the d erivation of the ou tp u t of the m od el, w hich, as d escribed in the next Section 3.3, is rep resented by d egrees of belief in the d ifferent d ep end ence levels. The FL p roced u re u sed in this w ork to associate the ou tp u t of the m od el to a given inp u t assessm ent is based on the Mam d am i fu zzy logic (Zio et al., 2009). Accord ingly, the d egree to w hich an inp u t assessm ent m atches a label involved in a ru le is com p u ted as the m axim u m valu e of the intersection of the inp u t assessm ent and label fu zzy sets (Figure 5, left), the d egree of activation of a ru le is com p u ted as the m inim u m valu e of the d egree to w hich each inp u t assessm ent m atches the labels involved in the ru les (Figu re 5, right) and the contribu tion to the ou tp u t of a given ru le is the m inim u m valu e betw een the ru le d egree of activation and the fu zzy set in the ru le ou tp u t (Figu re 5, right). Finally the ou tp u t fu zzy set is obtained by taking the u nion of all the involved ru le ou tp u t s.
Insert Figure 5. Left: example of computation of the degree to
which the Factor 1 input assessment matches the label “Low ” and
the Factor 2 input assessment the label “High” . Right:
computation of the degree of activation of the rule If “Factor 1” is
“Low” and “Factor 2” is “High” THEN “Dependence” is
“Medium” as minimum value between the degrees to which
Factor 1 and Factor 2 input assessments match the corresponding
labels in the rule.
3.3 Outputs
From the inp u t factors d escribing the context of tw o su ccessive actions w ith resp ect to their failu re d ep end ence, the m ethod p rod u ces tw o ou tp u ts. The first consists in the d egrees of belief (exp ressed in term s of p ossibility) for the d ifferent d ep end ence levels.
The TH ERP d ep end ence levels are u sed (N one, Low , Med iu m , H igh, Com p lete). This output show s how the analysts’ judgments translate into a possibilistic distribution of the d ep end ence levels.
The second typ e of ou tp u t gives qu antitative figu res to the d ep end ence level. This ou tp u t is the exp ected valu e of the cond itional p robability and its associated u ncertainty. These are the figu res that are inclu d ed into the PSA.
The op eration that allow s p assing from the first typ e of ou tp u t to the second is called d efu zzification (Zad eh, 1965) and , to be form alized , entails associating fu zzy sets and d egrees of m em bership to the levels of d ep end ence.
In the m ethod p rop osed in this p ap er, t his is d one by eliciting inform ation from the TH ERP hand book. In particu lar, Table 20-21 of (Swain & Guttman, 1983) suggests nom inal valu es and u ncertainty bou nd s for the cond itional p robability associated to each d ep end ence level. A p ossible association of fu zzy sets to the TH ERP d ep end ence levels consists in taking m em bership fu nctions w ith triangu lar shap e in log10 scale centered on the associated d ep end ence level nom inal valu es and w ith su p p orts given by the low er and u p p er confid ence bou nd s, a s reported in Figu re 6.
N ote that the p resented m ethod is still based on the five TH ERP d ep end ence levels, althou gh a finer characterization of the d ep end ence level cou ld have been obtained w ith m ore levels or by ad op ting a continu ou s d ep end ence scale (for exam p le, u sing Bayesian belief n ets). H ow ever, it w as d ecid ed to u se the TH ERP d ep end ence levels and the TH ERP relationship s betw een the levels and the cond itional H EPs because these are fam iliar to exp erts and analysts. This is an im p ortant p oint for the accep tance of the m ethod .
Insert Figure 6. Fuzzy sets and membership functions associated
to the THERP dependence levels. Elicited from THERP guidelines
(Swain
& Guttman, 1983).4 Building the dependence model: example on a
working model
The concep ts of the elicitation p rocess are here illu strated on a w orking m od el of d ep end ence, ap p licable for p ost-initiator H FEs of a nu clear p ow er p lant for fu ll-p ow er scenarios. The actu al expert elicitation exercise w ill be p erform ed in su bsequ ent w ork.
The inform ation that need s to be p rovid ed by the exp ert is:
The im p ortant factor relevant to d ep end ence and how they relate
Selection and p ositioning of the anchors
Im p act of the factors on the d ep end ence level
4.1 Determining input factors and anchors
The first inform ation to elicit from the exp ert concerns the im p ortant factors for d ep end ence and how they relate. An influ ence d iagram like the one in Figu re 7 is the resu lt: it show s the w orking m od el u sed to exp lore the m ethod ology.
Table 2 lists the inp u t factors, the qu alitative labels and the anchor p oints u sed in the w orking m od el. The exp ert has to locate the anchors on the inp u t scale, so as to bu ild the inp u t interface of Figu re 3.
Accord ing to the w orking m od el, three factors d irectly im pact the d epend ence level (Figu re 7): ―closeness in tim e‖, ―task related ness‖ and ―sim ilarity of p erform ers‖.
―Tasks relatedness‖ is further specified in terms of the ―similarity of cues‖ and
―similarity of functions/ goals‖, as Figure 7 shows. These factors include most, but not all, of the relevant factors. For exam p le, the u se of the sam e p roced u res m ay also have influ ence on the d ep end ence betw een tw o tasks. On the other hand , this w orking m od el w as consid ered to be of enou gh com p lexity for this illu strative ap p lication of the m ethod ology.
Note that the scale for factor ―closeness in time‖ does not relate to an absolute time scale (Figu re 3). For exam p le, the anchor p oint ―5 m inu tes‖ is m eant to be an exam p le of the concep t of closeness in tim e, rather than a tim e m ea su rem ent: a jud gm ent of ―8 minutes‖ could be also placed on the ―5 minutes‖ anchor position if the analyst feels that the situ ations are not d ifferent. Alternative anchors for ―closeness in tim e‖ are u nd er evalu ation, relating to typical tasks in N PP. In t his w ay, as for exam p le it w as d one in Kirw an (1997c), the ju d gem ent is not only connected w ith the nu m eric tim e sep aration .
Figu re 8 show s the resu lts of the exp ert elicitation of the inp u t factors and of the anchors. The Figu re show s the inp u t interface of the m od el as seen by the analyst.
Insert Figure 7. Influence diagram of the working model.
Insert Figure 8. Results from elicitation of the input factors and
the anchors: the dependence model as seen by the analyst.
4.2 Relationship between the input factors and dependence level
The exp ert know led ge on d ep end ence is converted into the FES ru les. This is d one by:
1) u sing statem ents given by the exp erts to fill som e of the ru les of the fu zzy exp ert system , and 2) filling the rest of the fu zzy ru les via a ru les interp olation app roach.
The statem ents from the exp ert can be of d ifferent form s. In this p ap er each statem en t evalu ates p articu lar com binations of the inp u t labels. When the exp ert is asked to evalu ate the selected contexts, he/ she is also show n the relative p ositioning of the anchors on the scale, so as to help him / her to contextu alize his/ her statem ents (Figu re 9). These evalu ations allow p artially filling the Table of ru les, as show n in Figu re 9 for
―task relatedness‖.
Then, the missing ―relationships‖ are filled in by an automatic ―interpolation‖
p roced u re w hich sm oothly sp read s the consequ ent labels over the fu zzy ru les (Marseguerra et al., 2004; Zio et al., 2009) (Figure 10).
Insert Figure 9. Table of rules for the intermediate variable Task
relatedness (partial fill from the expert statements)
Insert Figure 10. Table of rules for the intermediate variable Task
relatedness (complete fill by rules interpolation)
5 Use of the working model: dependence in
operators’ early response to ATWS
5.1 Scenario description
The case stu d y consid ered refers to a set of op erator actions requ ired to avoid excessive boron d ilu tion in the reactor cooling system in case of an Anticip ated Transient Withou t Scram (ATWS) scenario in a nu clear Boiling Water Reactor (BWR).
In the consid ered scenario, the op erators have su ccessfu lly initiated the Stand by Liqu id Control System (SLCS) to shu t the reactor d ow n. To facilitate the reactor shu t d ow n, the op erators are d irected by the p roced u res to increase the void ing by red u cing the level in the reactor to the Top of Active Fu el (TAF). Ad d itionally, they are requ ired to inhibit the actu ation of the Au tom atic Dep ressu rization System (ADS), w hich is activated by the signal of low w ater level in the reactor, generated w hile low ering the reactor w ater level to TAF. In case of failu re to inhibit the ADS, the reactor p ressu re w ou ld be au tom atically d ecreased and low p ressu re injection system s (Core Sp ray
System , CSS), w ou ld be activated . The injected w ater cou ld d ilu te the boron injected by the SLCS and the consequ ential failu re to control reactivity. In case of failu re to inhibit ADS actu ation , the op erators are called to control the level in the reactor u sing low p ressu re injection, trip p ing one of the CSS p u m p s and controlling the other p u m p . The signal to activate the ADS is generated abou t 7 m inu tes after the event of failu re to scram . At that p oint, the op erators have abou t 15 m inu tes to take actions to lim it the low p ressu re injection flow .
The p air of op erator tasks involved in the d ep end ence assessm ent are 1) inhibit the ADS and 2) control the reactor vessel level in ord er to p revent d ilu ting boron concentration after failure to inhibit the ADS. Both actions are d irected by the sam e em ergency p roced u re.
5.2 Five cases of analyst judgment
At first, three ju d gm ent cases are p resented , corresp ond ing to d ifferent p ossible interp retations of th e context by the analyst. The last tw o cases show how the m od el resp ond s to the variation of one inp u t from cases 2 and 3.
Case 1 – input judgments as point values on anchors
Figu re 11 show s an exam p le of this typ e of inp ut on the m od el interface and Figu re 12 show s the corresp ond ing fu zzification of the ju d gm ents by m eans of trap ezoid al MFs.
―Time‖ (upper left corner of Figure 11 and Figure 12): as said in the scenario d escrip tion of Section 5.1, the sep aration in tim e of the tw o actions is exp ected to be arou nd 15 m inu tes. The m ost reasonable anchor rep resenting this context is ―5 m inu tes‖, the conservative ju d gm ent closest to the real context.
―Cues‖ (upper right corner of Figure 11 and Figure 12): the initial cues for ADS inhibition are related to high reactor p ow er level d u e to the failu re to scram . The cu es for control of low p ressu re injection are related to the reactor vessel level, w hich has to be m anu ally m aintained . This situ ation m atches the anchor
―different indicators for different parameters‖.
―Goals‖ (low er right corner of Figure 11 and Figure 12): the tw o actions relate to the sam e fu nction (shu t d ow n the reactor by boron control), carried ou t via d ifferent system s. This situ ation m atches the anchor ―sam e fu nction by different systems‖.
―Performers‖ (low er left corner of Figure 11 and Figure 12): the action is carried ou t by the sam e team . This m atches the anchor ―sam e team ‖.
Figu re 13 rep orts the ou tp u t of the d ep end ence m od el as it resu lts from the above ju d gm ents. The m od el assigns a level of ―Low ‖, w ithou t u ncertainty (in the Figu re, the possibility of ―Low‖ is 1, w hile the possibility of all the other levels is 0). Without entering into the d etails of the u nd erlying fu zzy exp ert system , the reason for this resu lt is that the inp u t jud gm ents are su ch that only one ru le is activated , i.e.:
IF Tim e is Close AN D
Cu e sim ilarity is N one AN D
Sim ilarity of functions/ goals is H igh AN D Sim ilarity of p erform ers is H igh
TH EN Dep end ence is Low
Yet the analyst m ay not be satisfied w ith the jud gm ents of Figu re 11, as they d o not incorp orate entirely his/ her belief on the context. In p articu lar, am bigu ity and u ncertainty m ay com e w ith his ju d gm ents. The next cases show how the p resented m od el hand les these aspects.
INSERT Figure 11. Analyst input on the anchored scale for case
1: point estimates matching the anchors
INSERT Figure 12. Fuzzy input with trapezoidal MFs for case 1:
point estimates matching the anchors.
INSERT Figure 13. Output of dependence level for case 1: point
estimates matching the anchors.
Case 2 – input judgments as point values betw een labels or betw een anchors
This case show s how the m od el resp ond s to a ju d gm ent of the analyst w hich is in betw een labels. To better show the effect of this typ e of ju d gm ent, only one inp u t is given as an ―in betw een‖ judgment (i.e. similarity of cues), w hile the other ones are left to the valu es of Case 1.
Consid er the assessm ent on ―cu es‖ at the basis of the ju d gm ent in Case 1. An analyst m ay w ant to rep resent that cu es are not totally d ifferent as it is im p lied in the d escrip tion of Case 1. Ind eed , ADS actu ation is com m and ed by the signal of low w ater level in the reactor. Therefore, level in the reactor is also one of the p aram eters that the
op erators have to m onitor w hile inhibiting ADS. The context is therefore m ore am bigu ou s than that rep resented in Case 1. The analyst m ay therefore feel m ore confid ent to assign a point assessm ent som ew here in betw een the label ―N ON E‖, representing ―different indicators for different parameters‖ and the label ―LOW‖, rep resenting a som ew hat higher level of d ep end ence.
Figu re 14 show s the inp u t of the analyst on the m od el interface and Figu re 15 show s the corresp ond ing fu zzification of the ju d gm ents by m eans of trap ezoid al MFs.
Figu re 16 rep orts the ou tp u t of the d ep end ence m od el in form of the p ossibility valu es of the d ifferent levels of d ep end ence. It can be seen that the am bigu ity of the ju d gm ent reflects in th e m od el ou tp u t, w hich assigns possibility to both levels of ―low ‖ and
―medium‖ (in particular, 0.8 to LOW and 0.2 to MEDIUM). The relative possibilities valu es of the low and m ed iu m labels d ep end on the location of the inp u t assessm ent arrow of the analysis in Figu re 14.
The reason for this ou tp u t is that as a resu lt of the inp u t ju d gm ents, tw o ru les are activated , i.e.:
IF Tim e is Close AN D
Cu e sim ilarity is N one AN D
Sim ilarity of functions/ goals is H igh AN D Sim ilarity of p erform ers is H igh
TH EN Dep end ence is Low
IF Tim e is Close AN D
Cu e sim ilarity is Low AN D
Sim ilarity of functions/ goals is H igh AN D Sim ilarity of p erform ers is H igh
TH EN Dep end ence is MEDIUM
INSERT Figure 14. Analyst input on the anchored scale for case
2: point estimates between anchors.
INSERT Figure 15. Fuzzy input with trapezoidal MFs for case 2:
point estimates between anchors.
INSERT Figure 16. Output of dependence level for case 2: point
estimates between anchors.
Case 3 – range assessments (uncertainty)
This case show s how u ncertainty in the context can be rep resented in the ju d gm ent and how this reflects in the ou tp u t d ep end ence assessm ent. Again, only one inp u t is varied from the ju d gm ents at the basis of Case 2 to show this effect.
Consid er the ju d gm ent on the inp u t factor ―closeness in tim e‖. Accord ing to the scenario d escrip tion, the op erators have abou t 15 m inu tes to take actions to lim it the low p ressu re injection flow . In Case 1, in ord er to have the inp u t m atching an anchor, the conservative ju d gm ent of 5 m inu tes w as m ad e. H ow ever, m ore realistically an analyst may w ant to represent the uncertainty in the ―about 15 minutes‖ as an interval range betw een 5 and 20 m inu tes (Figu re 17 and Figu re 18).
Figu re 19 rep orts the ou tp u t of the d ep end ence m od el in form of the p ossibility valu es of the d ifferent levels of d ep end ence. It can be seen that as a consequ ence of the varied inp u t ju d gm ent, the ou tp u t assessm ent is sp read on the three valu es of ZERO, LOW, and MEDIUM, p eaked on the LOW level. The activated ru les are in fact:
IF Tim e is N either AN D
Cu e sim ilarity is N one AN D
Sim ilarity of functions/ goals is H igh AN D Sim ilarity of p erform ers is H igh
TH EN Dep end ence is Zero
IF Tim e is N either AN D Cu e sim ilarity is Low AN D
Sim ilarity of functions/ goals is H igh AN D Sim ilarity of p erform ers is H igh
TH EN Dep end ence is Low
IF Tim e is Close AN D
Cu e sim ilarity is N one AN D
Sim ilarity of functions/ goals is H igh AN D Sim ilarity of p erform ers is H igh
TH EN Dep end ence is Low
IF Tim e is Close AN D
Cu e sim ilarity is Low AN D
Sim ilarity of functions/ goals is H igh AN D Sim ilarity of p erform ers is H igh
TH EN Dep end ence is Med iu m
Insert Figure 17. Analyst input on the anchored scale for case 3:
range assessment
Insert Figure 18. Fuzzy input with trapezoidal MFs for case 3:
range assessment.
Insert Figure 19. Output of dependence level for case 3: range
assessment.
Cases 4 and 5 – response of the model to judgment variation on one input
This cases show how the m od el resp ond s to the variation of one inp u t from the valu es that w ere u sed in cases 2 and 3. N ote, these last tw o cases d o not corresp ond to a sp ecific d ep end ence analysis assessm ent, bu t they are p resented here as exam p les of sensitivity analysis.
The inp u ts for cases 4 and 5 are the sam e as those of cases 2 and 3, resp ectively, except for the judgment on factor ―similarity of cues‖, w hich is now centered on the anchor
―single indicator for the same parameter‖ (with falls on the in put label H IGH , w ithout overlap p ing to any other label).
The ou tp u t of the d ep end ence m od el is rep orted in Figu re 20: as exp ected , both ou tp u t assessm ents are shifted tow ard s higher d ep end ence levels, as a consequence that the inp u t ju d gm ent has been m oved tow ard s higher sim ilarity of cu es. In case 4, the level of H IGH is assigned w ithou t u ncertainty (Figu re 20, left), becau se the inpu t assessm ent is su ch to activate only one ru le:
IF Tim e is Close AN D
Cu e sim ilarity is H igh AN D
Sim ilarity of functions/ goals is H igh AN D Sim ilarity of p erform ers is H igh
TH EN Dep end ence is H igh
Instead , the ou tp u t d istribu tion of case 5 is spread over tw o levels (MEDIUM and H IGH ), as an effect of the u ncertainty in the ―closeness in tim e‖ ju d gm ent, w hich is exp ressed as an interval range (Figu re 20, right). For brevity, the ru les activated in case 5 are not rep orted .
Insert Figure 20. Output of dependence level for cases 4 (left) and
5 (right).
6 Discussion
6.1 Repeatability and traceability
The m otivation behind the d evelop m ent of the p resented m ethod has been to give rep eatability and traceability to the d ep end ence assessm ent am ong su ccessive hu m an actions affecting safety of installation .
Rep eatability com es from the fact that the p rop osed m ethod is based on an exp licit, com p u table m od el. Ind eed , as w ith DTs, the ju d gm ents are given on the inp u t factors and the ap p licable d ep end ence level follow s from the m od el and not from d irect exp ert ju d gm ent. With resp ect to the ju d gm ents to be p rovid ed on the inp u t factors, rep eatability benefits from p rovid ing anchor situ ations to the analyst: the m ore rep resentative and d efined the anchors are, the less controversial, and therefore variable, the inp u t ju d gm ents are.
Unlike DTs, traceability of the d ep end ence m od el is assu red by a system atic exp ert elicitation ap p roach, m ad e u p of tw o traceable step s (first fill and interpolation of the fu zzy logic ru les relating the inp u t to the d ep end ence level). Once the d ep end ence m od el is bu ilt, it is easy to go back and verify the base exp ert’s statem ents that originated the m od el. In p rinciple, the traceability of the p rocess to bu ild DTs cou ld be p otentially im p roved by p rovid ing better d ocu m entation, bu t this has been rarely the case.
6.2 Verification and validation issues
Mod el verification and valid ation are tw o essential step s in the d evelop m ent of any m ethod and are being taken into consid eration in the d evelop m ent of the p resent d ep end ence m ethod .
Verification is intend ed as the p rocess to m ake su re that the m od el rep resents correctly the experts’ knowledge (O’Keefe & Smith, 1987). After construction, the expert model has to be assessed (verified ) by the exp erts w hose exp ertise is cap tu red in the m od el. In this phase, feed back need s to be p rovid ed to the exp erts to allow them to d eterm ine w hether the exp ert system m od el is a good rep resentation of their know led ge, and if not, to m od ify the m od el ap p rop riately. Techniqu es for p rovid ing this feed back are cu rrently u nd er investigation by the au thors: they inclu d e visu alization techniqu es as w ell as im p ortance m easu res. Som e p relim inary resu lts can be fou nd in Pod ofillini et al. (2008).
A fund am ental step for the accep tance of the d ep end ence m od el is its valid ation . Yet, the em p irical valid ation of a hu m an error d ep end ence m od el is a very d ifficu lt task. In
ou tlined . The m ost significant H RA valid ation efforts have ad d ressed m ostly failu re probabilities for ―execution‖, that is, carrying out a series of actions or steps on a system . A review of valid ation efforts for a nu m ber of H RA m ethod s can be fou nd in Kirw an (1997b). These sou rces note the d ifficu lty of valid ating the failu re p robabilities for d ecision or d iagnosis that are p red icted w ith H RA m ethod s. This is d u e in p articu lar to the lack of reference d ata, w hich in tu rn is cau sed by the sensitivity of d ecision failu res to a broad range of variables (contextu al factors). Ind eed , d ata collection efforts have ad d ressed m ostly failu re p robabilities for ―execution‖, as, for exam p le, in Kirw an, et al. (2008), w here the focu s is on com m unication errors.
There are on -going efforts that shou ld im p rove the state of H RA d ata in the fu tu re.
One of these is an attem p t to analyze op erating exp erience and to obtain in this w ay em p irical relationship s betw een the factors and the observed hu m an failure events: the H u m an Error Rep ository and Analysis (H ERA) p roject and d atabase sp onsored by the U.S. N u clear Regu latory Com m ission (H allbert et al., 2006). A second effort is the International H RA Em p irical Stu d y being p erform ed by an international grou p of organizations jointly w ith the OECD H ald en Reactor Project, in w hich the p red ictions of H RA m ethod s are being com p ared w ith sim u lator d ata (Lois et al., 2008 and Dang et al., 2007). While both qu antitative p red ictions (the H EPs) and qu alitative p red ictions (the ―driving‖ or most important input factors identified in the H uman Reliability Analysis) w ill be ad d ressed in this w ork, the nu m ber of d ata p oints w ill not be su fficient to valid ate com p rehensively the relationship betw een the inp u t variables and the p red icted failu re p robabilities, a relationship rep resented by each H RA m ethod . Concerning the valid ation of a d ep end ence m od el, the basis or m echanism s that p otentially lead to d ep end ence w ithin a series of actions relate strongly to the d ecision - m aking of the p ersonnel associated w ith these actions. Therefore, one m ay anticip ate that d ata w ill be very d ifficu lt to collect. A m ajor reason is that the action and fa ilu re p robability of interest are cond itioned on a p reviou s p ersonnel failu re. Given the expected performance levels, this initial failure is relatively difficult to ―provoke‖
system atically and realistically su ch that the su bsequ ent p erform ance can be exam ined . In case the validation strategy w ould be to test the model’s predictions against sim u lator d ata, this p oses challenges also as to how the sim u lator exp erim ents shou ld be d esigned (e.g. w hat accid ent scenarios shou ld be sim u lated ).
This su ggests that a m od el of d ep end ence for H RA cannot be based on (bu ilt from ) a set of d ata from w hich the overall relationship betw een the inp u t variables and the ou tp u t variable of interest can be qu antitatively estim ated . For the sam e reason, it can be exp ected that the valid ation of su ch a m od el cannot be d one against a
com p rehensive set of d ata that is able to exp lore extensively the range of the m od el resp onse.
H ow ever, these d ifficu lties shou ld not d iscourage and som e w ay of test ing the em p irical basis of the m od el shou ld be p u rsu ed . Three concep t alternatives can be anticip ated here:
Valid ation of the m od el against a lim ited set of d ata, thu s valid ating only som e of the inp u t-ou tp u t relationship s, only those that cover the consid ered d ata.
Valid ation of the effect of ind ivid u al factors. This w ou ld ad d ress the qu estion if / how the effect of variations in one inp u t factor (averaging the effect of the other factors or keep ing these at fixed valu es) as anticip ated by the m od el com p ares w ith the em p irical d ata.
Valid ation of the relative strength of the factors and of their interactions. This w ou ld ad d ress the qu estion if the factors that are p red icted as being im portant by the m od el resu lt as being im p ortant also from the d ata.
It is exp ected that the exp erience w ith the m entioned International H RA Em p irical Stu d y w ill help in d efining h ow to go abou t the above concep t alternatives.
While valid ation against em p irical d ata has the m entioned challenges, alternative ap p roaches to valid ation are being consid ered for the short term .
Ind eed , there is a significant history w orking w ith the TH ERP d ep end ence ap p roach, in its original form u lation or su p p orted by DTs. This can be u sed to d raw som e conclu sions on the reasonableness of the nu m bers p rod u ced for d ep end ence calcu lations, i.e. on its so-called face valid ity (Kirw an, 1997a).
Data from exp erts can also be u sed to rep lace em p irical d ata for a valid ation exercise.
This can be d one in tw o w ays (w hich are not exclu sive). The first is to give case stu d ies to the exp erts and p artition the cases in tw o sets: one set is u sed to inform the relationship s and bu ild the m od el and the rest is u sed to test the p red iction cap ability of the m od el. The other w ay is to test the p red ictions against those from ano ther set of exp erts. These op tions w ill be consid ered in the d esign of the exp ert elicitation p rocess.
6.3 Additional discussion
It is w orth noting that other exp ert m od elling ap p roaches exist that can hand le d ep end ence am ong the inp u t factors. Probabilistic m od els su ch as influ ence d iagram s and Bayesian belief netw orks (Phillips et al., 1990) and connectionism networks
(Sträder, 2000) are some examples. Research is also being performed by the authors to com p are the p erform ance of these p robabilistic ap p roaches w ith Fu zzy ap p roaches.
Finally, n ote that the so-called second -generation H RA m ethod s (ATH EAN A (Cooper et al., 1996), MERMOS (Le Bot et al., 1998), CREAM (H ollnagel, 1998)) do not quantify d ep end ence based on cond itional H FEs p robabilities as the p resented m ethod assu m es.
Their com m on notion is that the likelihood of H FEs is d riven by p erform ance cond itions d eterm ined by the context w here the action takes p lace, rather than by intrinsic hu m an error p robabilities associated w ith the task. Follow ing this notion, the context m u st inclu d e p reced ing H FEs and the failu re p robability estim ated for any action shou ld reflect a) the effect of p reced ing H FEs on the scenario and on the operators’ situation assessment, and b) the relationships betw een the actions, w hich w ou ld inclu d e m any of the d ep end ence factors. In this w ay, a m od el of d ep end ence rem ains essential in a second -generation analysis even if the p rop osed d ep end ence assessm ent m ethod m ay not be ap p licable. As second -generation m ethod s have not yet been extensively ap p lied , im p roving d ep end ence assessm ent and qu antifying cond itional p robabilities rem ain issu es of m ajor concern.
Another w ay of hand ling d ep end ence is throu gh the u se of d ynam ic PSA tools, see e.g.
(Chang & Mosleh, 2007), w hich allow to d irectly sim u late the evolu tion of the system after each hu m an intervention and therefore, in p rincip le, to better d efine the context in w hich the d ep end ent actions are carried ou t.
7 Conclusions and Outlook
H u m an failu re d ep end ence assessm ent is a highly su bjective p art of H RA and efforts to im p rove the transp arency and rep eatability of the assessm ents are need ed . This p ap er p rop oses a d ep end ence m ethod that is based on an exp ert m od el, bu ilt from a transp arent exp ert elicitation p rocess. The exp ert m od el is a Fu zzy Exp ert system . This rep resentation has been selected since it is su itable for m od els m ostly bu ilt from exp ert ju d gm ent, as op p osed to em p irical d ata. A w orking m od el of d ep end ence has been d evelop ed to investigate the concep ts und erlying the p rop osed m ethod ; its internal relationship s have been set by the au thors. The exp ert elicitation exercise w ill be p erform ed in the fu tu re. The m od el has been ap p lied for d ep end ence a ssessm ent of tw o op erator actions in resp onse to an anticip ated transient w ithou t scram in a nu clear boiling w ater reactor.
Im p roving the transp arency and rep eatability of hu m an reliability d ep end ence assessm ent, w hile keep ing the m ethod p ractical to u se h ave been the goals of the research. The featu res of the m od el to achieve these goals are as follow s.
The p rop osed m ethod is based on an exp licit, com p u table m od el. As in a d ecision tree, an analyst is requ ired to give ju d gm ents on the inp u t factors of th e m od el, and not d irectly on the d ep end ence level (w hich is the ou tp u t of the m od el). While im p roving the rep eatability of the assessm ent, this is also exp ected to red u ce its u ncertainty, com p ared to the case of giving ju d gm ents d irectly on the d ep end ence level. Ind eed , u ncertainty on inp u t ju d gm ents exists as w ell, bu t w ith anchors to p rovid e references for the scale, the inp u ts can be m ad e less su bjective th an the d ep end ence level.
The com p u table m od el is bu ilt from a system atic exp ert elicitation ap p roa ch, m ad e u p of tw o traceable step s. This is im p ortant becau se, in connection w ith w hat said above, the accu racy of the resu lts p rod u ced by the m ethod d ep end s on the accu racy of the com p u table m od el. Given that em p irical valid ation of the m od el accu racy is extrem ely challenging, it is im p ortant that the bu ild ing p rocess is traceable, to allow scru tinizing the exp erts’ statem ents contained in the exp ert m od el.
With the u se of the exp ert m od el, the analyst’s inp u t ju d gm ents are d irectly and form ally converted into the ou tp u t, the (d iscrete) d ep end ence level. Com p ared to a d ecision tree-based ap p roach, the resu lt gives a d ep end ence assessm ent that can m ore closely reflect the analysts’ u nd erstand ing of the d ep end ence factors for a given set of tasks.
Anchor situ ations are p rovid ed as gu id ance for the analyst’s assessm ent of the inp u t factors. Using u p to five inp u t lingu istic labels for the factors, com p ared to the typical tw o of (binary) d ecision trees, m ay be p erceived as su bject to m ore su bjectivity as w ell as cu m bersom e by analysts and exp erts. H ow ever, the anchor p oints m ay cou nteract the effect of having m ore labels for each factor , by p rovid ing concrete references for d eterm ining the inp u t factor for the sp ecific case u nd er analysis.
N either the H RA analysts nor the exp erts w hose know led ge is rep resented by the exp ert m od el need to be fam iliar w ith Fu zzy Exp ert system s: no asp ect of the Fu zzy form alism is show n to analysts and exp erts and they interface only w ith the inp ut lingu istic labels and w ith the an chor p oints. This is very sim ilar to the interface of d ecision trees com m only u sed (e.g. SPAR-H and the EPRI H RA Calcu lator ®).
Once the m od el is finally bu ilt, v erification and valid ation are also im p ortant asp ects.
In p articu lar, valid ation of the d ep end ence m od el against em pirical d ata is significantly challenging, d u e to the d ifficu lty of collecting failu re p robability d ata.
H ow ever, it is exp ected that it w ill be p ossible to d raw som e conclu sions on the m od el face valid ity. Ind eed , althou gh the cu rrently u sed TH ERP-based ap p roaches m ay suffer from a nu m ber of lim itations, there is a significant exp erience in w orking w ith these
and this can be u sed to check the reasonableness of the resu lts p rod u ced by the p rop osed m ethod .
