HAL Id: hal-02870748
https://hal.archives-ouvertes.fr/hal-02870748
Submitted on 16 Jun 2020
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Copyright
Machine Learning for risk ranking automation in IRSN level 2 PSA
Guillaume Kioseyian, Marine Marcilhac Fradin
To cite this version:
Guillaume Kioseyian, Marine Marcilhac Fradin. Machine Learning for risk ranking automation in
IRSN level 2 PSA. International Conference on Probabilistic Safety Assessment and Management,
PSAM, Jun 2020, VENISE, Italy. �hal-02870748�
Machine Learning for Risk Ranking Automation in IRSN Level 2 PSA
Guillaume Kioseyian
Severe Accident Department, IRSN, France. E-mail: guillaume.kioseyian@irsn.fr Marine Marcilhac-Fradin
Severe Accident Department, IRSN, France. E-mail: marine.marcilhacfradin@irsn.fr
Level 2 PSA producing more and more output data, IRSN has been developing computing tools since 2017 to perform effective post-treatments and to provide in-depth analysis.
These tools have allowed the automation of numerous steps in the post-treatment of release categories generated by the accident progression event tree. The L2 PSA post-treatment is a crucial part to determine and analyse the risk ranking, and consequently to identify critical severe accident scenarios. The analysis of these specific accidental sequences points out the safety improvements to bring about, such as addition or modification of procedures or safety devices.
The toolbox developed has been recently completed with a Machine Learning algorithm-based-tool, using Regression Trees method. Thus, for future L2 PSA, a fully automated post-treatment of release categories is available, and the Farmer diagram can be automatically generated for each risk metric.
The results obtained with this new method are very satisfactory since the risk ranking automatically obtained is similar to that obtained manually. Moreover, the calculation time for this automatic grouping is about fifteen minutes whereas it is about eight days when done manually.
This application paves the way for other automations or process improvements of L2 PSA with the use of Machine Learning approaches.
Keywords: Machine Learning, L2 PSA, risk ranking, severe accident, radiological consequences.
1. Introduction
Level 1 Probabilistic Safety Assessment (L1 PSA) aims at identifying accidental sequences leading to core damage and at quantifying their frequency, whereas Level 2 PSA (L2 PSA) objective is to assess then the frequency and amplitude of radionuclides releases to the environment resulting from the progression of such accidents, so-called severe accidents.
For several years, IRSN, the technical and scientific support organization of the French Nuclear Safety Authority (ASN), has been developing L2 PSA for the Nuclear Power Plants (NPP) operated in France, with significant efforts to integrate a realistic modelling of the severe accident progression. L2 PSA is notably used at IRSN for safety review activities. It is carried out with a set of software tools, all designed by IRSN engineers and researchers, Guigueno et al. (2016). Thanks to these tools, each L1 PSA accidental sequence is extended by L2 PSA severe accident scenarios, with an assessment of radioactive releases and radiological consequences.
The main outcome of L2 PSA is a risk
“frequency versus radiological conséquences”
ranking of potential accidental sequences in a facility. Such a graph is called a Farmer diagram.
This step constitutes a key step in L2 PSA because the risk ranking obtained is decisive for identifying the sequences that deserve operator to modify or add safety devices or procedures.
Updating this risk ranking should highlight the gain obtained. Moreover, this risk ranking is used to communicate clearly and concisely towards non-L2 PSA specialists.
During the last years, the time required to complete this step has increased significantly.
Firstly, the number of severe accident scenarios whose radioactive releases and radiological consequences are assessed is today much more important. For instance, the last internal initiating events L2 PSA for the French 900 MWe Pressurised Water Reactors (PWR) has generated about 500,000 severe accident scenarios grouped in about 50,000 release categories (RC) in which the amplitude and kinetics of releases are supposed to be of the same order of magnitude. Secondly, the set of radiological risk metrics has been extended, Kioseyian et al. (2018). Today, four Farmer diagrams are available at the end of the L2 PSA:
- two metrics to assess the short-term radiological consequences (the effective dose and the thyroid equivalent dose);
Proceedings of the 30th European Safety and Reliability Conference and the 15th Probabilistic Safety Assessment and Management Conference.
Edited by Piero Baraldi, Francesco Di Maio and Enrico Zio
Copyright © 2020 by ESREL2020 PSAM 15 Organizers. Published by Research Publishing, Singapore
ISBN: 981-973-0000-00-0 :: doi: 10.3850/981-973-0000-00-0 esrel2020psam15-paper
2 Guillaume Kioseyian and Marine Marcilhac-Fradin - one metric to evaluate the long-term
radiological consequences (the contamination of soils with caesium-137);
one metric to facilitate the understanding of the results towards non-L2 PSA specialists (the International Nuclear and radiological Event Scale, INES).
In addition, it is planned to complete the current long-term risk metric in the coming years, with the French new indicators and guidance values for defining post-accident zoning, Kioseyian et al. (2018).
Due to the large amount of L2 PSA output data, developments have been made to automate as much as possible the realization of these four risk rankings. Nevertheless, at the time of those developments, one step could not be automated and was therefore performed “manually” by IRSN L2 PSA engineers. This is a time- consuming task. For that reason, important developments have recently been carried out to implement an automated process by means of Machine Learning (ML) algorithms.
This paper provides an overview of these developments. Then, the method used to automate the remaining step is described.
2. Farmer diagram building steps
At the end of the Accident Progression Event Tree (APET) quantification, the available results are the frequency of each release category and the radiological consequences they induced. At this step, the different Farmer diagrams mentioned in the introduction are available but not exploitable as they contain an excessive number of points. For instance, as explained above (cf. part 1), the L2 PSA for the French 900 MWe PWR has generated about 50,000 release categories.
Five steps are necessary in order to build usable Farmer diagrams with a reduced number of points. These points correspond to consistent groupings of RC. The proper realization of these steps is crucial, since they determine the Farmer diagram and therefore the final risk ranking acquired.
2.1 Step 1: containment failure modes for each release category evaluation
The first step consists in evaluating the different containment failure modes for each release category. This step is independent of the radiological risk metrics. Thus, whatever the number of risk metrics, this step has to be done only once.
This first post-processing step of the APET quantification has been automated thanks to software tools developed in R language. These tools consist in associating the release categories with the containment failures modes.
2.2 Step 2: evaluation of the dominating containment failure modes for each release category by using release pathways
For each release category, the software tools designed by IRSN provide the contribution of all possible release pathways of the PWR (containment, steam generator, filtered containment venting system, base-mat, etc.) to the radiological risk metrics values, which depend on the fission products groups. For instance, the current long-term risk metric, that is the contamination of soils with caesium-137, solely depends on caesium-137 whereas the thyroid equivalent dose, one of the short-term risk metrics, is essentially focused on iodine. To any release pathway is associated a retention coefficient for each radionuclide. Thus, the contribution of the release pathways is specific to each radiological risk metric.
For each release category, the second step consists in assessing the contribution to the given risk metric value of the containment failure modes by exploiting the release pathways.
Indeed, each containment failure mode is associated with one release pathway.
Containment failure modes with a contribution above a given threshold (set by the user) are then retained.
This step is repeated as many times as there are radiological risk metrics.
This second post-processing step of the APET quantification has been automated thanks to software tools developed in R language.
2.3 Step 3: grouping of the release categories by
dominating containment failure modes
The third step consists in grouping the release
categories by dominating containment failure
modes. As the latter are specific to the
radiological risk metrics, this step must be
repeated for each of them.
This post-processing step has been automated by means of tools in R language.
After this third step, the number of points on the Farmer diagram moves on from about 50,000 to about 100.
2.4 Step 4: splitting of groups with large risk metric value dispersion
For each group of each radiological risk metric obtained at the end of step 3, the dispersion of the metric value is analysed. If this dispersion is too large (over one decade), step 4 consists in splitting the group in order to reduce the variability, and make it acceptable in each sub- group.
At the time of the L2 PSA post-processing developments, the automation of this step could not be performed, and it was therefore done manually. This step is complex. Indeed, it requires assessing the influence of the available parameters to the risk metric value so as to retain the parameters that reduce the dispersion as efficiently as possible while having a physical meaning. To illustrate this last point, for the L2 PSA for the French 900 MWe PWR, many groups have been split according to the safety systems operating and their impact on the physical phenomena.
2.5 Step 5: keeping the groups with significant contribution to the global frequency or to the overall risk
At the end of the step 4, for each risk metric, the number of points on the Farmer diagram moves on from about 100 to about 150. This number is too large to be displayed in the different Farmer diagrams. For this reason, step 5 retains the groups with significant contribution to the global frequency or to the overall risk (product of the frequency by the risk metric value). This step is repeated for each radiological risk metric.
This fifth post-processing step of the APET quantification has been automated thanks to software tools developed in R language.
3. Use of Machine Learning to automate step 4
For the last L2 PSA for the French 900 MWe PWR, this step required at least two days of work to be done manually for each radiological risk metric. As IRSN considers four risk metrics, eight days were necessary at least to execute step 4.
As this step is costly and repetitive, IRSN has recently developed new tools based on Machine
Learning algorithms in order to automate the step 4.
3.1 Machine Learning algorithm specifications First of all, an in-depth review was undertaken of the available Machine Learning algorithms. This preliminary study allows to select the algorithm that best suits the objectives and expectations.
Firstly, as a reminder, the fourth step aims at reducing the radiological risk metric variability in a group. To this end, the release categories groups with large metric dispersions are split.
Because a target value is considered (i.e. the risk metric value), ML “supervised learning”
algorithms were preferred to “unsupervised learning” algorithms. In addition, as the target value is a numeric data, supervised learning algorithms that can be applied to regression learning tasks, in opposition to classification learning tasks, must be kept.
Taking into account this first requirement, the selected ML groups of algorithms are:
Regression Trees, Model Trees, Artificial Neural Networks and Support Vector Machines, Lantz (2019).
Secondly, the goal is not to obtain a predictive model. Indeed, high-performance tools, based on simplified physical models, are already available, Guigueno et al. (2016). They can assess in a few seconds the radioactive releases and radiological consequences induced by each release category. The second prerequisite is to use a ML algorithm that produces an understandable descriptive model.
Such a model will allow the user to observe at a glance what are the parameters used by the machine to split efficiently a group of RC. This requirement is very important because, as explained above (cf. part 2.4), the split of a group has to be performed while keeping a physical meaning. Thus, it is helpful for the user to facilitate access to information created by the machine in order to check the split building. For these reasons, Artificial Neural Networks and Support Vector Machines are excluded.
3.2 Regression Trees and Model Trees
Regression Trees and Model Trees offer a great interpretability since they use tree-like graph, well-known by PSA specialists, to explain the relationship between features (input variables) and a numeric outcome.
Regarding a group of RC that needs to be
split, the partition strategy is identical for both
Regression Trees and Model Trees,
Lantz (2019). Beginning at the “root node”,
4 Guillaume Kioseyian and Marine Marcilhac-Fradin these ML algorithms consist in separating the RC. The split is done according to the feature improving the best the homogeneity that results from the split outcome. This homogeneity is measured by statistics such as variance, which gives a measure of how the data is distributed from the mean. The criterion for stopping the split is met when the overall statistical improvement of the tree, induced by the addition of a new branch, is less than a threshold (set by the user in advance). The trees end with
“terminal nodes”, also known as “leaf nodes”
(they are simply called “leaf’ in the following paper).
The main difference between Regression Trees and Model Trees is in the value returned by the leaf nodes. In Regression Trees, this value corresponds to the average value of the sample that reaches the leaf. By contrast, in Model Trees, the leaf nodes do not terminate in a numeric prediction, but in a linear model.
Indeed, in Model Trees, a regression model is built at each leaf from the sample that reaches the leaf. Because of this, Model Trees are considered as an improvement of Regression Trees.
Model Trees may result in a more predictive accurate model. However, as explained above (cf. part 3.1), one of the goals is not to obtain a predictive model but a descriptive model that highlights the physical parameters influencing the most the risk metric value and displays the resulting structure in a human-readable format.
To this end, the use of Regression Trees and Model Trees is similar.
Finally, Regression Trees have been chosen as the ML algorithm to automate step 4.
Applications of Regression Trees are wide.
For example, they are used in finance, Andriyashin (2005), or medicine, Pouliakis et al.
(2015).
3.3 Implémentation on the L2 PSA for the French 900 MWe PWR
The use of Regression Trees to automate step 4 has thus been implemented on the L2 PSA for the French 900 MWe PWR.
The R package called rpart, providing an implementation of Regression Trees, has been used.
It should be noted that the goal of the paper is to demonstrate the effectiveness of the fully automated post-treatment method of the release categories, and not to present the French 900 MWe PWR L2 PSA results that would have to be fully described with the assumption retained and contextualized. Thus, all the values
presented in the following paper are standardized.
3.3.1 First application on a targetedgroup of release categories
To verify if Regression Trees are relevant to automate step 4, one specific group of release categories has been targeted. As this group presents a too large radiological risk metric dispersion value, it requires to be split after step 3. This group results from the realization of the third step for the effective dose metric. It brings RC whose only dominating containment failure mode is a containment leak via penetrations (that is a failure of containment isolation). This group has been selected due to its great complexity to be split manually for the French 900 MWe PWR L2 PSA results presentation.
This targeted group is composed of 5,000 RC. Its standardized effective dose dispersion extends from 1 to 250. The variability is therefore not acceptable and the group has to be split.
At this point, a matrix of 5,000 lines is available. Each line corresponds to a release category. In addition, this matrix is composed of about 50 columns (parameters) that define a release category (containment break size, containment pressure, containment spray system operating, pH of the sump water, state of the venting system, etc.), plus an additional one giving the value of the effective dose. Parameters are categorical variables with several modalities.
As mentioned above (cf. part 2.4), the split of a group has to ensure a physical meaning. For this reason, only about 15 parameters out of 50 were selected by expert judgement. Furthermore, to prepare the data for the learning process, those selected categorical variables have been converted into binary variables (values are 0 or 1). For instance, as illustrated in Table 1, for one given categorical variable, called X, with four modalities, four binary variables have been created.
Table 1. Transformation of categorical variables ,X
4 3 1 2
X 1 X 2 X 3 X 4
0 0 0 1
0 0 10
10 0 0
0 10 0
Then, a regression tree model has been built
from this updated matrix. This model,
represented in Figure 1, ends with four leaf
nodes. In each of them, the average standardized effective dose value of the release categories that have reached the leaf is showed, and the proportion of the total sample is specified. For instance, 49 % of the RC of the targeted group are in the fourth leaf, and the average standardized effective dose value in this sample is equal to 83.
\ PH_SUMP_WATER_POSTVR_2
