5. CASE STUDIES ON TRIAL APPLICATION OF THE KIND APPROACH
5.2. Generic case studies of hypothetical NESs
5.2.3. Comparison of two hypothetical NESs
The comparison of two NESs is extremely important; it can find wide application for solving practical problems, because the number of competing options offered by the decision maker for the final selection usually does not exceed two. At the same time, consideration of this case is characterized by significant risks regarding the indistinguishability of alternatives as well as the relatively low sensitivity of the ranking results to the model parameters.
The particulars of this case are that a comparison of two NESs using single-attribute value functions with local domains specified by means of the performance table will not depend on the forms of the single-attribute value functions. In this case, the values of an alternative multi-attribute value function may be evaluated by the following simple equation:
1
1if the NES is better then competitor on KI NES score , where weighting factor,
0 if the NES is worse then competitor on KI
NKI
This formula implies that in order to estimate the overall scores of alternatives (values of multi-attribute value functions), it is simply required to sum up the weighting factors for those indicators that, for a certain alternative, have scores superior to those of their competitors. This sum is the final value of a multi-attribute value function for the corresponding alternative.
Another specific feature of this case is that it is reasonable to use wider scoring scales for indicator evaluations because this provides a chance for more substantial differentiation of alternatives than it is possible to reach by using a narrow scale. This hypothetical case study is focused on the evaluation of the impact of the different scoring scales used for indicator evaluations and the single-attribute value function domains on the overall scores of the alternatives and the ranking results.
5.2.3.1. Description of considered NESs and assumptions
A comparative evaluation of two hypothetical NESs was performed in accordance with the recommendations given in the previous sections. A set of 19 unspecified KIs was involved in the comparative evaluation procedure.
Each indicator was evaluated on 2 and 10 point scoring scales. It was assumed that the two point scoring scale is the base option.
Linearly increasing functions defined on the local and global domains were chosen as single-attribute value functions for the base case option. A sensitivity analysis regarding the form of the single-attribute value function was carried out using exponential forms of single-attribute value functions.
5.2.3.2. Performance table
The values of all KIs for the considered NESs are presented in Table 5.7 (qualitative evaluation, and on 2 and 10 point scoring scales). In accordance with the assumptions made regarding the goals that are to be achieved by each indicator, a score value of 1 is the best possible indicator value, while a score value of 0 is the worst one, if a two point scoring scale is used. Accordingly, for a 10 point scoring scale, a score value of 10 is the best possible indicator value, while a score value of 1 is the worst one.
The performance table is also presented in the form of a value path for the case of the utilization of a two point scoring scale (Fig. 5.7). This figure shows variations in the values of KIs for the considered NESs demonstrating the merits and demerits of the options considered. It should be noted that this graph depends on the scoring scale selected for evaluation of the indicators, and in the case of a 10 point scoring scale, it will have a different representation for some indicators.
TABLE 5.7. PERFORMANCE TABLE High level
objectives Areas KIs abbr. Qualitative evaluation Two point
scoring scale 10 point scoring scale
NES-1 NES-2 NES-1 NES-2 NES-1 NES-2
Cost Economics E.1 xa 1 0 9 1
a x indicates that the NES provides the best performance for a corresponding KI.
b ~ indicates a KI for which both NESs show comparable performance, and which may be differentiated by using different scoring scales.
In devising Table 5.7, it was assumed that if the evaluation of a certain indicator for two alternatives in the relevant scale is the same, this indicator will be assigned a value of 0. According to the assumptions made about the form of the single-attribute value functions, this will not lead to a contribution of the corresponding indicators to the values of the multi-attribute value functions for the considered alternatives. Note that a 10 point scoring scale provides more subtle opportunities for evaluating the alternatives. For example, on a two point scoring scale, alternatives may have the same evaluation, while on a broader scale they may be awarded various scores.
5.2.3.3. Weighting factors
In order to evaluate the weights, it is assumed that at each level of the objectives tree, the importance of all high level objectives, the areas of evaluation and the KIs are identical. This option suggests that the experts and the decision maker need to agree that high level objectives characterized by the cost, performance and acceptability of the aggregated goals are equally important. Since evaluation of the weights is to be done based on consideration of the ranges over which the indicators may vary, it may be possible to achieve this goal of equal weights by adjusting the ranges over which the single-attribute value functions are defined. Expanding a range will typically increase the corresponding weight of an indicator, and vice versa.
TABLE 5.7. PERFORMANCE TABLE High level
objectives Areas KIs abbr. Qualitative evaluation Two point
scoring scale 10 point scoring scale
NES-1 NES-2 NES-1 NES-2 NES-1 NES-2
Cost Economics E.1 xa 1 0 9 1
E.2 ~b ~ 0 0 4 2
Performance
Waste management
WM.1 x 0 1 2 9
WM.2 x 0 1 1 10
WM.3 x 0 1 2 10
Proliferation resistance
PR.1 x 1 0 10 2
PR.2 x 0 1 1 10
PR.3 ~ ~ 0 0 2 3
PR.4 ~ ~ 0 0 4 3
Environment ENV.1 x 0 1 1 9
Country specifics
S.1 ~ ~ 0 0 3 1
S.2 ~ ~ 0 0 2 2
S.3 x 1 0 10 1
S.4 x 0 1 1 10
S.5 x 0 1 2 9
Acceptability Maturity of technology
M.1 ~ ~ 0 0 4 1
M.2 ~ ~ 0 0 1 1
M.3 ~ ~ 0 0 2 3
M.4 x 1 0 9 2
a x indicates that the NES provides the best performance for a corresponding KI.
b ~ indicates a KI for which both NESs show comparable performance, and which may be differentiated by using different scoring scales.
At the evaluation area level, in accordance with the high level objectives (cost (economics), performance (waste management, proliferation resistance, environment, country specifics), and acceptability (maturity of technology)), equal weighting factors were assigned to each evaluation area. These factors were determined based on the requirement that the sum of all the weighting factors for each area be equal to unity. As the table shows, the sum of weighting factors in one branch of the objectives tree is equal to unity.
At the final level (the level of the KIs), the weighting factors for each indicator included in the corresponding evaluation area were assumed to be equal as well. The weighting factors calculated in accordance with the above mentioned assumptions for each indicator are shown in Table 5.8. The table illustrates the procedure for hierarchical weighting. It can be seen that the final weighting factors obtained satisfy the condition that their sum be equal to unity. These values were used for a comparative evaluation in both the 2 and 10 point scoring scales.
5.2.3.4. Ranking results and their interpretation
The ranking results based on the above mentioned assumptions are presented in Fig. 5.8 for a two point scoring scale. The results from utilizing a 10 point scoring scale are discussed in the sensitivity analysis section. The alternative NES-1 is the preferred one, while alternative NES-2 is less attractive. The values of the multi-attribute value functions are 0.288 (NES-1) and 0.221 (NES-2); therefore, these alternatives can be considered with a certain degree of confidence as being distinguishable.
The results for decomposition of the analysis into individual components in accordance with the objectives tree structure are shown in Table 5.9, which makes it possible to interpret the ranking results. As shown in Table 5.9, the score for the aggregated high level objective performance is greater for the NES-2 than for the NES-1, but the scores for the aggregated high level objectives cost and acceptability are higher for the first alternative than for the second alternative. These advantages ensure that the first alternative is the preferred one.
A comparison of aggregated scores for all the evaluation areas provides an opportunity to examine the contribution of these areas to the values for the high level objectives. The corresponding evaluations are shown in Table 5.10 and Fig. 5.9.
The high level objectives cost and acceptability only include one evaluation area (economics and maturity of technology, respectively), while the corresponding values of the aggregated scores for the high and intermediate levels of the objectives tree are the same. In terms of the economics and maturity of technology areas, NES-1 is preferable to NES-2.
Regarding the corresponding area contribution to the high level objective performance, it should be noted that the first alternative is a less attractive option in terms of waste management, environment and country specifics, while these alternatives are identical in terms of proliferation resistance.
FIG. 5.7. Value path.
TABLE 5.8. WEIGHTING FACTORS High level
objectives High level
objective weights Areas Area weights Indicators Indicator weights Final weighting factors
Cost 0.333 Economics 1 E.1 0.5 0.167
E.2 0.5 0.167
Acceptability 0.333 Maturity of
technology 1
FIG. 5.7. Value path.
TABLE 5.8. WEIGHTING FACTORS High level
objectives High level
objective weights Areas Area weights Indicators Indicator weights Final weighting factors
Cost 0.333 Economics 1 E.1 0.5 0.167
E.2 0.5 0.167
Performance 0.333
Waste
management 0.25
WM.1 0.333 0.028
WM.2 0.333 0.028
WM.3 0.333 0.028
Proliferation
resistance 0.25
PR.1 0.25 0.021
PR.2 0.25 0.021
PR.3 0.25 0.021
PR.4 0.25 0.021
Environment 0.25 ENV.1 1 0.083
Country specifics 0.25
S.1 0.2 0.017
S.2 0.2 0.017
S.3 0.2 0.017
S.4 0.2 0.017
S.5 0.2 0.017
Acceptability 0.333 Maturity of
technology 1
M.1 0.25 0.083
M.2 0.25 0.083
M.3 0.25 0.083
M.4 0.25 0.083
FIG. 5.8. Ranking results.
5.2.3.5. Sensitivity analysis
(a) Impact of single-attribute value function forms
The overall scores for different assumptions regarding the scoring scales and domains are given in Table 5.11.
For a two point scoring scale, both the global and local domains provide the same ranks for the alternatives.
TABLE 5.9. HIGH LEVEL AGGREGATED OBJECTIVE SCORES
Cost score Performance score Acceptability score Overall score
NES-1 0.167 0.038 0.083 0.288
NES-2 0 0.221 0 0.221
TABLE 5.10. SECOND LEVEL SCORES
Economics Waste management Proliferation
resistance Environment Country specifics Maturity of technology
NES-1 0.167 0 0.021 0 0.017 0.083
NES-2 0 0.083 0.021 0.083 0.033 0
FIG. 5.9. Evaluation area scores.
TABLE 5.11. OVERALL SCORES FOR SCORING SCALES AND DOMAINS
Two point scoring scale 10 point scoring scale
Local domain Global domain Local domain Global domain
NES-1 0.288 0.288 0.575 0.375
NES-2 0.221 0.221 0.325 0.266
Utilization of a 10 point scoring scale provides more significant differences between the overall scores because this scale offers more precise indicator evaluation (for the local domain, these differences are 0.250 for a 10 point scoring scale and 0.067 for a 2 point scoring scale). When comparing two alternatives with a local domain of single-attribute value functions, the ranking results are not sensitive to the form of the single-single-attribute value functions:
both alternatives have the same ranks for any form of single-attribute value function with a local domain. The same is true for global domains of single-attribute value functions within a two point scoring scale.
If global domains have been chosen for all single-attribute value functions and if a 10 point scoring scale is used, the probability that the first alternative will have the first and second ranks will be equal to 93% and 7%, respectively. This also leads to a decrease in the difference between the values of the multi-attribute value functions: the overall scores for the NESs are equal to 0.375 and 0.266 for the first and second alternatives, respectively. Obviously, this situation makes the alternatives less distinguishable. This example demonstrates how the distinguishability of the alternatives is improved as a result of the utilization of local domains for single-attribute value functions and a wider scoring scale for indicator evaluations.
(b) Weight sensitivity
The results of the sensitivity analysis carried out based on the linear weights method are presented in Fig. 5.10.
These results correspond to the weighting factors for the high level objectives of the objectives tree for a two point scoring scale. The values of the weighting factors, at which the lines corresponding to different alternatives are crossed, divide the range of the weighting factors into areas where the alternatives being considered have different ranks. NES-1 looks more attractive than NES-2 owing to the lower values (less than 0.4) of the weighting factors for the performance objective. Such an analysis can be performed for both the intermediate level of the objectives tree (evaluation area level) and the KI level.
The analysis of the impact of the indicators’ weighting factor values on the ranking results makes it possible to identify indicators that could ultimately provide a change in the ranks of the alternatives. These are the indicators numbered 3, 4, 5, 7, 10, 14 and 15. However, in order to realize this rank reversal, it is necessary to increase the weighting factor values for the corresponding indicators at least twice.
In conclusion, the results of an expanded sensitivity analysis regarding the weighting factors for the high level objectives are presented. The analysis implied simultaneous variation of weighting factors for the high level objectives over a wide range and identification of the most preferred NES for corresponding weight combinations.
The NES attractiveness for different weight combinations is shown in Fig. 5.11. Based on this information, the preference probabilities for NES-1 and NES-2 (the relative square of corresponding areas) may be evaluated, and these are 63% and 37%, respectively.
The case studies presented, describing the comparison of five or two hypothetical NESs, have demonstrated the basic steps to be completed for a multi-criteria comparative evaluation of NESs based on the MAVT method within the framework of the KIND approach and represent a possible template for presenting and interpreting calculation results that may be implemented in the comparison of real systems.
5.3. INNOVATIVE VERSUS INNOVATIVE NUCLEAR ENERGY SYSTEMS