Uncertainty and sensitivity analysis

Given the uncertainties surrounding the selection of a cost-effective program of measures (e.g. Brouwer, 2005b), a separate uncertainty analysis was carried out on both the estimated costs and predicted impacts of the policy scenarios as presented in the previous section.

Economic experts from the Agricultural-Economics Institute (LEI) were asked to indicate the uncertainty surrounding the cost estimates. Based on our own understanding of the

groundwater model used to predict future groundwater quality in the Scheldt basin, we tried to roughly quantify the uncertainty surrounding the estimated environmental impacts. The uncertainties are expressed in an error percentage and run through another uncertainty model developed by Brouwer and Deblois (forthcoming) in Excel using Visual Basic macros to test the effect of uncertainty on the ranking of the policy scenarios in terms of their cost-effectiveness. The uncertainty margin reflects the minimum and maximum (interval) value of the effectiveness or cost at a specified probability or confidence level and underlying

probability distribution.

As often is the case, the uncertainty related to the environmental impact assessment is bigger than the uncertainty related to the estimated unit costs, primarily because of the complexity of the multidirectional biogeochemical processes in soil and groundwater. The scientific

uncertainty surrounding these complex processes is expressed in an ‘uncertain’ uncertainty range in Table 18, contrary to the fixed uncertainty percentages related to the unit costs. A further differentiation of the uncertainty between groundwater bodies requires a more in-depth investigation of the key uncertainty factors across the specific groundwater body areas.

Expert uncertainty assessment surrounding

Policy scenario groundwater quality

(%)

cost assessment (%)

A: livestock extensification 20-50 25

B: manure free zones 20-50 15

C: after-crop 20-50 15

D: optimization fertilizer application 20-50 15

E: nature development 20-50 45

Table 18: Uncertainties surrounding the predicted impacts on water quality in shallow

groundwater bodies in the Dutch Scheldt basin and the estimated unit costs of different policy scenarios

Important factors contributing to the uncertainty of the groundwater quality impact assessment include the fact that the model exercise starts from nitrate concentrations in shallow groundwater as derived from observations in the national monitoring network, assuming a linear relationship with nitrogen loads at land surface under the various policy scenarios. Another important factor is that the size of the different groundwater body areas differs substantially, ranging between 1,000 hectares for the northern creek system and 100,000 hectares for the clay-peat areas. Simulation results for larger areas are expected to produce smaller errors given the fact that even though the model was calibrated for the Scheldt basin many of the groundwater model relationships are based on average parameter values derived from national data and statistics (e.g. national found relationships between soil types, land use and fertilizer inputs). A third and final important factor is the uncertainty related to the correctness of the modeled horizontal and vertical hydrological and nutrient transportation fluxes, including the interactions between deeper and shallow groundwater and shallow groundwater and connected surface water. For example, base flow in the sand soil areas transports the main amount of water towards draining surface water, whereas in clay-peat areas the extent of influx from deeper groundwater may have been underestimated in the model. Moreover, not all parts of the various groundwater body areas will discharge their excess water in the same way, especially not if the area is large and characterized by spatial heterogeneity in topography, hydrology, and soil structure. The distinguished creek areas may have varying drainage characteristics depending on the structure of the shallow soil layers not revealed by general soil surveys. A similar problem may play a role in sand areas where sand

layers may reach a relatively great depth in some parts, while the soil in other parts may contain clay or peat layers, which may also not have been revealed by general soil surveys and analysis.

Monte Carlo analysis is used to randomly generate 2000 values from pre-specified probability distributions for the estimated average effectiveness and cost of each measure⁵. The Monte Carlo simulation method calculates multiple outcomes of a model by repeatedly sampling values from the probability distributions for the uncertain variables and using those values for the calculation. Based on these 2000 random values for both costs and effectiveness, the effectiveness of each measure is calculated, resulting in a large collection of

cost-effectiveness values per measure from which a new uncertainty margin is estimated. For every two policy scenarios the probability is calculated that one is more cost-effective than the other. The results of the uncertainty analysis are presented in Table 19.

Policy scenario

CE = Average cost-effectiveness of a policy scenario measured in €10³/mg NO3/litre/year

P(CEi<CEi-1): probability that policy scenario ‘i’ is more cost-effective than the second best policy scenario ‘i-1’

Table 19: Ranking of policy scenarios when accounting for uncertainty and the underlying probability that one policy scenario is more cost-effective than the next best scenario

5 Average cost-effectiveness of each policy scenario is calculated based on the average unit costs presented in Tables 13-17 in section 10.2 and the average reduction in nitrate concentration across all shallow groundwater bodies presented in tables 8-1 in section 9.3.2. Both the lower (20%) and upper bound (50%) uncertainty margin for groundwater quality have been analyzed, but only the results for the upper bound are presented here. Some differences are found when using the lower uncertainty margin for groundwater quality, but the main results remain the same. Different distributional assumptions are possible. Here we have no a-priori expectations regarding the distribution of either the environmental impacts on groundwater quality or the estimated unit costs and simply assume that the uncertainty is normal distributed.

The uncertainty analysis provides insight in the probability that one policy measure is more cost-effective than another policy measure, i.e. in the robustness of the ranking. This is shown in the last column in Table 19 for the ranking of the policy scenarios under uncertainty. The higher the probability that a policy scenario ‘i’ is more cost-effective than the next best policy scenario ‘i-1’, the more robust the ranking. Table 19 shows that it is 100 percent certain that optimization of fertilizer application is superior in terms of cost-effectiveness to manure free buffer zones, but the probability that manure free buffer zones are more cost-effective than growing after-crop is relatively low and just over 50 percent. The probability that after-crop is more cost-effective than livestock extensification is higher, namely 70 percent. Finally, livestock extensification is almost 100 percent more cost-effective than nature development on agricultural land.

Even if the priority ranking of different measures does not change as a result of incorporating uncertainty in the analysis, as shown in Table 19, the resulting probability distributions provide important information to policy makers, who will be interested to know how certain it is that one measure really is more cost-effective than another given the uncertainties involved.

If two policy measures are more or less equally cost-effective, but the uncertainty surrounding their cost and effectiveness is different, this is expected to play an important role in the selection procedure of programs of measures.

11. Public perception and valuation of the non-market benefits of groundwater