define what constitutes a significant change in project scope 6. evaluate transparency and accountability of P3s

1. Improve the definition of a P3—the latest draft of the Alternative Capital Financing Guidelines (the Guidelines) defines a P3 as “a cooperative venture based on contractual obligations between one or more public/private/not for profit partners that meets clearly defined public needs for the provision of goods or services through appropriate allocation of resources, risks and rewards”. This definition is very broad and could be interpreted to apply to most contracts.

More precise definition of P3 needed

Many people use a narrower definition than above, usually based on the five characteristics that we discussed in section 1 of this chapter.

This difference in interpretation creates confusion. Alberta could benefit by clearly stating what components (design, build, finance, operate, and own) of a project the private sector needs to be responsible for to have a P3. The Guidelines note that they “apply ONLY to P3 projects that involve private/voluntary sector investment and do not include outsourcing or design/build options”. But the

Guidelines do not note the processes that should exist for P3

arrangements that do not have a private or voluntary sector investment by financing.

We have heard differences of opinions as to what components of a capital project must be transferred to the private sector for such a project to be a P3. Ministries need a common understanding of what a P3 is so they can properly define processes based on the risks of the components involved. However, the definition should not be the deciding factor in the way to structure a project, as the decision should be based on providing the best value for money for the province.

Key prerequisites for P3s need to be enhanced

2. Determine key prerequisites to identify projects most suitable for P3s—the Guidelines list Alternative Capital Financing assessment criteria. They contain seven criteria that ask key questions to determine if a project is a suitable candidate for a P3. For example, one of the criteria is “risk” and one of the key questions is “what are the risks that may relate to a project and how does the alternative capital financing approach mitigate them or lead to a more effective risk management?”

The criteria and questions are a good first step, as they provide ministries with important factors that need to be considered when deciding if a P3 with alternative capital financing is appropriate.

Considering the lessons learned, we believe that the Guidelines should now be improved to allow decision makers to quickly identify or dismiss the P3 alternative by:

• providing more guidance as to what are the key prerequisites for a project to be a strong candidate for a P3 with a financing

component. Instead of just providing key questions, guidance should identify what answers support the use of a P3. In the example noted above, instead of asking if effective risk transfers can occur, the Guidelines should provide a detailed list of key risks the private sector is generally better able to manage, and some discussion of the magnitude of each and its effect on the decision of how to proceed with the project.

• developing a list of what types of capital projects and ministries would benefit the most from P3s. This would be an living document that would be improved over time as ministries gather better

information on many factors, such as where the private sector has demonstrated effective risk transfers and provided innovative solutions, and where the market has shown good competition.

Audits and recommendations Public Private Partnerships

Process

guidelines needed 3. Define when differences in key processes are appropriate—the draft Alternative Capital Financing Guidelines do not discuss the differences in the processes that can occur and the appropriateness of one process over another. The Ministry of Infrastructure used different processes for the Centre from the ones used by the Ministry of

Transportation for the Ring Road. The principal differences are how the successful party is selected and how the contracts are finalized after the final selection.

The Centre’s successful party was selected by evaluating the proposals and determining which one offered the best complete package. There was still much negotiation to be completed after the private party was selected. Also, the scope of the project changed substantially after the selection. The final private party was selected in October 2003, but the final contracts remain unsigned at the end of August 2004.

By contrast, the Ministry of Transportation negotiated the final details of the project when the three potential private parties were identified.

The Ministry is satisfied that all three parties are fully capable of completing the project. Therefore, when the decision is made later this year, it will be based on the lowest cost.

Both processes may be appropriate, based on the circumstances of each project. Conversely, one process may be more effective. The

Committee should compare the differences in the Ministries’ processes and then set guidelines for all major segments of the process, when appropriate.

4. Improve the timeliness of information—

4.1 Timeliness of information on alternatives

The draft Alternative Capital Financing Guidelines have flowcharts and analyses that provide ministries with good information as to the proper order of the processes. They also align the processes with the associated outputs, discuss who is to provide the outputs, and discuss what tools exist to produce the outputs.

The Guidelines, however, could be improved by giving a better description of the preferred timeline of each stage of the project, as illustrated by the following sections.

Better description of timeline needed at each stage of the

project 4.1.1 General discussion of timelines

4.1.2 Timelines in other jurisdictions

4.1.3 Review of Guidelines

4.1.4 Results from the audit of the Centre 4.1.5 Results from the audit of the Ring Road

4.1.1 General discussion of timelines—information to either support or refute the use of a P3 improves over time as a ministry receives expressions of interest and proposals, and negotiates contracts. But it is important to define the level of information that should be present at each stage of the project, (for example, the preliminary assessment stage and the first business case presented to Treasury Board) to allow an informed decision, while avoiding excessive costs in reviewing alternatives not selected.

Other places have two distinct early timelines

4.1.2 Timelines in other jurisdictions—we reviewed information on the procurement process timelines used in two countries with extensive P3 experience, the United Kingdom and Australia. In both cases, there are two distinct early timelines. The first is a preliminary assessment to determine whether a project is a P3 candidate. Based on predefined prerequisites, their processes help to quickly determine if choosing the P3 alternative is likely to produce value for money. The second is a business case that substantiates what the preliminary assessment assumed. In both countries, the government agencies complete the business case before formal discussions with the private sector (expressions of interest and proposals). The business case includes a detailed analysis of risks, costs, and benefits of different alternatives, based on past experience, and an informal review of the private sector market (for example, holding market forums, reviewing information from industry representative bodies). Both countries also then discuss how the business case is revisited after the proposals have been received to ensure the original business case is accurate, and the initial recommendation still holds.

4.1.3 Review of Guidelines—the two distinct early timelines noted in 4.1.2 coincide with the steps noted in the draft Alternative Capital Financing Guidelines. First, the preliminary assessment noted above is very similar to the opportunity paper that

ministries are expected to produce. Second, the business case is also very similar to the Detailed Alternative Capital Financing Analysis Template that the ministries are expected to produce.

The difference is in the level of detail provided at this stage. The Template could be clearer by stating that the analysis should be

Audits and recommendations Public Private Partnerships

sufficiently complete to show Treasury Board that the P3 is expected to produce value for money before going to the private sector market.

Both ministries found it difficult to produce an accurate business case without formally going to the market through the request for qualification and proposal processes for information on costs, benefits, and risk transfers. The difference between what the ministries did and what the two other countries do is that the ministries used the market to produce the business case instead of using the market to confirm the business case. The difference is important, because ministries can incur significant costs in negotiating with the private sector market and then later conclude that the government can best build the asset or provide the

service directly.

Risk assessment not

comprehensive by date necessary

4.1.4 Results from the audit of the Centre—as noted in the Centre’s timeline, the Ministry of Infrastructure prepared a business case in October 2002. The risk assessment contained in this business case could have been more complete by considering a broader range of risks and attempting to value them. A year later, in October 2003, the Ministry did complete a more comprehensive risk assessment that included an attempt to value the risks that would be transferred to the private sector. However, by October 2003, the request-for-proposal phase was complete and the Ministry was beginning final negotiations with the selected private partner.

4.1.5 Results from the audit of the Ring Road—as noted in the Ring Road’s timeline, the Ministry of Transportation completed the Ring Road’s business case in July 2003. The business case could have included the detailed analysis of the public sector

comparator, which is an analysis of what it would cost the government to produce comparable outputs to the P3. The Ministry retained an accounting firm to help prepare the public sector comparator based on assumptions, cost estimates, and risk ranges that the Ministry provided. This analysis was completed in the spring of 2004.

4.2 Overall analysis of alternatives

The key document in the draft Alternative Capital Financing

Guidelines that provides the business case for a P3 with a financing component is the Detailed Alternative Capital Financing Analysis

Template. It is very similar to the cross-ministry business case

template, except that it is updated to reflect the uniqueness of projects with an alternative capital financing component. The Template provides a ministry with the beginning of a sound methodology to evaluate different alternatives for service delivery.

Guidelines and Template can be improved

The Guidelines and the Template, however, could be improved by giving a better description of how to analyse risks and improve the overall analysis of different alternatives, as illustrated in the following sections.

4.2.1 General discussion of risk transfer 4.2.2 Analysis of risk transfer in guidelines 4.2.3 Overall analysis of alternatives in guidelines 4.2.4 Results from the audit of the Centre

4.2.5 Results from the audit of the Ring Road

Transferring risk from the government to private sector for a price

4.2.1 General discussion of risk transfer—it is commonly understood that a key prerequisite of a P3 is an effective risk transfer. The government transfers risk when it pays the private sector to assume a risk under contract. For example, the

government may transfer the construction risk of the project not being completed on time and within budget to the private sector.

In essence, the government pays the private sector to assume some project risks that traditionally the government assumed.

The risk transfer is analogous to an extended warranty on a consumer product, as a premium is paid to ensure a level of service. In some projects, the risk-transfer premium may be offset by private sector efficiencies, making the cost of a P3 alternative the same or less than the cost of a traditional service delivery alternative. However, a P3 that costs more than a traditional service delivery alternative could still represent value for money if the government pays an acceptable amount to avoid risks.

Guidelines could require better risk discussion

4.2.2 Analysis of risk transfer in guidelines—the Guidelines could be improved by updating the Detailed Alternative Capital Financing Analysis Template to require a more complete qualitative and quantitative discussion of risks. The Template could also include good examples of documentation to support the analysis of risks. The Template currently shows the risk analysis as a technical exercise of using the impact of a risk (high, medium or low dollar value) and the probability of the risk

Audits and recommendations Public Private Partnerships

(high, medium or low percentage) to calculate an expected value of the cost associated with the risk. Ministries should be

encouraged to supplement the technical exercise with a well-rounded analysis.

The qualitative discussion of risks could demonstrate whether the private party or the government can best mitigate a risk. The quantitative analysis could also be improved by showing a range of possible costs associated with a risk, or at least a best and worst case scenario and the likelihood of each.

Template could give examples of better cost-benefit analysis of alternatives

4.2.3 Overall analysis of alternatives in guidelines—the Guidelines could also be improved by updating the Detailed Alternative Capital Financing Analysis Template to encourage a more complete qualitative, quantitative, and overall discussion of the overall costs and benefits of each alternative to support the final recommendation. Again, the Template could be improved by providing good examples of this overall analysis. Similar to the calculation of expected risks, the Template’s calculation of overall expected costs of each alternative puts significant emphasis into a technical exercise of developing a single expected value. The Template does have a section called Qualitative Analysis—Non-Financial Benefits and Costs. But it is brief and only includes a few examples of non-financial benefits and costs. The Template could demonstrate to ministries how to use a good qualitative analysis in conjunction with the quantitative analysis to present a well-rounded analysis of the alternatives.

The qualitative discussion of risks could be improved by

expanding the discussion of the non-financial benefits and costs of the different alternatives. For example, ministries could use this analysis to justify why the benefits of one alternative could not be achieved by the other alternative. The quantitative

analysis could be improved by showing a range of possible costs associated with each alternative, using computer assisted tools, or at least a best and worst case analysis and the likelihood of each. These improvements, combined with the enhanced discussions of risk transfer in section 4.2.3 above, would allow ministries to present a well-rounded overall analysis of the costs, benefits and risks of each alternative.

Analysis of risk transfer and alternatives could be better

4.2.4 Results from the audit of the Centre—the Ministry of

Infrastructure could have improved both the analysis of the risk transfer and the overall analysis of the different alternatives. The quantitative analysis of the risk transfer and the overall analysis of costs of the alternatives could have used estimates of a range of costs rather than a single estimate.

$50 million difference could have been better explained

The Ministry could have better explained the difference between its initial calculation of the value of the risk transfer and total costs of the final P3 agreement that included financing. When the Ministry completed its risk assessment in October 2003, it

estimated the value of the risk transfer to be approximately

$34 million. Once costs were known with greater certainty, the P3 alternative with private financing had an additional cost of approximately $84 million as compared to the same alternative without private financing. The $84 million would be offset by the risk transfer and other benefits associated with private financing.

The Ministry could have provided a better analysis by comparing the $34 million to the $84 million and clearly noting what other private financing benefits exist, to justify the $50 million difference.

Better explanation of private financing needed

The Ministry could have improved the explanation of the

possible benefits of private financing, such as increased controls over contractors that the private financers would bring (for example, less risk of cost overruns, time delays, change in project scope). The Ministry could also have shown an offset to some of the interest expense. The reason for this is that there is an opportunity cost to the government using its own money in that if the government is not using it on the Centre, it can use the money on other projects or invest it.

Misperceptions of rising costs could have been avoided

The Ministry could have provided more detailed analysis of the various costs, risks, and benefits in the first business case. There appears to be a public perception that the capital costs of the project increased substantially over time, therefore making the P3 alternative with financing less attractive. However, we understand the estimated construction costs remained relatively stable over time, except when parts of the project were scaled back.

Audits and recommendations Public Private Partnerships

4.2.5 Results from the audit of the Ring Road—the Ministry of Transportation could also have improved its analysis of the risk transfer and the expected costs of the different alternatives, and its overall analysis of the different alternatives. The Ministry could have improved the Ring Road’s qualitative explanations of risks to clearly show why certain risks were transferred and others were retained. It also could have provided a quantitative analysis of the risks and presented a range of the total costs for each alternative in the business case, as described in greater detail above.

The overall analysis of the different alternatives could also have been improved. For example, the Ministry could have improved the analysis of the financing component of the project. The business case showed the interest rate implicit in the P3

arrangement and compared it to the government borrowing rate.

The Ministry could have improved the analysis by showing the total dollar value of interest and the net amount of interest once the time value of money is considered, and then clearly

explaining the expected benefits of the private financing, again, as described in greater detail above.

Guidelines could

5. Define what constitutes a significant change in project scope—the draft Alternative Capital Financing Guidelines do not discuss what constitutes a significant change in the scope of a project and what steps are involved to ensure value for money. A key factor in most P3s is that the private sector can provide the product or service more cost effectively than government. Generally, competition must exist to ensure cost effectiveness. When changes in the scope of a project occur after the successful proponent is chosen, a government increases the risk that the changes in the scope of the project will be overpriced because the competition element is no longer present.

It is not reasonable to expect every change in a project to occur before all contractors have had a chance to bid on a contract. Therefore, guidelines should describe a well-thought-out approach to deal with scope changes. The guidelines could define when the scope of a change is large enough to warrant re-tendering a project. They could also define what other steps can be used for smaller changes to ensure

It is not reasonable to expect every change in a project to occur before all contractors have had a chance to bid on a contract. Therefore, guidelines should describe a well-thought-out approach to deal with scope changes. The guidelines could define when the scope of a change is large enough to warrant re-tendering a project. They could also define what other steps can be used for smaller changes to ensure