CHBA Discussion Paper on the Canadian Construction Materials Centre and Code Conformity Assessment

(1)

https://doi.org/10.4224/21274823

READ THESE TERMS AND CONDITIONS CAREFULLY BEFORE USING THIS WEBSITE.

https://nrc-publications.canada.ca/eng/copyright

Vous avez des questions? Nous pouvons vous aider. Pour communiquer directement avec un auteur, consultez la

première page de la revue dans laquelle son article a été publié afin de trouver ses coordonnées. Si vous n’arrivez pas à les repérer, communiquez avec nous à [email protected].

Questions? Contact the NRC Publications Archive team at

[email protected]. If you wish to email the authors directly, please see the first page of the publication for their contact information.

NRC Publications Archive

Archives des publications du CNRC

For the publisher’s version, please access the DOI link below./ Pour consulter la version de l’éditeur, utilisez le lien DOI ci-dessous.

Access and use of this website and the material on it are subject to the Terms and Conditions set forth at

CHBA Discussion Paper on the Canadian Construction Materials Centre and Code Conformity Assessment

Moyes, Rowena E.

https://publications-cnrc.canada.ca/fra/droits

L’accès à ce site Web et l’utilisation de son contenu sont assujettis aux conditions présentées dans le site LISEZ CES CONDITIONS ATTENTIVEMENT AVANT D’UTILISER CE SITE WEB.

NRC Publications Record / Notice d'Archives des publications de CNRC: https://nrc-publications.canada.ca/eng/view/object/?id=5e1b49ec-0605-4801-bbce-e3764de96019 https://publications-cnrc.canada.ca/fra/voir/objet/?id=5e1b49ec-0605-4801-bbce-e3764de96019

(2)

CHBA Discussion Paper on the

Canadian Construction Materials Centre

and Code Conformity Assessment

October 2009 SUPPLEMENT

October 2009 Rowena E. Moyes [email protected]

(3)

Executive Summary

CHBA’s 2005 discussion paper on the CCMC and Code Conformity Assessment is still relevant today. Builders need evaluation systems to provide:

• process(es) allowing good products to enter the market quickly and inexpensively

• process(es) which weed out products or systems that will create serious callbacks or failures with possible builder liability

• no unnecessary documentation/ certification requirements at building permit stage

• no threat to regulatory approval for products and systems which have traditionally been accepted based on longstanding performance, non-referenced standards, etc.

• means of showing the builder took reasonable precautions to avoid problems with non-standard products or systems (due diligence)

There have been some changes to processes since 2005, including new guidelines on the scope and practice of product evaluations, risk assessment to guide product evaluations, some shifts to improve consistency, and an effort to clear out dormant files. Definition of objectives and functional statements in the 2005 codes appears to have helped focus CCMC evaluations, although there still appear to be cases of unnecessary requirements. A Business Review currently under way offers an opportunity to look at new options. A number of issues identified in the 2005 paper are still outstanding. It is still difficult to get local inspectors’ approval for alternatives, although the objective-based codes may have helped somewhat. Many people still seem to see the CCMC as the “only game in town” for innovative product and systems approval.

This is likely to be exacerbated by the growing demand for action on greenhouse gasses and climate change, especially reductions in use of fossil fuels and other carbon

generators. Energy efficiency and water conservation have already been added as objectives to the Ontario and British Columbia building codes and are expected in the National Building Code no later than 2012. It is possible that significant numbers of innovative products and systems from here and abroad will be seeking approvals. Some new approaches may be worth further examination, including:

• bringing building officials back into the process, with guidelines/protocols based on levels of risk

• recognizing good, accepted risk reduction tools

• allowing CCMC and others to prepare risk assessments as a stand-alone product

• accepting other test results

• anticipating new demand (core requirements for new classes of products, facilitating cross-jurisdictional approvals, etc.)

• reviewing manufacturing plant operations (for higher risk products)

• creating a new volunteer-based decision-making (rather than advisory) body, similar to the CCBFC

(4)

1. Scope

This paper is issued as a supplement to the Canadian Home Builders’ Association’s 2005 discussion paper on the Canadian Construction Materials Centre (CCMC) and Code Conformity Assessment. It is not a broad new investigation, but provides an update on specific process changes, challenges and opportunities, with particular emphasis on:

• changes to procedures

• experience with the objective-based codes which have come into use since 2005

• potential impact of new ‘green’ products and technologies

2. Summary from the March 2005 paper

CHBA’s initial discussion paper was a wide-ranging look at the CCMC, which is still relevant today. Anyone with an interest in this topic is encouraged to read the full 2005 document. We have reproduced its Executive Summary below.

1) The CCMC is designed to address a real need

Canadian building and fire codes define what products and systems must be used by law to protect health and safety in housing construction. Many of the

requirements refer to product, performance, design or installation standards, or to traditional approaches. However, as new products and systems are developed or brought in from other countries, they may use different approaches. The CCMC is a national body established to evaluate whether these products or systems meet the requirements of the building and fire codes.

2) The informal system of equivalency acceptance is breaking down

Traditionally, local building officials have accepted alternative products and systems which can show ‘equivalent’ performance to code, often based on manufacturers’ tests, etc. However, with the growing complexity of construction and with growing awareness of liability, officials in a number of localities have begun to back away from accepting equivalents. Or, they will only accept them a single project at a time, with code conformity confirmed by professional

engineers or architects. At the same time, provinces / territories have given their support to a national system of evaluation. In some cases, provinces have discontinued or restricted their own evaluation services.

3) Pressures are increasing

CCMC evaluations can take a long time. Evaluations and the associated tests can be extremely expensive – particularly if no performance data currently exists for a traditionally accepted product or system. In that case, someone wanting to use a new approach must not only test their own product or system but the traditional ones as well. As the informal acceptance system dries up, attempts to provide an ‘interim acceptance’ process have foundered because of liability concerns. Meanwhile, some products or systems which have just been accepted in the past with no specific code requirements, or those only partially covered by existing

(6)

standards, or even those with industry-recognized but not code-referenced standards, are suddenly being asked to provide evaluations.

4) Definition of role and scope is under revision

The existing policies do not contain clear enough definitions of what the CCMC evaluations cover and do not cover, or what they should require or not require. The Canadian Commission on Construction Materials Evaluation has been

examining this issue. New policies should address practicality, cost efficiency and how to relate tests and research to a ‘likelihood of problems’ factor. They should be based on simple risk assessment and management procedures which allow CCMC to do what is ‘reasonable’ and not ‘everything they can think of’.

5) Recommendations

Structural

a. Fix the problem of liability.

b. Apply the principles of Smart Regulation. CCMC/NRC and Advisory Commissions

c. Limit the scope of CCMC evaluations.

d. Define and use a simple process for risk rating, management and due diligence.

e. Clearly communicate the actual scope of CCMC evaluations. f. Create a practical avenue for review and/or appeals.

g. Investigate issues of ‘Equivalent’ or ‘Acceptable’ performance.1

h. Provide a functional alternative to the traditional acceptances offered by local/provincial officials.

i. Review CCMC’s capacity to perform the growing amount of work, and ensure it has the necessary resources.

Other jurisdictions

j. Resolve the issue of non-CCMC evaluations, and non-evaluated products. k. Investigate expanded use of alternative compliance acceptance methods.

1

In many cases, it is an advantage for new products and systems only to have to prove that their performance is ‘equivalent’ to current accepted solutions in Part B of the Codes. However, sometimes it would be easier to show that performance of a new product or system is ‘acceptable’ under the Codes’ objectives and functional statements. This is particularly true where the performance of traditional construction methods either has not been analyzed (which opens the question of who should pay for that analysis), or goes beyond those levels (i.e., the old methods would be considered ‘gold plated’ if introduced today).

(7)

3. Some changes to procedures

3.1 Limits to the scope of CCMC Evaluations

When CHBA’s initial paper was written in early 2005, the Canadian Commission on Construction Materials Evaluation (CCCME)2 was discussing draft simplified guidelines to help to define and focus the scope of CCMC’s evaluation services. Those have now been put into place. Here is the final wording:

1. Code explicit requirements shall be addressed. These requirements shall be addressed in relation to a product’s intended use, based on scientific justification and linked to objectives. (NOTE: Addressed is achieved by scientific justification (such as test evidence, calculable solutions, modelling, field studies) or

professional judgement that they attain the minimum level of performance against acceptable Code solutions with respect to the applicable objectives and the functional statements of the Code.

2. Evaluation criteria could be established on anticipated Code changes that have been approved by the Canadian Commission on Building and Fire Codes.

3. Once a requirement (other than proponent-requested claims) is established for an intended use, then that requirement shall be addressed for all similar products or materials, with the same intended use.

4. Proponent-requested performance claims beyond Code explicit requirements shall be subjected to the same scientific rigour as in point 1. (NOTE: The performance issues beyond the Code be identified separately in the evaluation strategy.) These guidelines are very similar to the proposals mentioned in CHBA’s 2005 paper. It appears that these guidelines are helping to reduce unnecessary work and delays, as long as the personnel involved do not apply them too conservatively.

3.2 Process for risk rating, management and due diligence

The National Building Code’s concept of “limiting the probability (... that people ...) will be exposed to an unacceptable risk” acknowledges that there are acceptable levels of risk. CHBA’s 2005 paper recommended a simple risk assessment process be used to guide CCMC’s decisions and requirements and define ‘due diligence’. The Association’s suggested model contained five levels of ‘likelihood of problems occurring’ and five levels of ‘probable hazard if they were to occur’. We suggested that scope should be limited and extensive tests should not be required in lower risk situations.

2

The CCCME is a body established by the National Research Council to provide policy for and technical advice to the CCMC. Its members represent a matrix of interests and expertise from across the country.

(8)

A new CCMC risk rating procedure was approved at the end of 2008, and has been applied as of January, 2009. A risk matrix form is filled out collectively by the technical team associated with the product type, as part of the initial review of the client’s

application. The risk assessment is revisited during the code analysis, and at other milestones. If the scope of the project changes, the risk analysis will also be reviewed. Ratings of 1 to 5 are applied on four criteria: probability of failure, severity of failure, impact on adjacent elements, and difficulty of detection.

Weightings are prepared by multiplying the scores; i.e. probability of failure x severity of failure x impact on adjacent elements x detecting failure. Totals are scored into one of three categories, as follows:

• 1-81 = low risk (described as products which “typically have no risk to the health and safety of the public”)

• 82-255 = medium risk (“having a minimal risk to the health, safety and welfare of the public”)

• 256-625 = high risk (“having a high potential to harm in regards to the health, safety, and welfare of the general public”)

The CCMC’s new risk assessment responds to issues raised by the CHBA and others. It can identify areas of concern early, and help to focus attention on key factors. However, some of the terms and definitions seem a bit worrisome, as is the use of “occupant discomfort” as a risk rating factor.

Also, some outcomes of the formula appear perverse. Taking an exaggerated example to make the point, it appears that a balcony guard made of corrugated cardboard would be rated 5 for likelihood of failure and 5 for severity of failure, but if it was very easy to detect could still end up with a rating well within the low-medium risk category.3 On the other hand, a sample risk assessment for a four-component system calculated risk by adding all the low to low-medium rating numbers for each individual component – 90, 72, 72 and 96 – together. This gave the system an overall score of 330, which is considered well into the high risk category.

These concerns can be magnified if personnel implement the system too conservatively. At the least, experience and outcomes should be monitored. See Appendix A for more discussion of this subject.

3

CCMC says it would assess the balcony guard as two risk components (attachment to the structure and railing). Failure of either would have a high impact on the other, resulting in a 5 rating for impact on adjacent elements. (If so, the rolling risk total would add up to 125, well within the 82 to 255 “medium risk” level.) CCMC also says the attachment to structure of this hypothetical corrugated cardboard guard would be given a 3 rating (moderate) for ease of detection, as “it is moderately reasonable to expect that the average person observing would consider complete failure imminent if part of the guard were to detach from the structure”. However the railing itself would be given a 5 (defined as “no chance of detection before failure”), as “a tear in the cardboard for example, may not necessarily lead to the same conclusion”.

(9)

3.3 Shift to reduce inconsistency

CCMC reports that there has been an ongoing shift towards more of a ‘team’ approach among evaluators. Since the introduction of the 2005 objective-based codes, new staff, new computer programs and the risk assessment process, evaluators dealing with the same areas of interest have been discussing issues together on a more frequent basis. As staff have realized that they have the support of their colleagues, they are reported to be getting more comfortable with this approach. This has not yet removed all complaints of inconsistent decisions by different staff members.

3.4 Clearing out the backlog

CCMC reports that they are making headway on reducing the backlog of applications for evaluation. New staff coming on board have been tasked with investigating old files and either moving them forward or closing them out. A new computer system also flags dormant/inactive files and prompts for decisions on action.

As of mid 2009, CCMC reports that its new document tracking system4 shows about 700 job files, including product listings (to existing standards), re-evaluations, and innovative product/system applications (alternatives to acceptable solutions listed in the National Codes). Approximately 100 of those are in the process of being closed. Reasons for closure can vary widely: some files go dormant because the person in charge at the applicant’s office has left the firm; some because applicants have given other projects higher priority; some because of mergers and acquisitions; some because applicants perceive the CCMC requirements as being impractical or too expensive, some because the applicant does not sign the contract or pay for the services. A number are shown as in ‘admin’, which includes some which are completed and in the process of being loaded to the website.

Of the slightly more than 500 active files, approximately half are waiting for action from the applicant. That leaves about 250 files being worked on by the staff of 14.

3.5 Additional process concerns

Some additional issues have arisen in the interviews for this paper.

Concerns over Re-Evaluations.

The CCMC website has an ‘Important Notes’ page, which says:

“Some of the Evaluation Reports and Listings appearing in this Registry show ‘Re-evaluation due’ dates that are past. CCMC is currently in the process of re-evaluating these products. In the interim, the evaluations remain valid for the conditions under which they were originally published.”

4

The computerized tracking system is still in the start-up phase and there may be some inconsistencies in the numbers.

(10)

Some manufacturers have found that potential users see the due dates, and the note on the listing that “Re-Evaluation is in Process”, and avoid the product. At the very least, the listing should repeat the statement that evaluations remain valid. The online material should be updated as soon as possible.

In addition, products must be re-evaluated every three years whether or not the Technical Guides (TGs) setting out tests and other requirements have been updated. This can be useful for identifying and checking impact of any changes to the production process. But if production hasn’t changed, re-evaluation is perceived as a waste of money.

Need for more outside review

Initial scope and evaluation plans are generally sent only to the Standing Technical Committee for quick outside review. For complex issues, there may be a need to go beyond this group, to draw on more specific expertise and knowledge of implications and costs. Also, when the TGs are revised, those plans do not go to the committee or other outside group. Where significant changes are planned in a TG, outside input should be sought.

Opportunities to facilitate standards development may be missed

In some cases, work on a TG can affect more companies than just the applicant. CCMC occasionally works with small groups of manufacturers to develop TGs. That process can help to identify wider concerns and issues any later product standard would have to address. There is an opportunity to ensure work on the TG could also be used towards a later product or system standard – which should encourage more manufacturers to pay a share of the TG costs. Reviewing work on more applications in the light of format and requirements for future product standards development could be very helpful.

Manufacturers and CCMC are both concerned about out-of-date Code contents – either acceptable solutions and/or referenced standards

CCMC is directed to compare performance of alternative solutions against the ones deemed acceptable in Part B. Sometimes it isn’t even possible to obtain samples of products meeting those standards any more. Sometimes traditional construction methods identified in the codes have attributes or performance levels which go beyond

performance requirement, or are irrelevant.

A CCMC listing number does not necessarily mean that a product or system will comply with the Code requirements

These listings just say the product conforms to an existing standard. Sometimes the Codes can require more than what is covered by the standard – sometimes only part of a standard could be referenced. This has spurred complaints that the format and contents are not useful either for the industry or for building inspectors. CCMC has been consider-ing addconsider-ing another level attestconsider-ing to conformance with both the standard and the code.

(11)

3.6 Business review

A consulting firm has been retained to review CCMC’s business practices in light of changes over the past 20 years. Particular interests include “the advent of green products, the concept of sustainability and pressures to increase energy efficiency”. Phase 1

consists of an environmental scan, web survey of manufacturers and building officials and select one-on-one interviews. Stakeholder consultations make up Phase 2, scheduled for completion by the end of 2009. The final report is expected by the spring of 2010.

4. Experience with objective-based codes

The 2005 national model codes took a brand new approach in a brand new format. A new Division A defines the Objectives and Functional Statements behind all the traditional prescriptive and performance requirements. The traditional contents were then renamed ‘acceptable solutions’, linked to appropriate objectives and functional statements, and placed in Division B. This was done for two main reasons:

• to facilitate acceptance of alternative acceptable solutions, by spelling out exactly what goal(s) and function(s) they would be expected to meet

• to define the codes’ core purpose (for the National Building Code, for

example, the four Objectives deal with Safety, Health, Accessibility, and Fire and Structural Protection of Buildings)

As pointed out in the last paper, building departments’ concerns about possible liabilities and the closing of provincial evaluation groups has significantly reduced the traditional avenues for acceptance of alternatives. There are some anecdotal reports that this situation may have eased slightly. This is attributed to the new objective-based codes format, or to some reduction in anxiety about liability, or both. However, the number of building departments demanding CCMC reports or listings – or individual site-specific engineering reports – for any deviation from strict code compliance is still perceived to be high.

As expected, introduction of objective-based codes does appear to have had some impact on the volume of CCMC applications. The number of applications to CCMC has varied from highs of 500+ per year in the early days to lows of 250 in 2001-4. In 2005, the number jumped to 314, possibly reflecting a desire by some manufacturers to get their evaluations for the new Code requirements completed early. In 2006, volume dropped back slightly to 261, then jumped again to 336 in 2007. The 2008 total was 284. CCMC notes that numbers alone do not tell the whole story; they do not show the complexity of the applications – which has increased appreciably.

There is a general perception that the 2005 National Codes’ definition of objectives, sub-objectives and functional statements, combined with the new CCMC guidelines, has helped to keep the scope of new product evaluations focused on explicit code

requirements. On the other hand, there are still reports of evaluation officers demanding tests apparently beyond that level.

(12)

With many building officials apparently refusing to accept anything other than CCMC reports for acceptance of alternative solutions, it is hard for manufacturers to fight unreasonable demands. Builders support justified caution – they do not want to use inadequate products – but CCMC staff must be very careful not to overstep or appear to overstep their role.

It is impossible to quantify the number of potential new products or systems which do not apply – through whatever possible acceptance route – because the process is considered too difficult, lengthy or expensive.

5. Potential impact of new ‘green’ products and technologies

Almost immediately after the 2005 National Codes were introduced, CCBFC received requests for changes to the National Building Code which did not fall within the scope of the four objectives identified in the new objective-based code.

The Canadian Commission on Building and Fire Codes and the Provincial-Territorial Policy Advisory Committee on Codes have been looking into the issue, and CCBFC is expected to add objectives for both of these issues by the time the 2015 codes are

published. The Ontario Building Code and the BC Building Code already include them. The push for new green technologies is definitely gathering strength. A key driver is climate change and the desire to address greenhouse gasses (GHGs). North America has lagged behind Europe in this, but appears ready to take new steps.

Energy efficiency technologies for buildings are one example. Carbon pricing and caps on GHG emissions by large emitters are another. Introduced by the European Union in 2005, cap and trade has recently become one of the Obama administration’s key policy goals in the United States. British Columbia, Manitoba, Ontario and Quebec are members of a cross-border group called the Western Climate Alliance, which is drafting core model policies for a widely based regional ‘hard’ cap and trade scheme to come into effect in 2012. The federal government has plans for a less ambitious ‘intensity cap’ scheme. A number of Canada’s other trading partners are introducing or investigating carbon pricing as well.

If these initiatives result in robust new programs, demand for new GHG-reducing technologies, especially fossil fuel energy conservation measures, can be expected to surge across the economy. Even if Cap and Trade schemes do not take off, the pressure for action on the environment means governments at all levels are likely to pursue ongoing measures to encourage or regulate green technologies, resource efficiency and reduced carbon emissions.

(13)

In this environment, manufacturers and importers are already working hard to come up with better ‘mousetraps’. As a result, pressure to get new products, equipment and systems into both new and existing buildings is expected to grow.

A surge in demand would definitely create challenges for the CCMC. It is appropriate for the Centre to prevent unsafe new products coming onto the market based on a solid scientific rationale. It is not appropriate for it to prevent good and necessary products because of inadequate funding or staffing, excessive technical requirements, or undue risk aversion/exposure.

It’s not uncommon to hear people say that the National Building Code doesn’t cover green technologies, so CCMC doesn’t have to worry about them – or not yet. But some of the most interesting new approaches are integrated into traditional building components, which the building codes do regulate.

Photovoltaics, for example, may be a key technology for reducing energy consumption in buildings – and helping electrical utilities achieve new government requirements for a growing percentage of electricity to come from renewable sources. New products in Europe include photovoltaics as an integral part of roofing or cladding systems, or sandwiched in glass in place of traditional glazing in windows, skylights, etc.

In France, the technical assessment body for building products apparently has assembled a new group of experts and produced core performance test requirements for these building-integrated-photovoltaics (BIPVs) in order to speed up evaluation of individual products. An assessment of this approach is beyond the scope of this report, but may be worth examining. How well could it work here? Should CCMC be considering similar approaches?

There is also an issue of cross-jurisdictional product evaluations. Building, fire and plumbing codes are not the only regulatory acceptance obstacles faced by new technologies. Products and systems to collect and reuse grey water, for example, also need approval from public health and other departments.

It may be very useful to have a small group that can coordinate and facilitate cross-jurisdictional new product evaluations. Is this a role for CCMC itself? If not, who could take it on? Should CCMC or another branch of IRC be promoting the concept and helping to bring together appropriate departments and agencies?

(14)

6. Responding to new challenges

A full examination of options for CCMC and evaluation of alternative approaches is well beyond the scope of this paper. Instead, this section raises a few issues, possible

approaches, and questions.

6.1 Bringing more building officials back into the process

The first recommendation in CHBA’s 2005 paper was: “Fix the problem of liability”. Potential liability appears to be one of the key reasons building officials are either refusing to consider alternative solutions without CCMC ‘approval’, or demanding engineers’ stamps on a site-by-site, application-by-application basis, or both.

This has dramatically reduced access to an important avenue for alternative solutions to become accepted. Bringing more building officials back into the process could do much to address unnecessary irritants and support innovation.

It is definitely possible to address building departments’ risk of liability. It requires a well-understood process to:

• identify risk factors

• identify tools to reduce the risks, and

• use the best, most cost effective tool or combination of tools in a reasonable, scientifically supportable way.

Unfortunately, it is the builders and manufacturers who have an incentive to go through this kind of process. There is far less incentive for municipal or provincial/territorial departments with responsibility for codes to move from a no-risk situation to a low-managed-risk situation (unless a surge in demand for green technology suddenly casts them as ‘obstacles in the race to fix climate change’.)

Who could champion this initiative? Who should prepare appropriate guidelines (see below)? What kind of regulatory support would they require, if any?

6.2 Providing guidelines/protocols for building officials

Courts will not assign negligence liability to an inspector or building department that approved a project which properly used the ‘deemed-to-comply’ acceptable solutions in Part B of the Code, even if the product or system fails. The inspector/department did what a reasonable person with the expected level of expertise would have done in the same circumstances (took due diligence) – which is the test courts use for assessing negligence. However, if the inspector has accepted an alternative solution, the situation gets more tricky. How can they prove what a reasonable person would have done then? Apparently, there is not a lot of guidance on this – for the building departments or for the courts.

Would it be helpful to produce notes, protocols or guidelines based on risk management principles? This would have to be done carefully, or it could worsen an already difficult

(15)

situation. But if done well such documents could become the standard for due diligence, reducing concerns about liability and improving acceptance of good alternative solutions. Should this be done through the Appendix of the National Codes or through a separate document?

6.3 Recognizing good, accepted risk-reduction tools in the protocols

It’s important to remember that the National Model Codes don’t operate in a vacuum. As illustrated in the CCBFC report Canada’s Construction System: the context for model codes5, there are multiple layers of responsibilities and activities playing a part. These include manufacturers’ own information, in-plant quality control, third-party testing, product, design and/or management standards, certification systems, design and best practice guides, detailed drawings and specifications, site tests, first- and third-party warranties, warranty and insurance program conditions, training and education (as well as contract obligations, other legal requirements, and individual and broader systems to protect consumers, respond to problems, and provide recourse if things go wrong.) A protocol for acceptance of alternative solutions should recognize and support these systems as means of risk management. Checklists could include advice on scoring/ evidence for such issues as: Are there tests to show performance? Were they done by reputable laboratories (listings of labs in Canada and other countries)? How often are they updated? Is there a recognized standard (not already called up in the National Codes)? Is the product/system certified by an accredited certification body, or other reputable group? Is it covered by an Other Regulatory Document (ORD), recognized by the Standards Council of Canada? Is there a history of successful product use? Has a recognized professional approved the product/system application? Can that approval be used generically rather than site-by-site? Is there a Standata for it? Has it been accepted for use in another jurisdiction with similar construction methods and expectations for health and safety? Does the application meet a design guideline from a reputable body? Is there a quality management program in effect? Is there a training program for installers? How strong is the proponent? Is there a warranty or insurance program?

The aim would be to balance the level of risk with the value of the kinds of risk

mitigation measures listed above. Scoring should be designed so that low risk products where the proponent is a longstanding, reputable manufacturer would only have to show the company’s product information, test results from a reputable laboratory, and an affidavit saying that the product or system conforms with the building code and there is a plant quality assurance program in place. For medium-risk products or systems, more risk mitigation could be required. Trade-offs should be possible: perhaps a new

manufacturer’s lack of track record could be balanced by generic engineering approvals and review of quality assurance, a new type of insurance, or similar.6

5

http://www.nationalcodes.ca/ccbfc/constructionsystem_e.pdf

6

Over the past couple of years, as provinces have adopted the new objective-based codes approach, chief building officials have begun approving a small number of alternative solutions. They should be collected as case studies, and used to identify process and analysis improvements.

(16)

Who should be the lead agency to prepare this kind of protocol? Would building departments recognize such a system? What regulatory support would be required, if any?

6.4 Allowing CCMC and others to prepare risk assessments as a stand-alone product

Any risk management starts with an assessment. If some building departments are not comfortable with assessing risk – or the proponent believes they are too conservative – could CCMC or other acceptable experts do so instead? This should be offered to manufacturers on a voluntary basis. Under such an approach, new products or systems where the risk of problems is assessed as low would be deemed acceptable alternative solutions without CCMC-type third-party code compliance evaluation, provided they have an acceptable score under the protocols discussed above. CCMC or the approved experts would also need a guideline setting out the steps to assess risk levels (current process, with amendments, might be acceptable). For higher risk products or systems, conditions could be applied, such as limiting the number of installations, requiring reports of any problems to be sent to the risk assessment body or expert for a period of years, etc. Would this help create a more efficient system? Which bodies should be qualified to prepare the outside risk assessments? Should they have to be third-party? Would building departments recognize such a system? What regulatory support would be required, if any?

6.5 Accepting other test results

An ongoing friction point in new product evaluations is the non-acceptance of tests performed to somewhat different standards than referenced in the codes, or performed in other countries by labs not listed by the Standards Council of Canada. There are a number of elements to this issue, including possible competitive advantage given to offshore manufacturers, and impact on Canadian testing labs if other countries’ results are

accepted here without reciprocal acceptance of Canadian results in those other countries. However, it does appear that some accommodations could be made.

For example, even strictly equivalent tests are only accepted if they were performed in the 1 – 1 ½ years preceding the application. Barring changes in test standards or product formulation, there seems little reason why tests up to three years old or even more should not be accepted. For tests using somewhat different standards, perhaps a simple risk assessment should be performed to examine what is the increased risk of problems because of the differences in the test and the increased severity? Where the increase in risk is assessed to be low, why can’t the test results simply be accepted? In marginal cases, could the test results be accepted but a condition be added that any problems must be reported to CCMC?

(17)

6.6 Anticipating new demand

As noted above, France has apparently produced core performance test requirements for evaluation of building-integrated photovoltaics in the wake of government policy to significantly increase renewable energy use. Alberta’s Standata can perform a similar role.

Should CCMC be anticipating demand for similar steps? If not, should another body? How would ‘core requirements’ be approached? Should it also be directed to design its work on certain TGs so as to support later standards development, where those are likely to be useful? Similarly, should the Centre encourage cross-jurisdictional approaches for products and systems which require approval from several different agencies?

6.7 Using outside expertise for full evaluations

Demand for evaluations is expected to grow in areas where CCMC does not currently have on-staff expertise. It might be very helpful for CCMC to be able to designate certain laboratories, research agencies or specific experts to perform all of the work for new product or system evaluations, including creation of the Technical Guides and

determination of compliance. This would also give the Centre flexibility to respond to changed volumes in more traditional areas.

Would this increase the efficiency of the system? Are there any concerns about using outside experts this way? How can they be addressed? Who would the outside

laboratories/agencies/experts be working for?

6.8 Reviewing manufacturing plant operations for higher risk products

Unlike most certification agencies, CCMC does not do regular plant visits to review operations. Interestingly, an Australian article7 says that Japan recently surveyed the manufacturers of 13,965 products which had received government approvals to find out if actual material specifications still matched the initial test results. “Around 50 companies reported 130 products differed from the ministry approved specification,” the article says. And of a sample of 52 products tested to date, five did not meet the performance

requirements. While there is no quantification of how much these products differed from the performance requirements, Japan is apparently introducing “continued monitoring, random checking and greater scrutiny by evaluation bodies”.

Should CCMC be performing plant visits? Should they be limited only to high risk products? How frequently should they be performed?

7

International Code Collaboration with Japan, in Australian Building Regulation Bulletin, Autumn

(18)

6.9 Creating a volunteer-based decision-making (rather than advisory) body

CHBA’s 2005 paper called for new policies to be “based on simple risk assessment and management procedures which allow CCMC to do what is ‘reasonable’ and not

‘everything they can think of’.”

There are some concerns that because CCMC is part of the National Research Council it could be held to a very high level of accountability in any case that went to court. As mentioned earlier, in determining liability courts usually base their decisions on what a ‘reasonable person’ with equivalent expertise would have done in the same

circumstances. Does that mean the courts would use the highest level of expertise inside the whole NRC to decide ‘reasonableness’? This potential liability appears to have choked up at least some of the decision-making process at CCMC.

Apparently, the current volunteer-based Canadian Council on Construction Materials Evaluation (CCCME) is an advisory body, rather than a decision-making one. It only gives input to the CCMC’s decisions.

Should there be a freestanding, volunteer-based, decision-making body, somewhat similar to the Canadian Commission on Building and Fire Codes (CCBFC) instead? Members of such a group could hear representations from proponents, opponents, and various research- and industry-based groups. CCMC would act as secretariat and give advice, but not be responsible for final decisions. Such a board should have more flexibility to make decisions, and its members should be protected from undue liability exposure under the federal government’s existing policies.

Could such a board function more effectively than the current system? What are the pros and cons?

6.10 Accrediting additional code evaluation and certification bodies

The Australian Building Code Board and New Zealand Department of Building and Housing have introduced a new system for evaluating whether innovative products and systems qualify as ‘acceptable alternatives’ under the two countries’ building codes8. In this system, the government-established Joint Accreditation System of Australia and New Zealand (JAS-ANZ) accredits qualifying certification bodies to evaluate innovative products and systems’ suitability under the codes, and to certify them with the CodeMark logo and certificate of conformity. The rules set out a system of risk analysis and

requirements based on risk levels. High risk products and systems, for example, require frequent plant monitoring plus construction site surveillance.

Legislation in Australian states and territories says that building approval agencies must accept products with this certification as alternative solutions. (Other avenues are

8

See http://www.abcb.gov.au/go/products/codemark/whatiscodemark for more information. Full rules for accreditation and certification are available here:

(19)

available for acceptance of innovative products and services, but officials are not obliged to perform the analyses or accept them.)

There are now two accredited CodeMark certification bodies in Australia. Interestingly, the Commonwealth Scientific and Industrial Research Organization, Australia’s

equivalent of the National Research Council, has decided to discontinue its building product and systems appraisals program (somewhat similar to CCMC) by December 2009. Apparently, this is part of a larger-scale reorganization of the CSIRO.

The Australia-New Zealand system appears to introduce competition into evaluations. Is this desirable? While there are reports of problems in Australia-New Zealand, are they due to the competitive approach or other issues? What are the criteria for evaluation agencies, such as expertise, staffing, insurance, oversight, etc.? How can competition in the evaluation sector improve access, timeliness, etc.?

7. Conclusion

CCMC fills an important role in the Canadian construction system. It has taken good steps to improve processes and move to a risk-based approach.

There are still some areas which need to be addressed for those products and systems going through the CCMC system. These include review of the risk analysis process, handling of re-evaluations, occasions when additional outside review may be advisable, opportunities to facilitate future standards, removing outdated requirements and standards from the national Codes, and addressing the information problem created when a CCMC product listing does not necessarily show code compliance.

An ongoing issue arises where lower- and medium-risk innovative products and systems do not require the full CCMC review. The less formal acceptance systems they need have been disappearing, and nothing has yet arisen to replace them. This situation is expected to get worse as new code objectives create new demand for products and systems from Canada and elsewhere to be approved for code use.

This paper raises several alternative approaches which may be useful within CCMC or as parallel processes. These include: providing guidelines and/or protocols for building officials on approving alternative solutions; recognizing good, existing and innovative risk reduction tools; allowing CCMC and others to prepare risk assessments as a stand-alone product; accepting other test results; anticipating new demand; using outside expertise for full evaluations; reviewing manufacturing plant operations for higher risk products; creating an outside volunteer-based, decision making (rather than advisory) body; and accrediting additional code evaluation and certification bodies.

CHBA suggests that the business review of CCMC should include discussion of potential usefulness of these alternatives.

(20)

Appendix A: More Details of the CCMC Process for Risk Rating

The building code’s concept of “limiting the probability (... that people ...) will be exposed to an unacceptable risk” acknowledges that there are acceptable levels of risk. CHBA’s 2005 paper recommended a simple risk assessment process be used to guide CCMC’s decisions and requirements. The suggested model contained five levels of “likelihood of problems occurring” and five levels of “probable hazard if they were to occur”. CHBA suggested that scope should be limited and extensive tests should not be required in lower risk situations. The sample risk assessment chart from that paper is shown below.

CHBA’s Proposed Risk Assessment Form

A new procedure has been approved and applied recently at CCMC. It is filled out collectively by the technical team associated with that product type after the evaluation officer does the initial review of the client’s application. The risk assessment is revisited during the code analysis, and at other milestones. If the scope of the project changes, the risk analysis will also be reviewed.

Ratings of 1 to 5 are applied on four criteria as shown below (it appears there is no zero rating):

CCMC’s new Risk Rating Chart

Column 1: Probability of Failure

Score Extent Description

5 Frequent Likely to occur frequently

4 Probable Likely to occur several times in the service life of a product 3 Occasional Likely to occur at least once in the service life of a product 2 Remote Unlikely to occur but possible

1 Improbable Occurrence very unlikely

Very Low Low Moderate High Very High

Very Low Low Moderate

High Very High

Hazard level if problems do occur

L ike lihood of pr obl em s oc cur ring

(21)

Column 2: Severity of Failure

5 Catastrophic Product fails and results in sudden fatal injury (i.e. Structural) 4 Major Product fails and likely results health and safety risk

3 Critical Product fails and may result in health and safety risk 2 Significant Product fails with likely impact to occupant discomfort 1 Functional

Failure

Product fails with unlikely impact to occupant discomfort

Column 3: Impact on Adjacent Elements

5 Catastrophic Adjacent element fails and results in sudden fatal injury (i.e. Structural) 4 Major Adjacent element fails and likely results health and safety risk

3 Significant Adjacent element fails and may result in health and safety risk 2 Critical Adjacent element fails with likely impact to occupant discomfort 1 Functional

Failure

Adjacent element fails with unlikely impact to occupant discomfort

Column 4: Detecting Failure

1 Very Obvious Almost certain to be detected before failure 2 Easily detectable Highly likely to be detected before failure 3 Reasonably Detectable Moderate chance of detection

4 Difficult to Detect Rarely detected before failure 5 Unlikely to Detect No chance of detection before failure

Weightings are prepared by multiplying the scores; i.e. probability of failure x severity of failure x impact on adjacent elements x detecting failure. Totals are scored into one of three categories, as follows:

• 1-81 = low risk

• 82-255 = medium risk

• 256-625 = high risk

CCMC says (italics added) that “In general ...:

• Low risk products typically have no risk to the health and safety of the public. Low risk products that have built in redundancies or a high factor of safety typically do not require the experience of skilled or specialized trades to employ in the field.

• Medium-risk products will be defined as having a minimal risk to the health, safety, and welfare of the general public. These types of products may or may not require specialized trades or a high level of skill to employ in the field.

• High-risk products will be defined as having a high potential to harm in regards to the health, safety, and welfare of the general public. These types of products

generally require specialized trades or a high level of skill to employ in the field. Clients whose products are considered and quantified as high risk will be directed

(22)

The CCMC’s risk assessment does respond to issues raised by the CHBA and others. However, there are some concerns about:

• definitions used

There seems to be a wide range between category 1 (“no risk”), category 2 (“minimal risk”) and category 3 (“a high potential to harm”), for example. Also, the use of “critical” and “significant” as descriptors for the middle category of risk ratings – normally used for “average”

• use of “occupant discomfort” as a risk factor

• the assumption that a product failure must cause adjacent elements to fail

• outcomes from the formula

It seems appropriate that a product with ratings of 1 in probability of failure, 1 in severity of failure, 5 in impact of failure on adjacent elements and 5 in difficulty of detection would score 25, for example, or very low risk. That this same rating would be given to a product with ratings of 5 in probability of failure, 5 in severity of failure, 1 in impact on adjacent elements and 1 in difficulty of detection seems perverse. There is a question whether columns 3 and 4 should even be considered for products with low scores in columns 1 and 2. Scoring for composite systems may also create counter-intuitive results. See below.

Scoring for systems with several components

In a sample risk assessment for a composite product recently provided by CCMC, the four components individually were scored at 90, 72, 72 and 96. (Using CCMC’s characterizations above, that would be minimal risk, no risk, no risk and minimal risk.) To calculate the rating for the product as a whole, those numbers were simply added together. That resulted in a final score of 330, or high risk. See below:

Sample CCMC Risk Assessment Table

Risk Assessment Results

Item (System) Risk Area (Component) Probability of Failure Severity of Failure Impact on Adjacent Elements Detecting Failure Total Score1 (Risk) Item 1 Component 1 2 3 3 5 90 Component 2 4 3 3 2 72 Component 3 2 3 3 4 72 Item 2 Component 4 2 4 3 4 96 Total 330

Adding risk factors together appears to produce a perverse result. The risk analysis process as a whole should be examined for shortcomings and modified as appropriate.