Classification of characterization/certification schemes

All of the major RM characterization/certification approaches, for total concentrations of constituents, can be classified in one of four categories. A fifth approach deals with method-specific schemes in which characterization is by a defined method giving a method-specific assessed property value. The following is a classification based on the author's interpretation and adaptation of descriptions of RM certification procedures in the literature [29, 30].

(1) Definitive method — one organization: a single definitive method used by a single organization, of the highest reputational quality, preferably applied in replicate by two or more highly skilled analysts, in more than one separate laboratories, working totally independently, preferably using different experimental facilities, with equipment and expertise to ensure traceability to the SI system. An accurately characterized, independently different, backup method, independently applied, is employed to provide additional assurance that the data are correct.

The term “definitive method” is applied to an analytical or measurement method that has a valid and well described theoretical foundation, is based on sound theoretical principles, (“first principles”), and has been experimentally demonstrated to have negligible systematic errors and a high level of precision. While a technique may be conceptually definitive, a complete method based on such a technique must be properly applied and demonstrated to deserve such a status for each individual application. A definitive method is one in which all major significant parameters have been related by a direct chain of evidence to the base or derived units of the SI. The property in question is either directly measured in terms of base units of measurement or indirectly related to the base units through physical or chemical theory expressed in exact mathematical equations. The written protocol indicates how each of these critical parameters in the measurement process has been controlled, how traceability to the base units has been accomplished, and what the bounds are to the limits of systematic error and thus uncertainty. Such methods, applied with high reliability, give 'true values' and provide the fundamental basis for accuracy in chemical analysis. Examples of definitive methods are: isotope dilution mass spectrometry, gravimetry (including fire assay analysis), coulometry and calorimetry.

Limitations of time, technical skills, specialized equipment and resources preclude the widespread use of the definitive methods. Furthermore, most analytical methods cannot be classified as definitive methods, usually because there is no straightforward theory relating all the experimental variables to the final result (e.g. the common techniques of atomic absorption and emission spectroscopies), or because effects, including matrix effects, are too complex to handle by theory. The certification of an RM by one measurement method requires a method of high scientific status,

laboratories of the highest quality and skilled analysts. The method must be sufficiently accurate to stand alone and reported results must have negligible systematic errors relative to end use requirements of the data. The acceptance of an RM certified in this way depends on the user community's confidence in the ability of the certifying agency to carry out the definitive method.

Independence of analysts and analyses in one organization is a fundamental question. It is important to have, even for the most reliable methods, more than one analyst/laboratory involved to avoid possible analyst/laboratory specific biases; certification by a single laboratory, without confirmation by another laboratory or method is risky. Measurement by a single definitive method is usually performed by two or more analysts working independently to minimize possible biases.

Frequently, an accurately characterized backup method is employed to corroborate the data. Some agencies feel that a certification campaign should not be based on a single measurement procedure and therefore do not normally certify values on the basis of a definitive method applied in one laboratory.

(2) Independent reference methods — one organization:Two or more independent reference methods, each based on an entirely conceptually different principle of measurement, independent in theory and experimental procedure, applied in replicate, within a single organization, of the highest reputational quality, by two or more expert analysts, working independently. The methods used can, naturally, include definitive methods. The results should be corroborated by a third or additional, independently different, accurately characterized, well established, thoroughly validated, definitive, reference or other methods.

A reference method is defined as a method of known and proven accuracy, thoroughly validated and experimentally demonstrated to have negligible systematic errors and a high level of precision. Its development involves removing the principal systematic errors of the measurement process, reducing them to tolerable levels, or when actual physical elimination is impossible, applying correction factors. The meaning of the term "independent" is that the basic theoretical and experimental principles on which one method rests, are entirely different from the principles of the other method(s).

Reference methods are generally arrived at by consensus and fairly extensive testing by a number of laboratories. For example, the flame atomic absorption method for Ca in serum developed under the leadership of NIST [41] was established after several interlaboratory comparison exercises.

The results were evaluated after each exercise and the procedure was changed as necessary. After five exercises, it was felt that the state of the art had been reached, with the reference method being capable of measuring Ca in serum with an accuracy of ±2% of the true value determined by IDMS (note that attainment of high accuracy and precision is not only a matter of the method but is a function of both the method and analyst expertise).

Since definitive methods are often unavailable, the multimethod approach is more frequently used in certification. A necessary condition for the certification of constituent concentrations is that determinations must be made by at least two independent, complementary, valid, reliable methods to avoid systematic errors associated with any one particular method or technique. Such measurements must agree within reasonable limits to permit certification. If significant discrepancies among analytical results from the different methods occur, additional work is carried out to reconcile them;

otherwise the property values cannot be certified. Every effort should be made to use methods based on more than one principle of measurement (three independent principles being desirable) and to engage trained and experienced analysts. The independent reference methods approach is based on the rationale that the likelihood of two independent methods being biased by the same amount and in the same direction is small. Therefore, when the results from two, three or more independent reference methods agree, one can have a high degree of assurance that they are likely to be accurate. The philosophy of basing RM results on determinations by at least two independent methods of analysis or on determinations by a definitive method is often referred to as 'the National Bureau of Standards approach'. It may be noted that NBS originally relied largely on approach 1, using isotope dilution mass spectrometry.

Concern on the independence of analysts in one organization still remains a fundamental question. Again it is important to have more than one analyst/laboratory involved to avoid possible analyst-specific and laboratory-specific biases. Characterization should be corroborated by additional methods or by laboratories in order to provide additional assurance that the data are correct.

(3) Independent reference and validated methods by selected expert analysts — multiple organizations and laboratories: Two or preferably three or more independent reference and/or validated methods, each based on an entirely conceptually different principle of measurement, independent in theory and experimental procedure, applied in replicate, by selected expert analysts, of high reputational quality and recognized competence working independently in an ad hoc network of laboratories participating in the collaborative interlaboratory characterization campaign under very carefully prescribed and controlled conditions. The methods used can, naturally, include definitive methods. All analytical methods are well characterized and established, thoroughly validated, of acceptable demonstrated accuracy and uncertainty, and the exercise is planned to incorporate widely different methods, based on different physical or chemical principles. The analysts, are carefully selected, on the basis of their established capabilities for the consistent production of precise and accurate results, reputation, expertise and experience in the specific field of analysis, familiarity with the matrix investigated, appreciation for the RM development concept, and a sense of healthy scepticism, and participate, on invitation, in the analytical campaign.

For the wide variety of materials and constituents in RMs, reference methods and definitive methods as well as in-house, single organization competencies are often not available. Thus the certifying agency cannot utilize certification approaches 1 or 2 but must resort to this approach relying on independent analysts and laboratories, using different (validated) methods. In this, a combination of definitive and reference methodologies applied by a single organization (approaches 1 and 2) is augmented by input from external analysts. This characterization philosophy is a variation of the two or more independent and reliable method approaches and can be briefly denoted as the expert analyst - different independent method approach. This characterization strategy is viable as long as the selection process selects analytical chemists with the requisite expertise (specific type of measurements and materials) and proven track records of performance, using definitive, reference or validated methods of analysis. Technical discussions with all participants before and after the exercise, as practiced by BCR, is beneficial.

The general premise behind this concept of certification by interlaboratory measurement is based on at least two assumptions: (a) There exists a population of laboratories that is equally capable in determining the characteristics of the RM to provide results with acceptable accuracy; (b) the differences between individual results, both within- and between-laboratories, are statistical in nature regardless of the causes (i.e. variation in measurement procedures, personnel, equipment, etc.). Each laboratory mean is considered to be an unbiased estimate of the characteristic of the material. The interlaboratory comparison mode has been widely used by national and other laboratories for the certification of RMs. The following guidelines, enunciated by NIST are instructive. According to NIST [40], this is a mode that must be used with the greatest restraint and under very carefully prescribed and controlled conditions. At that agency, this approach is used only when the following circumstances apply: (1) The RM under study is in a technical area that is well established and one where many good, reliable methods exist, (2) each of the laboratories in the network are of very high quality and are known to produce very reliable results, (3) each laboratory agrees to the conditions set forth by NIST, (4) NIST controls the experimental design and evaluation of data, (5) a previously issued RM, having similar properties to the RM candidate is used by each laboratory as an internal quality control check. When these conditions are met and maintained, this mode may be used with assurance to produce RMs of high accuracy and integrity. It must be assured that a wide range of reliable independent methods is covered with an absolute minimum of two but preferably three or more; it is furthermore beneficial and advocated that each method is used by at least two but preferably three independent analysts/laboratories. With entirely different basic principles used for the analysis, possible interferences or other systematic errors can reasonably be expected to be

different. Each analyst should use well established method(s), which demonstrate adequate performance in terms of trueness (no significant bias) and reproducibility (standard deviation consistent and explicable on the basis of random experimental errors). If definitive and reference methods exist for the particular constituent/matrix combination, they should be targeted for inclusion in the repertoire of methodologies sought for certification.

Although sophisticated methods may constitute the core methods for certification it is useful to include good, well executed routine methods. In order to further minimize systematic error, a conscious purposeful attempt should be made to get methods and procedures with wide-ranging and different sample preparation steps, including no decomposition as in instrumental neutron activation analysis and particle induced X ray emission spectrometry.

An overriding criterion regarding selection of methods and laboratories is the reputation, expertise and dedication of the scientist, analytical chemist, analyst or technologist conducting and responsible for the analyses. The choice is made on the basis of the analyst, not the laboratory, although the laboratory should have an acceptable reputation and environment conducive to good work. The laboratories, collaborating in the analytical campaign should be carefully selected, without political, regional, administrative or other constraints on the basis of reputation, experience in the specific field of analysis, familiarity with the matrix to be investigated and the availability of the required analytical technique. Participation is by invitation.

It is not necessary that participating laboratories be formally recognized, accredited or certified.

Measurement of the property of interest should be completed by, or under the supervision of a technically competent manager qualified either in terms of suitable academic qualifications or relevant work experience. The participating laboratory should consider the analysis as a very special one, to be performed with special attention and all possible care, and not have it performed as part of its regular routine.

(4) Volunteer analysts, various methods — multiple organizations and laboratories:A "round robin" exercise with the participation of volunteer analysts in many laboratories, volunteering freely to participate, or chosen either fairly completely at random or selectively according to some selection criteria, based on political, regional, administrative or other constraints, which may or may not be based on expertise or competence, due to an obligation to involve laboratories from a defined population of countries, regions or other groupings. Analytical methods used are varied, generally self-selected, and include reference, validated, non-validated, routine, as well as definitive methods, and this interlaboratory characterization exercise is carried out without imposition of prescribed conditions and controls. More in-depth statistical treatment is needed to deal with the wide range, and likely, discordant nature, of analytical results received. Calculations and reassessment of data reported in the literature, to arrive at estimates of concentrations and uncertainties, can be considered a component of this approach.

It is not always possible to have certification based on analyses done in-house or by selected laboratories according to strategies 1, 2 or 3 defined above. One must then resort to the last, and in the author's opinion, least preferred, mode of method-independent characterization, based on analysts, freely volunteering to participate, or selected without necessarily solid regard for expertise or competence, utilizing various methods. This approach, based on volunteer analysts and various methods in multiple organizations and laboratories, represents a round-robin type approach. Since no controls have been imposed upon the investigators, the limitations of such an approach and the data there from must be recognized. It must be appreciated that no mathematical processing can prove the validity of a concentration value derived from a mass of widely scattered data, the typical outcome of an exercise involving contributing analysts of varied backgrounds. Excellent insight into the problems associated with this approach has been provided by several experienced practitioners [34–37, 42–45].

Ingamells (1978) [46], in fact, suggested that the "round robin collaborative analysis" approach was a waste of time and effort and proposed instead a strategy involving only two mutually independent analysts, working in different laboratories and presumably using mutually independent methods. Parr

[44] felt that one of the criticisms that can be levelled against this type of certification procedure is that the participating laboratories are self-selected and some may have very little experience. He postulated that there could be considerable improvement in the analytical results if data were accepted only from experienced laboratories (e.g. approach 3 above). While some improvements in the confidence intervals associated with the certified values can sometimes be achieved in this way, the problem (of scattered data) certainly cannot be made to disappear; selection of laboratories can only be made on rather subjective grounds by the person responsible for certification. Abbey [34–37] has carried out many interesting statistical manipulations and calculations of literature-reported data for standard rocks, recalculating published recommended elemental concentration values. He clearly and forcefully observes that “Given a highly incoherent set of results for the determination of each constituent of a proposed reference sample, the originator is faced with the difficult problem of estimating the ‘true’ concentration. No known test can prove the validity of a concentration value derived from a mass of incoherent data” [37]. These observations apply equally well to all analytical endeavours.

(5) Method-specific; characterization by a specific, validated method by selected expert or experienced analysts — multiple organizations and laboratories: One specified analytical method applied in replicate, by selected expert or experienced analysts, of high reputational quality and recognized competence working independently in an ad hoc network of laboratories, participating under carefully prescribed and controlled conditions, giving a method-specific assessed property value. The analysts are selected, on the basis of their established reputation, expertise and experience with the method and the specific field of analysis and familiarity with the matrix investigated, and participate,on invitation, in the analytical collaborative interlaboratory characterization campaign.

In a few instances, RMs are certified for the value of a constituent or property that is method-dependent because existing technology or technical or scientific applications require this. In analytical chemistry, examples of this are the Kjeldahl technique for nitrogen, EPA mandated and other extraction procedures for leacheable toxic constituents, extraction procedures for soil nutrients and toxicants in agronomy and soil science, and various enzyme-based or enzyme measuring methods in clinical science. In such cases, demonstration of statistical control of the measurement process and agreement of results by independent analysts are the requirements for certification. As usually viewed by the RM scientist, the philosophy of certification rests on the concept of application of independent methodology to generate concordant results leading to one reliable value for the property. Such values are thus method-independent. Extractable and other concentrations generated by specific procedures are method-dependent, an idea which has to be rationalized with the fundamental method-independent concept in RM certification.