Traceability : implementing ILAC P 10 and ILAC P 14 – one accreditation body ’ s perspective

The International Laboratory Accreditation Cooperation (ILAC) has issued two policy documents on the subject of traceability and measurement uncertainty: ILAC P10[1] and ILAC P14[2]. All accreditation bodies (ABs) that have signed the ILAC Mutual Recognition Arrangement (MRA) are obligated to implement these policies. The authors describe one AB’s perspective on the difficulties encountered and lessons learned when implementing ILAC P10 and P14. This paper also offers insight into the AB’s decison-making process when creating requirements for accredited organizations with regard to measurement traceability and


Introduction
Traceability of measurement is a fundamental component in establishing equivalency of measurement results, whether it is for a National Metrology Institute (NMI) or an accredited calibration laboratory.Metrological traceability is defined by the VIM [3] as: " The property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty." The three important concepts in metrological traceability are the documented unbroken chain of calibrations, the measurement result and the measurement uncertainty.
The International Laboratory Accreditation Cooperation (ILAC) offered two documents on the subject : ILAC P10, which discusses its policy on the documented unbroken chain of calibrations, and ILAC P14, which discusses its policy on the measurement result and measurement uncertainty.These policies must be implemented by all ABs that have signed the ILAC MRA ; therefore, each AB must demonstrate that all of their accredited organizations use reference standards that are traceable to the International System of Units (SI) (when possible).This traceability is established with a documented unbroken chain of measurements through an acceptable National Metrology Institute (NMI) or an accredited calibration laboratory and includes an appropriate statement of measurement uncertainty.
The aim of this paper is to inform the reader of the practical applications and lessons learned in A2LA's implementation of ILAC P10 and ILAC P14 with its accredited laboratories.Areas discussed include how P10 is applied when there is a lack of certified or accredited reference materials, when an accredited calibration is unavailable or unattainable or when the calibration is performed by the owner of the reference standard or equipment "in-house".Also discussed are cases where the owner/user of the measuring and test equipment doesn't want measurement uncertainty and some of the challenges our accredited calibration laboratories encountered when implementing ILAC P14 in these instances.

ILAC P10
The original version of ILAC P10 was published in 2002 in an effort to harmonize key practices and policies between ABs in the area of traceability of measurements.In this policy document ILAC describes what it believes to be the ideal international system for traceability, while DOI: 10.1051/ C Owned by the authors, published by EDP Sciences, 2013 Web of Conferences also recognizing that many of the areas described required further development before they would be fully realized.The reason for this was because many identified areas were under the auspices of other entities, such as the BIPM[4] and CIPM [5].In 2002, it was recognized that the best that could be done was to identify key areas and then work with other parties to further develop the system for traceability.In 2013 an updated version of the policy was published that further describes the ideal system for traceability in more detail and reflects the consensus gained from the BIPM, CIPM, NMIs and the JCTLM [6].In this version, the policy was broadened to address metrological traceability requirements for not only ISO/IEC 17025:2005 [7] but also ISO 15189:2007 [8] and the policy is further recommended for other conformity assessment activities, such as inspection and product certification, where testing and/or calibration are involved.

Traceability defined
The 2013 version of P10 identifies three definitions related to traceability as taken from the VIM:

Metrological Traceability
Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty.

Metrological Traceability Chain
Sequence of measurement standards and calibrations that is used to relate a measurement result to a reference.

Metrological traceability to a measurement unit
Metrological traceability where the reference is the definition of a measurement unit through its practical realization.

P10 policy components
There are three key components to the ILAC policy on traceability but we will only cover those that relate to calibration and reference materials.The policy is paraphrased here: a) For Calibration equipment and reference standards that must be calibrated they shall be calibrated by: 1

Where traceability to an NMI or another accredited calibration laboratory cannot be established
What does an AB do when one of their accredited testing or calibration laboratories cannot achieve traceability directly to an NMI or accredited calibration laboratory?This routinely occurs when: 1. the only traceable calibration provider available is an approved NMI and/or 2. the owner of the reference standard cannot find an accredited calibration laboratory to calibrate the instrument or to calibrate it within a necessary range or specification.A2LA does allow exceptions to our traceability policy in the above noted cases by allowing the owner of the reference standard to use a non-accredited provider as long as they can provide A2LA with a complete "reverse traceability" study.
So just what is a "reverse traceability" study?This consists of a copy of each and every calibration certificate for all of the reference standards used between the owner's instrument and an approved NMI (usually NIST) or an accredited laboratory.If the owner's equipment is at the bottom of a very long traceability chain this could take a significant amount of time and effort to achieve.In these cases, the owner sometimes runs into a situation where, for example, the "fifteenth company" in the chain refuses to provide a copy of the calibration certificate(s) for the reference standard(s) used.This is typically due to confidentiality concerns and the fact that the owner of the equipment is not a direct customer.In some cases the "fifteenth company" wants to charge an exorbitant price for supplying a copy of the calibration certificates for the reference standards used, sometimes due to the time needed to retrieve the information.In both cases A2LA will often contact the company directly and can usually acquire a copy of the documents on behalf of the equipment owner upon signing a confidentiality agreement with the company in question.
It is important to note that A2LA retains copies of reverse traceability studies and catalogs them so that, where all things remain the same except the owner of the equipment or reference standard, the study could apply without that owner having to chase down the paper trail.
It is also important to note that, in cases where the accredited laboratory successfully submits a reverse traceability study, an exception is granted for the calibration interval of the instrument.
One drawback with this exception approach is that, because nearly all of the links in the chain are nonaccredited calibration providers, there is usually no information on measurement uncertainty until the receipt of an accredited calibration laboratory calibration certificate or an approved NMI calibration certificate.A statement of compliance is typically provided, however.It should be noted that A2LA does not accept, under any circumstances, traceability to an NMI that is not a signatory to the CIPM MRA and that does not participate in the measurement comparison activities of the CIPM.
You may ask why A2LA doesn't simply require the owner of the equipment or reference standard to use an approved NMI where an accredited calibration provider is not available.A2LA remains sensitive to the needs of our accredited calibration laboratories where use of the NMI would be too cost prohibitive and/or the time commitment involved in using the NMI would be too detrimental to the calibration laboratory's ability to provide timely service at a reasonable cost to its customers.In cases where the owner of the equipment does not elect to obtain the "reverse traceability" study for A2LA, their only other options are often to remove the capability from their scope of accreditation or use an approved NMI, such as the National Institute of Standards and Technology (NIST), to calibrate the reference standard in question.

What about reference materials?
In the case of reference materials, A2LA has a policy that the laboratory must use an accredited reference material from an ISO Guide 34:2009 accredited reference material producer (RMP) accredited by a signatory to the APLAC[12] Mutual Recognition Arrangement (MRA), where available.Internal reference materials are required to be checked in accordance with ISO/IEC 17025:2005.Deficiencies are only cited in cases where an accredited reference material is available but not used by the accredited laboratory.In cases where a reference material is in use because no accredited reference material was available at the time, the continuing challenge for A2LA accredited laboratories is keeping up with newly available accredited reference materials as more RMPs become accredited.

What about legal metrology?
In the United States all legal-for-trade weights and measures are required by regulation to be calibrated by a state government weights and measures laboratory.These include everything from gasoline pumps that measure fuel sold to automobile drivers to scales at the airport used to charge a customer for an overweight bag.Because of these regulations, calibration laboratories who service legal-for-trade weights and measures are required to send their reference standards to a state weights and measures laboratory for calibration.This presented a traceability problem since the state laboratories were not ISO/IEC 17025 accredited and A2LA policy requires the use of either a recognized NMI (one that participates in the BIPM KCDB) or an accredited calibration laboratory (capable of calibrating the reference standards in question) whose AB is a signatory to the ILAC MRA.
Nearly half of the active U.S. state weights and measures laboratories have gained accreditation through the National Voluntary Laboratory Accreditation Program (NVLAP).The National Institute of Standards and Technology (NIST), Office of Weights and Measures measurement assurance program as published in NIST Handbook 143, Program Handbook also provides a process whereby nearly all of the active weights and measures laboratories voluntarily implement ISO/IEC 17025.All of the state weights and measures laboratories that are recognized by the NIST Office of Weights and Measures are required to comply with program requirements that include completing NIST training, submitting regular evidence supporting claims of metrological traceability, and successfully participating in proficiency testing in each area of their measurement Scope.The current status of these laboratories is regularly updated and posted here: http://www.nist.gov/pml/wmd/labmetrology/lab-contactsac.cfm.It was on this basis that A2LA accepted the results of U.S. state weights and measures laboratories as meeting our traceability policy.

Internal or in-house calibration and traceability
What do you do about the testing or calibration laboratory that calibrates its own reference standards or equipment in-house ?How does an AB ensure traceability of the measurement ?First, we define an inhouse calibration as an internal calibration, not listed on an organization's scope of accreditation.For the calibration laboratory this means that, where the capability is found on the scope of accreditation, the calibration is considered as an external calibration, subject to the requirements of ISO/IEC 17025.

12001-p.3
A2LA requires all in-house calibrations to be supported by seven elements : a) The in-house laboratory must maintain documented procedures for the in-house calibrations ; b)

ILAC P14
ILAC P14 was written to harmonize among ABs the expression of measurement uncertainty on calibration certificates issued by accredited calibration laboratories and to harmonize scopes of accreditation issued by ABs.The goals of ILAC P14 are: • to ensure the GUM and the CMC [14] were applied uniformly by ILAC signatory members; and • to ensure metrological traceability is established in accordance with the VIM.Since the implementation deadline for the 2013 version of ILAC P14 is January 2014, this paper will only address the current (2010) version of the policy.

P14 policy components
There are three main policies in P14.The first is related to the estimation of measurement uncertainty, the second to scopes of accreditation of calibration laboratories and the third to statements of measurement uncertainty on calibration certificates.It was the third policy that was, by far, the most challenging for A2LA to implement among its accredited calibration laboratories.

Difficulties in implementing P14
Implementing ILAC P14 has been challenging for most ABs and A2LA is no exception.The issues encountered by A2LA, however, were wholly unexpected and ranged in nature from international interpretation of the policy to an unanticipated number of customers with contracts that ran counter to the requirements of P14.
When the new ILAC P14 was published, A2LA consulted our Measurement Advisory Committee along with our Criteria Council, which approves all new accreditation policies, in order to design an implementation plan for this new policy.During the course of the discussions, myriad questions arose on how this policy would be implemented for contract exclusions, statements of compliance and implicit uncertainty statements (such as a Test Uncertainty Ratio).
For example, would it still be acceptable to include only an implicit uncertainty statement on an endorsed calibration certificate (those that contain the A2LA Accredited symbol)?Or, would it still be acceptable to include only a statement of compliance on an endorsed calibration certificate, such as pass/fail?Furthermore, the biggest concern to A2LA was whether to allow our calibration laboratories to "opt-out" of any or all of the requirements of P14 via a contract with their customer; and, if so, under what circumstances.When making our decision, A2LA weighed this option and the sole use of implicit uncertainty statements or statements of compliance against the goals of P14 and, after careful consideration, A2LA decided that allowing an "opt-out" for any P14 requirements by means of a customer contract or the sole use of implicit uncertainty statements or statements of compliance would not meet the intent of metrological traceability.Since ISO/IEC 17025 already required the calibration laboratory to calculate and retain the measurement uncertainty and take it into account when making a statement of compliance, A2LA decided that adding the uncertainty to the calibration certificate would not be overly burdensome to our accredited laboratories in order to achieve metrological traceability.

12001-p.4
What this meant for A2LA calibration laboratories was that all calibration certificates endorsed with the A2LA Accredited symbol were required to also include the measurement result and measurement uncertainty in accordance with ILAC P14.Although A2LA did not expect that the transition to ILAC P14 would be simple, the problems we encountered once P14 was implemented were surprising.
These include: • varying interpretation of section 6.1 of ILAC P14 among ABs around the world; and • the need for some of our customers to make changes to their automated processes in order to accommodate the requirements of P14; and • an unexpected number of laboratories who had contracts with their clients to exclude measurement uncertainty from the final calibration certificate.

Varying interpretation of P14 among ABs
Not all Accrediting Bodies that are signatories to the ILAC Mutual Recognition Arrangement could agree on the interpretation of section 6.1 of P14.
• Section 6.1 states, "ISO/IEC 17025 requires calibration laboratories to report, in the calibration certificate, the uncertainty of measurement and/or a statement of compliance with an identified metrological specification or clauses thereof.
Accredited calibration laboratories shall report the uncertainty of measurement, in compliance with the requirements in 6.2 -6.5 of this section."Some ABs felt this language still allowed for either inclusion of the measurement uncertainty or incorporation of a statement of compliance with a specification in accordance with ISO/IEC 17025:2005.Others felt it clearly mandated the inclusion of the uncertainty of measurement on calibration certificates to support the 2008 version of the VIM.Because of the confusion over the language of the policy, some ABs delayed implementation of ILAC P14 until clarification was made.While this disagreement seemed contrary to the intent of the policy -that of ensuring that the GUM and CMC are applied uniformly among ILAC signatory members -it was important to clear up the meaning of this section so that all ABs could be in harmony when enforcing the policy.
Recently the new version of P14 was released with an addition to section 6.1 that states: "By exception, and where it has been established during contract review that only a statement of compliance with a specification is required, then the measured quantity value and the measurement uncertainty may be omitted on the calibration certificate.
The following shall however apply: • The calibration certificate is not intended to be used in support of the further dissemination of metrological traceability (i.e. to calibrate another device); • As specified in ISO/IEC 17025:2005 clause 5.10.4.2, the laboratory shall determine the uncertainty and take that uncertainty into account when issuing the statement of compliance; and • The laboratory shall retain documentary evidence of the measured quantity value and the uncertainty of measurement, as specified in ISO/IEC 17025 clauses 5.10.4.2 and 4.13, and shall provide such evidence upon request." This revision seems to have resolved the matter in dispute; however, what affect the revision will have on accredited calibration laboratories (if any) will not be known until implementation has been fully realized.

Inclusion of measurement uncertainty on calibration certificates
Another challenge for A2LA was the unexpected impact on the automated processes of our customers for inclusion of the measurement uncertainty on the calibration certificate.Calibration certificate templates that originally only reported a statement of compliance, such as pass/fail, needed to be changed to accommodate reporting measurement uncertainty.Similarly, calibration software used to calculate measurement uncertainty had to be amended for the prescriptive rounding rules (among other things) called for in ILAC P14.Accomplishing these changes involved a significant investment of resources by our customers, depending on how many templates were involved and how many uncertainty calculations were impacted.In many cases our customers didn't have the appropriate resources allocated in their budget and/or enough manpower to make the necessary changes in the time allotted by ILAC for implementation.In these cases A2LA has allowed our accredited laboratories more time to achieve compliance.

Clients who did not want or need measurement uncertainty
The remaining issue in implementing ILAC P14 was the number of customers who had existing contracts with their clients that allowed for the exclusion of measurement uncertainty from the calibration certificate.Since A2LA felt that allowing such exclusions would not meet the intent of metrological traceability, A2LA required all endorsed calibration certificates to contain the measurement uncertainty.
The calibration laboratories that had relied on these contract exclusions for years now found themselves having to calculate the measurement uncertainty and 12001-p.5 report it on the calibration certificate or, alternatively, only offer a non-accredited calibration certificate (those without the A2LA accredited symbol).Again, this required a significant investment in time and resources for the accredited laboratory to accomplish.Furthermore, many of the clients of our accredited calibration laboratories that did not want measurement uncertainty were very vocal in informing their calibration provider that they wanted it removed from their certificates.A great deal of time and effort has been made by A2LA to educate our customers and their clients on the new policy and on why it's important that the measurement uncertainty be reported on the endorsed calibration certificate.

Conclusion
With the implementation of every new policy there are unintended consequences for both the accrediting bodies and their accredited laboratories.This paper describes some of the difficulties encountered when implementing two ILAC policies, P10 and P14, and our key to success in overcoming these issues turned out to be flexibility and patience.
A2LA found that we had to allow some flexibility over requiring the use of an NMI when they are the sole provider of calibration service, to patiently provide assistance to our CABs to establish a reverse traceability study, and to allow extra time for implementation when the new policy significantly impacts the calibration laboratory's resources.