Risk calculations for conformity assessment in practice

. In 2012, the Joint Committee Guides in Metrology (JCGM) published novel guidance on the consideration of measurement uncertainty for decision-making in conformity assessment (JCGM 106:2012). The two situations of making a wrong decision are considered: the risk of accepting a non-conforming item, denoted as the customer risk, and the risk of rejecting a conforming item, denoted as the producer risk. In 2017, the revision of ISO 17025 obliged calibration and testing laboratories to “document the decision rule employed, taking into account the level of risk (such as false accept and false reject and statistical assumptions) associated with the decision rule employed, and apply the decision rule” in the context of the decision made about the conformity of an item. However, JCGM 106:2012 can in some cases be perceived as quite difficult to apply for non-statisticians as it mainly relies on calculations involving probability distributions. In order to facilitate uptake of the methodology of JCGM 106:2012, EURAMET is funding the project EMPIR 17SIP05 “CASoft” (2018 – 2020), involving the National Measurement Institutes from France, Sweden and the UK. The objective is to make the methodology accessible to organisations involved in decision-making in conformity assessment: calibration and testing laboratories, industrialists and regulation authorities. Where the customer or producer are concerned, there are two kinds of risks arising from measurement uncertainty: specific risk which concerns the risk of an incorrect decision for a particular item and global risk which is the risk of an incorrect decision for any item chosen at random. Both kinds of risk may involve prior information, taken into account through a so-called prior probability distribution, introducing the concept of a Bayesian evaluation of the risks. If a calibration and


Notion of risk
The problem of decision-making in conformity assessment arises in many industrial applications as soon as a measurement is performed. The measured value is then compared to a tolerance interval ; in order to decide whether the item (a product, an instrument, a material, etc.) can be considered as conforming to a specification. However, the uncertainty associated with such measured value makes it possible to take incorrect decisions, that is to accept a non-conforming item or to reject a conforming item. Such risks of incorrect decisions have to be quantified. To this extent, the Joint Committee for Guides in Metrology (JCGM) published in 2012 guidelines for such an evaluation [1]. Two kinds of risks are addressed: -The specific risk, defined as the probability of an incorrect decision for a particular item -The global risk, defined as the probability of an incorrect decision based on a future measurement result.
However, those calculations rely on relatively complex mathematical expressions which involve the evaluation of single or double integrals. Such complexity makes it difficult for practitioners to apply the methodology and to compute such risks. The aims of this paper are to explain which kind of risk should be considered and to introduce the software CASoft for calculating the risks. Such software is the target objective of an EMPIR (European Metrology Programme for Innovation and Research) SIP (Support for Impact Project) project [3], funded by EURAMET in order to facilitate uptake of the methodology by practitioners.
In this paper, we propose to distinguish between the specific risk and the global risk and we illustrate their calculations on the basis of a test case.

Probability distributions
Knowledge of the measurand is conveyed by two components: information that is available prior to measurement, also called prior information, and information supplied by the measurement. Both kinds of knowledge can be expressed using probability distributions. Using Bayes' theorem it is possible to determine the so-called posterior distribution: where : is a normalization constant which does not depend on is the prior distribution which does not depend on the measurement ; -| is the measurement distribution From the point of view of practitioners, the evaluation of measurement uncertainty often only relies on the measurement distribution itself, rather than on a convolution between the prior distribution and the measurement distribution. However, it is important to consider such convolution as it conveys all available knowledge about the measurand.
According to the application, both distributions may be available or not. In the case where only one is available, the posterior distribution is the same as the known distribution. In the case where both are available, the posterior distribution has to be computed. The posterior distribution may also be known if the prior and the measurement distributions are conjugate.

Specific risk
The scope of JCGM106:2012 [1] is that a decision is made on the basis of the measurement of a single quantity of interest. Only binary decision rules are considered, that is to say that the item must be either accepted (declared as conforming) or rejected (declared as nonconforming). There are no values for the quantity of interest that would lead to the state of no decisions (sometimes called the "doubt region").
Given that the only possible decisions are to accept or to reject the item, there are two associated specific risks: the specific consumer's risk, denoted here as * , which is the probability that an accepted item is non-conforming the specific producer's risk, denoted here as * , which is the probability that a rejected item is conforming.
Mathematically, both risks can be obtained as functions of the conformance probability , that is, the probability that a particular item is conforming : This expression involves the difference between the Cumulative Distribution Function (CDF) | evaluated respectively at and .
In the case where the tolerance interval is one-sided, the conformance probability is | when the measurement result is to be compared with an upper tolerance limit and it is 1 | when the result is to be compared with a lower tolerance limit .

Global risk
The global consumer's risk [1] is defined as "the probability that a non-conforming item will be accepted based on a future measurement". Accordingly, the global producer's risk is defined as "the probability that a conforming item will be rejected based on a future measurement". Such calculations can be very important for the industrialist in order to design their manufacturing process to ensure an efficient compromise between the risk of having conforming products rejected and the risk of "over-quality" of those products.
Such calculations involve not only the tolerance interval, which is given by the specification that the considered quantity is supposed to respect, but also the acceptance interval ; , which specifies the "acceptable" limits for the measured value.
Mathematically, the global consumer's risk is obtained as the value of a double integral: However, such double integrals can be simplified in order to make use of usual functions in probability: where | denotes the CDF of the measurement distribution. The same calculation is applied for . This approach is, in particular, used for those computations implemented in the CASoft software, which is scheduled to be available by the end of 2019. The same kind of calculation can be performed for the global producer's risk:

Calculation of a conformance probability
As a practical example, consider the control of the thickness of waste bags. European standard EN 13592:2003+A1 [4] indicates the lower tolerance limit 13.3 μm for the thickness at one particular location of a waste bag having a nominal thickness of 20 µm at that location. Suppose that the laboratory who performs the measurement has no prior knowledge about the thickness and is accredited for performing the measurement with an uncertainty of 1 μm. The CASoft Software enables these calculations to be performed and provides the end user with a graphical visualization of the considered risk. Figure 1 and Figure 2 show such graphs respectively for the second case and for the fourth case in Table 1.

Calculation of global risks
The manufacturer in the previous case sends to a laboratory a sample of its waste bags for testing. The laboratory is accredited for such measurements, with an associated measurement uncertainty of 1 μm. The acceptance interval used by the testing laboratory is the same as the tolerance interval and cannot be modified in order to limit the producer's risk because doing so would lead to an increase in the consumer's risk, resulting in more non-conforming waste bags accepted. Those bags would be less resistant due to them not being thick enough.
The quality control of the manufacturing process ensures that the waste bags are produced with a mean thickness of 15 μm and a standard deviation 2 μm. The application of Equation (8) yields a global producer's risk of 6.7 %. Such risk is represented in red in Figure 3. The quality manager made the observation that, despite their procedures and margins for manufacturing their products with a mean value of 15 µm, the conformity assessment decision was too often a rejection of the tested waste bags. As a result, the decision is made to use more material in their production process in order to shift the mean value of their produced waste bags from 15 μm to 17 μm. CASoft is used again for the computation of the risk incurred by this decision and the resulting global producer's risk is then 2.5 %. Such risk is represented in red in Figure 4. It can be seen that the joint distribution for the real value of the thickness and the measured value for conformity assessment is shifted towards the conforming region. As a consequence, the red area corresponding to the producer's risk falls into a region of lower probability.

CONCLUSION
The JCGM published in 2012 a reference document for decision-making in conformity assessment taking into account measurement uncertainty. The recent revision of the international standard ISO 17025:2017 [3] makes it even more important to address the problem of conformity assessment with uncertainty because it opens the possibility for entities involved in a conformity assessment problem to agree, prior to measurement, on the decision rule applied for the testing or the calibration.
The European project funded by EURAMET, entitled SIP CASoft, aims at providing the end users (industrialists, calibration and testing laboratories, regulation and standardization bodies, etc.) with software for the computation of the associated risks as well as communication activities designed to facilitate the uptake of the methodology. In particular, the developed software will cover the usual cases such as the consideration of Gaussian distributions and the presence or absence of prior information, as well as some situations that may occur in the field (other choices of probability distributions, the bivariate case, etc.).