Proficiency testing for calibration of multimeter

In 2018 and 2019, CT2M organize an inter-laboratory comparison of multimeter calibration in which 27 Europeans laboratories participated. Among them were calibration laboratories (accredited or not) but also laboratories performing themselves the calibrations of their multimeter. The principle of this inter-laboratory comparison is to circulate a multimeter from one laboratory to another in order to compare the calibration results (including correction and calibration uncertainty). The processing of the results is carried out according to the ISO 13528 statistical principles and in compliance with the requirements of ISO 17043. A final report in 2019 will indicate to the participants all the results in anonymous way and the performance scores to evaluate the ability of laboratories to carry out this calibration. This article will present the organization of this inter-laboratory comparison as well as the results obtained.


Introduction
Accreditation body require accredited laboratories to regularly participate in interlaboratory comparisons (ILC) to prove their ability to perform tests or calibrations. Since 2014, CT2M organized inter-laboratory comparison in various fields to meet that need.
In 2018, an inter-laboratory comparison was organized for the calibration of multimeter. 27 laboratories took part in this round which took place from April 2018 to April 2019. The participants were calibration laboratories (accredited or not) as well as testing laboratories carrying out internal calibrations and / or controls of their multimeter.
This proficiency testing scheme was organized in accordance with the requirements of ISO 17043 and the results processing and participant performance study were conducted according to the principles of ISO 13258. The results obtained are presented in this paper.
To ensure the stability of the multimeter during the ILC round, an accredited reference laboratory, which was not part of the participants, calibrated at the beginning and at the end of the round.

Procedure of the proficiency testing
Calibration method is not imposed. Laboratories were allowed to choose their internal method. Each laboratory used its procedure and was free to choose the number of répétitions. Calibration points are detailed as follows (Table 1) Informations about range, configuration and frequency were given to participants The participants had to determine the correction of each calibrated point. In addition, the following information could be mentioned : ✓ The expanded calibration uncertainty (k=2), ✓ Influence factors considering for the uncertainty calculation, ✓ Environmental conditions during calibration.

Hypothesis for data analysis
The purpose of the proficiency testing is to conduct an assessment of laboratory performance by comparing their results with each other and against a reference value. Assumptions were considered for the data analysis to make consistent conclusions on the results.
In accordance with the reference standards [1] [2] [3], the assumption of a normal distribution of the data series has been made. This hypothesis has been verified for for all results using the following two methods: The Grubbs test was performed on all corrections determined by each of the participant.
The purpose of this test is to identify a laboratory with an incoherent result compared to other participants. The test involves calculating the Grubbs parameter (G) and comparing it to the critical Grubbs values of ISO 5725-2 [3].
Application of this GRUBBS test highlighted: -11 outliers in continuous tension especially on low values -7 outliers in alternative tension -7 outliers in continuous current -11 outliers in alternating current especially on strong values -14 outliers in resistance.

Reference value and uncertainty
The reference values of this proficiency testing were obtained by a reference laboratory that is accredited according to ISO/IEC 17025.
It calibrated the multimeter at the beginning and at the end of the ILC round. The reference value xref is the average of both correction.
Its associated uncertainty Uref takes into account the uncertainty of the reference laboratory as well as the possible drift of the multimeter between the beginning and the end of the round.

Robust mean and robust standard deviation
The robust mean x* and the robust standard deviation S* are determined using the algorithm A defined in ISO 13528. The robust average and standard deviation are used to evaluate the Z-score of each participant.
The Table 2 shows the parameters obtained for all the tests : -x*: robust average -S*: robust standard deviation -Ux* : uncertainty on the robust average

Comparison between the robust averages and the reference values
When organizing an inter-laboratory comparison, it is important to ensure that the robust averages of the participants results are not significantly different from the reference values.
For instance, the table (tab.2) below show a good correspondence between the robust means and the reference values for calibration points.
In  Results in V Table 2.

assigned values in DC voltage
The number En highlights the difference between the robust average with its uncertainty and the reference value with its uncertainty ( Table 2). The number En between both the results are correct for all the calibration points.
It is therefore possible to conclude that there is no significant difference between the reference values (obtained by the reference laboratory) and the robust averages (participants results).

Participant results
For instance, the results for calibration point 1000V range 1000V are presented in the following graphs ( fig.1). They show the correction and their uncertainties associated with k=2 (error bars).

Fig. 1. Correction 1000V Range 1000V
For each calibration point, a histogram also shows the frequency of the results (number of laboratories) according to the class of values.

Participant performance
The laboratories performance is determined by the Z-score that is a standardized measure of the bias. This performance score is calculated using the following formula: The reference value x* is the robust average of the participants results, the value xlab is the value obtained by the laboratory and the parameter S* is the robust standard deviation of the participants results. For each participant, the z-scores were calculated for each calibrated points ( Fig. 2 -range 1000V). The Z-scores between -2 and -3 or 2 and 3 correspond to isolated result. The Zscores less than -3 or greater than 3 correspond to discordant result. If a laboratory obtains a positive (or negative) Z-score, this highlights an overall bias (systematic error) on the calibration process compared to the average of the participants.
An interpretation of the results in relation to the reference value was also performed using the number En. It is a parameter that lead to evaluate wthe difference between 2 values. The values are compared taking into account their associated expanded uncertainties. The results of the participants are therefore compared to the reference value. The number En is calculated using the following formula: xref is the reference value (reference laboratory), xlab is the laboratory value, -Uref is the uncertainty (k=2) on the reference value, -Ulab is the uncertainty (k=2) on the laboratory value.
The Figure 3 shows the results of all the numbers En for the calibrated point range 1000V. If a laboratory obtains a positive (or negative) number En on all the calibration points, this highlights an overall bias (systematic error) on the calibration process compared to the reference value. This bias is all the more so important as the number En is far from 0.

Fig. 3. Numbers En of each participant
The interpretation of the number En must be done with caution. Indeed some laboratories have significant uncertainties, which leads to a correct standardized deviation (less than 1 or more than -1) despite significant bias compared to the reference value.