Metrologically correct decision-making in vehicle testing

Résumé. Dans tous les tests les décisions d’approbation sont fondées sur les résultats de mesure qui sont comparés avec les niveaux de tolerance. L’ídée métérologique aussi prend en consideration l’incertitude de mesure. L’approbation d’un système de frein de voiture aussi signifie que le testeur de frein utilisé conforme à ses specification, lequel, en plus de cela, doit être déterminé par une calibration avec une incertitude suffissament basse. Ainsi nous pouvons discerner plusieurs niveaux d’incertitude -le test d’approbation – la calibration d’équipement – les moyens calibrés ou les références utilisées. Entre ces niveaux un certain proportion de refus, au moins de facteur 3, doit exister, pour que, finalement, une décision correcte (apprové/pas apprové) soit possible. Le document explique comment l’incertitude du résultat du test est reliée à la calibration de l’équipement du test, utilisant deux exemples de test de vehicules. Souvent la question, quelle incertitude peut être acceptée dans la calibration d’équipement, est posée par les organisations qui effectuent la calibration et aucune autorité peut déterminer les numéros généraux.


Introduction
The demand for regularly vehicle testing has existed long time and until quite recently this role was played by national organizations having a monopoly situation.What and how to test was mostly up to the testing organization (Bilprovningen in Sweden, TÜV in Germany etc.).The purpose still is to protect traffic safety and environment.The test programme has expanded over the years due to new functions, new fuels or improved test equipment.Ocular inspection and mechanical control have been complemented with measurement equipment to improve efficiency and trust in assessment.
The overall goal is to keep a certain technical standard and remove traffic dangerous vehicles or those that do not comply with environmental demands.Concerning the measurements performed an answer is requested whether or not a tested function complies with existing tolerances.But it is always the testing engineer that in every instant decides.And he has to state a "clear failure or deficiency".In order to rely on the measurement results the equipment needs to be traceably calibrated.But only very few strict levels are given concerning metrological demands, which gives space for metrologists to suggest how demands on uncertainty in different stages might look like to get a more neutral and less subjective decision.
One can state that there is an active market for the development of test equipment including various a Corresponding author: plau.gtbg@gmail.commeasurement quantities.Further the monopoly situation has changed to a market of vehicle testing institutions competing with each other where all are accredited (in Sweden by Swedac -Swedish Board for Accreditation and Conformity Assessment, in Norway by Norwegian Accreditation).Before this change a market of service organizations had established providing calibration services for those test equipment.Swedac which is a government authority responsible for quality and safety in Sweden as well accredits these organizations,.The regulation authority in Sweden is the Transportstyrelsen (Swedish Transport Agency) that sets up the rules for the testing.
Does this mean we have a regulated area that could be viewed similar to legal metrology?There the legislature sets the demands for measuring equipment to protect consumer interests.And there are accepted rules pushed forward by OIML what calibration laboratories need to achieve with respect to calibration and verification.

Goal for vehicle testing
Vehicles that do not comply with certain demands need to be repaired or must be removed from traffic; generally the most dangerous or most environmentally unfriendly.This is because a "clear deficiency" must be stated.In contrast to legal metrology rules this is interpreted as in case there is a grey-zone or some uncertainty a tested function cannot be undoubtedly classified as failed.The test should, however, be objective and measurements should serve this purpose as far as possible.
Can one with this perspective claim that low measurement uncertainty helps in decision-making?Of course, if a result is beyond a limit the instrument and method used have less and the object more responsibility for this, which makes a judgement more objective.The subjective "clear deficiency" does not disappear but is easier to state.The task to develop better test equipment, which can be calibrated more accurately, will make pass / no pass decisions easier.

Measurable quantities for approval/disapproval and demands
All together about 15 different measurements are performed depending on type of fuel, type and age of vehicle and what the inspection is aimed for.
The list contains: Angle and asymmetry of headlights; brake function for foot and hand brake individually on each wheel and possible non-balance between right and left side; retardation; different pressure points; various length, angle and noise measurements (only at special examinations); several gas concentrations in exhaust gas (CO, CO2, O2 and HC) to calculate the ʄ -value and check optimal combustion -general emission with OBD-system; opacity or light absorption coefficient in diesel fuel exhaust gas; leakage if natural gas is the fuel.
The general demand is that all equipment used for the measurement of a test quantity needs to be regularly calibrated against traceable standards and according to instrument suppliers or the authority responsible for drawing up regulations (in Sweden Transportstyrelsen).
Some of these statements are summarized shortly: Instruments for measuring exhaust gas from diesel engines must have sufficient accuracy.Noise measuring instruments must be of precision type class 2 or better.Length measurement equipment must be within ±5 mm at a length of 20 m.Measuring tapes has to follow accuracy class II and callipers follow a standard DIN 862.Timers need to have an accuracy of ±0,5 s per minute and angle measurement devices lie within a tolerance of ±0,5 °.When calibrating a retardation meter this should be done within ±3 %. [1] The list of real demands is short.The ʄ -value needs to lie in the interval 0,97 to 1,03.The two belonging gas concentration measurements that contribute to its calculation have a limit between 4,5 vol% and 0,3 vol% CO depending on the type of vehicle, motor and age and 100 ppm and 200 ppm HC depending on vehicle weight.Given an uncertainty for the measurement of the four gases CO, CO2, O2 and HC it would be possible to evaluate an uncertainty for the calculated ʄ -value (Brettschneiders equation).Concerning diesel engines a light absorption coefficient has to lie below K=3,5/m for turbo engines and below K=2,5/m for suction engines.For the brake test no limits are defined which brake force must be achieved.The breaks are operated until almost blockage and the test operator decides approval or not.The only explicit demand concerns a maximum non-balance between left and right wheel pair of not more than 30 %.
What can be stated from this short list is the following.The simpler the measurement the easier is it to quote suitable accuracies for the measurement instrument or the calibration service.Any uncertainty demand chain, however, should start with the measurement uncertainty necessary to decide if a result is inside or outside an allowed tolerance.But if there are no such values it is relative meaningless to express demands for an equipment or its calibration.

Chain of uncertainty demands in measurement and calibration
A general rule could be the following: In order to judge if a result is inside a tolerance limit the measured value plus the corresponding uncertainty have to be within the tolerance.This would mean the real value, which we never can know, has a probability of at least 95 % to be on the right side.The lower the uncertainty the nearer the result can approach the limit and still be within.If the test uncertainty amounts to maximum 1/3 of the limit value then the measurement uncertainty only accounts for roughly 5 % of the combination of measured value plus uncertainty.Thus one should try to achieve a turn down ratio of at least 3:1 between a limit and the corresponding measurement uncertainty.Figure 1 tries to visualize a demand chain starting from top and down to necessary standards.As mentioned several acceptance limits exist for carbon oxide in exhaust gas.How the 3:1 thumb rule would be applied to 5 such limits between maximum 4,5 and minimum tolerance 0,3 is demonstrated in table 1 showing what should be expected in terms of uncertainty in test and in calibration with respect to concentration in a reference gas of carbon dioxide.

Web of Conferences
How does this harmonize with findings in a real measurement starting the other way round, i.e. with the reference gas uncertainty that is given with a 2 %tolerance of the actual concentration?In the example the gas bottle holds 3,53 vol% CO with an uncertainty of 0,071 vol%.This is an ultimate limit how good a measurement could be and it is clear that there would not be a chance to test levels from 2 vol% and below.Uncertainty in measurement at test 1,5 1,17 0,66 0,33 0,1 Uncertainty in instrument calibration 0,5 0,39 0,22 0,11 0,033 Reference gas CO uncertainty at calibration 0,17 0,13 0,07 0,036 0,011 However, depending on circumstances the uncertainty in calibration can be better than a factor 3. Figure 3 below presents an uncertainty budget for the calibration of a 4-gas meter used at vehicle testing stations.
As can be seen there are seven contributions of which the uncertainty in the reference gas concentration u1 is the dominating one contributing with over 70 % to the final calibration uncertainty of 0,082 vol%.As the reference gas is the start for the calibration, the calibration uncertainty is the start for the measurement uncertainty (see figure 4).Here four contributions are important.First is u1 coming from the calibration amounting to about 50 % of the testing uncertainty.Second important is the last one u4.This contribution corresponds to the detected but not adjusted error in the 4-gas meter.As it is likely that the correction of -0,02 vol% is not performed in testing, this systematic error is added to the measurement uncertainty in a linear way (maxi-mum combination).This contribution could largely be avoided if the instrument were adjusted during calibration or if a corresponding correction were applied in testing, which most probably is not the case.Thus the total testing uncertainty actually could be reduced significantly.A picture how this finding matches the factor 3:1 rule of thumb is shown in figure 5. have this picture on the retina can be useful when demanding a calibration service to improve its calibration uncertainty or a testing facility to improve measurement uncertainty.Everything is depending on the uncertainty level of the reference gas concentration.Authorities not being aware of this hierarchically uncertainty chain might actually demand the impossible.
For HC-measurements with the two levels of 100 and 200 ppm the same chain could be demonstrated.

2 Brake force measurement
In brake testing the only defined demand is a maximum 30 % non-balance between measured left and right wheel brake force.A typical brake force is at least 2000 N and a 30 % non-balance corresponds to 600 N giving for example 1700 N for the left and 2300 N for the right wheel brake.One third would demand an uncertainty of 200 N at maximum for the difference measurement.Figure 6 shows a typical roller brake tester and figure 7 how a retarding reference force is applied to the motor axis using weights and a lever system.Using different weights or various lever arm length one can simulate different discrete brake forces.This is the basis to define the measurement model for the calculation of the calibration uncertainty.

30% at 2 kN = 600 N
It should be mentioned that sometimes also continuous brake forces are applied using calibrated load cells as reference and totally new brake test equipment is currently developed.
Assuming an experimentally determined reproducibility for the difference measurement of 150 N at a test station and a reasonable drift since last calibration of 60 N both for the right and left roller pair one would fulfil a 3:1 turn down ratio if each side could be calibrated with an uncertainty of maximum 72 N (see fig 8).
A reasonable calibration uncertainty for a 2 kN brake force is expect to lie between 30 and 50 N.This means the demand of U = 72 N during calibration of each side should be achievable.If a nonbalance of less than 600 N is detected the actual uncertainty is not questioned.
In the calculation above no correlation is assumed.In reality it is reasonable to assume a drift in the same direction for the right and left roller pair.And several uncertainty contributions in calibration are highly correlated reducing the uncertainty in the difference measurement.Among those one can mention the weight and length of all arms in the lever system (contributions u3 to u7 in the table below).
Thus, if the brake force at average is about 2000 N the testing uncertainty is acceptable.At higher brake forces the calibration uncertainty will increase steadily but so will of course also the uncertainty corresponding to a factor of 1/3 of the difference.
This means the calibration accuracy will most probably be sufficient the higher the brake force.However, the test station needs to show that the reproducibility when testing vehicle brakes for nonbalance between right and left is under control i.e. all personnel needs to perform the job the same way.Reproducibility here means different test persons, different cars and several repetitions in each case.

Summary
The area of vehicle testing is not regulated in a strict manner.Whereas there are detailed demands for a few measurements performed during vehicle testing the majority of these have no demands at all (no tolerances, no accuracy concerning the equipment used or the uncertainty in testing that should be achieved).
Even for the few test limits pointed out there are no demands for a measurement uncertainty, which is due to the variation in type and age of vehicle.This generates difficulties to define limits and prescribe suitable demands for the related measurement uncertainty during vehicle inspection.If, however, such demands existed the two examples can demonstrate how a maximum acceptable uncertainty in measurement and consequently in calibration could be derived.

Conclusions
Legal metrology has largely succeeded to internationally harmonize demands on the measurement quality when using certain equipment and thus generated trust that consumers can have confidence in mass or volume information in purchased products.A similar process regarding vehicle testing with its various test and decisions based on measurement results has only recently started in connection with the opening of an earlier monopolized market.Within the frame of legal metrology OIML has played an important role and national requirements have adopted the international demands.Within the frame of vehicle testing there are no such rules.There the suppliers of test equipment largely set goals mainly due to marketing concurrence what quality in testing can be achieved.Prescribing authorities (Transportsyrelsen in Sweden) probably lack competence and currently only point out that all equipment used in testing needs to be calibrated.Accreditation bodies (in Sweden Swedac) regulating this area since the end of the monopoly situation try to imprint by inspecting the accredited service organizations, but cannot set up goals for which uncertainties that are acceptable in testing and which are not.However, even if it is possible to construct reasonable uncertainty chains for testing and calibration from a pure metrological point of view there are two circumstances that prevent such a development.The one is the lack of clear limits for gono go borders in testing, which is intimately linked to the goal to not approve only at clearly stated deficiencies.Second a well-educated and motivated test personnel probably has better qualification to live up to this goal despite subjective judgements of what a "clear deficiency is than a strictly regulated demarcation.
DOI: 10.1051/ C Owned by the authors, published by EDP Sciences,

Figure 1 .
Figure 1.Uncertainty chain causing demands way down by starting at top.The same idea also holds for the calibration.The calibration uncertainty should be at least 3 times better than the demanded measurement uncertainty.Further the reference used for calibration should in turn have an uncertainty 3 times lower than the uncertainty aimed for in calibration.This is a rule of thumb before looking into details what makes up the uncertainty budget.

Figure 2 .
Figure 2. Typical 4-gas meter used at testing stations for calculating the ʄ -value

Figure 4 .
Figure 4. Uncertainty budget for measuring exhaust gas CO with a 4-gas meter testing a vehicle.

Figure 5 .
Figure 5. Example for a realistic measurement and preceding calibration uncertainty.

Figure 6 .
Figure 6.Roller brake tester for a wheel pair

Figure 7 .
Figure 7. Lever system to apply a reference force in roller brake tester calibration

Figure 8 .
Figure 8. Scheme for estimating a demand for calibration uncertainty.

Figure 9 .
Figure 9.An uncertainty budget valid for the calibration of a typical roller brake tester using weights