Smart Metrology: From the metrology of instrumentation to the metrology of decisions

Metrology is of much greater importance to the industrial world than may at first appear. Quality certification has often reduced industrial metrology to the model used in legal metrology, a model that is inappropriate to it. At the dawn of a new era, it is important to reposition the mission given to metrology and allow it to take its rightful place at the heart of the revolution of industrial practices ahead of us. This paper summarizes the new expectations that the company metrologist is likely to face. These are developed in greater detail in the book Smart Metrology From the metrology of instrumentation to the metrology of decisions (" Smart Metrology: De la métrologie des instruments à la métrologie des décisions") published by AFNOR in 2016.


Introduction
Mathematician Didier Nordon considers metrology to be a bridge between geometry (the science of shapes) and algebra (the science of numbers) [1].This bridge gives a quantity (value) for the different measurable entities: the length of a part, the concentration of an active substance in a solution, the energetic efficiency of a building, etc.This quantification, a process that is now universal thanks to the International Units system, involves a chain of decisions that starts with the definition of requirements (tolerances) and ends with the declaration of conformity once the intervening processes have all been successfully performed.This chain manipulates numbers that express the capability of entities to fulfil their purpose, whether it be the ability of a medication to heal or of a mechanical part to fit into the assembly it has been designed for.This is true for all entity types and all prescribed uses… In a perfect world, everything is 'algebraic'.For the entity to be considered compliant, it is sufficient that the value representing magnitude (in the sense of VIM3 [2], definition 1.1) fits within the tolerance range used to define the requirement.All that needs to be done is to test whether the value achieved is higher (>) than a minimum value and/or lower (<) than a maximum value.
However, in reality, the result of a measurement (or measurement process) cannot be reduced to a single number.In defining 2.9, VIM3 [2] specifies that a measurement result consists of ' all the values attributed to a measurand, together with all other relevant information available'.Therefore, we do not have a single value to deal with, but a number of possible values, some of which may be compliant while others are not.This inability to know the precise value attaching to the entity causes doubt concerning its compliance, and consequently creates a risk when a decision is taken, and this is true at every level of the chain referred to above.
The job of Smart Metrology, some of the principles of which are presented in this paper, is to evaluate the risks and find a balance based on the 'strictly necessary'.So much more than a simple 'objective', the 'strictly necessary' has become primordial, and Antoine de Saint-Exupéry gave a very clear explanation of why: 'We do not inherit the earth from our parents, we borrow it from our children.' The time to be 'Smart' is right now, and it is up to metrology to ensure its place at the heart of the Smart movement.

Risk, what still?
The notion of risk has an ever greater place in our lives, as well as in industry.The concept 'Customer risk' and 'Supplier risk' is often heard.And from this concept, of course, comes the principle of risk-sharing.Yet, this terminology is in fact somewhat deceptive.
It is measurement uncertainty, engendered by the imperfection of all the factors constituting a measurement result, that creates the risk.Since no measurement is exact, the reality is either higher or lower than the declared value.At the butcher's, it's easy enough to understand the principle of risk-sharing.Either a joint of meat is truly heavier than indicated by the scales (an error in favour of the customer and detrimental to the butcher), or it is smaller (and so it is the butcher who gains).In that world, the world of legal metrology (in other words the metrology of commercial transaction), risk-sharing is key to the whole strategy.Legal metrology, which has been State controlled for nearly 200 years, is about guaranteeing fair trade, by ensuring that the risks are the same for everybody, the butcher and the customer, in Paris as in Marseille.
In the industrial world, however, we are faced with a different issue.The example of a medical laboratory test result clearly illustrates the difference.Let us imagine the situation where a patient is actually ill (or 'noncompliant' to use the generalized term) but whose laboratory test reading gives a 'compliant' result (i.e.'not ill').This is a classic case of Customer risk (i.e. of a noncompliant entity mistakenly judged compliant) and it is easy to see that it is the patient here who, failing to receive the treatment that could improve his or her medical condition, suffers a potentially fatal prejudice.In the opposite case, the Supplier risk (i.e.compliant entity mistakenly judged non-compliant), the healthy patient is declared unwell and wrongly endures unnecessary treatment.If the treatment happens to be a three-month of cycle of chemotherapy, it's easy to see that the risk has nothing to do with the 'Supplier' (in this instance the laboratory that made the measurement) but that, quite the opposite, here too it is endured by the patient.
And so it is in every case.Bad decisions of the 'Supplier risk' type, when it comes down to it, are always detrimental to the customer.When mechanical parts are wrongly retouched or a batch of vaccines is mistakenly destroyed, it is the end customer who will pay in the price of compliant goods.Every company makes sure its decision errors are paid for by the customer one way or another, otherwise the company would not survive...In 2012 JCGM-WG1published the JCGM 106 document, which that same year became the ISO 98-4 Guide [3], and was translated and published in French in 2013.This standard explains that risks, both Customer and Supplier, are not only due to measurement uncertainties.In order for a Customer risk to exist, there have to be entities that are in reality non-compliant.Whatever the measurement uncertainty, if no entities are non-compliant the Customer risk is zero.On the other hand, the Supplier risk is not zero, because there is always a possibility, due to measurement uncertainty, of measuring entities as non-compliant that are in fact compliant.Therefore, to calculate a risk one needs to know the properties of the entity to be measured as well as the measurement uncertainties.This knowledge is called 'a priori knowledge'.And a priori knowledge is one of the keys to Smart Metrology.Smart Metrologists make a point of cultivating this knowledge, and this is something that differentiates them from their more traditional counterparts.
Without going into mathematical demonstrations that are outside the scope of this paper, it is nonetheless important to understand that the pattern of risks (Customer and Supplier) is not symetrical.In other words, while there might be a 1% gain on the Customer risk side, there could, depending on the specifics of the case, be a loss of several percent, even tens of percent, on the Supplier risk side.Since, as we have already seen, both types of risk are in the end borne by the customer, one obviously wonders about the sense of trying to put the gains on one side to the detriment of the other, knowing what the consequences will be.
Efficiency, which can be defined as effectiveness at the right price, is therefore to be found in the fine balance between 'Customer risk' and 'Supplier risk' that must be sought whatever the field.The job of Smart Metrology is to achieve this effectiveness through methods specific to it, although its mission does not end there.As measurement is at the very core of industrial decision making, from defining requirements to the declaration of conformity, Smart Metrology must clearly be present at every level within the company.The strictest need, the most efficient setting, the right decision, these are also the very core of Smart Metrology.

The current 'worst case' world…
Legal metrology, which for 200 years has been guaranteeing risk-sharing, has also inspired certain practices in industry.In a specific field (that of commercial transaction), where it was necessary to eliminate doubt a , legal metrology has done such a good job that we have wholeheartedly adopted the (false) idea that measurements are accurate.Since early childhood, in fact, every measurement result has been presented to us in the form of a single value and this daily repetition of a measurement expresses a unique value has led us to think, very often unconsciously, that the value expressed does truly represent without any degree of doubt reality.
Industry, which has come into being more recently than trade, has therefore grown up thinking that measurements are accurate -an erroneous belief that has nevertheless not stood in the way of its success.We produce cars, aeroplanes, processed foods, vaccines, medical laboratory tests and all that surrounds us, believing that measurements are accurate.The NF EN ISO 14253-1 standard [15] demonstrates at least in part what has happened.By comparing the recommendations of 'apparent common sense' to the reality of industrial practices, it is easier to understand all the unexplored potential of metrology.

Figure 3: Industrial practice
Our belief in the accuracy of measured values has led to specifications with the 'conformity zone' characteristic of the standard.A 'margin of safety' has been included between the real requirement (the specification) and the requirement expressed (tolerance 'on paper') … to be on the safe side!But reducing the tolerance has two very costly consequences: 1. Overly stringent production methods compared with actual requirement as defined.2. Massively increased Supplier risk (wrongly declaring non-compliant entities that actually are compliant) prejudicial to the interests of the customer.
Some might be reassured by the 'excess quality' as per the adage 'he who can do more can do less'.But this argument is misrepresentative because asking for too much means consuming and costing too much, and this, ultimately, constitutes a deficiency on the Quality scale!The customer expects the operation at the proper pricethat is the ultimate objective that every company should be aiming for.The Smart Metrologist, being a key component in efficient decision making, is de facto one of the mainstays of this ambition.

From worst-case metrology to Smart Metrology
Among the 'metrology' practices common in companies, especially those adopted in the wake of Quality guidelines such as ISO 9000, can be found costly practices that often serve no useful purpose despite their surface efficiency.We are going to examine three major issues encountered by metrology in order to demonstrate why they do not serve the real needs of the company and to show how Smart Metrology can offer different and far more efficient solutions.

4.1: Monitoring versus Calibration
One of the pillars of legal metrology is the periodic verification of the instrumentation upon which commercial transactions are based.From a legal point of view, anyone owning such equipment must, under pain of legal action, keep up to date a metrological logbook for each device.This logbook details the results of the periodic verifications prescribed by law.
The real risk, for the owner of this equipment, is not that the device fails a conformity check when the time comes, but that it is not checked before the due-date as per the relevant section of law.In this world, there are no impact studies to reveal whether or not the nonconformity has consequences for the customer.A little more, a little less, who cares!The only consideration at stake being 'equity' in the face of possible error, the only obligation is to meet the scheduled date.
In the industrial world, on the other hand, a nonconformity that becomes apparent at the moment of verification means all the possible consequences for previous production must be assessed.In this situation, corrective action could go as far as arranging the return of all entities subject to doubt, especially if the risk of the non-conformity might have serious consequences for the end customer.
Because the purposes of legal metrology and industrial metrology are so different, it stands to reason that their respective solutions should also be different.Metrological verification (a decision taken), which follows a calibration (revealing any deviation), validates only what has been.If the device is declared to be compliant, one may feel justified in saying "Phew", but only with regard to the past: nothing can be definitely affirmed as regards either the present or -indeed, even less so -the future.And yet, in industry, decisions are taken daily and the only way to be sure of their validity is if conformity is a daily fact, not merely a finding in a year or two, as is the case with period calibration/verification.
Smart Metrologists are perfectly aware of the need to meet expectations on a daily basis and do not hide behind practices that fall short of this.They aim to guarantee the conformity of instrumentation each and every day and, when wholly successful, only send equipment for calibration if in doubt about its health.In this they contrast with traditional metrologists, who often measure their performance by looking at the amount of equipment returning from verification 'compliant' (which only goes to show how pointless it was to send it off in the first place): Smart Metrologists work to completely opposite indicators.Their idea is to ensure the conformity of equipment at each instant and to send it off for calibration only when in doubt.If this doubt is justified (there is something wrong with the device and it needs to be repaired or rejected) and furthermore the company is not subject to customer or supplier claims resulting from the measurements undertaken, then the metrologist can justifiably assert the effectiveness of his or her power of detection.Consequently, only the strict minimum needs to be done in terms of periodic calibration, etc.In this case, the principle of 'conditional periodicity' comes into play: I only calibrate when in doubt!
In this context, it is not a matter of blindly calibrating/verifying, but being sure of the ability to detect an anomaly by some means or other.One of these is very simply to make colleagues aware of the fact that when a device receives a shock it should be brought in for inspection.What could be a simpler means of overseeing the equipment than insisting that a device found on the ground (bearing in mind that it cannot jump back up to the workbench by itself !) must be sent for checking…?No need for complicated maths or laborious statistics -just straightforward common-sense.
Sometimes, monitoring methods can be quite complicated, and may even need to be formally recorded with a view to detecting and therefore arresting a drift.
Here, Smart Metrology can make use of existing literature on the subject [4][5].
Even if documentation exists to define periodicity [6][7][8], none of it pretends to have the answer when it comes to predicting the date of an incident.Only consistent monitoring makes it possible to satisfy the ongoing need in industry for sound decisions based on measurement results.For this reason, Smart Metrologists do not concentrate their efforts on following a more or less arbitrary schedule for calibration/verification.Their priority is having the means to detect an anomaly when it occurs and to ensure this detection ability is effectual on a daily basis.Risk management, the fact of seeing or not seeing an anomaly, is therefore at the heart of each day's activity.

4.2: The 'famous' MPE
The question of MPE (Maximum Permissible Error) is a recurring one in the field of metrology.By contrast with legal metrology, which knows in advance the regulatory uses of instrumentation, industrial metrology needs to be able to assess measurement uncertainties in order to ascertain its needs in terms of the metrological performance of a device.A great many metrologists would no doubt love to find an immutable value for this maximum error in some standard or other… Sadly for them, only the metrologists in question can possibly know what it should be!Every measurement process (Figure 4) is unique and the impact of the very measuring device itself on the quality of the final result (the measurement uncertainty) inescapably different.Let us remember also that the statistics required to estimate measurement uncertainties are not so complex as to exclude all but the confirmed mathematician from understanding and using them.
Put simply, it is a matter of realizing that the variants (standard deviation squared) are added when dealing with independent random error and that the averages are added algebraically when dealing with errors of the systematic type.Plus the fact that measurement errors are often a combination of these two types of error… It is easy enough to see that the concept of MPE, borrowed from and effectual in legal metrology, does not make much sense in the industrial world.If the instrument error is of the random type, we need to know the variance, and the type of mathematical 'gymnastics' frequently used to get from MPE to variance can be somewhat questionable.If the error is of the 'systematic' type, we need to know this to correct it, or to add it linearly to the uncertainty produced by random effects.

Figure 4: Synoptic of a measurement process
Moreover, claiming to have found a maximum error by evaluating the device against just a few measurement points (from three to ten, rarely more) is also pure nonsense.VIM3[2] suggests a new definition of calibration that will make it possible, once it has become widespread, to answer the question of 'systematic/random' and of the number of calibration points.The French College of Metrology Collège (CFM: Français de Métrologie) has published a guide [9] that deals with this question.It also provides free downloadable software for metrologists at its site, M-CARE, with which all the necessary calculations can be done.
Only the company metrologist can be in control of the measurement processes upon which the company relies… Thinking that a standard written by others can satisfy specific in-house requirements without adaptation can be a costly mistake, whether the error be towards 'excess quality' (the norm prescribes stricter MPEs than are needed) or that of sub-quality (the standard prescribes less strict MPEs than the actual need).
It is of course possible to define MPE, differentiating if necessary between random and systematic effects, for instruments of the same type used interchangeably within the company.In this case, verification serves to ensure that the performance of each device does not deteriorate and that each can continue to be used in the same way as any other without negative consequences for the company.But whatever the particulars of the situation, assessment of the overall uncertainty taking into account all the factors involved (measurand, operators, environment, device, propagation, etc.) is necessary for efficient decision making to be possible.For this reason, standards that deal with the assessment of measurement uncertainties [10][11][12][13] are indispensable tools in Smart Metrology.
To conclude on the subject of MPEs, it is worth remembering that the NF EN ISO 14978 standard [14] states: 'Standards for specific measuring equipment, with the exception of a few examples (i.e.ISO 1938 and ISO 3650)

, shall not include any numerical values for MPEs and MPLs but shall include empty tables for MPE or MPL values as a guidance for the user of the standard'.
The standard is therefore quite explicit about the fact that it is up to users to define the metrological performance of their instrumentation, and not the standard…

4.3: Capability
The question of taking into account measurement uncertainties when declaring conformity is another recurring issue in industrial metrology.We saw above how the NF EN ISO 14253-1 standard [15] suggests dealing with uncertainty and we saw that tolerances as they stand are typically 'conformity zones': once again, therefore, it is not a solution to remove all uncertainty at the risk of losing 'room for manoeuvre'!In the domain of SPC ( Statistical Process Control), the question of the capability of a manufacturing procedure to produce entities in accordance with required tolerances is analysed from a statistical point of view.This is an approach to mass production, where measuring each entity would be impossible (not to say prohibitively costly).It is worth remembering that MSP uses averages and manufacturing dispersion as a means of establishing the aptitude to produce compliant articles via contractual objectives and calculated indicators, and a proportion of non-conformity is accepted.SPC therefore has two problems to deal with, namely the effects of sampling and measurement uncertainties.For the latter, it suggests that measurement reliability can be achieved via the concept of the 'suitability of the measurement process'', also known as 'capability'.In this context, the objective is clear: the measurement errors have to be negligible in view of the dispersive nature of the procedure in question.This characteristic of measurement error can therefore by summarized as the ratio of measurement uncertainty to dispersion.It is written thus: In other words, if the dispersion of the procedure is appropriate to the required tolerance, the standard uncertainty of the measurement process of the samples must remain below 10% of the required tolerance.
The value to choose for C is, however problematic, because the very real underlying risk is of making a wrong decision (Customer risk or Supplier risk) and the  tells us that this simple coefficient is not enough… Once again, a universal magic number for ‫ܥ‬ would suit a good number of metrologists, but there is no point pretending, there can be no such thing, since it depends on: x The Customer and Supplier risk-sharing established between the relevant parties x Characteristics of the production procedures.
The Smart Metrologist's approach to the problem is to try to balance the risks, since both will be suffered by the customer.The ISO standard GUIDE 98-4[3] explores a system of 'guardbands', i.e. a strategy of 'revising' the tolerance expressed.The standard explains how this revision can be driven by the aim of achieving a given Customer risk, despite a nonnegligible uncertainty.The authors advocate (cf.[17]) recognizing the fact that, since both risks are detrimental to the customer, the guardbands need to be driven by optimization of the weighted sum of risks.Defined in this way, the decision limits (the new tolerance after application of guardbands) are set in a rational and documented way, with understanding of the reality at stake.Once again, Smart Metrology demonstrates its interest in facilitating effective decision control.

Smart Metrology and Big Data
The start of the 21 st century saw the birth of a new industrial revolution, the digital revolution.The majority of experts agree that this revolution is inexorable, and probably positive, at any rate for the economic world.This new technological innovation consists in the capacity to store data from diverse sources (notably via smart objects) in volumes never previously achieved and to utilize it by means of ever greater computing capabilities via Artificial Intelligence techniques (A.I).
From a world poorly-supplied with information, where industry, when making decisions, had to make do with little data (and not always as reliable as hoped), we are moving to a very different situation where the concern is 'How to make use of the masses of information available?'.This changing world demands a correspondingly different mode of thought in order to produce the best results.The 'worst case' style of thought is very expensive, as we have seen, as much from an economic point of view as from that of sustainable development.Now that industry has so much information and a better understanding of the world, it can optimize its practices and move towards the 'strict minimum'.Risk management is a useful tool that should be used to achieve this very necessary improvement.Is it not said, moreover, that opportunity is the positive side of risk?
In this new environment known as 'Big Data', which extends from the collection of data to the I.A. algorithms, the pioneers (notably G.A.F.A) soon realized that veracity of data was the most essential criterion to make sure of.And that is indisputably the role of metrology: guaranteeing the reliability of measured data.No longer lost in isolated questions about MPEs and periodicity, the Smart Metrologist must be the guarantor for the company of reliable data production, every single day.
Guaranteeing day-to-day reliability means being able to detect an anomaly the moment it arises; being armed against the slow, perhaps barely visible, drift of a measuring device; but, perhaps above all, it means being in control of each of the factors that affect the measurement process.This process involves, notably, the operator.The operator, in fact, is one of the 'key' factors of the measurement process, because the operator is the most likely person, perhaps the only one in a position to detect an anomaly.The fact of finding a device on the floor, as in the example given above, should lead to immediate action that only the user can instigate.Increasing the awareness of operators should not be limited to mishaps or abnormal utilization of a device, but should extend to 'a priori knowledge', the importance of which we explained above.Developing the capacity of the operator to single-handedly detect an anomaly is the only true guarantee of the reliability of data.A dubious value (i.e.noticeably different from what the operator expects) should lead to an action, for example cross-measurement with another device, or by another operator.
Improving understanding of the entity measured by being able to make use of a priori knowledge is also one of the objectives of the Smart Metrologist.Without going into mathematical details, the metrologist must always remember that the measured value is the algebraic sum of the true value sought and of the measurement error produced at time of measurement.For this reason, the measured value can be expressed by an almost infinite number of combinations ܸ ܴ݈݁ܽ + ܸ ‫݀݁ݎݑݏܽ݁݉‬ .By knowing the probability of one true value or another (a priori knowledge) and the probability of one measurement error or another (knowledge of the measurement uncertainty), it becomes possible to calculate the probability of each of the possible combinations (which is simply the product of the probabilities ܸ ܴ݈݁ܽ ; ܸ ‫݀݁ݎݑݏܽ݁݉‬ ).Once this calculation has been made, it is a matter of looking for the most probable combination in order to give as 'true' value the most likely value, bearing in mind that which was actually measured.This is the application by Smart Metrology of what statisticians call Bayesian inference [18].

And to conclude: the role of the Smart Metrologist in industry
Despite thirty years of quality certification, metrology has not yet done justice to its potential.Having concentrated on practices that, as we have explained, do not further its real objectives, it has got by till now by meeting, as a minimum requirement, the expectations of the auditor.A historic opportunity is now open to it.Without reliable data, industry will not be able to benefit from the new technologies that are becoming available to it thanks to Big Data .By neglecting these new assets companies not only lose their competitive edge but, and perhaps more importantly, fail to restrict themselves to the 'strict minimum' required of humanity at this critical time in history, in order to ensure the preservation of resources.
Smart Metrologists need to get out of the office and think about whether or not expectations are truly being met, not just those of the auditors, but those of their entire ecosystem.Efficient decisions are at the heart of industrial performance and of sustainable development.Since, as we have already said, metrologists are the linchpins of this necessity, it is up to them to fulfil their role wholeheartedly.
And so, their place is not in front of a computer where they programme, execute, and congratulate themselves on meeting often arbitrary dates and MPEs.They should be where the machines are, there where measurements are produced, to ensure that these really are representative of the reality they are intended to describe.Their role is to control all the factors that contribute to producing a measurement.From the theoretical method to the operator, from environmental conditions to the intrinsic quality of the measurand to be quantified, the metrologist must make sure that everything is under control, and realize that a simple green label stuck to a device, even if it does serve a purpose, is not enough .