Metrological characterization of 3D imaging systems: progress report on standards developments

. A significant issue for companies or organizations integrating non-contact three-dimensional (3D) imaging systems into their production pipeline is deciding in which technology to invest. Quality non-contact 3D imaging systems typically involve a significant investment when considering the cost of equipment, training, software, and maintenance contracts over the functional lifetime of a given system or systems notwithstanding the requirements of the global nature of manufacturing activities. Numerous methods have been published to “help” users navigate the many products and specifications claims about “quality”. Moreover, the “best” system for one application may not be ideally suited for another application. T he lack of publically-available characterization methods from trusted sources for certain areas of 3D imaging make it difficult for a typical user to select a system based on information written on a specification sheet alone. An internationally-recognized standard is a vehicle that allows better communication between users and manufacturers. It is in this context that we present a progress report on standards developments to date in the diverse, but finite, world of non-contact 3D imaging systems from the nanometre to the 100 m range.


Introduction
Ever since the first experiment in digital optical threedimensional (3D) imaging in the late 1960's, numerous approaches have been published to "help" users understand the main optical distance-measurement techniques (see Figure 1), and to navigate the many commercial products and claims about the "quality" of 3D imaging systems. By 3D imaging system, we mean those non-contact measurement instruments used to produce a 3D representation, e.g. a dense point cloud of the visible surfaces of an object or a site. The final output is usually a set of (x, y, z) coordinates in a Cartesian coordinate system. The digital surrogate of a surface is obtained with respect to the instrument at a standoff distance, within a finite volume of interest, with a certain measurement uncertainty, and, with a known sampled spatial resolution. These imaging systems capture not only the geometry of visible surfaces but also intensity or colour images [1,2,3,4].

A tool to communicate in a global economy: international standards
Though 3D imaging technology has been around for a few decades, it is still considered to be an emerging technology. As such, published characteristic values and the methods used to compute them can vary significantly amongst different commercial systems and point in time.
From the perspective of the 3D imaging system's manufacturer, the measurement limits of a system should be obtained using standardized methods so that the manufacturer can provide specifications that mean something to the customer. For the customer, the system needs be suited to its intended purpose, or better yet, it has to be fit for purpose. An internationally-recognized documentary standard is a tool that allows better communication between users and manufacturers especially in a global commercial context. It is a document "established by consensus and approved by a recognized body, that provides, for common and repeated use, rules, guidelines or characteristics for activities or their results, aimed at the achievement of the optimum degree of order in a given context" and "standards should be based on the consolidated results of science, technology and experience, and aimed at the promotion of optimum community benefits" [5]. To achieve these goals, metrology provides the foundation by which accurate and reliable measurements can be captured in a standard document that will help achieve this overarching goal of fit for purpose.

The Emergence of Physical Standards
The appearance of physical or material standards arose from the need to transform the raw data acquired with a 3D imaging system into a form that was more appropriate for data processing, i.e., using a Cartesian coordinate system. This first stage was system calibration, although the expression was often used loosely and not according to the more recent definition in the VIM, which includes an uncertainty assessment [6]. In many cases, spheres had only to look like spheres and planes needed to only look flat. The second stage was also the first step in evaluating the accuracy of a given system immediately after its calibration, possibly using physical standards distinct from those used in the calibration stage, and in a second step that involved performing a periodic verification in order to avoid frequent recalibration. In documentary standards, these two steps became formalized as the acceptance and reverification steps. The period from the late 1960's until the early 1990's saw the development of many different types of physical material references for calibration, acceptance and reverification of 3D imaging systems. These artefacts were made of wood, plastic, aluminum and steel, and were formed into basic shapes (see Figure 2). For small measurement volumes, the reference artefacts were portable, or at least transportable, and for larger volumes arrays of targets with known coordinates were more useful.
The early part of the 1990's saw the formation, by the Intelligent Manufacturing Systems (IMS) program, of an international precompetitive R&D collaboration in the area of advanced manufacturing [7]. In 1993, a worldwide assessment of commercial rapid-prototyping (RP) technologies was initiated. One of the test cases was to assess the capability of commercial RP technologies using two test parts that were fabricated from a CAD model. One of these test parts was scanned at the NRC Canada in September of 1993 and is shown on Figure 2c. Although the test was specifically for RP technologies, it became clear among the participants that in order to compare 3D data acquired with 3D imaging systems a common language and set of standardized tests needed to be developed to ensure clear and concise communication among the partners spread across the globe.

The modest goal of this paper
It is in this context that we present a progress report on standards developments to date in the diverse, but finite, world of non-contact 3D imaging systems and solutions. We describe document and physical standards in the framework of non-contact measuring instruments that produce dense point clouds of the visible surfaces of an object or a site. These surfaces may be engineered by some manufacturing process, handmade/facticius or subdivo surfaces.

Early Work on Standards
It is interesting to note that the three main optical distance-measurement techniques used in commercial systems (see Figure 3) occupy distinct measurement ranges or volumes with some overlap in both scan range or volume and types of applications [1][2][3][4]8]. Here we identify them with generic terms linked to the measurement range distance: x nano/micro range (< 10 mm), x close range (10 mm to 2 m), and, x mid-to-long range (> 2 m). a) b) c) Figure 3. Representative commercial systems capable of producing dense 3D images, typically within a few seconds: a) low-coherence interferometry (image courtesy of Novacam Technologies Inc), b) hand-held triangulation system, c) portable time-of-flight system.
The evolution of digital 3D imaging closely followed advances made in the field of solid-state electronics, photonics, and computer vision and graphics. One obvious change that has been instrumental in the growth of digital 3D technology is the availability of affordable and fast digital computers; not to mention the invention of compact and efficient light sources. Furthermore, the possibility to process these dense point clouds in an efficient and cost-effective way has opened up a multitude of applications in areas as diverse as industrial, cultural, commercial and military activities. These areas, in return, have created new challenges, identifying new areas where the science and the technology aspects need improvement.
The mid 1990's and early 2000's saw the emergence of different groups interested in preparing written documentary standards and proposing physical standards using a standardized vocabulary. The people involved in these groups were primarily from industry and academia. They covered fields of expertise as diverse as surveying, computer vision/graphics, photogrammetry, coordinate and surface metrology and manufacturing in general. We now look at some influential pre-international standards work following the three measurement range distances given above.

Nano/micro Range
In the nano/micro range, a detailed surface topography was known to be critical to the functionality of a mechanical part, specifically form, waviness and roughness. Surface profiles had been employed since the 1930's, and by the 1980's it had become clear that areal surface topography parameters were required in order to provide much better insight into predicting the functionality of a surface [9,10,11]. In 1990 the project "Development of Methods for the Characterisation of Roughness in Three Dimensions" started under the leadership of Birmingham University. This project produced a publication in 1993, the "Blue book" which contained the Birmingham 14 Parameters [9]. By 2002, the International Organization for Standardization (ISO) Technical Committee (TC) 213, dealing with Dimensional and Geometrical Product Specifications and Verifications, formed working group (WG) 16 to address standardization of areal surface texture measurement methods [12]. It is important to remember that the term texture used in the context of microtopography does not mean the same as reflectance information of a surface as it is the case in the other areas covered by digital 3D imaging.

Close Range
In the close range, 1996 saw the formation of two working groups, i.e., one from the German Society for Photogrammetry and Remote Sensing (DGPF) and the other from the German Society for Measurement and Automatic Control (VDI/VDE-GMA). Together they initiated a joint venture to develop a set of guidelines for the acceptance and verification of optical 3-D measuring systems. This working group (within VDI -The association of German engineers) assembled more than 30 experts from well-known system manufacturers, universities and users who were working with optical 3-D measuring systems. This pioneering work resulted in 2002 in the VDI/VDE 2634 Part 1 [13][14].
In 2004, The International Association of CMM Manufacturers carried out a cooperative project known as the Optical Sensor Interface Standard (OSIS) [15][16]. The main goal was to realize a common interface standard for CMMs. The standard is structured into three sub-workgroups, dealing with different aspects of the integration of optical sensors: WG1-Mechanical/electrical interface, WG2-Data integration and WG3-Specification, classification and performance verification for optical distance probes. The WG3standard appeared in 2006 [16]. Building on the work of VDI and OSIS, the ISO/TC 213 Dimensional and geometrical product specifications and verification has since acquired momentum with multiple standards within ISO 10360 for CMMs , which use multiple types of probing devices such as optical distance sensors, are being developed [17].

Mid-to-long Range
In 2006, ASTM committee E57 was established to develop standards for the performance evaluation of 3D imaging systems for the mid-to-long range [18][19]. The committee's focus was on standards for 3D imaging systems typically used for applications including, but not limited to, construction and maintenance, surveying, mapping and terrain characterization, manufacturing, transportation, mining, mobility, historic preservation, and forensics. The ASTM E57 committee consists of four subcommittees: Terminology, Test Methods, Best Practices, and Data Interoperability. Around that time, the ISO 17123, describing field test procedures for different geodetic systems, was expanded to include simplified field tests for mid-to-long range systems [20]. This work followed the ISO 17123 philosophy of finding simplified and full test procedures for these systems, known as terrestrial laser scanners (TLS), that can be performed in the field rather than in a laboratory setting. This work yielded a proposal for the ISO 17123 Part 9 [21]. Laboratory procedures for testing surveying and construction instruments may be covered in the future by ISO 16331 [22] but no information was available for this present publication.

What is Next?
The following section presents a progress report on standards developments based on methodologies to evaluate the accuracy of non-contact three-dimensional imaging systems. The pioneering work described in the sections above represent the basis for many of the documentary and physical standards adopted by the standardization communities involved with 3D imaging.
It is important to mention that the 3D imaging systems covered by the standards in the next section are all based on a structured illumination source [1][2][3], e.g. laser, SLED, fringe projection, etc... Towards the end of the paper, we look at newly-available stereo and multibaseline dense 3D imaging methods based on unstructured illumination [4].

Current Standards
Current work on the definition of internationallyrecognized standards and one national guideline will be described in this section. In particular, we discuss work performed by the ISO/TC 213, VDI/VDE, ASTM E57 and ISO/TC 172/SC 6 committees. Based on current market data, we conclude that there is a necessity to accelerate the creation of standards in a critical area: the close range, where mobile 3D scanning systems are used as stand-alone units or in conjunction with a second imaging system to increase the measurement-volume capability. This future standard, if adapted to industrial needs, will increase user confidence and will support market growth in this measurement range that is critical in many industrial processes.

Documentary Standards
ISO-TC 213 provides to the practitioner a series of documents related to profile measurement on a surface. Profile measurement acquired by a stylus instrument relates to the measurement of a line across a surface. That profile can then be represented mathematically as a height function with lateral displacement, z(x). These two-dimensional (2D) profiles are dissected into their intrinsic form (P is used in front of parameters, e.g., Pa), waviness (W is used in front of parameters, e.g., Wa) and roughness (R is used in front of parameter, e.g., Ra) characteristics. Most surfaces are a combination of all three characteristics and it is customary to assess them separately by various spatial-filtering techniques [23][24]. The filters are described by the cut-off wavelengths /min and /max or by the corresponding spatial bandwidth kmax and kmin. The regions are separated by applying spatial filtering -typically using Gaussian, double Gaussian or 5-th order Butterworth filters. For reasons given earlier, a surface is a 3D entity and, hence, should be studied as such [25]. The ISO-TC 213 has produced surface texture documentary standards as part of its scope. They are encapsulated in the ISO 25178 Geometrical product specifications (GPS) -Surface texture: Areal. Surface texture is measured using either a series of profiles or using areal measurement. Areal surface texture measurement is the measurement of an area on the surface that can be represented mathematically as a height function with displacement across a plane, z(x,y) [3]. Usually that surface representation is captured as a regular grid where a surface is sampled at constant X, Y increments (see Figure 4). Table 1 lists the ISO specification standards relating to the measurement and characterisation of areal surfaces.
The ASME has also published a comprehensive documentary specification standard, B46.1, which includes some areal analyses [26]. The terminology used for areal measurement differs from that of single profile measurement. Instead of using terms like cut-off frequency (or length), form, waviness and roughness, the ISO 25178 proposes to use nesting index as the index corresponding to the cut-off wavelength of a linear filter, or to the scale of the structuring element of a morphological filter. Therefore, a surface image is decomposed by removing any base shape or a form such as a tilt, a cylinder, a sphere, etc. This mathematical operation is referred to as an Foperator (ISO 25178-2:2012-3.1.4.3) in that the "form" is factored out of the "raw" measurements prior to any additional filtering operations. After the F-operator, a Gaussian filter, the S-Filter (ISO 25178-2:2012-3.1.4.1), can be applied to the data to limit short spatial wavelengths, and the L-Filter (ISO 25178-2:2012-3.1.4.2) applied to limit long spatial wavelength structures prior to analysis [27]. Typical operations are illustrated in Figure 5. The naming convention of 3D or Areal parameters defined in ISO 25178 start with 'S' for surface or 'V' for volume.  Depending on the measurement technology used, lens characteristics, camera pixel resolution and electronics, there is a minimal spatial wavelength structure that may be measured. As a rule of thumb, the smallest spatial wavelength that can be extracted consists of five measured resolution elements along a given direction [28]. The largest spatial wavelength that can be measured is limited to the full extent of the measured field. For example, if a measurement is made over a 256 Pm × 128 Pm lateral region (field of view) then the largest spatial wavelength that can ideally be measured is 256 Pm × 128 Pm. For a resolution element of 0.25 Pm, the smallest spatial wavelength is 1.25 Pm. The topic of surface metrology using areal measuring instruments is growing rapidly thanks to innovations in the manufacturing of improved instruments, the availability of processing software and the different major initiatives around the world that are yielding numerous publications. Of particular interest, we note the geometrical imperfections of areal measuring instruments that are sometimes left in commercial units because of their high complexity and the need for traceability of the measurements. Therefore it has been clear to many that proper calibration of a given instrument is required [29]. Calibrating an instrument in a metrological sense [6] requires • calibrated areal physical measurement standards (ISO 25178-70), • a means for calibrating the geometric deviations of the instrument (ISO 25178-600), • software measurement standards for calculating areal surface texture parameters (ISO 25178-71), and • methods for calculating the uncertainties associated with areal surface-topography-measuring instruments and surface-texture parameters [28]. a) b) Figure 7. Evaluation of the field curvature a confocal imaging instrument (50 × lens) on a section of an optical flat using some ISO 25178 parameters, a) parameters before curvature correction, b) parameters of the surface after correction.
The main metrological characteristics of areal surface topography measuring instruments are flatness deviation (see Figure 6-7), linearity errors, amplification coefficient, perpendicularity of the axes, resolution of the measurements along the axis of operation and, measurement noise [28]. We can add two more issues to the list: stability of the translation movements needed in an instrument [30] and traceability of the topography measurements [29,31]. As in many scientific endeavours, questioning the state of knowledge is fundamental. As pointed out by many authors, surface metrology and, in particular, surface topography measurement still has many open or unresolved questions [32].

Physical Standards and Material Measures
A newly-published ISO standard addresses reference artefacts or "material measures" that can be used for periodic verification and adjustment of areal surface texture instruments (ISO 25178-70). The standard defines a material measures as follows: Def. material measure: <surface texture> dedicated manufactured workpiece intended to reproduce or supply, in a permanent manner during its use quantities of one or more given kinds, each with an assigned quantity value. Source: ISO 25178-70:2014-

(Term and definition -3.1)
A material measure is sometimes called a calibration sample, calibration specimen, calibration standard, standard artefact, physical measurement standard or physical standard. The material characteristics of the 13003-p.5 material measure must not significantly affect the measurement carried out on it. Two groups of artefacts are proposed according to the type of the measurement method: types of profile material measures and types of areal material measures. The physical standards should be uniquely identified. Serial number, type and nominal values of the measurands are recommended to be engraved on the standard and/or standard's casing. According to ISO 25178-70, a material measure can be used for two different purposes: x calibration of the metrological characteristics, followed by assessment of the measurement uncertainty, x user adjustment of the instrument, which establishes corrections of the measured quantities. For example, bi-dimensional (areal) physical measurement standards are shaped like perpendicular/ circular/star-shape grooves, as cross gratings, spheres, planes or irregular surfaces.
The physical realization of the material measure listed in ISO 25178-70 presents some manufacturing challenges. NPL in the UK has created a kit containing artefacts that address the calibration of topographymeasuring instruments according to ISO 25178-600. At the moment, the commercialized box contains an optical flat, step heights, lateral grids, star patterns (for testing resolution), sphere on plane, and an artefact with a deterministic surface [31]. As noted by researchers, surfaces are made of a complex combination of spatial frequencies so calibrating the spatial frequency response of the instruments offer a more viable solution. For the assessment of the measuring systems used to measure micro-components, such as micro-gears and micro-optical components with features smaller than 0.5 mm down to 1 Pm, PTB (Physikalisch Technische Bundesanstalt) in Germany has developed a series of calibrated micro-artefacts which have "cooperative" surfaces in the context of tactile and optical probing [33][34]. They can also be measured by x-ray computed tomography. The micro-contour standard made of stainless steel and micro-cube made of titanium with spherical calottes shows different radii, angles and distances that are calibrated with known uncertainties. The artefacts are commercially available. a) b) Figure 9. PTB calibrated artefact, a) Micro-contour standard of stainless steel (50 mm × 3 mm × 20 mm) with a profile measured at NRC Canada with a triangulation-based laser scanner, b) Micro-cube (10 mm × 10 mm × 10 mm) made of titanium with spherical calottes 1 [33][34].
More research groups and industries are working on different physical standards for the calibration, acceptance and reverification of nano-micro topographymeasuring instruments, some compatible with the ISO 25178, others with the proprietary manufacturing of micro-components 2 . Nanospheres and microspheres are examples of traceable silica size standards created by NIST in the USA 3 .

Close Range
Acceptance and reverification tests of optical 3D imaging systems are fundamental for manufacturers (Acceptance) and users (reverification) wishing to 1) prove the applicability of a given system to the task at hand (fitness-for purpose), 2) compare different instruments using fair and appropriate methodologies and metrics, 3) manage instrument warranty issues, and, 4) reduce costs through effective use of 3D imaging systems. Figure 10 lists typical system parameters that are derived from specifications and physical standards.
The various physical standards, along with the different tests methods, are used to quantify the spatial accuracy of close-range 3D optical imaging systems. The documentary and physical standards described below are mainly used to assess the accuracy of indication of size (probing error, sphere spacing error, flatness, length measurement error). Form is evaluated as part of a probing error (sphere and flat surface). At the moment, the national guideline and international standards in this range of 3D imaging cover those basic shapes. In what follows, we limit our description to those documentary standards directly linked to optical 3D imaging and summarize the three essential issues found in a metrological procedure: define the measurand, the calibration and the traceability. Figure 10. Typical 3D imaging system parameters that need characterisation.

Documentary Standards
As mentioned in section 2, the German VDI/VDE has been very active in defining standards for coordinate metrology and, in particular, for coordinate measuring machines (CMMs) equipped with optical probing (2D and 3D) capability: x VDI/VDE 2617 Part 6.  [35] proposes a revision to the ISO 10360-2 [17] tests specific to coordinate measuring machines that have been equipped with an optical distance sensor (ODS). The list of ODS systems is fairly complete and includes both triangulation-and interferometry-based optical sensors that can capture a single coordinate, a set of coordinates on a line and a complete area composed of 3D coordinates. VDI/VDE 2617-6.2 is composed of characteristic tests such as probing error and error of indication for size measurement.
The VDI/VDE 2634 [14] Parts 1-3 closely follow the recommendations in the VDI/VDE 2617 but separates itself from the CMM so that the measuring systems can be mobile. Part 2 and Part 3 are important to manufacturers and users of optical 3D imaging systems because these two parts of the guideline verify system compliance with required performance specifications. This is done through acceptance tests performed by the manufacturers and verification tests by the users. A user will typically perform this task at regular intervals and will document the performance over time. Part 2 addresses single-view optical systems based on area scanning and Part 3, multiple-view systems. Area scanning is based on triangulation methods, which include fringe projection, moiré techniques, and photogrammetry or scanning systems with area-based measuring capabilities.
We now look at the main guideline characteristics. Details are available in the documentary standards. All the tests are performed in normal modes of operation and conditions for a particular system. Operating mode refers to adjustments and configuration options of the system and conditions of operation denote external influences on the system. Furthermore, all artefacts used must be calibrated with respect to their dimensions (size) and form (shape). Their properties must have little effect on the quality parameters being evaluated. Both artefacts and system need to have reached a steady state temperature close to the measurement volume (thermal soak time); otherwise, mathematical corrections for thermal variation will need to be applied.
The quality parameters for the tests are probing error, sphere-spacing error and flatness measurement error. Table 2 lists the symbols proposed by the VDI/VDE 2634 Part 2. The operations listed above are usually performed on a single point cloud or multi-view registered point clouds; however, not all 3D imaging systems provide data in that format. Hence, the VDI/VDE 2634 briefly discusses the use of polygonised or triangulated data files. More information would be needed in the future because many more systems primarily use that data format.
Filtering and preprocessing of the measured values are allowed only if they are part of the boundary conditions for the characteristic test, or it is a routine event in the operation of the system. The guideline allows for the removal of no more than three out of one thousand measured points, which is more like a pruning operation. If filtering similar to ISO 25178 is applied then structure resolution will be affected and proper 13003-p.7 documentation will need to be created. In any case, a final report would be drafted upon completion of the acceptance and reverification tests. For the evaluation of conformity and non-conformity with specifications, ISO 14253-1 [36] needs to be used and ISO 23165 provides the instruction for the estimation of test uncertainty [37].
In this guideline, the quality parameters for the acceptance tests are accompanied by a clear definition, the recommended artefact, the procedure, the method to calculate the results, and the interpretation of the results. The reverification of the optical 3D measuring system under conditions similar to those used for the acceptance tests ensures long-term compliance with limits for the quality parameters as specified by the user. Trends can be detected for preventive maintenance or changes in equipment and/or the environment. The recommended procedure is analogous to that described above. This guideline allows more leeway to the user. The quality parameters may be specified by the user or the number of trials may be reduced or repeated. a) b) c) Figure 11. Location of artefact for the evaluation of the quality parameters according to VDI/VDE 2634 Part 2, a) example of ten locations for probing error, b) recommended locations for sphere-spacing error, c) recommended locations for flatness measurement error.
The probing error, both for form (PF) and size (PS), describes the characteristic error of the system within a small portion of the measuring volume and so attempts to evaluate the intrinsic properties of the 3D imaging sensor. Using a sphere of a given dimension, form is given by the range of radial distance between the measured points and a best-fit sphere. The fit is performed according to the least-squares method with unconstrained (that is, unspecified or undefined) radius. The size is the difference between the measured (or more accurately, the estimated) and calibrated diameter of the sphere. At least ten uniformly-distributed, but arbitrary, positions of the sphere within the volume are measured (see Figure 11a).
where W is the quality parameter (P, F) and Form is the form deviation of the artefact, i.e. either a sphere or flat plane. We use k=2 to compute U as per ISO 14253-1.
The sphere-spacing error, SD, verifies the lengthmeasuring capability of the system and ensures traceability. SD is the difference between the measured (estimated from the point cloud of data using a best-fit sphere-fitting algorithm with known, or constrained, radii) and the calibrated values of the distance between the centres of two spheres. For example, a ball bar, or simply two spheres, that are at a fixed and calibrated distance from each other can be used. LP and DP are specified as a function of LO. The radii of the spheres and the spacing between them must be known and recorded on a certificate that includes the measurement uncertainties. Seven positions within the measurement volume are recommended for the artefact (see Figure  11b). The interpretation of the results and handling of erroneous data are performed in a manner similar to that used for probing error. The basic equation for the expanded test uncertainty U in the case of SD is of the following type: where HI accounts for different uncertainties e.g. calibration of the length artefact, thermal expansion, temperature, mounting, etc... Here we use k=2 to compute U as per ISO 14253-1.
The quality parameter flatness measurement error, F, is the range of the signed distances of the measured points from the best-fit plane calculated using the leastsquares method. The artefact is a parallelepiped with a width of at least 50 mm and length no less than 0.5×LO. A calibration certificate stating the flatness must be provided and its value should not affect the quality parameter being evaluated. No less than six different orientations of the artefact should be measured (see Figure 11c). The interpretation of the results and handling of erroneous data are performed in a manner similar to that used for probing error.
The VDI/VDE 2634 Part 3 applies to a very important case scenario found in many applications: multiple views or multiple images (3D) generated by repositioning of the sensor and/or of the object to be measured. These 3D images are registered together in a common Cartesian coordinate system using reference markers (e.g. contrast targets) or the actual shape of the object through an ICP 4 algorithm. Exclusions are described in Part 3 that differentiate it from VDI/VDE 2617 Part 6.2 in some aspects. Part 3 of VDI/VDE 2634 13003-p.8 extends the acceptance testing and reverification of Part 2 by covering the effect that re-positioning of the sensor or the object exerts on the quality parameters. The basic principle is that the artefacts are measured in such a way that any 3D image from a view point with respect to the sensor is always different. A single 3D image sees a volume smaller than the whole system measuring volume. Table 3 lists the symbols proposed by the VDI/VDE 2634 Part 3.
The probing error consists of two influence factors: errors of a single 3D image as per Part 2 and the errors due to the transformation of the different 3D images in a unique coordinate system. Again both form and size describe the quality parameter. The test sphere is measured on a random basis in at least three arbitrary positions within the measuring volume (see Figure 12a). In each position, the sphere must be measured from at least five sensor positions (directions) in order to capture the surface in a complete way (see Figure 12b). The evaluation and assessment is performed in a manner similar to Part 2. The quality parameter sphere-spacing error is used to test the capability of the system of performing length measurements from several different single 3D images or area scans. The length artefact is composed of two spheres and needs to cover a large section of the measuring volume. If possible, the measurement of those two spheres in a single 3D image should be avoided. SD is the difference between the measured (in actuality, estimated from the point cloud of data using best-fit sphere-fit algorithm with free, or unconstrained, radii) and the calibrated values of the distance between the centres of two spheres. The interpretation of the results and solution to erroneous data are performed in a manner similar to previous quality parameters. Two features of this test need to be mentioned. The first feature is the fact that SD assumes free radii, which is different from the assumption used in Part 2. The second feature is that the results of the averaging effect due to the fitting process where the probing error is not contained in the sphere spacing error. To obtain a more complete picture of the length errors and as explained in ISO 10360, the length measurement error needs to be determined.
The quality parameter length measurement error, E, is computed using ball bars, ball beams, gauge blocks, step gauges or ball plates. What is sought is the behaviour of the measuring system over the whole measurement volume. The definition of E provided in Part 3 is in agreement with ISO 10360. E is the error with which the length of a material measure can be determined with the measuring system if the measurement were to be carried out by bidirectional probing of two points from opposite directions on nominally parallel faces vertical to one of the two surfaces. The actual procedure requires a reference artefact that is fully characterised, i.e., the calibration certificate must contain both the spacing and the form of all probing elements and the roughness of the surfaces should be negligibly small. The reference artefact is measured in seven different positions as shown on Figure 12c. The procedure is adapted according to the type of artefact, i.e. gauge blocks only or ball bars/ball plates. If gauges blocks are used, the two measuring faces of the gauge block are recorded in the middle of the measuring faces. This is achieved by selecting a point from the point cloud that is nearest to the centre of the faces. For the evaluation of E, only one single measured point is used per face, i.e. E=La-Lr. If ball bars and ball plates are used instead, the same procedure as for the sphere-spacing error is applied. To be consistent with the definition of E, the averaging effect is countered by adding the measurement of a short gauge block with each length and used to compute a correction value. Part 3 proposes three methods to apply this corrective value to the SD.
It has been observed that the value calculated using the proposed methods yields an error of indication for the length measurement that can be dominated by the probing errors; therefore, keeping the probing error and the sphere-spacing error distinct may help the users in obtaining more meaningful characteristics [16]. The VDI/VDE tests include instructions on how to interpret 13003-p.9 the results and a method for handling erroneous data using a process similar to that described for previous quality parameters. As mentioned above, the reverification of the optical 3D measuring system under similar conditions to those used for the acceptance tests ensures long-term compliance with limits for the quality parameters as specified by the user.
We now return to 3D imaging systems attached to a CMM. Here, the ISO standard is covered instead of the VDI/VDE 2617 Part 6.2. The ISO/TC 213 recently published the documentary standard ISO 10360-8 describing acceptance and reverification tests for CMMs with optical distance sensors [38]. The ODS include both single point measuring sensors and area measuring sensors, e.g., laser point scanners, laser line scanners, fringe/pattern projection systems, and moiré-based systems. Co-axial measurement with interferometry and confocal-based systems are also covered (see Figure 1a). Unlike off-axis measurement systems like triangulationbased systems (see Figure 1b), co-axial measurement systems provide 3D data along a single axis.
Two technical objectives are covered in this part of the ISO 10360: test the error of indication of a calibrated test length in the whole volume and test the errors of the optical distance sensor in the sensor local volume of measurement. These tests should ensure direct traceability to the unit length, the metre, when the CMM is used in similar length measurement situations.
Furthermore, ISO 10360-8 specifies: x the performance requirements that can be assigned by the manufacturer or the user of the CMM, x the manner of execution of the acceptance and reverification tests to demonstrate the stated requirements, x the rules for verifying conformance, and x the applications for which the acceptance and reverification tests can be used [38]. This methodology and associated physical standards (artefacts), is similar to the VDI/VDE 2634 but there are some important changes in the way the tests are implemented. c) use all measured points from the test artefact.
Option b) means that 5 % of the measured points are eliminated to determine Pform.Sph.D95%:j:ODS. This option serves a similar function to eliminating outlier data points. The choice depends on how probing is performed, i.e., either by representative points (25) or by point cloud.  Figure 13) Each of the metrological characteristics listed in Table 4 is associated with a MPE or MPL value depending on the situation. It is important to note that the term maximum permissible limit (MPL) is used in the ISO standard: this term, unlike the term maximum permissible error (MPE), is used when the test measurement results are not errors per se. Hence, testing an MPL specification does not require the use of artefacts with a relevant calibration. The subscript "j" indicates the motion of the ODS. The ODS is allowed two types of motion: translation (see Figure 13a) and articulated (see Figure 13b) and the third state is when the ODS sees an area without the need to move (see Figure 13c). The metrological characteristics listed in Table 4 are performed in a manner that resembles the VDI/VDE 2634 but there are important differences in the details. For example, the probing form error PForm.Sph.1×25:j:ODS can be performed using either a spherical or local test flat (planar artefact) and with 25 representative points. A least-squares-fitting routine determines if we get radial distances or normal distances respectively. The probing dispersion value PForm.Sph.D95%:j:ODS is the smallest width of a spherical shell, or the smallest separation between two parallel planes, that encompasses 95 % of all the data points. The "probing size error" and "probing size error All" both use a test sphere and its calibrated diameter. The "length measurement error" is either EBi:j:ODS or EUni:j:ODS with qualifier "Bi" or "Uni" indicating that it is associated with the bidirectional ("Bi") or unidirectional ("Uni") length measurement error. The standards tell us 13003-p.10 that a calibrated test length may be either bidirectionally calibrated or unidirectionally calibrated. The flat form measurement error EForm.Pla.D95%:j:ODS is the smallest distance between two parallel planes that covers 95 % of the points measured on a global test flat. This is because the standard distinguishes between a local test flat and a global test flat. For all of these tests, the MPE and MPL may be expressed in one of three forms as described in ISO 10360-1 and ISO 23165 except for EBi:j:ODS,MPE and EUni:j:ODS,MPE. There is one last metrological characteristic that is specific to articulated motion. The standard defines the articulated location value LDia.5×25:Art:ODS, which is the diameter of the minimum circumscribed sphere of the centres of five spheres. Measurements are taken from five different articulating angles on one test sphere located anywhere in the measuring volume. This is a location value and, hence, a maximum permissible limit (MPL) is used.
ISO 10360-8 defines the requirements for metrological characteristics from the points of view of the manufacturer and the user. These requirements include the environmental conditions, operating conditions, metrological characteristics, workpiece loading effects. A section on acceptance tests and reverification tests provide the practitioner with more details regarding choices and implementation. This is followed by a chapter on compliance with specifications, a chapter on the application of this standard and by two normative annexes. It is worth noting that: x PForm.Sph.D95%:j:ODS and PSize.Sph.All:j:ODS do not apply for CMMs with point measuring optical distance sensors. x the size of the test sphere must be between 10 mm and 51 mm in diameter. x a material standards selection flowchart for probing tests is provided. x the diameter and form of the test sphere and the form of the local test flat must be calibrated and it is recommended that the form error does not exceed 20 % of the corresponding MPE or MPL x the form deviation and the roughness of the test sphere and the local test flat must be taken into account using ISO 14253-1 when performing acceptance or reverification tests. x the measured area on the test sphere should cover the widest cone angle of the sphere specified by the manufacturer. The cone angle of the region of the sphere on which the points are selected must be stated in the datasheet. The measured area of the flat on which the points are selected must be stated in the datasheet. x the disclosure of minimum density or minimum number of points necessary to perform a test is disclosed. x the manufacturer may, at their discretion, specify additional MPEs for special operation conditions, e.g. filters. x when probing by representative points, select 25 areas from the measuring area on the test sphere or the local test flat and reduce to a representative point for each area. If a CMM equipped with area measuring sensors is tested, a point reduced from the measured points in no larger than 5 mm 2 area must be used as the representative point. The manufacturer must provide the software tools to reduce the data to a representative point. x software needs to compute the unconstrained (with the radius being unrestricted) Gaussian (i.e. leastsquares) associated sphere. x when probing by point cloud either 95% of "All" the points are used. Furthermore, length measurement error and flat form measurement error require extra attention. For the length measurement error that exists for some types of CMMs, it may be impractical to obtain bidirectional measurement results. Therefore, unidirectional length measurement specifications may be allowed and the topic is covered in an annex. For this test, five different calibrated test lengths are placed in each of seven different positions (locations and orientations) within the measuring volume of the CMM. Each length is measured three times, for a total of 105 measurements. When five lengths per measurement line are measured unidirectionally, either of the two methods may be used for the calculation of Ebi:j:ODS. Artefacts that represent a calibrated test length are treated in more detail in one of the standard's annexes.
In flat form measurement error, one has to consider that a data (3D image) merging operation known as 'stitching' can be useful in real measurement situations. This occurs when measuring a workpiece with larger geometry than the sensor area. Stitching may be performed in several different ways, but ISO 10360 limits stitching to those based on simple data merging by referring three dimensional coordinates obtained by Cartesian CMMs (mechanical 3D image registration). The test plane must be placed in two positions along diagonals. The number of measured points must be at least 25 and be evenly distributed over the test plane. When one looks closely at the specification that 25 representative points must be picked even with an area measuring sensor and that a representative point created from the measured points is no larger than 5 mm 2 in area, it makes one wonder if this is achievable in reality. Real area and line scanners generate thousands of 3D coordinates and filtering functions may be implemented within the scanner head so implemented with no option to disable them.
The acceptance test looks at the performance of a CMM with optical distance sensors and those equipped with area measuring sensors. Data rejection and repeated measurements are well explained in the document. Reverification tests and interim checks need to be part of a user's quality assurance system.

Other Documentary Standards
Several documents pertaining to 3D imaging systems or sensors have been prepared by other organisations. In 13003-p.11 particular, the committee ASME B89.4 5 -Coordinate Measuring Technology had a project team 4.14 on noncontact scanning probes. The work has since been consolidated into B89.4.11 -Probes and Probe Changers. The German standard DIN 32877:2000-08 6 covers Optoelectronic measurement of distance, profile and form. ASTM E57 committee has currently no activity in the short range. ASTM Committee F04 7 on Medical and Surgical Materials and Devices, initiated by surgeons, is working a document. The committee F04.05 is standardizing a technique to measure and report the accuracy of surgical navigation and robotic positioning devices for Computer Assisted Orthopaedic Surgical Systems. The ISO published the ISO 9283 8 standard of Performance Criteria and Related Test Methods for industrial manipulators. Many robot manufacturers and users utilize this standard to determine the requirements and performance of their products. ISO 12836:2012 8 specifies test methods for the assessment of the accuracy of digitizing devices for computer-aided design/computer-aided manufacturing (CAD/CAM) systems for indirect dental restorations.

Physical Standards
As described previously, the documentary standards recommend assessing the accuracy of indication of size using quality parameters like probing error (form and size), sphere spacing error, flatness and length measurement error [39]. The German guideline VDI/VDE 2634 and the international standard ISO 10360-8 both require only basic shapes, i.e., spheres and planes in simple or structured arrangements.
The physical standards must be measurable by the particular optical distance sensing technology used either triangulation-or interferometry-based. They are chosen for their physical and optical qualities [8,[40][41]. We need to keep in mind that optical sensors can capture a single coordinate, a set of coordinates on a line or a complete area composed of 3D coordinates in a singleview or multiple-view procedure. Because of their measurement principles, interferometry-based and confocal/focus-based ODS are capable of measuring opaque, transparent (see Figure 6) and translucent materials. Triangulation-based techniques require opaque materials [1][2] so that testing triangulation system requires reference artefacts that have cooperative optical surface characteristics, such as negligible volume scattering (the optical and mechanical surface should coincide), colour compatible with the light source, scattering characteristics like diffuse Lambertian and no 3D surface texture (striation may cause diffractive effect with laser sources). Metallic surfaces are sometimes treated to make them diffusely reflecting by increasing their surface roughness (see Figure 4). The roughness of the surfaces should be negligibly small so to have little effect on the quality parameters being evaluated and have negligible impact on the test uncertainty (see ISO 10360-8: part 6.2.2). Some surface treatments, like vapour blasting, light particle blasting, or spray particle coating are able to change a specular surface into a diffusely-reflecting surface. They need to be dimensionally stable without being too bulky and constructed such that the relevant sensor parameters can be extracted. The test procedure requires a reference artefact that is fully characterised, i.e., the calibration certificate shall contain size, spacing and form related information.
Form and size error: x A sphere with certified form and diameter values (see Figure 14b). x A flat plane is used for flatness measurement and as an alternative to the sphere form error test. The flat plate can be a machined or lapped plate, a 1-2-3 steel block, a 4-ways parallel block, an optical flat, a quartz block with chrome oxide deposition or any flat surface that is certified with a flatness characteristic (see Figure 14a). Length Measurement Error: x Two spheres mounted on a rigid axis: The accuracy of a 3D digitizer is evaluated using a two spheres centres distance test using a ball bar, or simply two spheres that are at a fixed and calibrated distance from each other (see Figure 14c). x Two parallel faces mounted in a rigid manner: An important way of finding the distance error between spheres is to evaluate the distance between two parallel flat surfaces, such as the two faces of a gauge block (see Figure 14a). In the situation that form deviation and roughness of the reference artefact may influence the test results, it must be taken into account when proving conformance or nonconformance with the specification (see Equations (1)-(4)). There are other commercially-available artefacts that are more or less compatible with either the VDI/VDE 2634 or ISO 10360-8. Figure 15a shows a VDI/VDE 2634 compatible artefact where the original artefact is fine for CMM measurements but not for triangulationbase imaging systems. Figure 15b presents a photograph of an attempt to coat the surfaces with a power and in Figure 15c a vapour-blasted process was applied to artefact. Both latter cases void the calibration certificate provided by the manufacturer.

The Mid-to-long Range
We have two different approaches to the acceptance and reverification of an optical 3D imaging system's performance characteristics for the mid-to-long range. One approach, based on field testing, originates from the surveying world. The other is the result of efforts to validate systems for industrial and construction applications in a more controlled environment using reference instruments to ensure measurement traceability. We will concentrate our discussion on the latter standard's activity. In any case, these standards describe methods to evaluate the performance of singledetector laser-based scanning systems based on the principle of time of flight. Most of these systems perform hemispherical scanning, typically with a field of view of 360°x 320°. Data rates are in excess of 10 000 3D coordinates per second and sometimes reach 1 million points per second.

Documentary standards:
Technical Committee 172 of ISO, concerned with Geodetic and surveying instruments, has a new project approved (17123-9 on terrestrial laser scanners) and a possible committee in the works (ISO 16331-2 also on terrestrial laser scanners) [20]. Many practitioners are waiting to see when official documents will be published. In the meantime, ASTM E57 has been very active in preparing and promoting documentary standards [18,19,42]. The E57 committee is structured as follows:  Figure 17. Planes arrangement for protocol ASTM E2938-15 Standard.
We now describe the E2938-15 document on a Test Method for Evaluating the Relative-Range Measurement Performance of 3D Imaging Systems in the Medium Range. It pertains to the following: x Measures error along the range direction only x Applies only to medium-range TLS x Measures distance between the geometric centers of planes The E2938-15 test involves placing two target plates along the LOS as illustrated in Figure 17 such that the Line of Sight (LOS) passes through ‫ܥ‬ (the geometric center of the nearer plate), ‫ܥ‬ (the geometric center of the farther place), and ܱ (the origin of the TLS), and the normal vectors of the target-plate faces are aligned along the LOS with target plate face normal vectors directed toward ܱ. The arrangement can also be generated using a single target plate moved to ‫ܥ‬ and ‫ܥ‬ as required. The reference instrument (for example, a laser tracker) is used to assist in achieving this setup as closely as is practically achievable. In all cases the target plate is rigidly mounted so that it does not move when measured by either the reference instrument (RI) or the instrument under test (IUT).

Determining the Reference Distance
A spherically-mounted retro-reflector (SMR) is used to obtain position measurements at widely-distributed points on the target-plate face so that a plane can be accurately fit to the measurement data obtained using the RI. Additional position measurements are obtained for all four sides bounding the target-plate face so that the bounds of the target-plate face can be established.
Once the five planes have been generated, these planes are used to estimate the reference geometric center of the target plate face. The geometric center can be defined as the theoretical center of mass, or centroid, of a representation of the target plate face using a structure of uniform density with bounds defined by the target plate edge. In this case, the geometric center would be the theoretical point on the uniform density structure that if placed on a balance point would result in the structure being in perfect balance.
The geometric center of the target-plate face is estimated at ‫ܥ‬ to obtain = [ ‫ݔ‬ ‫ݕ‬ ‫ݖ‬ ] ் and at ‫ܥ‬ to obtain = [ ‫ݔ‬ ‫ݕ‬ ‫ݖ‬ ] ் . The reference distance obtained from the RI is then found to be where ‖‖ ଶ = √• is the Euclidean or L 2 norm.

Determining the Test Distance
The IUT measurement of the distance between ‫ܥ‬ and ‫ܥ‬ involves estimating the position of the geometric centroid of the target plate face in the IUT reference frame. Centroid estimating is a two-stage process: fitting of a plane to the target plate face such that it is not affected by measurement results obtained from near the edges of the target plate face (see section 5.5), and establishing the bounds on the plane such that they coincide with the edges of the target plate face. This requires that the measurement results obtained by the IUT include all four visible edges and that the edges are clearly distinguishable from the background such that all measurement results associated with the target plate face, including its edges, can be visually extracted from the scan data by the operator.
In the first stage, the operator visually selects a region on the target plate face that is sufficiently far from the edges that the resulting measurement set is unlikely to include edge-affected measurement results. A best-fit plane of infinite bounds is then fit to the measurement results in the user-selected region using the Total Leastsquares method.
In the second stage, all measurement results more than two standard deviations of the plane-fit residuals from best-fit plane are removed from the scan of the target plate face. A bounding box is then constructed around the remaining measurement results and used to establish the bounds on the best-fit plane. Specifically, the target plate face edges are estimated to be the edges of the smallest rectangular bounding box that completely encloses all of the data. The geometric center of the bounded best-fit plane becomes the estimate of the geometric center of the target plate face. Geometric center of the target-plate face is estimated at ‫ܥ‬ to obtain ෝ = ‫ݔ[‬ ො ‫ݕ‬ ො ‫̂ݖ‬ ] ் and at ‫ܥ‬ to obtain ෝ = ‫ݔ[‬ ො ‫ݕ‬ ො ‫̂ݖ‬ ] ் . The reference distance is then found to be ݀ መ ூ் = ฮ ෝ − ෝ ฮ ଶ (6)

Range Measurement Conformance Test
The relative range error is calculated to be the signed difference between the IUT and RI-generated distances between the geometric centres of the same target plate faces using 13003-p.14 ‫ܧ‬ = ݀ መ ூ் −݀ ோூ

(7)
Only if ห‫ܧ‬ ห≤ RMPE, as specified by the manufacturer, is the IUT considered being in conformance with the manufacturer specifications. The test is repeated three times and the IUT must be found to be in conformance for all three tests; otherwise the IUT is considered to not be in conformance with the manufacturer's specifications.
We have reviewed standards for only one class of laser scanner operating in the mid-to-large volume and for land-based applications. A classification would include Mobile Laser Scanner (MLS) also land-based systems but mounted on moving vehicles, and Aerial Laser Scanners (ALS) which are primarily mounted on aircrafts but recently have become viable on UAVs.

The Missing Link in Close Range
From the previous listing of standards, one can see that the nano/micro and mid-long ranges are making significant strides in developing documentary standards. For close range systems, there appears to be no standards, not even under development, that addresses more complex surfaces used in the aerospace, automotive and medical industries. These surfaces, also known as freeform surfaces, assume more organic shapes that fulfill aesthetic requirements in certain fields and perform a function on actual components in specific fields like in aerospace [43]. It is well known that a range discontinuity on a surface affects the accuracy of 3D imaging systems [2]. Furthermore, continuity between surfaces may have an impact on the profile measuring capability of a 3D imaging system [8,[43][44]. It is in this context that other types of artefacts have been tested. Some were created in general workshops others offer measurement traceability in National measurement Institutes (NMI) like the NPL-Freeform 9 [45] or the PTB Micro-cube [33] made of titanium with spherical calottes 10 . The idea of finding a surface with curves that challenge the measurement process in an optical 3D imaging system has been on the mind of researchers for some time. Performing high quality measurements (at least with some traceability) of real world complex surfaces with changes in curvature (with NURBS) using a sampled network of (x,y,z) coordinates represents a topic worth investigating [46]. The working group known as 'Métrologie Grand Sud' 11 has been conducting an intercomparison of optical 2D and 3D imaging systems since 2000. The group has produced a large plate 370 mm × 500 mm made of common geometric and organic shapes made of aluminum and the surface treatment was selected according to criteria compatible with 3D imaging systems. The intercomparison includes the effect of filtering on the measurement of radii and flatness. Tests similar to the VDI/VDE 2634 part 2 were also conducted.
The NRCC has been working on a set of test procedures and artefacts to characterize the capability of a 3D imaging system to accurately measure the geometric properties of an object's surface. The approach consists of scanning a series of calibrated artefacts in which the uncertainties of the associated characteristic reference values are much less than the measurement uncertainties produced by the instrument under test and under specified measurement conditions. All of the metallic and coated-glass artifacts selected for the characterization target are grouped on the same plate in a target case for portability. To define a set of test procedures that is practical, simple to perform and easy to understand, they decided to use a terminology that is already well-known in the manufacturing field, i.e., geometric dimensioning and tolerancing (GD&T) [8,47].

Image-based Dense 3D Surface Reconstruction Techniques
The emergence of dense 3D surface reconstruction techniques based only on 2D images has not yet been supported by the definition of an internationally recognized standard. Rigorous photogrammetric and powerful computer vision algorithms have joined their respective strengths to the generation of quasiautomatically-textured dense 3D point clouds from 2D images in single/multiple baseline arrangements and are now commercially available (see Figure 21). Low-cost and professional-level software packages and opensource solutions, including web services, have become very popular in diverse fields. This technology is readily available and is applied to all three measurement ranges mentioned above using a wide variety of instruments. Dense 3D reconstructions are possible using everything from scanning electron-microscopes images (SEM) [48] to aerial images. Many different types of images have been used and are generated with metric, semi-metric, and consumer-grade photographic cameras [2,4,49]. Some tests of a public domain software package were performed within the National Research Council (NRC) of Canada Metrological Laboratory, built specifically for 3D imaging metrology work [50]. The test-object used, an ad-hoc 3D artefact (see Figure 22a), was characterized by important depth variations, different textures and materials, and the presence of both large-and small-scale details on the surfaces. The 3D scene included contrast targets, scale bars and spheres. A digital camera (4368 × 2912 pixels) equipped with a fixed focal length lens (50 mm) was used in the experiments. The 3D artefact was acquired at a focusing distance of about 4.75 m. A comparison was made between the point cloud generated with the dense 3D package and a 'reference' instrument that created a similar point cloud. That point cloud was acquired with a hemispherical laser scanner. Reference coordinates were also generated but with a laser tracker on some contrast targets. Both target location values and point clouds comparison was performed. An accuracy assessment of the image orientation phase expressed as a standard deviation calculated from ten control points and twenty-seven 2D images yielded Vx=0.47 mm, Vy=1. The color-coded error maps shown on Figure 22b indicate that the largest deviations from the reference data are mainly located at sharp surface gradients, such as the ones corresponding to the edges between the vertical walls of the corners and to the small grooves among the bricks. This is similar to sudden changes of curvature and depth on the surface object. The color scale ranges from -5 mm (violet) to +5 mm (red). These sharp edges are problematic for active laser scanners as well as with many other 3D imaging techniques. This is especially true when the spot diameter is large compared to the structural (lateral) resolution being analyzed. In the present situation, there may be a mismatch between the structure resolution of the laser scanner and the imagebased 3D point clouds. Figure 22b demonstrations that the image-based reconstruction can resolve small creases on the wallpaper glued on the two surfaces (top section of ad-hoc 3D artefact).

13003-p.16
Many elements of 3D imaging at the instrument or processing level, and in the different operating ranges, are common. The different standards (documentary and physical) and guidelines mentioned in the previous sections are tailored to a particular field of application but there are issues that all most face and answer.

What are we Measuring Anyway?
This question is fairly complex and requires a complete understanding of all the possible sources of error that can affect measurement results and the "true" nature of the measurement process. The field of surface metrology, with the recent publication of areal surface standards, i.e., ISO 25178, needed to address the difference between a mechanical and an electromagnetic surface. Indeed optical probing of surfaces behaves in a different way compared to a stylus tip (conically-shaped or hemispherical). For example, ISO 25178-3 considers the two cases: sphere radius and lateral period limit as a function of maximum sampling distance for a given S-Filter nesting index.
Light interacts with the environment-matter interface in a much different way compared to tactile probes. A typical user needs to understand a measurement situation in terms of light absorption, transmission, reflection, scattering and diffraction. An expert user may need to look at the physics behind the process. Looking at the underlying assumption made with active optical time-offlight-based measurements that the imaged surface is opaque, diffusely reflecting and uniform, one should expect unreliable measurements on some types of surface if translucent. Marble and some polymeric materials depart from the ideal hypothesis by exhibiting two critical optical properties: translucency and in some cases non-homogeneity. Figure 23 two 3D reconstructions, one from a time-of-flight laser scanner (section 3.3) and a second with a multi-baseline imagebased method (section 4). The effect of different translucent materials on the optical distance capability of the two methods is fairly obvious. The wall is made of the same marble, a few blocks are very recent additions and the rest are the original. That wall had been restored recently and should be fairly flat or at least with a small curvature. Figure 23a shows an apparent displacement of about 15 mm of the blocks made using new marble and the 2D image-based reconstruction on Figure 23b shows only a global curvature of the wall. Some of the polymerbased spheres shown on Figure 18 give similar systematic errors when measured with time-of-flight laser scanners. On the other hand, optical triangulationbased 3D imaging systems give some but reduced systematic errors but they exhibit an increased measurement noise level compared to measuring an opaque material [1][2]. (Scale: blue is -20 mm and green-yellow +8 mm)

External frame of reference (EFOR)
In applications where a large measurement volume needs to be accessed but with low measurement uncertainty, a single 3D imaging camera can't always be found. One solution is to rely on the 3D shape of an object and to simultaneously register all the 3D images covering the large object in a common coordinate system. Three-dimensional image registration is known to produce cumulative errors (bias errors) especially on surfaces with dull topographies. In surface metrology (section 3.1), images are routinely stitched together to form a larger one. Typically arrays of 5×5 images are stitched together using the system's translation stage or some local processing. In 3D object and site modelling (sections 3.2, 3.3 and 4), an ICP algorithm is used to minimize the difference between two or more pairs of point clouds. The number of 3D images can easily be in the hundreds. By the way, the term "registration" is defined in the ASTM-E57.
ASTM-E57-E2544 -11a registration, n-the process of determining and applying to two or more datasets the transformations that locate each dataset in a common coordinate system so that the datasets are aligned relative to each other.
The next-best solution to achieving a large measurement volume with relatively small measurement uncertainty is to help the registration process with a more robust external frame-of-reference strategy and/or a device or set of devices. Methods include x adding known features like coded or non-coded contrast targets and spheres in the total field of view of interest and scanning them at the best resolution possible, x combining the 3D imaging camera with the small field of view with an accurate 3D positioning system (optical or mechanical) with a large enough field of view/span and which relies on targets or markers or encoders. Accurate optical 3D systems can be one of the following or all of them: The only available complete solution at the international level and with a traceability route (EFOR plus 3D scanner) is with the ISO 10360. Although the other solutions can yield a measurement range in the 0.1 m to 20 m range with a local measurement quality of a close range system operating at a range of 0.1 m, at the moment no traceability route is covered by a documentary standard and a set of physical standards at the international level for those combined systems.

Hidden filtering and the parameterization of a 3D surface
The optical measuring techniques listed in section 1 and their implementation into systems covered by documentary standards described in section 3 are meant to measure a distance from a given probing sensor location to a point or set of points on the surface of a workpiece. It is the scanning mechanism or the projection technique used that defines a complete surface s(x,y,z) using a surface parameterization given by where (u, v) are the variables representing the two degrees of freedom of the scanning/projection mechanism (e.g. two deflection angles) and the variable (w) depends on the distance-measuring method (anglederived for triangulation, range for TOF and interferometry), (x, y, z) are the computed coordinates from these variables and the parameters extracted after calibration of the 3-D imaging system, and, T is the transpose matrix operator. The pose (EFOR) composed of the rotation matrix and translation vector has been omitted for simplicity but can be important to preserve the 3D data set orientation in space i.e. point of view of the 3-D imaging system. a) b) c) Figure 24. Some topological organizations of 3D data sets, the parameterization of a surface varies according to the scanning method, a) regular grid where a surface is sampled at constant X,Y increments (e.g. CMM), b) random profiles acquisition from a hand-held line scanner, c) regular profiles acquisition from a line scanner mounted on a translation stage.
It is important to remember that the surface is not necessarily sampled on a regular (x, y) grid as shown on Figure 24a. This is the case in the ISO 10360-8 where an optical distance sensor is used to acquire 3D coordinates using a CMM. In another implementation, intersecting lines (profiles) on the surface can be acquired using a manually operated mechanical arm with appropriate joint encoders or with some optical device that can track the profile scanner (Section 5.2 and Figure 24b).The results are unorganized 3D data sets that are transformed into a polygonised or triangulated data files and delivered to the user. Figure 24c shows the case where the sequence of line by line acquisition may be done at constant spatial intervals but the sampling within a line follows a constant angular sampling criterion. Not shown here, hemispherical scanners yield constant angular sampling along the two scanning directions. The issue here is that when a user gets a file containing (x,y,z) coordinates, these coordinates originate from raw measurements that are not necessarily obtained in a Cartesian coordinate system. A certain amount of fixed or data-dependent adjustment, filtering or interpolation has already been applied to the measurements. These may be disclosed by the manufacturer but are usually part of the confidential information of the instrument. A full calibration in the sense of the VIM would be a better route to follow and not just the delivery of a file with (x,y,z) data.

Optical structural resolution: axial and lateral
The ability to distinguish two objects or features from each other is fundamental in optical 3D metrology. This is often called the structure resolution limit of detection of an instrument. It is the ability of a 3D optical sensor to react to a change in the surface topography. Structure (VDI/VDE) or structural (ISO) resolution is conveniently divided into two components: a lateral part and an axial (longitudinal) part. The former corresponds to a perpendicular component and the latter to the parallel component, both with respect to the 3D sensor's optical axis. For on-axis (projection and collection paths are collinear) measurement systems we find those based on time-of-flight and interferometry principle and also some through-the-lens triangulation systems, such as focusvariation and confocal systems, are in this category. Wide-baseline triangulation systems need careful attention because the projection and collection paths are not collinear. According to ISO 25178-600, other expressions exist for resolution which encompasses several parameters and functions to actually quantify the topographic spatial resolution. They depend on the application and the method of measurement. These include lateral-period limit, stylus-tip radius, lateral resolution, width limit for full-height transmission, small-scale fidelity limit, Rayleigh criterion, Sparrow criterion and Abbé.Surface metrology instruments are covered in more details in the ISO 25178 series of documentary standards. For example, the "lateral-period limit" is the spatial period of a sinusoidal profile at which the height response of the 13003-p.18 instrument transfer function falls to 50%. Here, height is linked to spatial frequency or in other words a maximum slope may be specified for a given instrument.
Optical lateral resolution can be given by the Rayleigh or Sparrow limits or Abbé definition depending on the coherence of the source [3,51]. The numerical aperture is fundamental to understand lateral resolution and depth of field for a diffraction-limited imaging system. According to basic physics, the angular aperture is the angle of the cone of light entering an optical system from a point on a surface being measured. The numerical aperture, AN is the sine of the half aperture angle multiplied by the refractive index n of the surrounding medium: ‫ܣ‬ ே = sin ߙ In the close range, systems covered by the VDI/VDE 2617-6.1 which perform lateral optical measurements, discuss optical structure resolution axial and lateral using simple tests and a basic Fourier signal analysis. The VDI/VDE 2634 has no information on this topic. The ISO 10360-8 discusses structural resolution influence quantities in more detail, a test method and provides some examples of material standards. An optical transfer function analysis may give us a more realistic picture of the system's resolution [52]. As it is proposed in surface metrology using areal instruments, a star pattern can help determine the lateral resolution [31]. There are other patterns like gratings, steps or spheres but the star pattern seems to appear often in many experiments from the nano/micro range to the mid-to-large range systems [1,4,11,40,41]. Some authors are exploring superresolution techniques which allow the acquisition of images with a higher resolution than the diffraction limit [52].
Without dwelling in the details of instrument, we find that axial resolution (longitudinal) depends on the measurement principle selected (see Figure 1), the surface topography/reflectance being evaluated and environmental conditions. Signal to noise, surface microtopography, and speckle limit the depth measurement ability of a system [1][2][3][4]11].

Outlier Removal and Data Anomalies
The last topic covered in this paper is related to the idea that when one has to deal with dense 3D information originating from a 3D imaging system, not all the 3D data should be used when extracting a feature, e.g., in sphere fitting, plane fitting, a height parameter like Sq. As seen earlier, documentary standards allow for filtering and pruning of the data set. This can take the form of a removal of 3 points out of 1000 points, selecting 95% of the points, or by filtering using linear, robust or morphological operators. The removal of measured points stems from the fact that real structures (on surfaces) may be confused with by-products of the measurement process. This situation occurs in all 3D imaging systems from the nano/micro range to the midto-long range. These anomalies are sometimes called artifacts, outlier points, or blunders. We list some of them using common names: step edge (whiskers), ghost/phantom points, contrast-related, laser or Fringe spread on steep surfaces, edge curl, mixed pixels, multiple reflections, dust, rain, snow. The visible effect on a profile is observed as a sharp range discontinuity that correspond to a reflectance change (see Figure 26a), and rounding or overshoot on edges (see Figure 26b) [53]. a) b) Figure 26. Simple discontinuities on a surface, a) sharp depth (range) discontinuities appear with a reflectance change on a flat surface, b) rounded or overshoots (edge curls) appear on step edges on a surface.

Conclusion
We presented a progress report on standards developments to date in the diverse, but finite, world of non-contact 3D imaging systems from the nanometre to the one-hundred metre range. The current sets of internationally-recognized standards and some national guidelines represent a fundamental way to ensure better communication about a system's specifications between users and manufacturers. The documentary and physical standards produced to date help to solve a part of an issue faced by companies or organizations that integrate noncontact three-dimensional (3D) imaging systems into their production pipeline: deciding in which technology to invest. Quality non-contact 3D imaging systems typically involve a significant investment when considering the cost of equipment, training, software, and maintenance contracts over the functional lifetime of a given system or systems, notwithstanding the requirements of the global nature of manufacturing activities. The nano/micro range seems to converge to a detailed list of methods for the calibration and reverification of 3D imaging systems. For close-range systems there are some gaps in the existing documentary and physical standards. These gaps are found in the lack of methods for structure resolution evaluation, performance assessment on free-form surfaces or GD&T related evaluation, accuracy of 3D imaging camera mounted on external frames of reference (EFOR), dynamic range and signal to noise characteristics, just to name a few. Mid-to-long range systems appear to converge towards both total station-and laser trackercompatible methods. Some aspects of structure resolution may be considered in the near future. While many engineering needs require only geometry information, there is a need, particularly in the field of Heritage, for the collection of colour information in registration with the geometry. The global nature of manufacturing and engineering services activities is surely a driving force behind continuing the work of defining standards. We may, in the near future, need to consider creating an interdisciplinary committee that would be mandated to harmonize the terminology and test methods in the field of 3D imaging in its entirety.