Deep learning and 3D optical metrology improve complex feature detection and ranking

France Abstract. ​ With the increasing need of measurement automation in many industrial sectors, 3D optical metrology is now required to integrate options like surface feature detection and classification that were previously performed by visual inspectors. The recent combination of deep learning algorithms with 3D optical metrology improves characterization and classification of


Focus-Variation Technology
Focus-Variation combines the small depth of focus of an optical system with vertical scanning to provide topographical and color information from the variation of focus. The main component of the system is a precision optics containing various lens systems that can be equipped with different objectives, allowing measurements with different resolution. Due to the small depth of field of the optics only small regions of the object are sharply imaged. To perform a complete detection of the surface with full depth of field, the precision optics is moved vertically along the optical axis while continuously capturing data from the surface. This means that each region of the object is sharply focused. Algorithms convert the acquired sensor data into 3D information and a true color image with full depth of field. This is achieved by analyzing the variation of focus along the vertical axis.

Deep learning integration
The integration of machine learning algorithms to such systems gives optical profilers and µCMMs the ability to learn how to detect and classify surface features using large data sets instead of using hard coded rules. The deep learning algorithm uses neural layers arranged in different categories, were each connection between neurons is associated with a weight.
These weights allow to compute an output prediction based on input dataset values. During the necessary training phase of the machine learning algorithm, the use of a large number of input datasets with known output allows to automatically adjust the weights of the neural connections by iteration, in order to obtain the right prediction associated to each input dataset.

Applications
Already developed deep learning modules make it possible to automate inspection operations that previously included a visual inspection.
Automatic defect measurement module allows automatic measurement and display of the number, length, depth, volume and height of defects.
Even small defects in components (e.g.rotor blades) or component assemblies (e.g. motors) can have critical effects on safety, as a concentration of stress is created here. The maximum depth of a defect determines whether the component needs to be repaired or disposed of. Heat development, which can lead to the melting of the turbine, as well as crack formation can be consequences of insufficient quality control due to inadequate measurements. Focus-Variation combined with deep learning algorithms makes it possible to quantitatively detect surface defects as well as component edges and visualize them in 3D. Typically, the technology is used to measure width, length, depth and volume of fractures, scratches, etc.  Grinding grain analysis allows the segmentation or separation of abrasive grains and binder. Automatic analysis of the grain tip protrusion is performed as well as additional evaluation of grain size distribution and chip space volume. Shot peening analysis allows automatic evaluation of surface coverage rate on a pinned surface.
Shot peening is one of the most effective methods of improving fatigue resistance and preventing microcracks. As of today, the Almen Strip Test is a common method for the analysis of irradiation intensity. During one process cycle, the Almen Strip, consisting of spring steel, is shot peened. The processed surface is formed into an arc. The blasting process is then evaluated by manually. measuring the sheet height. Here the irradiated surface is not directly assessed, e.g. by determining the percentage affected by the abrasive. The irradiation process is characterized only indirectly by the curvature of the Almen Strip. Such indirect tests, performed by different users, show measurement deviations of up to 15%. This cumbersome, inaccurate test can be significantly improved. The combination of image analysis applications and 3D metrology enables the automatic evaluation of the degree of coverage of blasted surfaces and the verification of the necessary processing time including beam intensity. The repeatable, precise measurement takes place directly on the component and creates a direct evaluation based on metrological data. Laser treatment analysis allows automatic verification that the specified surface laser treatment has been achieved for the surface to function Surface treatment by laser is intended to increase the safety of cleaning processes in preparation for the following surface processes such as bonding or passivation. Part of quality assurance is to verify whether a component has been treated with the correct laser intensity. Since the laser intensity coupled into the component depends not only on the beam power of the laser itself, but also on the surface reflectivity of the component, simply monitoring the laser output power is insufficient. A direct analysis of the 3D surface structure of the lasered component using algorithms that have automatically learned the correct structure from positive and negative examples can reliably solve this task.

Fig. 5.
Evaluation of a lasered surface structure. Above: 60% and thus insufficiently lasered surface; below: 100% and thus sufficiently lasered surface Conclusion As a conclusion, such deep learning modules combined with optical 3D high resolution systems represent two major advances in the field of industrial metrology. First, they make it possible to avoid visual inspection and therefore the associated operator influence, in order to systematize controls. Secondly, the combination of deep learning modules with robust, repeatable and environmentally insensitive 3D optical metrology systems makes it possible to standardize the use of high-resolution optical metrology systems in production,