Least squares best-fit geometric elements taking into account uncertainty structure

National Physical Laboratory, Hampton Road, Teddington, Middlesex TW11 0LW, UK

^a Corresponding author: ian.smith@npl.co.uk

Abstract

In coordinate metrology, a key activity is fitting a geometric surface to coordinate data. By far the most common fitting criterion has been ordinary least squares (OLS). OLS is appropriate if the covariance matrix associated with the data is a diagonal matrix. However, in recent years much effort has been devoted to developing more realistic uncertainty models for coordinate data and it is now timely to develop fitting algorithms that take into account this richer uncertainty structure. For many measuring systems, the uncertainty characterisation is such that the covariance matrix associated with coordinate data can be represented in a compact, factored form that makes explicit the contribution of random and systematic effects associated with the measuring system. This compact representation enables the least squares fitting algorithm to be implemented using essentially the same computational components as for the OLS fitting scheme.