Estimating the standard uncertainty contribution of the straight-line fit algorithm used to determine the position and the width of a graduation line

摘要

We propose an implementation of the straight-line fit algorithm to evaluate data obtained as part of measurements of line structures, e.g. a grating line, which allows us to determine, at a conventional threshold value, the coordinates of the edges, the position and the width of its photometric profile as well as the related standard uncertainty contribution. These are derived for the (1 - alpha)100percent confidence intervals with alpha E [0, 1]. The implementation is demonstrated using data obtained by the PTB Nanometer Comparator (Fluegge and Koning 2001 Proc. SPIE 4401 275-83). The investigation revealed that large parts of the uncertainty of the position and the width of the line are due to the choice of the fit interval and the uncertainty of the determination of the minimum and maximum level of the photoelectric signal of the line.