A novel general methodology for studying and remedying finite precision error with application in Kalman filtering

摘要

Least squares (LS) techniques, like Kalman filtering, are widely used in environmental science and engineering. In this paper, a new general approach is introduced for the study of the generation, propagation and accumulation of the quantization error in any algorithm. This methodology employs a number of fundamental propositions demonstrating the way the four operations addition, multiplication, division and subtraction, influence quantization error generation and transmission. Using these, one can obtain knowledge of the exact number of erroneous digits with which all quantities of any algorithm are computed at each step of it. This methodology offers understanding of the actual cause of the generation and propagation of finite precision error in any computational scheme. Application of this approach to all Kalman type LS algorithms shows that not all their formulas are equivalent concerning the quantization error effects. More specifically, few generate the greater amount of quantization error. Finally, a stabilization procedure, applicable to all Kalman type algorithms, is introduced that renders all these algorithms very robust.