Outlier detection and sequence reconstruction in continuous time series of ocean observation data based on difference analysis and the Dixon criterion

摘要

In light of the specific characteristics of the long-term record and complex sequence nature of the ocean observation data, a new method was developed based on the original Dixon detection criteria to be specifically detect and remove data outliers. This method combines the two traditional methods of data quality control and Dixon detection theory and assumes that the second-order differential sequence of parameter measurements passes an appropriate stationarity test. Thus, the measurement attributes are considered to be in the same physical state and to occupy a small range of time and space, equivalent to a parallel observation test. Provided that the observations over a small range of time and space correspond to the record of a sequence covering a short period of time, this short time sequence is treated as a sliding window in the proposed new method. Outliers are detected based on lookup-table after an index parameter Q is calculated within the sliding window. A correlation analysis and the test results show that the proposed new method can effectively instantiate a sequence of outliers characterized by different phases. Compared with other existing methods, the new method proved to be computationally efficient and easily programmable for practical implementation. Further, this method preserves the original data because the outliers are replaced by an inverse distance-weighted average of the recorded data within the window, while other data were intact.