A machine learning approach to quality control oceanographic data

摘要

Sensor errors are inevitable when measuring the ocean; thus, a reliable dataset of observations requires a quality control (QC) procedure capable of detecting spurious measurements. While manual QC by human experts minimizes errors, it is inefficient to handle large datasets and vulnerable to inconsistencies between different experts. Although automatic QC addresses some of these issues, the traditional methods result in high rates of false positives. Here, I propose a machine learning approach to automatically QC oceanographic data based on the Anomaly Detection technique. Multiple tests are combined into a single multidimensional criterion that learns the behavior of the valid measurements and identifies bad samples as outliers. When applied to 13 years of hydrographic profiles, the Anomaly Detection resulted in the best classification performance, reducing the error by at least 50%. The Anomaly Detection approach introduced here was implemented in the Python package CoTeDe, an open-source framework to quality control oceanographic data.