Control chart and Six sigma based algorithms for identification of outliers in experimental data, with an application to particulate matter PM10

摘要

Outliers, which can have significant effects on further analysis and modelling, occur between continuously measured environmental data. Most methods for outlier detection depends on model or distribution of observed variable. However the distribution of environmental variables cannot be estimated quite often. This paper presents two procedures, which do not impose restrictions on the distribution of analysed variable, and which permit the intervals of the environmental observations, where the outliers occur, to be detected. The proposed procedures are based on smoothing original data and subsequent analysis of the residuals. The output of both methods is an interval of observations, where the residual process behaves substandard, and whose quality must be further manually assessed. Thus the value of the proposed methodology is that the number of observations for manual data control is reduced. Both methods are applied to problem of detection outliers in hourly PM10 measurements. However, the methodology is general and can be applied to different type of data whose quality control is required.