A NOVEL ENTROPIC-GROUPING TECHNIQUEFOR OUTLIERS DETECTION

摘要

Outlier detection identifies highly deviate observations, which may jeopardise the underlyingdistributions in hydrologic analyses. Most conventional outlier detection methods require the assumptionsof a probability distribution and a chosen level of significance. A new hybrid non-parametric group-basedtechnique called Entropic-grouping is proposed for outlier detection. A concept is suggested whereaccordancy among elements can be recognized by combining individual elements in a certain manner. Ahigher level of accordancy among elements can be achieved by removing the element(s) (i.e., outlier(s))violating the concept of agreement in grouped data sets. Accordancy measures of Canberra, City-block,Euclidean, Minkowski and Mahalanobis are also employed for evaluation based on a decision rule oftrading off between losing the information from reducing the observation size to exchange forimprovement of the overall accordancy. In the case study, the grouping approach demonstrates a certainagreement with eight conventional methods. The advantages of using the group based outlier detectiontechniques areas are: no subjective postulation about the underlying probabilistic distribution; noinvolvement of error based conceptual ideology for interpreting the inherent nature of flood data sets; andthe size, sequence or completeness of the testing data set will not be a concern in the analysis.