Anomaly detection in data mining. Hybrid approach between filtering-and-refinement and DBSCAN

摘要

Anomaly detection is a domain that represents the key for the future of data mining. We will try to present some key anomaly detection methods applicable in the data mining process. Some methods are existing techniques as the DBSCAN algorithm and some have just been presented to the public recently and could be the answer to future anomaly detection development. One example is the filtering-and-refinement approach, a new general two stage technique for more efficient and effective anomaly detection. This paper will try to illustrate the strengths and weaknesses of the classical techniques presented but as we will see the results are completely dependent on the data sets that are analyzed. We will emphasize on efficiency, robustness and accuracy. We will also try to demonstrate a hybrid approach obtained by combining the filtering-and-refinement method with the DBSCAN algorithm. In our experiments we pursued to compare the performance of the normal DBSCAN algorithm with the performance of the hybrid one. Our results indicate that the hybrid method is more accurate in terms of detecting anomalies and far superior in terms of speed than the normal DBSCAN algorithm.