Large dataset summarization with automatic parameter optimization and parallel processing for local outlier detection

摘要

As one of themost important research problems of data analytics and data mining, outlier detectionfrom large datasets has drawn many research attentions in recent years. In this paper, weinvestigate the interesting research problem of summarizing large datasets for supporting efficientlocal outlier detection. To summarize large datasets, efficient summarization algorithms areproposed that produce a highly compact summary of the original dataset, which can be appliedto detect local outliers from future similar datasets. A novel automatic parameter optimizationmethodis proposed toproduce theoptimal setup of thekey parametersused in the summarizationalgorithm. Parallel processing methods are also proposed to accelerate significantly the summarizationprocess. The experimental evaluation results demonstrate that our proposed algorithmsare highly scalable for large datasets and effective in producing dataset summary for local outlierdetection.