Incremental anomaly detection using two-layer cluster-based structure

摘要

Anomaly detection algorithms face several challenges, including processing speed, adapting to changes in dynamic environments, and dealing with noise in data. In this paper, a two-layer cluster-based anomaly detection structure is presented which is fast, noise resilient and incremental. The proposed structure comprises three main steps. In the first step, the data are clustered. The second step is to represent each cluster in a way that enables the model to classify new instances. The Summarization based on Gaussian Mixture Model (SGMM) proposed in this paper represents each cluster as a GMM. In the third step, a two-layer structure efficiently updates clusters using GMM representation, while detecting and ignoring redundant instances. A new approach, called Collective Probabilistic Labeling (CPL) is presented to update clusters incrementally. This approach makes the updating phase noise-resistant and fast. An important step in the updating is the merging of new clusters with existing ones. To this end, a new distance measure is proposed, which is a modified Kullback-Leibler distance between two GMMs.