A DCT based approach for detecting novelty and concept drift in data streams

摘要

Data streams are one of the most challenging environments for machine learning. In many applications, the high volume data streams have an inherent concept drift over time. Identifying novel classes and detecting the occurrence of concept drift in such an environment is a major challenge. In this paper, a new method has been proposed to detect novelty and handle concept drift with limited required memory and storage space. The method is based on clustering algorithm. It uses Discrete Cosine Transform to build compact generative models which are then used to detect novel classes and concept drift effectively. The proposed method has been evaluated with seven common data sets from various domains. The results indicate its superior performance when compared with existing methods in terms of novelty and drift detection, computational complexity and memory requirements.