Data streams may encounter data distribution changes, which can significantly impair the accuracy of models. Concept drift detection tracks data distribution changes and signals when to update models. Many drift detection methods apply thresholds to distinguish between drift or non-drift streams and to claim their method outperforms others with non-aligned drift thresholds. We consider that selecting a proper drift threshold could be more important than developing a new drift detection algorithm, and different drift detection algorithms may end up with very similar performance with aligned drift thresholds. To better understand this process, we propose a novel threshold selection algorithm to align the drift thresholds of a set of algorithms so that they are all at the same sensitivity level. Based on comprehensive experiment evaluations, we observed that several state-of-the-art drift detection algorithms could achieve similar results by aligning their thresholds, providing a novel insight to explain how drift detection algorithms contribute to data stream learning. We noticed that a higher detection sensitivity improves accuracy for data streams with frequent distribution change. The evaluation results are showing that drift thresholds should not be fixed during stream learning. Rather, they should adjust dynamically based on the prevailing conditions of the data stream.
展开▼
