An incremental intrusion detection system using a new semi-supervised stream classification method

摘要

In recent years, the utilization of machine learning and data mining techniques for intrusion detection has received great attention by both security research communities and intrusion detection system (IDS) developers. In intrusion detection, the most important constraints are the imbalanced class distribution, the scarcity of the labeled data, and the massive amounts of network flows. Moreover, because of the dynamic nature of the network flows, applying static learned models degrades the detection performance significantly over time. In this article, we propose a new semi-supervised stream classification method for intrusion detection, which is capable of incremental updating using limited labeled data. The proposed method, called the incremental semi-supervised flow network-based IDS (ISF-NIDS), relies on an incremental mixed-data clustering, a new supervised cluster adjustment method, and an instance-based learning. The ISF-NIDS operates in real time and learns new intrusions quickly using limited storage and processing power. The experimental results on the KDD99, Moore, and Sperotto benchmark datasets indicate the superiority of the proposed method compared with the existing state-of-the-art incremental IDSs.