Zero-day Attack Identification in Streaming Data: Nearest Neighbor Heuristics and Dynamic Semantic Network Generation in the Spark Eco-system

摘要

Intrusion Detection Systems (IDS's) have been in existence for many years now, but they fall short in efficiently detecting zero-day attacks. Over the past decade, anomaly detection has attracted wide attention of numerous researchers to overcome the shortcomings of IDSs (Intrusion Detection Systems) in detecting zero-day attacks. In this research, we design an organic combination of Semantic Link Networks (SLN) and Dynamic Graph generation for the zero-day attacks on the fly into one comprehensive system. Furthermore, to deal with increasing volumes of network traffic and improve full packet analysis efficiency, we employ Spark Streaming platform for parallel detection. To substantiate the performance of zero-day attack detection process; we calculate the relevance of each feature in KDD'99 intrusion detection datasets. Compared to the previous studies on Zero-day attack identification, we witnessed comparably good results as we employed semantic learning and reasoning on top of the training data and also collaborative classification methods.