Toward a monitoring and threat detection system based on stream processing as a virtual network function for big data

摘要

The late detection of security threats causes a significant increase in the risk of irreparabledamages and restricts any defense attempt. In this paper, we propose a sCAlable TRAfficClassifier and Analyzer (CATRACA). CATRACA works as an efficient online Intrusion Detectionand Prevention System implemented as a Virtualized Network Function. CATRACA is based onApache Spark, a BigData Streaming processing system, and it is deployed over theOpen PlatformforNetwork Functions Virtualization (OPNFV), providing an accurate real-time threat-detectionservice. The system presents a friendly graphical interface that provides real-time visualizationof the traffic and the attacks that occur in the network. Our prototype can differentiate normaltraffic from denial of service (DoS) attacks and vulnerability probes over 95% accuracy underthree different datasets. Moreover, CATRACA handles streaming data under concept driftdetection with more than 85% of accuracy.