AutoCorrel II: A neural network event correlation approach

摘要

As a follow-up to our earlier model Autocorrel I, we have implemented a two-stage event correlation approach with improved performance. Like Autocorrel I, the new model correlates intrusion detection system (IDS) alerts to automate alert and incidents management, and reduce the workload on an IDS analyst. We achieve this correlation by clustering similar alerts, thus allowing the analyst to only consider a few clusters rather than hundreds or thousands of alerts. The first stage uses an artificial neural network (ANN)-based autoassociator (AA). The AA's objective is to attempt to reproduce each alert at its output. In the process, it uses an error metric, the reconstruction error (RE), between its input and output to cluster similar alerts. In order to improve the accuracy of the system we add another machine-learning stage which takes into account the RE as well as raw attribute information from the input alerts. This stage uses the Expectation-Maximisation (EM) clustering algorithm. The performance of this approach is tested with intrusion alerts generated by a Snort IDS on DARPA's 1999 IDS evaluation data as well as incidents.org alerts.