Finding Gold in the Sand: Identifying Anomaly Indicators Though Huge Amount Security Logs

摘要

Security devices produce huge amount of logs and far beyond the processing speed of human beings. This paper introduces a flexible, powerful, and unsupervised approach to detecting anomalous behavior in large-scale security logs. We provide an adaptive log extraction mechanism which could extract keywords and support similar log grouping. We propose an anomaly detection framework, named AIFinder, which supports different anomaly detection algorithms. To evaluate the effectiveness and efficiency of our framework, we conduct several experiments and run three anomaly detection algorithms. The results demonstrate that AIFinder could process security logs in a real-time manner with acceptable precision and recall.