The extraction of security situation in heterogeneous log based on Str-FSFDP density peak cluster

摘要

In order to reduce the false alarm rate in the process of security events extraction and discover a wide range of anomalies by scrutinising various logs, an improvement of Str-FSFDP (a fast search and find of peak density based data stream) clustering algorithm in heterogeneous log analysis is presented. Because of the advantages in data attribute relationship analysis for mixed attributes data, this algorithm can classify log data into two types whose corresponding distance measure metrics are designed. Twelve attributes are defined in the unified XML format for clustering in this paper. These attributes are divided by the characteristics of each type of log and the importance of expressing a security event. To match the new micro cluster characteristic vector mentioned in the Str-FSFDP algorithm, this paper uses time gap to improve the UHAD (unsupervised anomaly detection model) framework. The time gap is designed as a threshold value based on micro cluster strategy. Experimental results reveal that the framework using Str-FSFDP clustering algorithm with time threshold can improve the aggregation rate of the log events and reduce the false alarm rate.