A Spark Platform-based Intrusion Detection System by Combining MSMOTE and Improved Adaboost Algorithms

摘要

With the development of information security technology, intrusion detection has become a new research field. Aiming at the problem of high misjudgment rate and slow processing speed of traditional intrusion detection algorithm, this paper proposes an intrusion detection system based on Spark platform using MSMOTE and improved Adaboost algorithms. The logical venation of this paper are: Firstly, it uses the Synthetic Minority Oversampling Technique (MSMOTE) to pre-process the unbalanced data sets. Secondly, it adds the sample point misjudgment rate into the Adaboost algorithm, and improve the weight of sample points and weak classifier according to the noisy data, meanwhile it classifies the data set. Finally, by using Spark platform, it takes the robust classifier weight obtained from the improved Adaboost algorithm as the standard weight, and it process the subsets of each node in parallel. According to the experimental results of KDD99 data set, the new intrusion detection method raised by this paper can decrease the system error rate and maintaining a high accuracy rate, besides it boosts the system processing speed effectively.