A method of evaluating and deploying machine learning models for anomaly detection of a monitored system includes providing a plurality of candidate machine learning algorithms configured for anomaly detection of the monitored system. For each type of anomalous activity, a benchmarking dataset is generated, which comprises samples drawn from a pool of negative samples, and a smaller number of samples drawn from a relevant pool of positive samples. For each combination of candidate machine learning algorithm with type of anomalous activity, the method includes drawing a plurality of training and cross-validation sets from the benchmarking dataset. Using each of the training and cross-validation sets, a machine-learning model based on the candidate algorithm is trained and validated using the cross-validation set, with average precision as a performance metric. A mean average precision value is then computed across these average precision performance metrics. A ranking value is computed for each candidate machine learning algorithm, and a machine learning algorithm is selected from the candidate machine learning algorithms based upon the computed ranking values. A machine learning model based on the selected algorithm is deployed a to a monitoring system, whereby the monitoring system executes the deployed machine learning model to detect anomalies of the monitored system.
展开▼
