Learning Decision Trees from Synthetic Data Models for Human Security Behaviour

摘要

In general, in order to predict the impact of human behaviour on the security of an organisation, one can either build a classifier from actual traces observed within the organisation, or build a formal model, integrating known existing behavioural elements. Whereas the former approach can be costly and time-consuming, and it can be complicated to select the best classifier, it can be equally complicated to select the right parameters for a concrete setting in the latter approach. In this paper, we propose a methodical assessment of decision trees to predict the impact of human behaviour on the security of an organisation, by learning them from different sets of traces generated by a formal probabilistic model we designed. We believe this approach can help a security practitioner understand which features to consider before observing real traces from an organisation, and understand the relationship between the complexity of the behaviour model and the accuracy of the decision tree. In particular, we highlight the impact of the norm and messenger effects, which are well-known influencers, and therefore the crucial importance to capture observations made by the agents. We demonstrate this approach with a case study around tailgating. A key result from this work shows that probabilistic behaviour and influences reduce the effectiveness of decision trees and, importantly, they impact a model differently with regards to error rate, precision and recall.