A protocol-specific constraint-based intrusion detection system

摘要

With the advancement of new technologies, the frequency of malicious attacks is also growing rapidly. Even networks without external connections cannot hide from these attacks. Constant monitoring of a network is vital for an organization's security system. Among numerous monitoring techniques, network behavior analysis has become popular. Normal traffic patterns of a network can be modeled as network constraints. The violation of these constraints indicates the possibility that an intrusion has occurred. Expressing network vulnerabilities is not an easy task. Sometimes, it is more complicated than recognizing an intruder's attack pattern. Currently, network administrators use Intrusion Detection System (IDS) rules to define security concerns regarding their networks. An IDS rule finds it difficult to detect sophisticated multi-packet intrusions. Constraints compared to IDS rules possess better expressiveness to describe a network behavior for defending the network against different attacks. Evaluating constraints in an efficient manner is a key to achieving a better IDS. Numerous constraint checking techniques provide good performance in solving constraints. However, they are not always effective in checking constraints with dynamic information. In this thesis, we propose a protocol specific constraint-based IDS to detect intrusions in a network. We investigate two protocols used in the Data Distribution Service (DDS) and identify their vulnerabilities. These two protocols are Internet Group Management Protocol (IGMP) and Real-Time Publisher Subscriber Protocol (RTPS). We develop constraints to protect a network against attacks that may exploit these vulnerabilities. For checking these constraints, a naive tree-based technique along with an optimized version is presented. Both techniques have the adaptability to cope with a continuous update of relevant network behavior information from a network traffic. The structure and life cycle of the constraint trees are explained in detail. A Domain Specific Language (DSL) is designed to express these constraints. An experimental private network is built which simulates network traffic similar to an Air Traffic Control System (ATC). Finally, we present how this IDS evaluates network constraints against the traffic generated from the experimental network and prevents attacks.