A fast multi-patterns parallel matching algorithm for massive HTTP data processing

摘要

The development of data services in wireless mobile networks leads to the tremendous growth of net users, making user behavior grow rapidly. And it brings a great opportunity for researchers to analyze user behavior through large-scale network traffic, which is not only significant for Internet Service Providers (ISP) to optimize resource allocation, but also can provide users with more customized service. The analysis of user behavior is based on the extraction of user characteristics, and multi-patterns URL matching is the foundation. However, the efficiency of extracting user behavior from massive network traffic data is still a huge challenge problem. This paper focuses on the efficiency of extracting user characteristics and proposes a novel algorithm, Multi-Patterns Parallel Matching on HTTP Traffic (MPPM) that takes advantage of the hash map in data searching, and it can extract user behavior from massive HTTP traffic more effective and faster than conventional methods with the same accuracy. Experiments are conducted by using real-world HTTP traffic data collected from the ISP networks. It is demonstrated that the proposed algorithm is superior to the known methods, as well as the capacity of dealing with massive HTTP traffic data. The implementation of MPPM algorithm will be a solid base to build a high-performance analysis engine of user behavior for massive HTTP data processing.