Distributed Big Data Driven Framework for Cellular Network Monitoring Data

摘要

The smart monitoring system (SMS) vision relies on the use of ICT to efficiently manage and maximize the utility of network infrastructures and services in order to improve the quality of service and network performance. Many aspects of SMS projects are dynamic data driven application system where data from sensors monitoring the system state are used to drive computations that in turn can dynamically adapt and improve the monitoring process as the complex system evolves. In this context, a research and development of new paradigm of Distributed Big Data Driven Framework (DBDF) for monitoring data in mobile network infrastructures entails the ability to dynamically incorporate more accurate information for network monitoring and controlling purposes through obtaining real-time measurements from the base stations, user demands and claims, and other sensors (for weather conditions, etc.). The proposed framework consists of network probes, data parsing application, Message-Oriented Middleware, real-time and offline data models, Big Data storage and Decision layers., and Other data sources. Each Big Data layer might be implemented using comparative analysis of the most effective Big Data solutions. In addition, as a proof of concept, the roaming users detection model was created based on Apache Spark application. The model filters streaming protocols data, deserializes it into Json format and finally sends it to Kafka application. The experiments with the model demonstrated and acknowledged the capacities of the Apache Spark in building foundation for Big Data hub as a basic application for online mobile network data processing.