Analysis of Commuting Characteristics of Mobile Signaling Big Data Based on Spark

摘要

The study of commuting mode is of great significance for reducing urban traffic pressure and constructing intelligent city. However the commonly used research methods are slow in computing when dealing with large-scale mobile signaling data. A method of parallel clustering and statistics using Spark is proposed. In this method, a large amount of data is cleaned on Hive and denoised. The user data is divided into different areas through the K-Means algorithm on the Spark, and then the spatial-temporal statistics are carried out in the different partition area. Finally, the location of the user's place of residence and work and the length of commuter distance and time are obtained, which can be used to divide users from the traditional nine-to-five and non-nine-to-five and provide an effective reference for urban planning and traffic congestion.