Floating car data processing model based on Hadoop-GIS tools

摘要

Urban travel characteristics is an essential apart of urban crowd flow analysis. Floating car data, known as “FCD”, have been recorded taxis' routine trajectories in cities, which can draw the characteristics of urban crowd flow. As a large volume of datasets, floating car data need an efficient way to process and analyze. A Hadoop-GIS methodology is used in this paper, consists of two major tools which provided by ESRI's `the GIS tools for Hadoop' - Spatial Framework for Hadoop and Geoprocessing Tools for Hadoop. Hadoop-GIS uses a spatial query languages HiveQL, an SQL-like language in Hive with schema transparently converting queries to MapReduce. More than 1 billion floating car data in Beijing over 17 days in November, 2014, is adopted in this paper, generated from about 32,000 GPS-equipped taxicabs. An aggregation analysis proves that Hadoop-GIS could process floating car data effectively and efficiently. After data cleansing and pretreating of floating car data, the taxi trajectories, which terminate at Tsinghua University, are first extracted through Hadoop-GIS to discover the crowd flow patterns in Beijing, aiming to provide suggestion for city vehicles management. Proved by experiments, the FCD processing model, which based on Hadoop-GIS, could fulfill the processing requirements of large scale floating car data sets and has the spatial analyzing capability of parallel processing. These features improve the storage and processing abilities of large scale spatial data greatly.