An improved algorithm for clustering uncertain traffic data streams based on Hadoop platform

摘要

During the development of intelligent transportation systems, traffic data has the characteristics of streaming, high dimension and uncertainty. In order to realize the query of uncertain traffic data streams in a distributed environment, the authors design the algorithm of Uncertain Traffic Data Stream Parallel Continuous Query algorithm (UTD-SPCQ). Firstly, the sliding window mode is applied to realize the data receiving and buffering in the data stream environment, so as to adapt to the MapReduce computing framework of the Hadoop distributed structure. Then, the impact of the high dimensionality and uncertainty of the data on the feature analysis of the dataset is reduced, through the dimension reduction and data rewriting. Finally, a multi-attribute data point RePoint is newly defined, to solve the problem of data dimension increase caused by data rewriting. Experiments show that this algorithm optimizes the traditional density-based clustering algorithm, and make it more adaptable to parallel continuous queries for uncertain traffic data streams, and can fully consider the newly generated streaming traffic data.