Data Aggregation and Visualization Technique for Traffic Sensor Data

摘要

A wealth of information is captured by traffic sensors but extracting and representing the said information is a challenge. We developed a data processing tool in Apache Spark to aggregate the data points recorded by the sensors and enrich it with geographical information as well. We also developed a tool in Processing to aid the visual analysis of this data set. It plots the paths identified in the transformed data as a subway map, while still preserving the relative locations of each sensor. The transformed data is also suitable for further analysis using existing tools such as Tableau. We use all three of these tools in conjunction to solve the VAST challenge 2017 - mini challenge 1.