City digital pulse: a cloud based heterogeneous data analysis platform

摘要

In recent years, increasing attention has been paid to developing exceptional technologies for efficiently processing massive collection of data. This is essential in the research on smart city, which involves various types of data generated by different kinds of sensors (hard and soft). In this paper, we propose a cloud-based platform named City Digital Pulse (CDP), where a unified mechanism and extensible architecture are provided to facilitate the various aspects in big data analysis, ranging from data acquisition to data visualization. We instantiate the proposed system using multi-model data collected from two social networks, namely Twitter and Instagram, which can provide instant geo-tagged data. Data analysis is performed to detect human affections from user uploaded content. The information revealed from the collected social data can be visualized at multiple dimensions through a well-designed Web application. This allows users to easily sense changes in human affective status and identify the underlying reasons. This offers priceless opportunities to improve the decision making in many critical tasks using the detected attitudes in the social messages, such as promotion strategy for companies or new policy making for the government. Our experiment results confirm the effectiveness of the proposed architecture and algorithms.