Big data platform for air quality analysis and prediction

摘要

With the advance of industry, air quality (AQ) is increasingly becoming worse. There are increasingly AQ monitors device have been deployed around country for monitoring air-quality all year long. To estimate and predict AQ, such as PM (particulate matter) 2.5, become an important issue for government to improve people's quality of life. As we can know, there are many factors can affect the AQ, such as traffic, factory exhaust emissions, weather, incineration of garbage, and so on. In most well-developed countries, these pollution sources are monitored for future environmental policy making. In this paper, we will propose a semantic ETL (Extract-Transform-Load) framework on cloud platform for AQ prediction. In the platform, we exploit ontology to concretize the relationship of PM 2.5 from various data sources and to merge those data with the same concept but different naming into the unified database. We implement the ETL framework on the cloud platform, which includes computing nodes and storage nodes. The computing nodes are used to execute data mining algorithms for predicting, and storage modes are used to store retrieved, preprocessed, and analyzed data. We utilize restful web service as the front end API to retrieve analyzed data, and finally we exploit browser to show the visualized result to demonstrate the estimation and prediction. It shows that the big data access framework on the cloud platform can work well for air quality analysis.