Design and implementation of monitoring data storage and processing scheme based on distributed computing

摘要

The monitoring system in cloud computing environment will generate a large number of monitoring data in real time, how to achieve efficient storage and processing of monitoring data in the big data environment is particularly important. To solve this problem, a scheme of monitoring data storage and processing based on Hadoop is proposed. The scheme adopts the HBase database to store the timing monitoring data, and enhance the field method of wide table storage model improved HBase database to improve storage efficiency monitoring data; the traffic data, using MapReduce distributed computing to improve the processing efficiency. The experimental results show that the scheme is scientific and effective, and it can improve the processing speed of the monitoring system under the massive monitoring data, and solve the bottleneck problem of the monitoring data in the cloud computing environment.