Stochastic Configuration Networks Based Adaptive Storage Replica Management for Power Big Data Processing

摘要

In the power industry, processing business big data from geographically distributed locations, such as online line-loss analysis, has emerged as an important application. How to achieve highly efficient big data storage to meet the requirements of low latency processing applications is quite challenging. In this paper, we propose a novel adaptive power storage replica management system, named PARMS, based on stochastic configuration networks (SCNs), in which the network traffic and the data center (DC) geodistribution are taken into consideration to improve data real-time processing. First, as a fast learning model with less computation burden and sound prediction performance, the SCN model is employed to estimate the traffic state of power data networks. Then, a series of data replica management algorithms is proposed to lower the effects of limited bandwidths and a fixed underlying infrastructure. Finally, the proposed PARMS is implemented using data-parallel computing frameworks (DCFs) for the power industry. Experiments are carried out in an electric power corporation of 230 million users, China Southern power grid, and the results show that our proposed solution can deal with power big data storage efficiently and the job completion times across geodistributed DCs are reduced by 12.19 on average.