基于Hadoop的电网监控信息流分布式处理研究

摘要

针对智能电网调度系统中大数据集监控信息流存取困难问题，提出一种基于Hadoop云计算框架的分布式集群处理新方法。通过分析电网监控系统中信息流特性，提取3类关键信息流；综合利用分布式文件系统HDFS和映射聚合模型Map/Reduce，建立云集群分布式处理平台，实现监控信息的高效并行处理。以某配电网断面量测记录数据集为例进行测试，结果表明：该方法处理效率相比传统客户/服务器数据库方法提高了约1.4倍，集群规模越大，效果越好，有效避免了大数据集信息处理响应延迟问题。%In order to solve problems of large data sets access for monitoring information flow in smart grid dis-patching system,this paper proposes a distributed cluster processing method based on Hadoop cloud computa-tion framework for monitoring information flow. By analyzing information flow characteristics of power grid monitoring system,the three kinds of key processing stream were extracted;By comprehensively using distrib-uted file system HDFS and the parallel programming model Map/Reduce,the cluster distributed information processing platform was established,and information efficient parallel processing was achieved. Taking power grid dispatching measurement logging data set as an example,the test results show that the new method process-ing efficiency is improved 1.4 times compared with traditional client/server database method,and is related to cluster node scale,which can avoid response delay problem of large data sets information processing.