面向海量数据的在线流数据服务框架

摘要

To handle the problem of sharing massive streaming data between different applications,we proposed an online streaming data service framework.By analysing and abstracting the scene of sharing streaming data by multi-application,we designed a three-tier online streaming data service framework consisting of data tier,management tier and interface tier.The framework divides the streaming data management into four components including index building,registrar,matcher and memory cleaning,and manages the streaming data by a special data management engine.The presentation of the framework gave a solution of uniform streaming data management to the scene of streaming data sharing by multi-application.Experimental results showed that,relative to traditional database,the writing speed of the proposed service framework increased nearly 5 times,and the reading speed increased by 4 times.%针对多应用共享海量流数据的问题，提出一个在线流数据服务框架。通过对多应用共享流数据场景进行分析和抽象，设计具有数据层、管理层和接口层的在线流数据服务框架。框架将流数据管理划分为索引建立、注册器、匹配器和内存清理四个组成部分，并且通过一个专用的数据管理引擎对流数据进行管理。框架的提出为多应用流数据共享场景提供了一种统一管理流数据的方案。实验结果表明，相对与传统数据库，在线流数据服务框架在流数据写入速度能提高近5倍，在流数据的读取速度上提高4倍左右。