互联网开源数据存储与分析技术研究

摘要

当前网络环境中的开源数据量巨大，传统数据采集技术难以满足对开源数据的采集和分析需求。文章提出采用云存储的方法进行开源数据收集处理，从而解决了开源数据存储的难题。同时，文中阐述了基于云存储的开源数据计算分析技术设计思路，对开源数据存储与计算各环节进行了分析研究。%Great open source data currently in the network environment, the traditional data acquisition technology to meet the open source data collection and analysis requirements. This paper put forward the method of open source cloud storage of data collecting and processing, in order to solve the problem of open source data storage. At the same time, this paper expounds the design idea of open source data analysis technique based on cloud computing, based on the analysis of the open source data storage and calculation of each link.