一种基于NoSQL的气象非结构化数据产品存储方法

摘要

A NoSQL storage design for meteorological grid data products is introduced.An analysis is made of the meteorological structure of grid data products and their application scenarios,and the column storage database in NoSQL is designed to increase the efficiency of storage and access through optimizing the way of table store.According to the type,time and forecast effectiveness of data products,we work out the primary key column,attribute column,and data column,then,split the big grids of data column,use the grid sequence number to record data column names,and design data compression strategies to improve the overall efficiency of data access.Taking numerical prediction products as an example,we make use of Aliyun OTS to simulate tests.The results show that the column storage database based on NoSQL technology can meet with the requirements of the underlying data support of product service platform,which has a certain reference significance for the extension and application of key technologies,such as distributed data environment,cloud computing,and so on.%基于一种面向气象非结构化数据产品的NoSQL存储方法,综合气象网格化数据产品的结构及其应用场景,设计了NoSQL中的列存储数据库,并通过优化表格存储方式提高存储、访问效率.根据数据产品的类型、时间、预报时效等信息,制定主键列,属性列和数据列,设计数据存储模型.对数据列大网格进行切分,用网格点序号标记数据列名,并制定数据压缩策略,从而提高数据访问的整体效率.以气象数值预报产品为例,利用阿里云的OTS产品进行模拟测试,结果表明,基于NoSQL技术的列存储数据库可以满足产品服务平台的数据支撑要求,为分布式数据、云计算等技术的推广应用提供参考.