多源定量遥感产品并行处理系统设计及实现

摘要

The remote sensing data has many characteristics such as unstructured,formatted,huge amount of a single image and less amount of data records,the amount of most remote sensing data isn't significantly reduced after processing,and large-scale remote sensing data processing is needed.Based on the existing hardware resources,this paper designed and implemen-ted an integrated parallel system for multi-source quantitative remote sensing production in centralized cluster.In order to solve the problem of the parallel about the serial algorithm designed and developed by the remote sensing scientists and the require-ment of large scale data management,the system designed a parallel processing system based on task parallelism and an appli-cation-aware's parallel file system.The system integrated more than 30 kinds of preprocessing algorithms for multiple raw data and 50 kinds of production algorithms for multi-source quantitative remote sensing products developed by remote sensing science workers,to realize the process of workflow production,and solved the problem of large-scale production of the multi-source remote sensing quantitative products on demand.And the production process was carried out.It is confirmed that the platform has the reliability and validity in the management and processing of large scale remote sensing data.%针对遥感数据非结构化、有格式、单景数据量大、记录总量小、大部分处理过程后数据量不显著减少的特点和大规模遥感数据处理的迫切需求，基于已有硬件资源，设计和实现了集中式集群计算的多源定量遥感产品生产系统。系统为解决遥感科学工作者设计开发的算法进行并行计算的问题以及大规模数据管理的需求，设计了基于任务并行的处理系统和面向应用的并行文件系统。系统集成了遥感科学工作者开发的30多种原始数据的预处理算法和50多种多源定量遥感产品生产算法，解决了多源定量遥感产品规模化流程化按需生产的难题，并进行了产品生产，证明了系统在管理和处理大规模遥感数据时的可靠性和有效性。