海量遥感数据的高可靠并行处理方法

摘要

With the development of cloud computing technology , a quick solution to the application of remote sensing data is provided for earth observation ,however ,in the process of scheduling ,because of the reliability the cloud computing cluster can not seriously affect the efficiency of data processing . Scheduling method based on the principal deputy version makes remote sensing data abstracted as the DAG task .In orden to meet the system schedulability under the premise of reliability ,at the cost of cloud computing virtual nodes and communication links as the goal ,the earliest start time of major and minor version of the task is analyzed ,to avoid the task assigned to the node failure rate higher on execution . Experiments show that under the premise of ensuring reliable scheduling ,it can improve not only the degree of fairness of DAG scheduling ,but effectively shorten the average Makespan scheduling .%云计算技术不断发展，为对地观测的遥感数据的应用提供快速的解决方法。然而在调度过程中，云计算集群的不可靠性会严重影响数据处理效率。为此，研究一种基于主副版本的调度方法，将遥感数据抽象为DAG任务，在满足系统可调度性前提下，以云计算虚拟节点及通信链路的可靠性代价为目标，分析主版本任务的开始时间条件，以此限定副版本任务的开始时间，避免节点调度过程中由于发生故障而导致任务无法正常执行的问题。仿真实验表明，文中方法在保证遥感任务可靠调度的前提下，任务的执行周期（Makespan）能达到最佳。