MapReduce functions to remote sensing distributed data processing-Global vegetation drought monitoring as example

摘要

Global change models for different applications are developed, according to the principle of remote sensing technology. Data for these models are generally remote sensing image, which is multiplatform, multidimentional, multiband, and multisource. Moreover, such data may be in different parts of the world and perhaps up to terabyte or petabyte level. Therefore, a data-intensive computing problem in the global change has emerged. Distributed computing infrastructures are suitable to store large-scale datalike satellite images that have to be written only once and read frequently. The emergence of the cloud computing technology brings new information architecture, and global change models implemented in the cloud platform provide users with stable, effective, on-demand cloud computing services. In this paper, the experiment is carried out on the cloud framework based on open cloud computing platform-Hadoop. In addition, on this framework, it achieves a prototype example for monitoring global vegetation drought conditions. Oriented to a variety of remote sensing data, we propose an abstract data format to achieve the unified description of remote sensing data. The data abstraction is to discretize the multidimensional remote sensing data for easy-distributed storage and computation. Using MapReduce paradigm, the complexity of remote sensing algorithms is resolved. Experimental results show that based on the parallel programming method, good scalability changing with the amount of processed data in the Hadoop distributed environment.