Using MapReduce for scalable and distributed processing of scientific XML data.

摘要

A seamless and intuitive search capability for the vast amount of datasets generated by scientific experiments is critical to ensure effective use of such data by domain specific scientists. There exists a critical need for an easy-to-use and scalable framework, specialized for scientific data. The work in this thesis makes use of the MapReduce model in XML metadata indexing for scientific datasets We present an indexing structure that scales well for large-scale MapReduce processing. We present performance results using two MapReduce implementations, Apache Hadoop and LEMO-MR, to emphasize the flexibility and adaptability of our framework in different MapReduce environments.