Towards Trusted Services: Result Verification Schemes for MapReduce

摘要

Recent development in Internet-scale data applications and services, combined with the proliferation of cloud computing, has created a new computing model for data intensive computing best characterized by the MapReduce paradigm. The MapReduce computing paradigm, pioneered by Google in its Internet search application, is an architectural and programming model for efficiently processing massive amount of raw unstructured data. With the availability of the open source Hadoop tools, applications built based on the MapReduce computing model are rapidly growing. In this work, we focus on a unique security concern on the MapReduce architecture. Given the potential security risks from lazy or malicious servers involved in a MapReduce task, we design efficient and innovative mechanisms for detecting cheating services under the MapReduce environment based on watermark injection and random sampling methods. The new detection schemes are expected to significantly reduce the cost of verification overhead. Finally, extensive analytical and experimental evaluation confirms the effectiveness of our schemes in MapReduce result verification.