Scalable Distributed RDFS Reasoning Using MapReduce and Bigtable

摘要

The reasoning over massive RDF data has a great advancement in last few years. Many methods have been proposed in past several years, including the method with MapReduce. But the current MapReduce approach contains four reasoning steps and avoids data duplication by special data processing and partitioning. Our work is to propose an algorithm for RDFS reasoning with MapReduce and Bigtable. Through the optimization of RDFS rules' applying sequence in map and reduce methods, our approach can complete RDFS closure reasoning without special data preprocessing and partitioning in only one MapReduce reasoning step. We have implemented our method on Hadoop and HBase with 3 nodes. We compute the RDFS closure over different datasets and our practice enjoys faster speed and better speedup, calculating RDFS closure of 260 million triples in 50 minutes, about 15 minutes faster than WebPIE.