Graph based knowledge discovery using MapReduce and SUBDUE algorithm

摘要

Knowledge Discovery is the process of extracting useful and hidden information. Extracting knowledge from data represented in the form of graphs is emerging in this new generation. Graphs are used to model and solve many real world problems. In this work, we aim to show how skills data from resumes is modelled into a variant of graph data structure called conceptual graph using MapReduce programming model. Resumes are taken as data source because they are the ones containing skill-sets of candidates. Initial storage and pre-processing is done in a big data framework using Hadoop Distributed File System (HDFS) and MapReduce. SUB Structure Discovery Using Examples (SUBDUE), a popular graph mining algorithm is used for retrieving common skill-sets. The results obtained from real-world dataset of resumes clearly demonstrate the potential of graph mining algorithms in skill set analytics. Proposed approach is able to extract common skill-sets. Common skill-set extraction is useful for course curriculum designers as well as job seekers.