An automatic skills standardization method based on subject expert knowledge extraction and semantic matching

摘要

The job market is rapidly changing. Artificial Intelligence and automation technologies are reshaping the career market. Everyday, new jobs appear and new skills are added to the scope of existing job profiles. At the same time, some skills that once were assumed to be "must-haves" for particular jobs are no longer requested and some jobs are even becoming obsolete. The speed of changes as well as the increasing complexity of the job market introduce a key new challenge: there is no clear definition for a particular job in terms of skills and scope and consequently, people holding the same job title cannot be assumed to be actually doing the same thing. In addition, applicants find difficult to develop career paths, as the mapping of skills to particular jobs are fuzzier than ever before. In this article, we present a novel approach to homogenize the job definition, gathering first subject matter expertise using semantic expansion techniques on collaborative wikies, applying a word embeddings supported method to mine the skills from existing job posts and finally executing a semantic matching algorithm to converge to a consistent skills mapping. In order to show how our method performs, we apply it to one of the most popular, yet heterogeneous modern jobs, thedata scientistand discuss the results obtained for the English speaking market.