Extracting and denoising concept mentions using distributed representations of concepts

摘要

A method and apparatus are provided for automatically analyzing candidate concepts extracted from a first source text against a reference concept set comprising a plurality of concepts by obtaining a vector representation for each of the concepts in the first concept set and the reference concept set and performing a natural language processing (NLP) analysis comparison of the candidate concepts to the reference concept set to determine a similarity measure corresponding to each candidate concept and validating one or more of the candidate concepts based on the similarity measure for each candidate concept meeting a minimum similarity threshold requirement.