Improving NER in Social Media via Entity Type-Compatible Unknown Word Substitution

摘要

Named entity recognition (NER) is a fundamental task for information extraction (IE), and current state-of-the-art methods try to address this issue and achieve high performance on clean text (e.g., newswire genres). However, most of these algorithms do not generalize well when they transit to the noisy domain such as social media. To alleviate the noisy expression in social media data, we present a novel word substitution strategy based on constructing an entity type-compatible (ETC) semantic space. We substitute unknown words with the ETC words found by deep metric learning (DML) and nearest neighbor (NN) search. Comprehensive experiments show that the proposed framework achieves state-of-the-art performance on the W-NUT2017 dataset and the novel strategy brings good generality to multiple NER tools and previous works.