This paper integrates the current Google's most powerful NLP transfer learning model BERT with the traditional state-of-the-art BiLSTM-CRF model to solve the problem of named entity recognition. A bi-directional LSTM model can consider an effectively infinite amount of context on both sides of a word and eliminates the problem of limited context that applies to any feed-forward models. Google's model applied a feedforward neural network, causing its performance to weaken. We seek to solve these issues by proposing a more powerful neural network model named BT-BiLSTM. The new neural network model has obtained F1 scores on three Chinese datasets exceeds the previous BiLSTM-CRF model, especially on the value of recall. It shows the great value of the combination of large scale none-labelled data pre-trained language model with named entity recognition, which inspire new ideas on other future work.
展开▼
