Most languages have no established writ ing system and minimal written records. However, textual data is essential for nat ural language processing, and particularly important for training language models to support speech recognition. Even in cases where text data is missing, there are some languages for which bilingual lexicons are available, since creating lexicons is a fun damental task of documentary linguistics. We investigate the use of such lexicons to improve language models when tex tual training data is limited to as few as a thousand sentences. The method involves learning cross-lingual word embeddings as a preliminary step in training monolin gual language models. Results across a number of languages show that language models are improved by this pre-training. Application to Yongning Na, a threatened language, highlights challenges in deploy ing the approach in real low-resource en vironments.
展开▼
