Inserting Punctuation to ASR Output in a Real-Time Production Environment

摘要

The output of a speech recognition system is a continuous stream of words that has to be post-processed in various ways, out of which punctuation insertion is an essential step. Punctuated text is far more comprehensible to the reader, can be used for subtitling, and is necessary for further NLP processing, such as machine translation. In this article, we describe how state-of-the-art results in the field of punctuation restoration can be utilized in a production-ready business environment in the Czech language. A recurrent neural network based on long short-term memory is employed, making use of various features: textual based on pre-trained word embeddings, prosodic (mainly temporal), morphological, noise information, and speaker diarization. All the features except morphological tags were found to improve our baseline system. As we work in a real-time setup, it is not possible to employ information from the future of the word stream, yet we achieve significant improvements using LSTM. The usage of RNN also allows the model to learn longer dependencies than any n-gram-based language model can, which we find essential for the insertion of question marks. The deployment of an RNN-based model thus leads to a relative 22.6% decrease in punctuation errors and improvement in all metrics but one.