Learning to Forget: Continual Prediction with LSTM

摘要

Long short-term memory (LSTM; Hochreiter & Schmidhuber, 1997) can solve numerous tasks not solvable by previous learning algorithms for recurrent neural networks (RNNs). We identify a weakness of LSTM net- works processing continual input streams that are not a priori segmented into subsequences with explicitly marked ends at which the network's in- ternal state could be reset.