We introduce a novel neural easy-first decoder that learns to solve sequence tagging tasks in a flexible order In contrast to previous easy-first decoders, our models are end-to-end differentiable. The decoder iteratively updates a "sketch" of the predictions over the sequence At its core is an attention mechanism that controls which parts of the input are strategically the best to process next We present a new constrained softmax transformation that ensures the same cumulative attention to every word, and show how to efficiently evaluate and backpropagate over it. Our models compare favourably to BILSTM taggers on three sequence tagging tasks.
展开▼
