Detecting Quotable Sentences from Text Using Syntactic Token Augmentation and Recurrent Neural Networks

摘要

From the beginning of the literary era, 'Quotations' have been inspiring writers and avid readers. The information outburst starting right from the printing of books to digitization of such media, hints towards the innumerable occurrences of unexposed quotations. In this paper, we explore an approach to detect and extract quotable sentences from large text corpses. Analysis of quotations from the gold standard sources suggests that quotable sentences can be detected with their intrinsic properties of sentence structure and style of speech that makes the quotations meaningful, concise, and effective. However, subjectivity of this property makes this problem more convoluted than any other binary text classification problem. To resolve that, we describe a method to preserve sentence semantics during the process of filtering and classification of sentences. Our language model with Recurrent Neural Networks outperforms the other approaches to detect quotations with 84.30% accuracy, which is a significant 2.8% improvement with our own statistical model. The work also adds to the understanding of the impact of manually designed features on text classification problems.