Visual summarization of online news through topics using Latent Dirichlet Allocation

摘要

One way to approach text summarization challenges is to assume documents are built around different sets of topics. In this paper, a new alternative for constructing a graph of words and topics is introduced. By using Latent Dirichlet Allocation (LDA) and Gibbs sampling, it is possible to estimate the probability of a word belonging to a set of topics. This method does not require a fixed number of topics, and as an unsupervised method, can calculate an optimal number of topics in a given text corpus. For this study, a network graph was constructed with nodes as words and arcs representing the distance between them, color-coded topics, and interactive features.