Mining Information from Heterogeneous Sources: A Topic Modeling Approach

摘要

In recent years, the phenomenal growth and popularity of social media, news and discussion websites has led to a vast number of information sources available online. These sources generate massive amounts of real-time content on a daily basis making it increasingly difficult to glean true and useful information from them. Automatically categorizing and compressing important contextual information from these sources is crucial for tasks such as web document classification and summarization. Therefore, in this paper, we propose a novel topic modeling framework- Probabilistic Source LDA which is designed to handle heterogeneous sources. Probabilistic Source LDA can compute latent topics for each source, maintain topic-topic correspondence between sources and yet retain the distinct identity of each individual source. Therefore, it helps to mine and organize correlated information from many different sources. At the same time, it aids in automatically reducing noise and redundancy in the information gathered. Using real data on the US elections 2012, we demonstrate that our Probabilistic Source LDA method can extract highly relevant latent topics while maintaining topic-topic congruence between different sources.