Diversifying trending topic discovery via Semidefinite Programming

摘要

Discovering trending topics from the Web has attracted much attention in recent years, due to users' increasing need for time-sensitive information. A large body of the existing research focuses on detecting trends by examining search traffic fluctuations, and the queries with large traffic increments would be detected as trends. The weakness of this approach is that the trends are dominated by popular fields such as celebrities, as those related queries have large search traffic. Consequently, other topics such as travel and shopping are rarely regarded as trends. In this paper, we present a scalable diversified trending topic discovery system with a MapReduce implementation. The trending topics are discovered based on three criteria: diversity, representativeness, and popularity. We explicitly model these three factors in our objective function and propose an efficient Semidefinite Programming algorithm to solve the corresponding optimization problem. To the best of our knowledge, no prior work in the literature tackles trending topic diversification. We conduct a comprehensive set of experiments with case studies to demonstrate the effectiveness of our approach. The proposed system has been successfully tested in the real-world operational environment, yielding significant improvement in traffic over the existing production system.