DIMENSION OPTIMIZATION IN SINGULAR VALUE DECOMPOSITION-BASED TOPIC MODELS

摘要

Techniques are described for analyzing text. Embodiments tokenize a plurality of documents into a plurality of sets of terms. An average top dimension weight corresponding to the plurality of documents is calculated based on performing singular value decomposition (SVD) factorization for a plurality of dimension counts. An average inverse top dimension top term ranking for the plurality of documents is further calculated based on the SVD factorization for the plurality of dimension counts. A number of dimensions is determined based on the average top dimension weight and the average inverse top dimension top term ranking. A topic model is built for the plurality of documents based on the number of dimensions. The topic model is adapted to identify patterns of terms that correspond to semantic topics in at least the plurality of documents.