Building Topic Models in a Federated Digital Library Through Selective Document Exclusion

摘要

Building topic models in federated digital collectionspresents numerous challenges due to metadatainconsistencies. The quality of topical metadata is difficultto ascertain and is interspersed with often irrelevantadministrative metadata. In this study, we propose a way toimprove topic modeling in large collections by identifyingdocuments that convey only weak topical information.These documents are ignored when training topic models.Their topical associations are instead inferred modeltraining. A method is outlined for identifying weaklytopical documents by defining runs of similar documents ina collection. In preliminary evaluation using a corpus fromthe Institute of Museum and Library Services DigitalCollections and Content aggregation, results show anincrease in coherence among words in topics. In showingthis, we demonstrate that it may be beneficial to inducetopic models using less, higher-quality data.