A General Framework for Fast Textual Document Preprocessing

摘要

Due to the large amount of textual documents available from heterogeneous sources. Knowledge Discovery in Text (KDT) has emerged as a challenging application. A critical phase of a KDT Process is the Pre-processing phase. In this work we present a framework for textual document Pre-Processing aimed at the extraction of linguistic, syntactic and structural relevant features. Further, we describe OPM, a software developed by Exeura s.r.l., that efficiently implements the framework based on a parallel processing technique.