Email document parsing method and apparatus

摘要

A preferred example of the process flow of the inventive method (1) is depicted in FIG. 1). The first step (2) of the method (1) is to import an email document (3) to be parsed. In the preprocessing step (10) the email (3) is processed to determine the presence of any header text (5) (excluding any header text that may be within the embedded reply chain) or attachments 4, including attached email documents, if any. Once the header text (5), attachments (4) or other forwarded materials have been identified in the preprocessing step (10), these components of the email (3) are categorized by the computer (51) as non-author composed text. Next the process flow of the parsing computer (51) moves to the step of normalization (11). This entails processing the email document (3) to ascertain whether it is in a preferred format and, if the email document (3) is not in the preferred format, converting at least some of the information within the email document to the preferred format. The parsing computer (51) now progresses through several analysis steps, referred to as the segmentation step (12), the linguistic analysis step (13) and the punctuation analysis step (14). The results of these analysis steps (12) to (14) are recorded in suitable memory or storage means accessible to the CPU of the parsing computer (51). In the segmentation step (12) the text of email (3) is split into paragraphs, and the paragraphs are split into sentences. The linguistic analysis step (13) includes identification of predefined words and phrases of various types. In the punctuation analysis step (14) the parsing computer (51) analyses the text at the character level so as to check for use of sentence punctuation marks and other predefined characters. At the completion of the analysis steps (12) to (14), the process flow proceeds to step (15), in which the analysed email document, including any annotations that have been inserted, is saved into the memory of the computing apparatus, along with any extraneous results of the analysis. Next a number of features are defined at step (18). Typically, a feature is a descriptive statistic calculated from either or both of the raw text and the annotations. At step (19) the features extracted at step (18) are converted into data structures associated with segments of the text. At step (20) the machine learning system receives the data structures and associated lines of text as input and is responsive to that input so as to categorise each line of text as broadly falling into one of two categories: author composed text or non-author composed text.