System, methods, and data structure for quantitative assessment of symbolic associations in natural language

摘要

A method for quantitatively assessing associations between different terms in a natural language includes obtaining a first group of sentences, at least some of which comprise an object name, parsing each of the first group of sentences to identify a subject and a predicate, assigning a sentence type to each of the first group of sentences according to the location of the object name in the sentence, tokenizing each of the first group of sentences to produce a plurality of tokens that includes a jth token, and adding a weighting coefficient to a parameter token_j_count for each sentence that includes the jth token, wherein the weighting coefficient is dependent on the sentence type. A cumulative value of the parameter token_j_count obtained from the first group of sentences is divided by the total number of sentences in the first group to produce an internal association strength for the jth token.