首页>
外国专利>
Word boundary probability estimating, probabilistic language model building, kana-kanji converting, and unknown word model building
Word boundary probability estimating, probabilistic language model building, kana-kanji converting, and unknown word model building
展开▼
机译:词边界概率估计,概率语言模型构建,假名汉字转换和未知词模型构建
展开▼
页面导航
摘要
著录项
相似文献
摘要
Calculates a word n-gram probability with high accuracy in a situation where a first corpus), which is a relatively small corpus containing manually segmented word information, and a second corpus, which is a relatively large corpus, are given as a training corpus that is storage containing vast quantities of sample sentences. Vocabulary including contextual information is expanded from words occurring in first corpus of relatively small size to words occurring in second corpus of relatively large size by using a word n-gram probability estimated from an unknown word model and the raw corpus. The first corpus (word-segmented) is used for calculating n-grams and the probability that the word boundary between two adjacent characters will be the boundary of two words (segmentation probability). The second corpus (word-unsegmented), in which probabilistic word boundaries are assigned based on information in the first corpus (word-segmented), is used for calculating a word n-grams.
展开▼