A METHOD FOR DIACRITISATION OF TEXTS WRITTEN IN LATIN- OR CYRILLIC-DERIVED ALPHABETS

摘要

The presented invention is related to the method for the recovery of diacritical marks in texts written in any of the languages using Latin- or Cyrillic-derived alphabets with diacritical marks. The embodiment of the invention presented in this document uses multiple information sources (topical information and information on semantic proximity) in the task of diacritisation, which is recognised and treated as a classification task. The invention relies on classification based on topical information provided by text categorisation, the information on semantic proximity of particular words in the text, as well as morphological information. At word level the classification task is limited to the calculation of the semantic score of each particular word interpretation. The actual recovery of the diacritical marks is carried out only at the sentence level (or possibly some higher level such as paragraph or the entire text), with the assumption that users, when adapting a text to a non-diacritised setting, consistently use one of the existing conventions, rather than switching between different conventions.