Entropy rate of Thai text and testing author authenticity using character combination distribution

摘要

This paper has two main goals. The first goal is to estimate the entropy rate of Thai text which is found to be roughly 2 bits/character. The second goal is to come up with methods for text authentication based on probability distribution and information theoretic quantities. Using proposed methods, we found that digital books composed by the same author give close numerical values, while those from different authors give much higher differences. Among the three techniques under consideration, we found that the entropy-based method provides the best test. Thirty Thai text sources of various styles are tested to increase reliability of the study. Additionally, the comparison of the effectiveness of proposed methods is shown here.