Predictive Entropy of Hindi and Tamil

摘要

Any sequence of written symbols is called a message for convenience. The information associated with any one symbol depends on the relative frequency of occurrence and the average over the various symbols of a language is termed entropy. A more accurate description should take into account a larger number of previous symbols or blocks of previous symbols and their effect on the succeeding symbol. The increasing dependency of the symbols on those preceding them reduces the entropy per symbol.