ZIPF'S LAW ACROSS LANGUAGES OF THE WORLD: TOWARDS A QUANTITATIVE MEASURE OF LEXICAL DIVERSITY

摘要

Zipf's law (Zipf, 1949) has been argued to differ systematically between texts and languages (Popescu et al., 2009), to change throughout time (Bentz, Kiela, Hill, & Buttery, forthcoming) and to reflect language complexity (Baixeries, Elvev?g, & Ferrer-i-Cancho, 2013). Furthermore, we argue that Zipf's law can be used as cross-linguistic, quantitative measure of the lexical diversity of languages (in parallel to biodiversity indices). In this context, lexical diversity is defined as the breadth of word forms used to encode a constant information content. A quantitative measure of lexical diversity can help to a) model factors driving the evolution of lexical encoding strategies on historical and evolutionary timescales, b) to determine the range of lexical diversities in natural languages and distinguish them from other symbolic encoding systems and animal communication.