Data Acquisition for, and Analysis of, Word Frequencies in the English Language

摘要

A process of data acquisition and analysis of text files for the purpose of following the insurgence and growth of words in the English language is reported. Pitfalls that were encountered and the solutions that were found to these problems are also discussed. The question of whether the Zipf-Mandelbrot law applies to the distribution of words in a language over time is examined. Several other results and future research ideas are discussed.