Physical Interpretations of Information Measures

摘要

In this section, we make use of the elementary background that was established in the previous section, and we draw certain analogies between statistical physics and information theory, most notably, with Shannon theory. These analogies set the stage for physical interpretations of information measures and their properties, as well as fundamental coding theorems. As we shall see, these physical interpretations often prove useful in gaining new insights and perspectives, which may be beneficial for deriving new analysis tools.