A unified matrix representation for calculating entropy quantities

摘要

In this paper a unified framework for presenting various entropy quantities needed in information theory is given. This method allows an easy, elegant and compact presentation and lends itself to easy implementation using standard mathematical/engineering software. Examples illustrating the use of this technique are given together with a general MATLAB program for calculating entropies as well as the channel capacity. The technique was preferred over classical representations by undergraduate students taking a first course on digital communications.