Communication and coding vis-a-vis information theory vis-a-vis probability and statistics

摘要

The motivation behind this paper is threefold, as follows. Firstly, it attempts to make a comprehensive exposition of the important role of elementary 'information theory', relevant to 'communication coding'. Secondly, the paper tries to illustrate comprehensively and convincingly how 'information theory' is a bridge between two different disciplines, namely 'communication and coding' and 'probability and statistics'. Thirdly, and most importantly, the paper leads to the conclusion that while we are trying to solve a problem in a particular area, e.g. 'electrical engineering', we ought to have a 'research group' with scientists from all relevant areas, for instance economics, probability and statistics, geology etc. The point is: by marrying expertise in seemingly diverse fields of knowledge we would vastly improve our chances of arriving at a gainful solution to the problem at hand, which would be practically optimal.