Cybernetic and Statistical Concepts of Entropy and Correlation Relating to Systems Analysis and Synthesis (Partitions of Discrete Information Spaces)

摘要

This report deals with an information theoretical model devised to yield the information content of a system by partitioning the system (parameters) into mutually exclusive sets (subsystems). The information content conveyed by the subsystems is then correlated with the information conveyed by the system as a whole (the latter is tantamount to observing the system parameters simultaneously). The cybernetic and statistical concepts of entropy and correlation have a wide application to many systems problems that are amenable to probabilistic models. These concepts can be fruitfully applied to assess and evaluate the degree of organization and disorganization of systems. The following related topics are also considered in some detail: extension of Shannon's fundamental inequality; the bearing of entropy and correlation on the analysis and synthesis of systems; consistency of information spaces and their underlying probability spaces; development of information correlators to measure the degree of independence and mutual dependence of the parameters (subsystem) under investigation; the extent to which system parameters can be grouped into disjoint sets (subsystems); partitioning procedures which are often dictated by considerations of time, cost, and effort. (Author)