AXIOMATIC DERIVATION OF THE MUTUAL INFORMATION PRINCIPLE AS A METHOD OF INDUCTIVE INFERENCE
Abstract
The Mutual Information Princip le (MIP) has already been used in various areas, as a generalization of the Maximum Entropy Principle (MEP), in the very common situation where our measurements of a random variable contain errors having some known average value. An axiomatic derivation of the MIP is given below, in order to place it in a rigorous mathematical framework with the least possible intuitive arguments. The procedure followed is similar to the one proposed by Shore and Johnson for the Minimum Cross‐entropy Principle, and some relationships between the two methods of inductive inference are pointed out.
Citation
AVGERIS, T.G. (1983), "AXIOMATIC DERIVATION OF THE MUTUAL INFORMATION PRINCIPLE AS A METHOD OF INDUCTIVE INFERENCE", Kybernetes, Vol. 12 No. 2, pp. 107-113. https://doi.org/10.1108/eb005646
Publisher
:MCB UP Ltd
Copyright © 1983, MCB UP Limited