Maximum Entropy Modelling of Mixture Distributions

Abstract

Shannon’s entropy is usually defined separately for discrete, and for (absolutely) continuous random variables. However, many random variables encountered in practice have mixed (discrete‐continuous) distributions. A simple expression for the entropy of random variables with mixed (discrete‐continuous) distributions is derived in terms of the usual entropy definitions. In addition, the maximum entropy problem in the general setting of mixture distributions is discussed.

Citation

Politis, D.N. (1994), "Maximum Entropy Modelling of Mixture Distributions", Kybernetes, Vol. 23 No. 1, pp. 49-54. https://doi.org/10.1108/03684929410050577