To read the full version of this content please select one of the options below:

kth‐order Markov chain‐based approximation of the Shannon entropy of Gaussian photon‐counting processes

Bonawentura Kochel (Department of Toxicology, Wroclaw University of Medicine, Wroclaw, Poland)

Kybernetes

ISSN: 0368-492X

Article publication date: 1 October 2004

Abstract

A method for approximation of the Shannon entropy of Gaussian photon‐counting processes with infinite history was constructed on the memory function of these processes, described by autoregressive‐integrated moving average (ARIMA) models. Most frequently, photon‐counting processes are stationary or nonstationary multidimensional Gaussian discrete‐time stochastic ones which justify the use of the ARIMA models. Starting from the memory function, a memory time‐equivalent finite autoregressive representation of a given process with infinite history, i.e. a stationary finite‐order Gaussian Markov chain, was determined, then corresponding autocorrelation matrices were calculated from the truncated memory function using the Yule‐Walker equations, and an autocorrelation‐based formula for approximation of the entropy of the process through the entropy of its stationary Markovian representation was given. An ARMA(1,1) process together with its stationary (MA(1)) or nonstationary (IMA(0,1,1)) boundary cases were considered to demonstrate opposite changes in the entropy as the memory time increases at a fixed variance of the process: the entropy was found to decrease for stationary processes and increase for nonstationary ones. It was also found on experimental examples (perturbed human neutrophils and yeast cells) that those changes can be reversed by opposite changes in the process variance. The method allows us to determine, at any desired accuracy, the Shannon entropy of time‐discrete stochastic processes, and reveals new aspects of the relationship between the process' stationarity, memory, entropy and heteroskedasticity.

Keywords

Citation

Kochel, B. (2004), "kth‐order Markov chain‐based approximation of the Shannon entropy of Gaussian photon‐counting processes", Kybernetes, Vol. 33 No. 9/10, pp. 1491-1509. https://doi.org/10.1108/03684920410546701

Publisher

:

Emerald Group Publishing Limited

Copyright © 2004, Emerald Group Publishing Limited