Estimation of Probability Density Function, Differential Entropy and Other Relative Quantities for Data Streams with Concept Drift

摘要

In this paper estimators of nonstationary probability density function are proposed. Additionally, applying the trapezoidal method of numerical integration, the estimators of two information-theoretic measures are presented: the differential entropy and the Renyi's quadratic differential entropy. Finally, using an analogous methodology, estimators of the Cauchy-Schwarz divergence and the probability density function divergence are proposed, which are used to measure the differences between two probability density functions. All estimators are proposed in two variants: one with the sliding window and one with the forgetting factor. Performance of all the estimators is verified using numerical simulations.