The Generalized Cross Entropy Method, with Applications to
Probability Density Estimation
Z.I. Botev and D.P. Kroese
The fundamental problem in statistical learning is to determine the
simplest model that explains a given set of empirical data and which
uses as few assumptions as possible. Many classical approaches to the
statistical learning problem impose extra (artificial) assumptions in
order to provide a unique and well-behaved solution of the problem.
In this paper we describe a simple and general framework for
statistical modelling which unifies many recent advances in Monte
Carlo statistical methods. The approach combines
information-theoretic ideas based on generalized cross-entropy
principles with constrained functional optimization fundamentals. The
effectiveness of the approach is demonstrated through an application
to density estimation and data modeling.
Keywords: Cross entropy, information theory, Monte Carlo simulation,
statistical modeling, kernel smoothing, functional optimization,
bandwidth selection, calculus of variations}