In this article, we will explore the topic of Continuous Bernoulli distribution from different perspectives and approaches. Continuous Bernoulli distribution is a very important issue today, as it affects a wide spectrum of people and contexts. Over the next few lines, we will analyze the importance of Continuous Bernoulli distribution, its impact on society and some possible solutions or approaches to address this issue effectively. Through reflection and critical analysis, we will seek to better understand Continuous Bernoulli distribution and its relevance in everyday life.
Probability density function | |||
Notation | |||
---|---|---|---|
Parameters | |||
Support | |||
where | |||
CDF | |||
Mean | |||
Variance |
In probability theory, statistics, and machine learning, the continuous Bernoulli distribution is a family of continuous probability distributions parameterized by a single shape parameter , defined on the unit interval , by:
The continuous Bernoulli distribution arises in deep learning and computer vision, specifically in the context of variational autoencoders, for modeling the pixel intensities of natural images. As such, it defines a proper probabilistic counterpart for the commonly used binary cross entropy loss, which is often applied to continuous, -valued data. This practice amounts to ignoring the normalizing constant of the continuous Bernoulli distribution, since the binary cross entropy loss only defines a true log-likelihood for discrete, -valued data.
The continuous Bernoulli also defines an exponential family of distributions. Writing for the natural parameter, the density can be rewritten in canonical form: .
The continuous Bernoulli can be thought of as a continuous relaxation of the Bernoulli distribution, which is defined on the discrete set by the probability mass function:
where is a scalar parameter between 0 and 1. Applying this same functional form on the continuous interval results in the continuous Bernoulli probability density function, up to a normalizing constant.
The Beta distribution has the density function:
which can be re-written as:
where are positive scalar parameters, and represents an arbitrary point inside the 1-simplex, . Switching the role of the parameter and the argument in this density function, we obtain:
This family is only identifiable up to the linear constraint , whence we obtain:
corresponding exactly to the continuous Bernoulli density.
An exponential distribution restricted to the unit interval is equivalent to a continuous Bernoulli distribution with appropriate[which?] parameter.
The multivariate generalization of the continuous Bernoulli is called the continuous-categorical.