Bernoulli Distribution

Hao Helen Zhang

doi:10.4135/9781506326139

Entry
Reader's guide
Entries A-Z
Subject index

Return to Entries

Bernoulli Distribution

By: Hao Helen Zhang
In:The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation
Chapter DOI:https://doi.org/10.4135/9781506326139.n81
Subject:Education

Request Permissions

Show page numbers Hide page numbers

The Bernoulli distribution is the range of probabilities for two possible outcomes. It is a central statistical concept. This entry describes the Bernoulli distribution and Bernoulli random variables and explains the relationship between the Bernoulli distribution and the binomial distribution.

Suppose a random experiment has two possible outcomes, either success or failure, where the probability of success is p and probability of failure is q = 1 − p. Such an experiment is called a Bernoulli experiment or Bernoulli trial. For a Bernoulli experiment, define a real-valued random variable X which takes two values as: X = 1 if success and X = 0 if failure. Such a random variable X is called a Bernoulli random variable. The probability distribution of X is given by Pr(X = 1) = p and Pr(X = 0) = 1 − p. This distribution is called a Bernoulli distribution, denoted by Bernoulli(p). It is named after Jacob Bernoulli, a Swiss mathematician of the 17th century.

The term success here means the outcome meets some special condition, and it is not based on a moral judgment. The following are some examples of Bernoulli random variables.

Toss a coin once. Two possible outcomes are “heads” and “tails.” Suppose heads happens with probability p, while tails happens with probability 1 − p. Let X be a random variable such that X = 1 if the outcome is heads, and X = 0 if the outcome is tails. Then X is a Bernoulli random variable and its distribution is Bernoulli(p). When a fair coin is tossed, we have p = q = 0.5.
Roll a die once. Let X be a random variable which takes two values: X = 1 if the Number 3 occurs, and X = 0 otherwise. Then X is a Bernoulli random variable. If the die is balanced, then the probability distribution of X is Bernoulli(1/6).
In clinical trials, let X represent a patient’s status after a certain treatment as, X = 1 if a patient survives, and X = 0 otherwise. Then X is a Bernoulli random variable.

Statistical Properties

Assume a random variable X follows a Bernoulli(p) distribution. Its probability mass function is given by P(X = 1) = p and P(X = 0) = 1 − p. Equivalently, it is expressed as

$\begin{matrix} P (X = x) \\ = p^x (1 - p)^{1 - x}, \ quad x = 0, 1 . \end{matrix}$

Its expectation is E (X) = p, variance is Var (X) = p(1 − p), and skewness is \frac{1–2p}{\sqrt{pq}}. The moment generating function is

$M_X (t) = E (e^{X t}) = 1 - p + p e^t .$

The characteristic function is

$\ phi_X (t) = 1 - p + p e^{i t} .$

The family of Bernoulli distributions \{Bernoulli (p), 0\le p\le 1 \} is an exponential family.

Estimation of p

Suppose we take a random sample of size n, X_1, \cdots, X_n, from Bernoulli(p). Then an estimator for p is given by the sample mean:

$\begin{matrix} \ bar {X} = \ frac {1} {n} \ sum_{i = 1} \land n X_i, i .e ., \\ \ hat {p} = \ bar {X} . \end{matrix}$

Because the sample mean is unbiased for the population mean p, we have E(\hat{p}) = p. By the law of large numbers, \hat{p} is also a consistent estimator for p. In other words, the sample proportion of successes from n experiments can consistently estimate the success probability p. The estimator \hat{p} is also the maximum likelihood estimator.