Multivariate Normal Distribution

Probability Density Function

If X = (X1, …, Xn)′ is a multivariate normal random vector, denoted X ∼ N(μ, σ) or X ∼ Nn(μ, σ), then its density is given by

where μ = (μ1, …, μn)′ = E(X) is a vector whose components are the expectations E(X1), …, E(Xn) and σ is the nonsingular variance–covariance matrix (n × n) whose diagonal terms are variances and off-diagonal terms are covariances:

Note that the covariance matrix σ is symmetric and positive definite. The (i, j)th element is given by and

An important special case of the multivariate normal distribution is the bivariate normal.

If , where and , then the bivariate density is given by

Let X = (X1, X2)′ the joint density can be rewritten in matrix notation as

Multivariate Normal Density Contours

The contour levels of fx(x), that is, the set of points in n for which fx(x) is constant, satisfy

These surfaces are n-dimensional ellipsoids centered at μ, whose axes of symmetry are given by the principal components (the eigenvectors) of σ. Specifically, the length of the ellipsoid along the ith axis is c√λi, where λi is the ith eigenvalue associated with the eigenvector ei (recall that eigenvectors ei and eigenvalues λi are solutions to σei = λiei for i = 1, …, n).

Some Basic Properties

The following list presents some important properties involving the multivariate normal distribution.

• The first two moments of a multivariate normal distribution, namely μ and σ, completely characterize the distribution. In other words, if X and Y are both multivariate normal with the same first two moments, then they are similarly distributed.

• Let X =(X1, …, Xn)′ be a multivariate normal random vector with mean μ and covariance matrix σ, and let . The linear combination is normal with mean E(Y) = α′μ and variance

  

  Also, if α′X is normal with mean α′μ and variance α′σα for all possible α, then X must be a multivariate normal random vector with mean μ and covariance matrix σ (X ∼ Nn(μ, σ)).

• More generally, let X =(X1, …, Xn)′ be a multivariate normal random vector with mean μ and covariance matrix σ, and let A ∊ m × n be a full rank matrix with m ≤ n, the set of linear combinations Y = (Y1, …, Ym)′ = AX is multivariate normally distributed with mean Aμ and covariance matrix AσA′. Also, if Y = AX + b where b is a m × 1 vector of constants, then Y is multivariate normally distributed with mean Aμ + b and covariance matrix AσA'.
• If Xi and Xj are jointly normally distributed, then they are independent if and only if Cov (Yi, Yj) = 0. Note that it is not necessarily true that uncorrelated univariate normal random variables are independent. Indeed, two random variables that are marginally normally distributed may fail to be jointly normally distributed.

• Let Z = (Z1, …, Zn)′ where Zi ∼ i.i.d.N(0, 1) (where i.i.d. = independent and identically distributed). Z is said to be standard multivariate normal, denoted Z ∼ N(0, In), and it can be shown that E[Z] = 0 and V(Z) = In, where In, denotes the unit matrix of order n. The joint density of vector Z is given

  ...