Multivariate normal distribution

In probability theory and statistics, the multivariate normal distribution, multivariate Gaussian distribution, or joint normal distribution is a generalization of the one-dimensional (univariate) normal distribution to higher dimensions. One definition is that a random vector is said to be k-variate normally distributed if every linear combination of its k components has a univariate normal distribution. Its importance derives mainly from the multivariate central limit theorem. The multivariate normal distribution is often used to describe, at least approximately, any set of (possibly) correlated real-valued random variables, each of which clusters around a mean value.

Definitions

Notation and parametrization

The multivariate normal distribution of a k-dimensional random vector

X

=

(

X

1

,

…

,

X

k

)

T

{\displaystyle \mathbf {X} =(X_{1},\ldots ,X_{k})^{\mathrm {T} }}

can be written in the following notation:

X

∼

N

(

μ

,

Σ

)

,

{\displaystyle \mathbf {X} \ \sim \ {\mathcal {N}}({\boldsymbol {\mu }},\,{\boldsymbol {\Sigma }}),}

or to make it explicitly known that

X

{\displaystyle \mathbf {X} }

is k-dimensional,

X

∼

N

k

(

μ

,

Σ

)

,

{\displaystyle \mathbf {X} \ \sim \ {\mathcal {N}}_{k}({\boldsymbol {\mu }},\,{\boldsymbol {\Sigma }}),}

with k-dimensional mean vector

μ

=

E

⁡

[

X

]

=

(

E

⁡

[

X

1

]

,

E

⁡

[

X

2

]

,

…

,

E

⁡

[

X

k

]

)

T

,

{\displaystyle {\boldsymbol {\mu }}=\operatorname {E} [\mathbf {X} ]=(\operatorname {E} [X_{1}],\operatorname {E} [X_{2}],\ldots ,\operatorname {E} [X_{k}])^{\mathrm {T} },}

and

k

×

k

{\displaystyle k\times k}

covariance matrix

Σ

i

,

j

=

E

⁡

[

(

X

i

−

μ

i

)

(

X

j

−

μ

j

)

]

=

Cov

⁡

[

X

i

,

X

j

]

{\displaystyle \Sigma _{i,j}=\operatorname {E} [(X_{i}-\mu _{i})(X_{j}-\mu _{j})]=\operatorname {Cov} [X_{i},X_{j}]}

such that

1

≤

i

≤

k

{\displaystyle 1\leq i\leq k}

and

1

≤

j

≤

k

{\displaystyle 1\leq j\leq k}

. The inverse of the covariance matrix is called the precision matrix, denoted by

Q

=

Σ

−

1

{\displaystyle {\boldsymbol {Q}}={\boldsymbol {\Sigma }}^{-1}}

.

Standard normal random vector

A real random vector

X

=

(

X

1

,

…

,

X

k

)

T

{\displaystyle \mathbf {X} =(X_{1},\ldots ,X_{k})^{\mathrm {T} }}

is called a standard normal random vector if all of its components

X

i

{\displaystyle X_{i}}

are independent and each is a zero-mean unit-variance normally distributed random variable, i.e. if

X

i

∼

N

(

0

,

1

)

{\displaystyle X_{i}\sim \ {\mathcal {N}}(0,1)}

for all

i

=

1

…

k

{\displaystyle i=1\ldots k}

.

Centered normal random vector

A real random vector

X

=

(

X

1

,

…

,

X

k

)

T

{\displaystyle \mathbf {X} =(X_{1},\ldots ,X_{k})^{\mathrm {T} }}

is called a centered normal random vector if there exists a

k

×

ℓ

{\displaystyle k\times \ell }

matrix

A

{\displaystyle {\boldsymbol {A}}}

such that

A

Z

{\displaystyle {\boldsymbol {A}}\mathbf {Z} }

has the same distribution as

X

{\displaystyle \mathbf {X} }

where

Z

{\displaystyle \mathbf {Z} }

is a standard normal random vector with

ℓ

{\displaystyle \ell }

components.

Normal random vector

A real random vector

X

=

(

X

1

,

…

,

X

k

)

T

{\displaystyle \mathbf {X} =(X_{1},\ldots ,X_{k})^{\mathrm {T} }}

is called a normal random vector if there exists a random

ℓ

{\displaystyle \ell }

-vector

Z

{\displaystyle \mathbf {Z} }

, which is a standard normal random vector, a

k

{\displaystyle k}

-vector

μ

{\displaystyle {\boldsymbol {\mu }}}

, and a

k

×

ℓ

{\displaystyle k\times \ell }

matrix

A

{\displaystyle {\boldsymbol {A}}}

, such that

X

=

A

Z

+

μ

{\displaystyle \mathbf {X} ={\boldsymbol {A}}\mathbf {Z} +{\boldsymbol {\mu }}}

.