Normal-Wishart distribution

Normal-Wishart
Notation	$(𝝁,𝜦)\sim \mathrm{N}\mathrm{W}({𝝁}_0,\lambda ,𝐖,\nu )$
Parameters	${𝝁}_0\in {ℝ}^D$ location (vector of real) $\lambda >0$ (real) $𝐖\in {ℝ}^{D\times D}$ scale matrix (pos. def.) $\nu >D-1$ (real)
Support	$𝝁\in {ℝ}^D;𝜦\in {ℝ}^{D\times D}$ covariance matrix (pos. def.)
PDF	$f(𝝁,𝜦|{𝝁}_0,\lambda ,𝐖,\nu )=𝒩(𝝁|{𝝁}_0,(\lambda 𝜦{)}^{-1})𝒲(𝜦|𝐖,\nu )$

In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).

Definition

Suppose

$𝝁|{𝝁}_0,\lambda ,𝜦\sim 𝒩({𝝁}_0,(\lambda 𝜦{)}^{-1})$

has a multivariate normal distribution with mean ${𝝁}_0$ and covariance matrix $(\lambda 𝜦{)}^{-1}$, where

$𝜦|𝐖,\nu \sim 𝒲(𝜦|𝐖,\nu )$

has a Wishart distribution. Then $(𝝁,𝜦)$ has a normal-Wishart distribution, denoted as

$(𝝁,𝜦)\sim \mathrm{N}\mathrm{W}({𝝁}_0,\lambda ,𝐖,\nu ).$

Characterization

Probability density function

$f(𝝁,𝜦|{𝝁}_0,\lambda ,𝐖,\nu )=𝒩(𝝁|{𝝁}_0,(\lambda 𝜦{)}^{-1})𝒲(𝜦|𝐖,\nu )$

Properties

Marginal distributions

By construction, the marginal distribution over $𝜦$ is a Wishart distribution, and the conditional distribution over $𝝁$ given $𝜦$ is a multivariate normal distribution. The marginal distribution over $𝝁$ is a multivariate t-distribution.

Posterior distribution of the parameters

After making $n$ observations ${𝒙}_1,\dots ,{𝒙}_n$, the posterior distribution of the parameters is

$(𝝁,𝜦)\sim \mathrm{N}\mathrm{W}({𝝁}_n,{\lambda }_n,{𝐖}_n,{\nu }_n),$

where

${\lambda }_n=\lambda +n,$

${𝝁}_n=\frac{\lambda {𝝁}_0+n\overline{𝒙}}{\lambda +n},$

${\nu }_n=\nu +n,$

${𝐖}_n^{-1}={𝐖}^{-1}+\sum _{i=1}^n({𝒙}_i-\overline{𝒙})({𝒙}_i-\overline{𝒙}{)}^T+\frac{n\lambda }{n+\lambda }(\overline{𝒙}-{𝝁}_0)(\overline{𝒙}-{𝝁}_0{)}^T.$

Generating normal-Wishart random variates

Generation of random variates is straightforward:

1. Sample $𝜦$ from a Wishart distribution with parameters $𝐖$ and $\nu$
2. Sample $𝝁$ from a multivariate normal distribution with mean ${𝝁}_0$ and variance $(\lambda 𝜦{)}^{-1}$

Related distributions

• The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision.
• The normal-gamma distribution is the one-dimensional equivalent.
• The multivariate normal distribution and Wishart distribution are the component distributions out of which this distribution is made.

References

• Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.