Normal-Wishart distribution

Normal-Wishart

Notation	$(\boldsymbol\mu,\boldsymbol\Lambda) \sim \mathrm{NW}(\boldsymbol\mu_0,\lambda,\mathbf{W},\nu)$
Parameters	$\boldsymbol\mu_0\in\mathbb{R}^D\,$ location (vector of real) $\lambda > 0\,$ (real) $\mathbf{W} \in\mathbb{R}^{D\times D}$ scale matrix (pos. def.) $\nu > D-1\,$ (real)
Support	$\boldsymbol\mu\in\mathbb{R}^D ; \boldsymbol\Lambda \in\mathbb{R}^{D\times D}$ covariance matrix (pos. def.)
PDF	$f(\boldsymbol\mu,\boldsymbol\Lambda|\boldsymbol\mu_0,\lambda,\mathbf{W},\nu) = \mathcal{N}(\boldsymbol\mu|\boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^{-1})\ \mathcal{W}(\boldsymbol\Lambda|\mathbf{W},\nu)$

In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).

Definition

Suppose

$\boldsymbol\mu|\boldsymbol\mu_0,\lambda,\boldsymbol\Lambda \sim \mathcal{N}(\boldsymbol\mu|\boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^{-1})$

has a multivariate normal distribution with mean $\boldsymbol\mu_0$ and covariance matrix $(\lambda\boldsymbol\Lambda)^{-1}$, where

$\boldsymbol\Lambda|\mathbf{W},\nu \sim \mathcal{W}(\boldsymbol\Lambda|\mathbf{W},\nu)$

has a Wishart distribution. Then $(\boldsymbol\mu,\boldsymbol\Lambda)$ has a normal-Wishart distribution, denoted as

$(\boldsymbol\mu,\boldsymbol\Lambda) \sim \mathrm{NW}(\boldsymbol\mu_0,\lambda,\mathbf{W},\nu) .$

Characterization

Probability density function

$f(\boldsymbol\mu,\boldsymbol\Lambda|\boldsymbol\mu_0,\lambda,\mathbf{W},\nu) = \mathcal{N}(\boldsymbol\mu|\boldsymbol\mu_0,(\lambda\boldsymbol\Lambda)^{-1})\ \mathcal{W}(\boldsymbol\Lambda|\mathbf{W},\nu)$

Properties

Marginal distributions

By construction, the marginal distribution over $\boldsymbol\Lambda$ is a Wishart distribution, and the conditional distribution over $\boldsymbol\mu$ given $\boldsymbol\Lambda$ is a multivariate normal distribution. The marginal distribution over $\boldsymbol\mu$ is a multivariate t-distribution.

Generating normal-Wishart random variates

Generation of random variates is straightforward:

1. Sample $\boldsymbol\Lambda$ from a Wishart distribution with parameters $\mathbf{W}$ and $\nu$
2. Sample $\boldsymbol\mu$ from a multivariate normal distribution with mean $\boldsymbol\mu_0$ and variance $(\lambda\boldsymbol\Lambda)^{-1}$

Related distributions

• The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision.
• The normal-gamma distribution is the one-dimensional equivalent.
• The multivariate normal distribution and Wishart distribution are the component distributions out of which this distribution is made.

References

• Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.