Print this chapterPrint this chapter

Probability Distributions and their Stories

Continuous Multivariate distributions

Multivariate Gaussian, a.k.a. Multivariate Normal, distribution

  • Story. This is a generalization of the univariate Gaussian.
  • Example. Finch beaks are measured for beak depth and beak length. The resulting distribution of depths and length is Gaussian distributed. In this case, the Gaussian is bivariate, with \(μ=(μ_d,μ_l)\) and \(\begin{align}
    \mathsf{\Sigma} = \begin{pmatrix}
    \sigma_\mathrm{d}^2 & \sigma_\mathrm{dl} \\
    \sigma_\mathrm{dl} & \sigma_\mathrm{l}^2
    \end{pmatrix}
    \end{align}\)    .
  • Parameters. There is one vector-valued parameter, \(μ\), and a matrix-valued parameter, \(Σ\), referred to respectively as the mean and covariance matrix. The covariance matrix is symmetric and strictly positive definite.
  • Support. The K-variate Gaussian distribution is supported on \(\mathbb{R}^K\).
  • Probability density function.

    \(\begin{align}
    f(\mathbf{y};\boldsymbol{\mu}, \mathsf{\Sigma}) = \frac{1}{\sqrt{(2\pi)^K \det \mathsf{\Sigma}}}\,\mathrm{exp}\left[-\frac{1}{2}(\mathbf{y} - \boldsymbol{\mu})^T \cdot \mathsf{\Sigma}^{-1}\cdot (\mathbf{y} - \boldsymbol{\mu})\right].
    \end{align}\)

  • Usage The usage below assumes that mu is a length K array, Sigma is a K×K symmetric positive definite matrix, and L is a K×K lower-triangular matrix with strictly positive values on teh diagonal that is a Cholesky factor.

    • Package Syntax
    NumPy np.random.multivariate_normal(mu, Sigma)
    SciPy scipy.stats.multivariate_normal(mu, Sigma)
    Stan multi_normal(mu, Sigma)
    NumPy Cholesky np.random.multivariate_normal(mu, np.dot(L, L.T))
    SciPy Cholesky scipy.stats.multivariate_normal(mu, np.dot(L, L.T))
    Stan Cholesky multi_normal_cholesky(mu, L)


  • Related distributions.
    • The Multivariate Gaussian is a generalization of the univariate Gaussian.
  • Notes.
  • The covariance matrix may also be written as \(Σ=S⋅C⋅S\), where
    \(S=\sqrt{diag(Σ)}\),
    and entry \(i\), \(j\) in the correlation matrix C is

    \(C_{ij}=σ_{ij}/σ_iσ_j\).

    Furthermore, because \(Σ\) is symmetric and strictly positive definite, it can be uniquely defined in terms of its Cholesky decomposition, \(L\), which satisfies \(Σ=L ⋅ LT\). In practice, you will almost always use the Cholesky representation of the Multivariate Gaussian distribution in Stan.