The Exponential Family of Distributions

Fit Analyses

The Exponential Family of Distributions

The distribution of a random variable Y belongs to the exponential family if its probability (density) function can be written in the form

$f(y ; \theta, \phi) = \exp( \frac{y \theta-b(\theta)}{a(\phi)} + c(y , \phi) )$

where ${\theta}$ is the natural or canonical parameter, $\phi$ is the dispersion parameter, and a, b and c are specific functions.

The mean and variance of Y are then given by (McCullagh and Nelder 1989)

$E(y) = \mu = {b'}(\theta)$

$\rm{Var}(y) = a(\phi) {b"}(\theta)$

The function ${{b"}(\theta)}$ can be expressed as a function of $\mu$ , $b"(\theta) = V(\mu)$ ,and it is called the variance function. Different choices of the function ${b(\theta)}$ generate different distributions in the exponential family. For a binomial distribution with m trials, the function ${a(\phi) = \phi/m}$ .For other distributions in the exponential family, ${a(\phi) = \phi}$ .

SAS/INSIGHT software includes normal, inverse Gaussian, gamma, Poisson, and binomial distributions for the response distribution. For these response distributions, the density functions f(y), the variance functions ${V(\mu)}$ , and the dispersion parameters $\phi$ with function ${a(\phi)}$ are

Normal: $f(y) = \frac{1}{\sqrt{2 \pi} \sigma} \exp( -\frac{1}2( \frac{y-\mu}{\sigma} )^2 )$ for $-\infty\lt y\lt\infty$

$V(\mu) = 1$

$a(\phi) = \phi = {\sigma}^2$
Inverse Gaussian: $f(y) = \frac{1}{\sqrt{2\pi y^3} \sigma } \exp( -\frac{1}{2 \mu^2y} ( \frac{y-\mu}{{\sigma}} )^2 )$ for y > 0

$V(\mu) = \mu^3$

$a(\phi) = \phi = {\sigma}^2$
Gamma: ${f(y) = \frac{1}{y {\Gamma}({\nu})} (\frac{{\nu}y}{\mu})^{{\nu}} \exp(-\frac{{\nu}y}{\mu}) } {for y \gt 0}$

${V(\mu) = \mu^2}$

${a(\phi) = \phi = {\nu}^{-1}}$
Poisson: ${f(y) = \frac{\mu^y e^{-\mu}}{y! } } {for y=0, 1, 2, ... }$

${V(\mu) = \mu}$

${a(\phi) = \phi = 1}$
Binomial: ${f(y) = {m \choose r} \mu^r (1-\mu)^{m-r} } {for y= r/m, r=0, 1, 2,..., m}$

${V(\mu) = \mu (1-\mu)}$

${a(\phi) = \phi / m = 1/m}$

Top of Page