The FMM Procedure

The Form of the Finite Mixture Model

Suppose that you observe realizations of a random variable Y, the distribution of which depends on an unobservable random variable S that has a discrete distribution. S can occupy one of k states, the number of which might be unknown but is at least known to be finite. Since S is not observable, it is frequently referred to as a latent variable.

Let $\pi _ j$ denote the probability that S takes on state j. Conditional on , the distribution of the response Y is assumed to be $f_ j(y;\alpha _ j,\bbeta _ j|S=j)$ . In other words, each distinct state j of the random variable S leads to a particular distributional form and set of parameters $\{ \alpha _ j,\bbeta _ j\}$ for Y.

Let $\{ \balpha , \bbeta \}$ denote the collection of $\alpha _ j$ and $\bbeta _ j$ parameters across all j = 1 to k. The marginal distribution of Y is obtained by summing the joint distribution of Y and S over the states in the support of S:

$\displaystyle f(y;\balpha ,\bbeta ) =$	$\displaystyle \sum _{j=1}^ k \Pr (S=j)\, f(y;\alpha _ j,\bbeta _ j\|S=j)$
$\displaystyle =$	$\displaystyle \sum _{j=1}^ k \pi _ j f(y;\alpha _ j,\bbeta _ j\|S=j)$

This is a mixture of distributions, and the $\pi _ j$ are called the mixture (or prior) probabilities. Because the number of states k of the latent variable S is finite, the entire model is termed a finite mixture (of distributions) model.

The finite mixture model can be expressed in a more general form by representing $\balpha$ and $\bbeta$ in terms of regressor variables and parameters with optional additional scale parameters for $\bbeta$ . The section Notation for the Finite Mixture Model develops this in detail.