The FMM Procedure (Experimental)

The Form of the Finite Mixture Model

Suppose that you observe realizations of a random variable $\text{[math]}$ , the distribution of which depends on an unobservable random variable $\text{[math]}$ that has a discrete distribution. $\text{[math]}$ can occupy one of $\text{[math]}$ states, the number of which might be unknown but is at least known to be finite. Since $\text{[math]}$ is not observable, it is frequently referred to as a latent variable.

Let $\text{[math]}$ denote the probability that $\text{[math]}$ takes on state $\text{[math]}$ . Conditional on $\text{[math]}$ , the distribution of the response $\text{[math]}$ is assumed to be $\text{[math]}$ . In other words, each distinct state $\text{[math]}$ of the random variable $\text{[math]}$ leads to a particular distributional form $\text{[math]}$ and set of parameters $\text{[math]}$ for $\text{[math]}$ .

Let $\text{[math]}$ denote the collection of $\text{[math]}$ and $\text{[math]}$ parameters across all $\text{[math]}$ = $\text{[math]}$ to $\text{[math]}$ . The marginal distribution of $\text{[math]}$ is obtained by summing the joint distribution of $\text{[math]}$ and $\text{[math]}$ over the states in the support of $\text{[math]}$ :

	$\text{[math]}$	$\text{[math]}$
	$\text{[math]}$	$\text{[math]}$

This is a mixture of distributions, and the $\text{[math]}$ are called the mixture (or prior) probabilities. Because the number of states $\text{[math]}$ of the latent variable $\text{[math]}$ is finite, the entire model is termed a finite mixture (of distributions) model.

The finite mixture model can be expressed in a more general form by representing $\text{[math]}$ and $\text{[math]}$ in terms of regressor variables and parameters with optional additional scale parameters for $\text{[math]}$ . The section Notation for the Finite Mixture Model develops this in detail.