The Form of the Finite Mixture Model

Suppose that you observe realizations of a random variable , the distribution of which depends on an unobservable random variable that has a discrete distribution. can occupy one of states, the number of which might be unknown but is at least known to be finite. Since is not observable, it is frequently referred to as a latent variable.

Let denote the probability that takes on state . Conditional on , the distribution of the response is assumed to be . In other words, each distinct state of the random variable leads to a particular distributional form and set of parameters for .

Let denote the collection of and parameters across all = to . The marginal distribution of is obtained by summing the joint distribution of and over the states in the support of :


This is a mixture of distributions, and the are called the mixture (or prior) probabilities. Because the number of states of the latent variable is finite, the entire model is termed a finite mixture (of distributions) model.

The finite mixture model can be expressed in a more general form by representing and in terms of regressor variables and parameters with optional additional scale parameters for . The section Notation for the Finite Mixture Model develops this in detail.