### Conjugate Sampling

Conjugate prior is a family of prior distributions in which the prior and the posterior distributions are of the same family of distributions. For example, if you model an independently and identically distributed random variable using a normal likelihood with known variance ,

a normal prior on

is a conjugate prior because the posterior distribution of is also a normal distribution given , , , and :

Conjugate sampling is efficient because it enables the Markov chain to obtain samples from the target distribution directly. When appropriate, PROC MCMC uses conjugate sampling methods to draw conditional posterior samples. Table 59.6 lists scenarios that lead to conjugate sampling in PROC MCMC.

Table 59.6: Conjugate Sampling in PROC MCMC

Family

Parameter

Prior

Normal with known

Variance

Inverse gamma family

Normal with known

Precision

Gamma family

Normal with known scale parameter (, , or )

Mean

Normal

Multivariate normal with known

Mean

Multivariate normal

Multivariate normal with known

Covariance

Inverse Wishart

Multinomial

Dirichlet

Binomial/binary

p

Beta

Poisson

Gamma family

In most cases, Family in Table 59.6 refers to the likelihood function. However, it does not necessarily have to be the case. The Family is a distribution that is conditional on the parameter of interest, and it can appear in any level of the hierarchical model, including on the random-effects level.

PROC MCMC can detect conjugacy only if the model parameter (not a function or a transformation of the model parameter) is used in the prior and Family distributions. For example, the following statements lead to a conjugate sampler being used on the parameter mu:

parm mu;
prior mu ~ n(0, sd=1000);
model y ~ n(mu, var=s2);


However, if you modify the program slightly in the following way, although the conjugacy still holds in theory, PROC MCMC cannot detect conjugacy on mu because the parameter enters the normal likelihood function through the symbol w:

parm mu;
prior mu ~ n(0, sd=1000);
w = mu;
model y ~ n(w, var=s2);


In this case, PROC MCMC resorts to the default sampling algorithm, which is a random walk Metropolis based on a normal kernel.

Similarly, the following statements also prevent PROC MCMC from detecting conjugacy on the parameter mu:

parm mu;
prior mu ~ n(0, sd=1000);
model y ~ n(mu + 2, var=s2);


In a normal family, an often-used and often-confused conjugate prior on the variance is the inverse gamma distribution, and a conjugate prior on the precision is the gamma distribution. See Gamma and Inverse-Gamma Distributions for typical usages of these prior distributions.

When conjugacy is detected in a model, PROC MCMC performs a numerical optimization on the joint posterior distribution at the start of the MCMC simulation. If the only sampling methods required in the program are conjugate samplers or direct samplers, PROC MCMC omits this optimization step. To turn off this optimization routine, use the PROPCOV=IND option in the PROC MCMC statement.