Prior Distributions
Previous Page|Next Page

The QLIM Procedure

Prior Distributions

Priors for Heteroscedastic Models
Standard Distributions

The PRIOR statement is used to specify the prior distribution of the model parameters. You must specify a list of parameters, a tilde $\mbox{({\scriptsize {$\sim $}})}$, and then a distribution with its parameters. You can specify multiple PRIOR statements to define independent priors. Parameters that are associated with a regressor variable are referred to by the name of the corresponding regressor variable.

You can specify the special keyword _REGRESSORS to consider all the regressors of a model. If multiple prior statements affect the same parameter, the prior that is specified is used. For example, in a regression with three regressors (X1, X2, X3) the following statements imply that the prior on X1 is NORMAL(MEAN=0, VAR=1), the prior on X2 is GAMMA(SHAPE=3, SCALE=4), and the prior on X3 is UNIFORM(MIN=0, MAX=1): 

...
prior _Regressors ~ uniform(min=0, max=1);
prior X1 X2 ~ gamma(shape=3, scale=4);
prior X1 ~ normal(mean=0, var=1);
...

If a parameter is not associated with a PRIOR statement or if some of the prior hyperparameters are missing, then the following default choices are considered:

Table 22.2: Default values for prior distributions.

PRIOR distribution

$\Variable{Hyperparameter}_1$

$\Variable{Hyperparameter}_2$

$\Variable{Min}$

$\Variable{Max}$

Parameters Default Choice

NORMAL

MEAN=0

VAR=1E6

$-\infty $

$\infty $

$\Variable{Regression-Location-Threshold}$

IGAMMA

SHAPE=2.000001

SCALE=1

$>0$

$\infty $

$\Variable{Scale}$

GAMMA

SHAPE=1

SCALE=1

$\Variable{0}$

$\infty $

UNIFORM

    

$-\infty $

$\infty $

BETA

SHAPE1=1

SHAPE2=1

$-\infty $

$\infty $

T

LOCATION=0

DF=3

$-\infty $

$\infty $


See the section Standard Distributions for density specification.

Priors for Heteroscedastic Models

The choice of the prior distribution for a heteroscedastic model is particularly interesting. Based on the notation provided in section HETERO Statement, you need to provide a prior for $\bgamma $. This prior is enough to induce different $\sigma _ i^2$ into the analysis. The resulting inference is a compromise between two cases: the inference based on the entire sample and the inference based on a single unit $\mb {z}_{i}$. The degree of compromise is determined by $\pi (\bgamma )$.

This type of modeling is similar to a method called “hierarchical Bayes,” in which the prior is characterized by two levels: one for each individual $\pi (\sigma _ i^2|\bgamma )$ and one for the entire population $\pi (\bgamma )$. In this scenario the degree of compromise between the information provided by a unit and the information provided by the entire sample is determined by the data.

The choice of the prior might not be straightforward, and it can heavily affect sampling performance. Depending on how the heteroscedastic effects are modeled, the default priors are

$\displaystyle \textnormal{if } \left[1+\exp (\Strong{z}^{}_{i}{\bgamma })\right], $
$\displaystyle $
$\displaystyle \pi (\gamma _ j)= \Strong{normal} \left\{ \Variable{mean}=\frac{1}{\bar{z}_ jJ}\left[\log \left(\frac{\varepsilon ^4}{1+\varepsilon ^2}\right)\right],\Variable{var}=\frac{1}{{\bar{z}_ j}^2J}\left[\log \left(\frac{1+\varepsilon ^2}{\varepsilon ^2}\right)\right]\right\} $
$\displaystyle \textnormal{if } \left[\exp (\Strong{z}^{}_{i}{\bgamma })\right], $
$\displaystyle $
$\displaystyle \pi (\gamma _ j)= \Strong{normal} \left\{ \Variable{mean}=\frac{1}{\bar{z}_ jJ}\left[\log \left(\frac{1}{2}\right)\right],\Variable{var}=\frac{1}{{\bar{z}_ j}^2J}\left[\log \left(2\right)\right]\right\} $
$\displaystyle \textnormal{if } \left(1+\Strong{z} ^{}_{i}{\bgamma }\right), $
$\displaystyle $
$\displaystyle \pi (\gamma _ j)= \Strong{normal} \left\{ \Variable{mean}=0,\Variable{var}=\frac{1}{{\bar{z}_ j}^2J}\right\} $
$\displaystyle \textnormal{if } \left(\Strong{z} ^{}_{i}{\bgamma }\right), $
$\displaystyle $
$\displaystyle \pi (\gamma _ j)= \Strong{normal} \left\{ \Variable{mean}=\frac{1}{\bar{z}_ jJ},\Variable{var}=\frac{1}{{\bar{z}_ j}^2J}\right\} $
$\displaystyle \textnormal{if } \left[1+(\Strong{z}^{}_{i}{\bgamma })^2\right], $
$\displaystyle $
$\displaystyle \pi (\gamma _ j)= \Strong{normal} \left\{ \Variable{mean}=\frac{(\varepsilon ^2-1/2)^{1/4}}{\bar{z}_ jJ},\Variable{var}=\frac{\varepsilon -(\varepsilon ^2-1/2)^{1/2}}{\bar{z}_ j^2J}\right\} $
$\displaystyle \textnormal{if } \left[(\Strong{z}^{}_{i}{\bgamma })^2\right], $
$\displaystyle $
$\displaystyle \pi (\gamma _ j)= \Strong{normal} \left\{ \Variable{mean}=\frac{(1/2)^{1/4}}{\bar{z}_ jJ},\Variable{var}=\frac{1-(1/2)^{1/2}}{\bar{z}_ j^2J}\right\} $

where $\bar{z}_ j=\frac{1}{n}\sum _{i=1}^ nz_{ij}$, $\forall j$, and $\varepsilon $ is a small number (by default, $\varepsilon =0.1$ for the EXPONENTIAL link function and $\varepsilon =0.71$ for the QUADRATIC link function).

The priors for the EXPONENTIAL and QUADRATIC link functions are not straightforward. To understand the choices, do the following:

  1. Assume that

    \[ \Strong{z}^{}_{i}{\bgamma }=z_{i1}{\gamma _1}+\ldots +z_{iJ}{\gamma _ J}\approx \bar{z}_{1}{\gamma _1}+\ldots +\bar{z}_{J}{\gamma _ J},\qquad \forall i \]

  2. Set the priors according to the link function type:

    • For the EXPONENTIAL link function, set

      $\displaystyle \textnormal{E}\left[\exp (\Strong{z}^{}_{i}{\bgamma })\right] $
      $\displaystyle \approx $
      $\displaystyle \textnormal{E}\left[\exp (\bar{z}_{1}{\gamma _1})\right]\times \ldots \times \textnormal{E}\left[\exp (\bar{z}_{J}{\gamma _ J})\right]=\varepsilon $
      $\displaystyle \textnormal{V}\left[\exp (\Strong{z}^{}_{i}{\bgamma })\right] $
      $\displaystyle \approx $
      $\displaystyle \textnormal{E}\left[\exp (2\bar{z}_{1}{\gamma _1})\right]\times \ldots \times \textnormal{E}\left[\exp (2\bar{z}_{J}{\gamma _ J})\right]-\varepsilon ^2=1 $

      Assume a normal prior for $\pi (\gamma _ j)$, and set

      $\displaystyle \textnormal{E}\left[\exp (\bar{z}_{j}{\gamma _ j})\right] $
      $\displaystyle = $
      $\displaystyle \varepsilon ^{\frac{1}{J}},\forall j $
      $\displaystyle \textnormal{E}\left[\exp (2\bar{z}_{j}{\gamma _ j})\right] $
      $\displaystyle = $
      $\displaystyle (1+\varepsilon ^2)^{\frac{1}{J}},\forall j $

      Based on the properties of the lognormal distribution, the prior hyperparameters for $\gamma _ j$ can be derived. Notice that $J$ is the number of regressors that are used in the heterogeneous regression. If the intercept is excluded, then $\varepsilon =1$.

    • For the QUADRATIC link function, set

      $\displaystyle \textnormal{E}\left[(\Strong{z}^{}_{i}{\bgamma })^2\right] $
      $\displaystyle \approx $
      $\displaystyle \left[\textnormal{E}\left(\bar{z}_{1}{\gamma _1}+\ldots +\bar{z}_{J}{\gamma _ J}\right)\right]^2+\textnormal{V}\left[\bar{z}_{1}{\gamma _1}+\ldots +\bar{z}_{J}{\gamma _ J}\right]=\varepsilon $
      $\displaystyle \textnormal{V}\left[\exp (\Strong{z}^{}_{i}{\bgamma })\right] $
      $\displaystyle \approx $
      $\displaystyle \textnormal{E}\left[(\bar{z}_{1}{\gamma _1}+\ldots +\bar{z}_{J}{\gamma _ J})^4\right]-\varepsilon ^2=1 $

      Assume a normal prior for $\pi (\gamma _ j)$. Based on the properties of the normal distribution, the preceding expressions return

      $\displaystyle \textnormal{E}\left[\bar{z}_{1}{\gamma _1}+\ldots +\bar{z}_{J}{\gamma _ J}\right] $
      $\displaystyle = $
      $\displaystyle (\varepsilon ^2-1/2)^{1/4} $
      $\displaystyle \textnormal{V}\left[\bar{z}_{1}{\gamma _1}+\ldots +\bar{z}_{J}{\gamma _ J}\right] $
      $\displaystyle = $
      $\displaystyle \varepsilon -(\varepsilon ^2-1/2)^{1/2} $
      $\displaystyle \varepsilon $
      $\displaystyle > $
      $\displaystyle (1/2)^{1/2} $

      The prior hyperparameters for $\gamma _ j$ can be derived by setting

      $\displaystyle \textnormal{E}\left[\bar{z}_{j}{\gamma _ j}\right] $
      $\displaystyle = $
      $\displaystyle \frac{(\varepsilon ^2-1/2)^{1/4}}{J},\forall j $
      $\displaystyle \textnormal{V}\left[\bar{z}_{j}{\gamma _ j}\right] $
      $\displaystyle = $
      $\displaystyle \frac{\varepsilon -(\varepsilon ^2-1/2)^{1/2}}{J},\forall j $

      Notice that $J$ is the number of regressors that are used in the heterogeneous regression. If the intercept is excluded, then $\varepsilon =1$. It is important to emphasize that the restriction $\varepsilon >(1/2)^{1/2}$ is likely to introduce some distortion because $\varepsilon $ cannot be any “small” number.

Standard Distributions

Table 22.3 through Table 22.8 show all the distribution density functions that PROC QLIM recognizes. You specify these distribution densities in the PRIOR statement.

Table 22.3: Beta Distribution

PRIOR statement

BETA(SHAPE1=$a$, SHAPE2=$b$, MIN=$m$, MAX=$M$)

 

Note: Commonly $m=0$ and $M=1$.

Density

$\frac{(\theta -m)^{a-1} (M-\theta )^{b-1}}{B(a,b)(M-m)^{a+b-1}}$

Parameter restriction

$a>0$, $b>0$, $-\infty <m<M<\infty $

Range

$ \left\{ \begin{array}{ll} \left[ m, M \right] & \mbox{when } a = 1, b = 1 \\ \left[ m, M \right) & \mbox{when } a = 1, b \neq 1 \\ \left( m, M \right] & \mbox{when } a \neq 1, b = 1 \\ \left( m, M \right) & \mbox{otherwise} \end{array} \right. $

Mean

$ \frac{a}{a+b}\times (M-m)+m$

Variance

$ \frac{ab}{(a+b)^2(a+b+1)}\times (M-m)^2$

Mode

$ \left\{ \begin{array}{ll} \frac{a-1}{a+b-2}\times M+\frac{b-1}{a+b-2}\times m & a > 1, b > 1 \\ m \mbox{ and } M & a < 1, b < 1 \\ m & \left\{ \begin{array}{l} a < 1, b \geq 1 \\ a = 1, b > 1 \\ \end{array} \right. \\ M & \left\{ \begin{array}{l} a \geq 1, b < 1 \\ a > 1, b = 1 \\ \end{array} \right. \\ \mbox{not unique} & a = b = 1 \end{array} \right. $

Defaults

SHAPE1=SHAPE2=1, $\Variable{MIN}\rightarrow -\infty $, $\Variable{MAX}\rightarrow \infty $


Table 22.4: Gamma Distribution

PRIOR statement

GAMMA(SHAPE=$a$, SCALE=$b$ )

Density

$\frac{1}{b^ a\Gamma (a)} \theta ^{a-1} e^{-\theta /b} $

Parameter restriction

$ a > 0, b > 0 $

Range

$[0,\infty )$

Mean

$ab$

Variance

$ab^2$

Mode

$(a-1)b$

Defaults

SHAPE=SCALE=1


Table 22.5: Inverse-Gamma Distribution

PRIOR statement

IGAMMA(SHAPE=$a$, SCALE=$b$)

Density

$ \frac{b^ a}{\Gamma (a)} \theta ^{-(a+1)}e^{-b/\theta } $

Parameter restriction

$ a > 0, b > 0$

Range

$ 0<\theta <\infty $

Mean

$\frac{b}{a-1},\qquad a > 1$

Variance

$\frac{b^2}{(a-1)^2(a-2)},\qquad a>2$

Mode

$ \frac{b}{a+1}$

Defaults

SHAPE=2.000001, SCALE=1


Table 22.6: Normal Distribution

PRIOR statement

NORMAL(MEAN=$\mu $, VAR=$\sigma ^2$)

Density

$ \frac{1}{\sigma \sqrt {2\pi }} \exp \left( - \frac{(\theta - \mu )^2}{2\sigma ^2}\right) $

Parameter restriction

$ \sigma ^2 > 0 $

Range

$ -\infty <\theta <\infty $

Mean

$\mu $

Variance

$\sigma ^2$

Mode

$\mu $

Defaults

MEAN=0, VAR=1000000


Table 22.7: $t$ Distribution

PRIOR statement

T(LOCATION=$\mu $, DF=$\nu $)

Density

$\frac{\Gamma \left(\frac{\nu +1}{2}\right)}{\Gamma \left(\frac{\nu }{2}\right)\sqrt {\pi \nu }}\left[1+\frac{(\theta -\mu )^2}{\nu }\right]^{-\frac{\nu +1}{2}} $

Parameter restriction

$ \nu > 0 $

Range

$ -\infty <\theta <\infty $

Mean

$\mu , \textnormal{ for }\nu >1$

Variance

$\frac{\nu }{\nu -2}, \textnormal{ for }\nu >2$

Mode

$\mu $

Defaults

LOCATION=0, DF=3


Table 22.8: Uniform Distribution

PRIOR statement

UNIFORM(MIN=$m$, MAX=$M$)

Density

$ \frac{1}{M-m}$

Parameter restriction

$-\infty <m<M<\infty $

Range

$ \theta \in [m, M]$

Mean

$ \frac{m+M}{2} $

Variance

$\frac{(M-m)^2}{12}$

Mode

Not unique

Defaults

MIN$\rightarrow -\infty $, MAX$\rightarrow \infty $


Previous Page|Next Page|Top of Page
Copyright © SAS Institute Inc. All Rights Reserved.