Estimating Fixed and Random Effects in the Mixed Model :: SAS/STAT(R) 13.1 User's Guide: High-Performance Procedures

Estimating Fixed and Random Effects in the Mixed Model

ML and REML methods provide estimates of $\bG$ and $\bR$ , which are denoted $\widehat{\bG }$ and $\widehat{\bR }$ , respectively. To obtain estimates of $\bbeta$ and predicted values of $\bgamma$ , the standard method is to solve the mixed model equations (Henderson, 1984):

$\left[\begin{array}{lr} \bX ’\widehat{\bR }^{-1}\bX & \bX ’\widehat{\bR }^{-1}\bZ \\*\bZ ’\widehat{\bR }^{-1}\bX & \bZ ’\widehat{\bR }^{-1}\bZ + \widehat{\bG }^{-1} \end{array}\right] \left[\begin{array}{c} \widehat{\bbeta } \\ \widehat{\bgamma } \end{array} \right] = \left[\begin{array}{r} \bX ’\widehat{\bR }^{-1}\mb {y} \\ \bZ ’\widehat{\bR }^{-1}\mb {y} \end{array} \right]$

The solutions can also be written as

$\begin{align*} \widehat{\bbeta } & = (\bX ’\widehat{\bV }^{-1}\bX )^{-} \bX ’\widehat{\bV }^{-1}\mb {y} \\ \widehat{\bgamma } & = \widehat{\bG }\bZ ’\widehat{\bV }^{-1} (\mb {y} - \bX \widehat{\bbeta }) \end{align*}$

and have connections with empirical Bayes estimators (Laird and Ware, 1982; Carlin and Louis, 1996). Note that the $\bgamma$ are random variables and not parameters (unknown constants) in the model. Technically, determining values for $\bgamma$ from the data is thus a prediction task, whereas determining values for $\bbeta$ is an estimation task.

The mixed model equations are extended normal equations. The preceding expression assumes that $\widehat{\bG }$ is nonsingular. For the extreme case where the eigenvalues of $\widehat{\bG }$ are very large, $\widehat{\bG }^{-1}$ contributes very little to the equations and $\widehat{\bgamma }$ is close to what it would be if $\bgamma$ actually contained fixed-effects parameters. On the other hand, when the eigenvalues of $\widehat{\bG }$ are very small, $\widehat{\bG }^{-1}$ dominates the equations and $\widehat{\bgamma }$ is close to $0$ . For intermediate cases, $\widehat{\bG }^{-1}$ can be viewed as shrinking the fixed-effects estimates of $\bgamma$ toward $0$ (Robinson, 1991).

If $\widehat{\bG }$ is singular, then the mixed model equations are modified (Henderson, 1984) as follows:

$\left[\begin{array}{lr} \bX ’\widehat{\bR } ^{-1}\bX & \bX ’\widehat{\bR } ^{-1} \bZ \widehat{\bG } \\ \widehat{\bG }’\bZ ’\widehat{\bR } ^{-1}\bX & \widehat{\bG }’\bZ ’\widehat{\bR } ^{-1}\bZ \widehat{\bG } + \bG \end{array}\right] \left[\begin{array}{c} \widehat{\bbeta } \\ \widehat{\btau } \end{array} \right] = \left[\begin{array}{r} \bX ’\widehat{\bR } ^{-1}\mb {y} \\ \widehat{\bG }’\bZ ’\widehat{\bR } ^{-1}\mb {y} \end{array} \right]$

Denote the generalized inverses of the nonsingular $\widehat{\bG }$ and singular $\widehat{\bG }$ forms of the mixed model equations by $\bC$ and $\bM$ , respectively. In the nonsingular case, the solution $\widehat{\bgamma }$ estimates the random effects directly. But in the singular case, the estimates of random effects are achieved through a back-transformation $\widehat{\bgamma } = \widehat{\bG }\widehat{\btau }$ where $\widehat{\btau }$ is the solution to the modified mixed model equations. Similarly, while in the nonsingular case $\bC$ itself is the estimated covariance matrix for $(\widehat{\bbeta },\widehat{\bgamma })$ , in the singular case the covariance estimate for $(\widehat{\bbeta },\widehat{\bG }\widehat{\btau })$ is given by $\bP \bM \bP$ where

$\bP = \left[\begin{array}{cc} \bI & \\ & \widehat{\bG } \end{array}\right]$

An example of when the singular form of the equations is necessary is when a variance component estimate falls on the boundary constraint of $0$ .

The HPLMIXED Procedure

Estimating Fixed and Random Effects in the Mixed Model