QIC goodness of fit statistic for GEE models

Contents:

Purpose / History / Requirements / Usage / Details / Limitations / Missing Values / References

NOTE: Beginning in SAS 9.2, the QIC statistic is produced by PROC GENMOD. Beginning in SAS 9.4 TS1M2, QIC is available in PROC GEE.

PURPOSE:

The %QIC macro computes the QIC and QICu statistics proposed by Pan (2001) for GEE (generalized estimating equations) models. These statistics allow comparisons of GEE models (model selection) and selection of a correlation structure.

HISTORY:

Version	Update Notes
1.2	More adjustment of macro termination criteria.
1.1	Updated checks for errors that should terminate the macro. Fixed new version check to work in SAS 8.
1.0	Initial coding.

REQUIREMENTS:

Base SAS^®, SAS/STAT^®, and SAS/IML^® software in SAS 8.2 or later.

USAGE:

Follow the instructions in the Downloads tab of this sample to save the %QIC macro definition. Replace the text within quotes in the following statement with the location of the %QIC macro definition file on your system. In your SAS program or in the SAS editor window, specify this statement to define the %QIC macro and make it available for use:

   %inc "<location of your file containing the QIC macro>";

Following this statement, you may call the %QIC macro. See the Results tab for an example.

The parameters available with the %QIC macro are described below. Note that if any parameter argument you wish to specify contains commas or semicolons, you should enclose the argument in %str( ).

Required parameters

The following macro parameters are required. These options directly correspond to same-named options or statements in PROC GENMOD. For more details on their use, see the GENMOD documentation:

class=

Specify the SUBJECT= variable, any CLASS variables in the model, and the WITHIN= variable if specified.

response=

Specify the response variable.

dist=normal | binomial | gamma | poisson | igaussian

Specify the response distribution. Only the distributions listed above are supported.

subject=

Specify the variable or effect that defines the subjects or clusters.

type=     or

logor=

Specify the correlation structure or log odds ratio structure. Specify either type= or logor=, but not both.

Optional parameters

The following macro parameters are optional. All available options are described in the GENMOD documentation.

data=

Specify the data set to analyze. If omitted, the last-created data set is analyzed.

Poptions=

Specify any options needed in the PROC GENMOD statement.

model=

Specify the model. That is, list the effects which would normally appear in the MODEL statement after the response and equal sign. If omitted, an intercept-only model is fitted.

Moptions=

Specify any options needed in the MODEL statement other than DIST=.

within=

Specify the WITHIN= variable or effect.

Roptions=

Specify any options needed in the REPEATED statement other than SUBJECT=, TYPE=, or LOGOR=.

out=

Specify the name of the OUTPUT OUT= data set which will contain predicted values. If omitted the data set is named _OUTR.

p=

Specify the name of the variable containing predicted values in the OUTPUT OUT= data set. If omitted, the variable is named _P.

Ooptions=

Specify any additional statistic keyword options in the OUTPUT statement other than P=.

weight=

Specify a weight variable for the WEIGHT statement if desired.

freq=

Specify a frequency variable for the FREQ statement if desired.

stmts=%str( statements )

Specify any CONTRAST, ESTIMATE, FWDLINK, INVLINK, ASSESS, LSMEANS statements desired. The complete statements should appear within %str( ) as shown. BY, DEVIANCE, and VARIANCE statements are not supported.

outqic=

Names a data set containing the QIC and QICu statistics. If omitted, the data set is named _QIC.

appendto=

Specify the name of the data set to which the current model's QIC statistics should be added.

label=

Specify a label for the current model's QIC statistics when appended to the APPENDTO= data set. The label can be up to 40 characters long. If omitted the default label is structure / model, where structure is the value of the TYPE= or LOGOR= option and model is the value of MODEL=.

QICoptions=NOPRINT | INDMODEL

NOPRINT suppresses all displayed results from the macro. INDMODEL requests that the results of fitting the independence model (its covariance matrix is needed to compute QIC) be displayed.

The version of the %QIC macro that you are using is displayed when you specify version (or any string) as the first argument. For example:

    %QIC(version, data=six,
         class=case city,
         response=wheeze, model=city, dist=bin,
         subject=case, type=ind)

DETAILS:

The QIC (Quasilikelihood under the Independence model Criterion) statistic proposed by Pan (2001) and further discussed by Hardin and Hilbe (2003) is analogous to the familiar AIC (Akaike's Information Criterion) statistic used for comparing models fit with likelihood-based methods. Since the generalized estimating equations (GEE) method is not a likelihood-based method, the AIC statistic is not available.

QIC can be used to find an acceptable working correlation structure for a given model. However, Hardin and Hilbe (2003) recommend the use of QIC only to choose among otherwise equally suitable structures. They provide several guidelines based on the nature of the data for selecting suitable structures that should be applied first.

QIC and the related QICu statistic can be used to compare GEE models — that is, for model selection. QICu approximates QIC when the GEE model is correctly specified. QICu, defined as Q+2p, adds a penalty (2p) to the quasilikelihood (Q), where p is the number of parameters in the model. Models do not need to be nested in order to use QIC or QICu to compare them. Note that QICu should not be used for selecting a working correlation structure.

When using QIC or QICu to compare two structures or two models, the model with the smaller statistic is preferred.

When using the %QIC macro, the following message will appear but can be ignored:

WARNING: Iteration limit exceeded.

This message appears because GENMOD is run with parameters restricted to a set of values and this message is a normal consequence. The message does not indicate a problem.

The %QIC macro attempts to check for a later version of itself. If it is unable to do this (such as if there is no active internet connection available), the macro will issue the following message:

   QIC: Unable to check for newer version

The computations performed by the macro are not affected by the appearance of this message.

LIMITATIONS:

Multinomial models and models with user-defined distributions (via the VARIANCE and DEVIANCE statements) are not supported. BY processing is not supported. For binomial data in which each observation is a single trial, the response variable must be numeric and use the values 1 (for event) or 0 (for nonevent). Note that GENMOD does not impose this restriction.

MISSING VALUES:

Since GENMOD is used within the macro, missing values are handled as described in the GENMOD documentation.

REFERENCES:

Pan, W. (2001), "Akaike's information criterion in generalized estimating equations," Biometrics, 57, 120-125.

Hardin, J.W. and Hilbe, J.M. (2003), Generalized Estimating Equations, Chapman & Hall/CRC: New York.

Thall, P.F. and Vail, S.C. (1990), "Some Covariance Models for Longitudinal Count Data with Overdispersion," Biometrics, 46, 657 - 671.

Ware, J.H., Dockery, Spiro A. III, Speizer, F.E., and Ferris, B.G., Jr. (1984), "Passive Smoking, Gas Cooking, and Respiratory Health of Children Living in Six Cities," American Review of Respiratory Diseases, 129, 366 - 374.

---

These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.

---

---

These sample files and code examples are provided by SAS Institute Inc. "as is" without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. Recipients acknowledge and agree that SAS Institute shall not be liable for any damages whatsoever arising out of their use of this material. In addition, SAS Institute will provide no support for the materials contained herein.

EXAMPLE 1: Selecting a correlation structure

This example uses the repeated measures data from the "Six Cities" study of the health effects of air pollution (Ware et al., 1984). These data are discussed and analyzed in the Getting Started section of the GENMOD documentation. Below, a series of calls of the %QIC macro are made to fit the model using several possible correlation structures. Note that the unstructured correlation or log odds ratio structures are not possible as they result in singularities.

   %inc "<location of your file containing the QIC macro>";

   %qic(data=six, Poptions=desc,
        class=case city(ref='portage') smoke age,
        response=wheeze, model=city smoke age , dist=bin,
        subject=case, type=ind,
        p=pred, QICoptions=noprint, appendto=summary)
   %qic(data=six, Poptions=desc,
        class=case city(ref='portage'),
        response=wheeze, model=city smoke age, dist=bin,
        subject=case, type=exch,
        p=pred, QICoptions=noprint, appendto=summary)
   %qic(data=six, Poptions=desc,
        class=case city(ref='portage'),
        response=wheeze, model=city smoke age , dist=bin,
        subject=case, type=ar,
        p=pred, QICoptions=noprint, appendto=summary)
   %qic(data=six, Poptions=desc,
        class=case city(ref='portage'),
        response=wheeze, model=city smoke age , dist=bin,
        subject=case, type=mdep(3),
        p=pred, QICoptions=noprint, appendto=summary)
   %qic(data=six, Poptions=desc,
        class=case city(ref='portage'),
        response=wheeze, model=city smoke age , dist=bin,
        subject=case, logor=exch,
        p=pred, QICoptions=noprint, appendto=summary,
        label=%str(ALR:exch / city smoke age))
   %qic(data=six, Poptions=desc,
        class=case city(ref='portage'),
        response=wheeze, model=city smoke age , dist=bin,
        subject=case, logor=fullclust,
        p=pred, QICoptions=noprint, appendto=summary)
   
   proc sort data=summary;
     by QIC;
     run;
   proc print data=summary noobs;
     var label QIC;
     run;

Assuming all of the above structures are reasonable for the data, the results below suggest that the exchangeable alternating logistic regression (ALR) model and the AR1 GEE model are the best structures since they have the smallest QIC values. Note the use of the LABEL= option to create a label for the exchangeable ALR model. Otherwise, this model's label would be identical to the default label used for the exchangeable GEE model. Notice also that there is not a great deal of difference among the QIC values for the various models.

                            label                 QIC
                  
                  ALR:exch / city smoke age     87.1322
                  ar  / city smoke age          87.1512
                  exch  / city smoke age        87.2175
                  fullclust / city smoke age    87.3960
                  mdep(3)  / city smoke age     87.6385
                  ind  / city smoke age         87.8350

EXAMPLE 2: Model selection

This example uses the Progabide data from Thall and Vail (1990) which are also used in the "Log-Linear Model for Count Data" example in the GENMOD documentation. All possible models are fit and the QIC and QICu values are displayed.

   %inc "<location of your file containing the QIC macro>";

   %qic(data=progabide,
        class=id,
        response=y, model=time, dist=poisson, Moptions=offset=lnperiod,
        subject=id, type=exch,
        QICoptions=noprint, appendto=ProgModels)
   %qic(data=progabide,
        class=id,
        response=y, model=trt, dist=poisson, Moptions=offset=lnperiod,
        subject=id, type=exch,
        QICoptions=noprint, appendto=ProgModels)
   %qic(data=progabide,
        class=id,
        response=y, model=time trt, dist=poisson, Moptions=offset=lnperiod,
        subject=id, type=exch,
        QICoptions=noprint, appendto=ProgModels)
   %qic(data=progabide,
        class=id,
        response=y, model=time*trt, dist=poisson, Moptions=offset=lnperiod,
        subject=id, type=exch,
        QICoptions=noprint, appendto=ProgModels)
   %qic(data=progabide,
        class=id,
        response=y, model=time time*trt, dist=poisson, Moptions=offset=lnperiod,
        subject=id, type=exch,
        QICoptions=noprint, appendto=ProgModels)
   %qic(data=progabide,
        class=id,
        response=y, model=trt time*trt, dist=poisson, Moptions=offset=lnperiod,
        subject=id, type=exch,
        QICoptions=noprint, appendto=ProgModels)
   %qic(data=progabide,
        class=id,
        response=y, model=time|trt, dist=poisson, Moptions=offset=lnperiod,
        subject=id, type=exch,
        QICoptions=noprint, appendto=ProgModels)
   
   proc sort data=ProgModels;
     by QICu;
     run;
   proc print data=ProgModels noobs;
     run;

The results indicate that models TIME and TIME TRT are best. Their two QICu values differ by less than the 2p penalty term in the QICu statistic since the two models differ in size by only one parameter and their QICu values differ by less than 2. Both models differ by much more than this when compared to the next best model which includes TIME and the interaction.

                  label                       QIC        QICu
                  
                  exch  / time             -678.310    -683.027
                  exch  / time trt         -674.885    -682.031
                  exch  / time time*trt    -664.364    -669.138
                  exch  / time|trt         -659.587    -668.657
                  exch  / trt              -634.009    -637.728
                  exch  / trt time*trt     -627.698    -637.207
                  exch  / time*trt         -620.818    -625.824

Right-click on the link below and select Save to save the %QIC macro definition to a file. It is recommended that you name the file qic.sas.

Download and save qic.sas

The %QIC macro computes the QIC and QICu statistics proposed by Pan (2001) for GEE (generalized estimating equations) models. These statistics allow comparisons of GEE models (model selection) and selection of a correlation structure.

Type:

Sample

Date Modified:	2017-02-15 10:26:45
Date Created:	2007-08-14 03:03:15

Operating System and Release Information

Product Family	Product	Host	SAS Release
			Starting	Ending
SAS System	SAS/STAT	z/OS
		Z64
		OpenVMS VAX
		Microsoft® Windows® for 64-Bit Itanium-based Systems
		Microsoft Windows Server 2003 Datacenter 64-bit Edition
		Microsoft Windows Server 2003 Enterprise 64-bit Edition
		Microsoft Windows XP 64-bit Edition
		Microsoft® Windows® for x64
		OS/2
		Microsoft Windows 8 Enterprise 32-bit
		Microsoft Windows 8 Enterprise x64
		Microsoft Windows 8 Pro 32-bit
		Microsoft Windows 8 Pro x64
		Microsoft Windows 8.1 Enterprise 32-bit
		Microsoft Windows 8.1 Enterprise x64
		Microsoft Windows 8.1 Pro
		Microsoft Windows 8.1 Pro 32-bit
		Microsoft Windows 95/98
		Microsoft Windows 2000 Advanced Server
		Microsoft Windows 2000 Datacenter Server
		Microsoft Windows 2000 Server
		Microsoft Windows 2000 Professional
		Microsoft Windows NT Workstation
		Microsoft Windows Server 2003 Datacenter Edition
		Microsoft Windows Server 2003 Enterprise Edition
		Microsoft Windows Server 2003 Standard Edition
		Microsoft Windows Server 2003 for x64
		Microsoft Windows Server 2008
		Microsoft Windows Server 2008 R2
		Microsoft Windows Server 2008 for x64
		Microsoft Windows Server 2012 Datacenter
		Microsoft Windows Server 2012 R2 Datacenter
		Microsoft Windows Server 2012 R2 Std
		Microsoft Windows Server 2012 Std
		Microsoft Windows XP Professional
		Windows 7 Enterprise 32 bit
		Windows 7 Enterprise x64
		Windows 7 Home Premium 32 bit
		Windows 7 Home Premium x64
		Windows 7 Professional 32 bit
		Windows 7 Professional x64
		Windows 7 Ultimate 32 bit
		Windows 7 Ultimate x64
		Windows Millennium Edition (Me)
		Windows Vista
		Windows Vista for x64
		64-bit Enabled AIX
		64-bit Enabled HP-UX
		64-bit Enabled Solaris
		ABI+ for Intel Architecture
		AIX
		HP-UX
		HP-UX IPF
		IRIX
		Linux
		Linux for x64
		Linux on Itanium
		OpenVMS Alpha
		OpenVMS on HP Integrity
		Solaris
		Solaris for x64
		Tru64 UNIX