The LOGISTIC Procedure

Score Statistics and Tests

Subsections:

To understand the general form of the score statistics, let $\mb{g}(\bbeta )$ be the vector of first partial derivatives of the log likelihood with respect to the parameter vector $\bbeta $, and let $\bH (\bbeta )$ be the matrix of second partial derivatives of the log likelihood with respect to $\bbeta $. That is, $\mb{g}(\bbeta )$ is the gradient vector, and $\bH (\bbeta )$ is the Hessian matrix. Let $\bI (\bbeta )$ be either $-\bH (\bbeta )$ or the expected value of $-\bH (\bbeta )$. Consider a null hypothesis $H_0$. Let ${\widehat{\bbeta }}_{H_0}$ be the MLE of $\bbeta $ under $H_0$. The chi-square score statistic for testing $H_0$ is defined by

\[ \mb{g}’({\widehat{\bbeta }}_{H_0})\bI ^{-1}({\widehat{\bbeta }}_{H_0})\mb{g} ({\widehat{\bbeta }}_{H_0}) \]

and it has an asymptotic $\chi ^2$ distribution with r degrees of freedom under $H_0$, where r is the number of restrictions imposed on $\bbeta $ by $H_0$.

Residual Chi-Square

When you use SELECTION= FORWARD, BACKWARD, or STEPWISE, the procedure calculates a residual chi-square score statistic and reports the statistic, its degrees of freedom, and the p-value. This section describes how the statistic is calculated.

Suppose there are s explanatory effects of interest. The full cumulative response model has a parameter vector

\[ \bbeta =(\alpha _{1},\ldots ,\alpha _{k},\beta _1,\ldots ,\beta _ s)’ \]

where $\alpha _{1},\ldots ,\alpha _{k}$ are intercept parameters, and $\beta _1, \ldots , \beta _ s$ are the common slope parameters for the s explanatory effects. The full generalized logit model has a parameter vector

\begin{eqnarray*} \bbeta & =& (\alpha _{1},\ldots ,\alpha _{k},\bbeta _{1}’,\ldots ,\bbeta _{k}’)’ \quad \mbox{with} \\ \bbeta _{i}’ & =& (\beta _{i1},\ldots ,\beta _{is}), \quad i=1,\ldots ,k \end{eqnarray*}

where $\beta _{ij}$ is the slope parameter for the jth effect in the ith logit.

Consider the null hypothesis $H_0\colon \beta _{t+1}=\dots =\beta _{s}=0$, where $t < s$ for the cumulative response model, and $H_0\colon \beta _{i,t+1}=\dots =\beta _{is}=0, t<s, i=1,\ldots ,k$, for the generalized logit model. For the reduced model with t explanatory effects, let ${\widehat{\alpha }_{1}}, \ldots , {\widehat{\alpha }_{k}}$ be the MLEs of the unknown intercept parameters, let ${\widehat{\beta }}_{1}, \ldots , {\widehat{\beta }}_{t}$ be the MLEs of the unknown slope parameters, and let $\widehat{\bbeta }_{i(t)}’ = ({\widehat{\beta }}_{i1}, \ldots , {\widehat{\beta }}_{it}), i=1,\ldots ,k$, be those for the generalized logit model. The residual chi-square is the chi-square score statistic testing the null hypothesis $H_0$; that is, the residual chi-square is

\[ \mb{g}’({\widehat{\bbeta }}_{H_0})\bI ^{-1}({\widehat{\bbeta }}_{H_0})\mb{g} ({\widehat{\bbeta }}_{H_0}) \]

where for the cumulative response model ${\widehat{\bbeta }}_{H_0}=({\widehat{\alpha }_{1}},\ldots ,{\widehat{\alpha }_{k}}, {\widehat{\beta }}_{1},\ldots ,{\widehat{\beta }}_{t},0,\ldots ,0)’$, and for the generalized logit model ${\widehat{\bbeta }}_{H_0}=({\widehat{\alpha }_{1}},\ldots ,{\widehat{\alpha }_{k}}, {\widehat{\bbeta }_{1(t)}}’,\bm {0}_{(s-t)}’,\ldots {\widehat{\bbeta }_{k(t)}}’,\bm {0}_{(s-t)}’)’$, where $\bm {0}_{(s-t)}$ denotes a vector of $s-t$ zeros.

The residual chi-square has an asymptotic chi-square distribution with $s-t$ degrees of freedom ($k(s-t)$ for the generalized logit model). A special case is the global score chi-square, where the reduced model consists of the k intercepts and no explanatory effects. The global score statistic is displayed in the "Testing Global Null Hypothesis: BETA=0" table. The table is not produced when the NOFIT option is used, but the global score statistic is displayed.

Testing Individual Effects Not in the Model

These tests are performed when you specify SELECTION= FORWARD or STEPWISE, and are displayed when the DETAILS option is specified. In the displayed output, the tests are labeled "Score Chi-Square" in the "Analysis of Effects Eligible for Entry" table and in the "Summary of Stepwise (Forward) Selection" table. This section describes how the tests are calculated.

Suppose that k intercepts and t explanatory variables (say $v_1, \ldots , v_ t$) have been fit to a model and that $v_{t+1}$ is another explanatory variable of interest. Consider a full model with the k intercepts and $t+1$ explanatory variables ($v_1,\ldots ,v_ t,v_{t+1}$) and a reduced model with $v_{t+1}$ excluded. The significance of $v_{t+1}$ adjusted for $v_1,\ldots ,v_ t$ can be determined by comparing the corresponding residual chi-square with a chi-square distribution with one degree of freedom (k degrees of freedom for the generalized logit model).

Testing the Parallel Lines Assumption

For an ordinal response, PROC LOGISTIC performs a test of the parallel lines assumption. In the displayed output, this test is labeled "Score Test for the Equal Slopes Assumption" when the LINK= option is NORMIT or CLOGLOG. When LINK=LOGIT, the test is labeled as "Score Test for the Proportional Odds Assumption" in the output. For small sample sizes, this test might be too liberal (Stokes, Davis, and Koch 2000, p. 249). This section describes the methods used to calculate the test.

For this test the number of response levels, $k+1$, is assumed to be strictly greater than 2. Let Y be the response variable taking values $1, \ldots , k, k+1$. Suppose there are s explanatory variables. Consider the general cumulative model without making the parallel lines assumption

\[ g(\mbox{Pr}(Y\leq i~ |~ \mb{x}))= (1,\mb{x}’)\bbeta _ i, \quad 1 \leq i \leq k \]

where $g(\cdot )$ is the link function, and ${\bbeta }_ i=(\alpha _{i}, \beta _{i1}, \ldots , \beta _{is})^\prime $ is a vector of unknown parameters consisting of an intercept $\alpha _{i}$ and s slope parameters $\beta _{i1}, \ldots , \beta _{is}$. The parameter vector for this general cumulative model is

\[ \bbeta =(\bbeta ’_1,\ldots ,\bbeta ’_ k)’ \]

Under the null hypothesis of parallelism $H_0\colon \beta _{1m}=\beta _{2m}=\cdots =\beta _{km}, 1 \leq m \leq s $, there is a single common slope parameter for each of the s explanatory variables. Let $\beta _1,\dots ,\beta _ s$ be the common slope parameters. Let ${\widehat{\alpha }_{1}}, \ldots , {\widehat{\alpha }_{k}}$ and ${\widehat{\beta }}_1, \ldots , {\widehat{\beta }}_ s$ be the MLEs of the intercept parameters and the common slope parameters. Then, under $H_0$, the MLE of $\bbeta $ is

\[ {\widehat{\bbeta }_{H_0}}=({\widehat{\bbeta }}’_1,\ldots ,{\widehat{\bbeta }}’_ k)’ \quad \mbox{with} \quad {\widehat{\bbeta }}_ i=({\widehat{\alpha }_{i}},{\widehat{\beta }}_1,\ldots , {\widehat{\beta }}_ s)’ \quad 1 \leq i \leq k \]

and the chi-square score statistic $\mb{g}’({\widehat{\bbeta }}_{H_0})\bI ^{-1}({\widehat{\bbeta }}_{H_0})\mb{g}({\widehat{\bbeta }}_{H_0})$ has an asymptotic chi-square distribution with $s(k-1)$ degrees of freedom. This tests the parallel lines assumption by testing the equality of separate slope parameters simultaneously for all explanatory variables.