The LOGISTIC Procedure
- Overview
- Getting Started
-
Syntax
PROC LOGISTIC Statement BY Statement CLASS Statement CONTRAST Statement EFFECT Statement EFFECTPLOT Statement ESTIMATE Statement EXACT Statement EXACTOPTIONS Statement FREQ Statement LSMEANS Statement LSMESTIMATE Statement MODEL Statement ODDSRATIO Statement OUTPUT Statement ROC Statement ROCCONTRAST Statement SCORE Statement SLICE Statement STORE Statement STRATA Statement TEST Statement UNITS Statement WEIGHT Statement -
Details
Missing Values Response Level Ordering Link Functions and the Corresponding Distributions Determining Observations for Likelihood Contributions Iterative Algorithms for Model Fitting Convergence Criteria Existence of Maximum Likelihood Estimates Effect-Selection Methods Model Fitting Information Generalized Coefficient of Determination Score Statistics and Tests Confidence Intervals for Parameters Odds Ratio Estimation Rank Correlation of Observed Responses and Predicted Probabilities Linear Predictor, Predicted Probability, and Confidence Limits Classification Table Overdispersion The Hosmer-Lemeshow Goodness-of-Fit Test Receiver Operating Characteristic Curves Testing Linear Hypotheses about the Regression Coefficients Regression Diagnostics Scoring Data Sets Conditional Logistic Regression Exact Conditional Logistic Regression Input and Output Data Sets Computational Resources Displayed Output ODS Table Names ODS Graphics
-
Examples
Stepwise Logistic Regression and Predicted Values Logistic Modeling with Categorical Predictors Ordinal Logistic Regression Nominal Response Data: Generalized Logits Model Stratified Sampling Logistic Regression Diagnostics ROC Curve, Customized Odds Ratios, Goodness-of-Fit Statistics, R-Square, and Confidence Limits Comparing Receiver Operating Characteristic Curves Goodness-of-Fit Tests and Subpopulations Overdispersion Conditional Logistic Regression for Matched Pairs Data Firth’s Penalized Likelihood Compared with Other Approaches Complementary Log-Log Model for Infection Rates Complementary Log-Log Model for Interval-Censored Survival Times Scoring Data Sets Using the LSMEANS Statement
- References
| Iterative Algorithms for Model Fitting |
Two iterative maximum likelihood algorithms are available in PROC LOGISTIC. The default is the Fisher scoring method, which is equivalent to fitting by iteratively reweighted least squares. The alternative algorithm is the Newton-Raphson method. Both algorithms give the same parameter estimates; however, the estimated covariance matrix of the parameter estimators can differ slightly. This is due to the fact that Fisher scoring is based on the expected information matrix while the Newton-Raphson method is based on the observed information matrix. In the case of a binary logit model, the observed and expected information matrices are identical, resulting in identical estimated covariance matrices for both algorithms. You can specify the TECHNIQUE= option to select a fitting algorithm, and specify the FIRTH option to perform a bias-reducing penalized maximum likelihood fit. Note for generalized logit models that only the Newton-Raphson technique is available. For conditional logistic regression, see the section Conditional Logistic Regression for a list of methods used.
Iteratively Reweighted Least Squares Algorithm (Fisher Scoring)
Consider the multinomial variable 
such that
 |
 |
With 
denoting the probability that the jth observation has response value i, the expected value of 
is 
where 
. The covariance matrix of is 
, which is the covariance matrix of a multinomial random variable for one trial with parameter vector 
. Let 
be the vector of regression parameters; in other words, 
. Let 
be the matrix of partial derivatives of with respect to . The estimating equation for the regression parameters is
 |
where 
, 
and 
are the weight and frequency of the 
th observation, and 
is a generalized inverse of 
. PROC LOGISTIC chooses as the inverse of the diagonal matrix with as the diagonal.
With a starting value of 
, the maximum likelihood estimate of is obtained iteratively as
 |
where , 
, and are evaluated at 
. The expression after the plus sign is the step size. If the likelihood evaluated at 
is less than that evaluated at , then is recomputed by step-halving or ridging as determined by the value of the RIDGING= option. The iterative scheme continues until convergence is obtained—that is, until is sufficiently close to . Then the maximum likelihood estimate of is 
.
The covariance matrix of 
is estimated by
 |
where 
and 
are, respectively, and evaluated at . 
is the information matrix, or the negative expected Hessian matrix, evaluated at .
By default, starting values are zero for the slope parameters, and for the intercept parameters, starting values are the observed cumulative logits (that is, logits of the observed cumulative proportions of response). Alternatively, the starting values can be specified with the INEST= option.
Newton-Raphson Algorithm
For cumulative models, let the parameter vector be 
, and for the generalized logit model let 
. The gradient vector and the Hessian matrix are given, respectively, by
 |
 |
 |
|||
 |
|
 |
where 
is the log likelihood for the th observation. With a starting value of , the maximum likelihood estimate of is obtained iteratively until convergence is obtained:
 |
where 
and 
are evaluated at . If the likelihood evaluated at is less than that evaluated at , then is recomputed by step-halving or ridging.
The covariance matrix of is estimated by
 |
where the observed information matrix 
is computed by evaluating at .
Firth’s Bias-Reducing Penalized Likelihood
Firth’s method is currently available only for binary logistic models. It replaces the usual score (gradient) equation
 |
where 
is the number of parameters in the model, with the modified score equation
 |
where the 
s are the 
th diagonal elements of the hat matrix 
and 
. The Hessian matrix is not modified by this penalty, and the optimization method is performed in the usual manner.