PROC SURVEYLOGISTIC: Iterative Algorithms for Model Fitting

The SURVEYLOGISTIC Procedure

Iterative Algorithms for Model Fitting

Two iterative maximum likelihood algorithms are available in PROC SURVEYLOGISTIC to obtain the pseudo-estimate $\text{[math]}$ of the model parameter $\text{[math]}$ . The default is the Fisher scoring method, which is equivalent to fitting by iteratively reweighted least squares. The alternative algorithm is the Newton-Raphson method. Both algorithms give the same parameter estimates; the covariance matrix of $\text{[math]}$ is estimated in the section Variance Estimation. For a generalized logit model, only the Newton-Raphson technique is available. You can use the TECHNIQUE= option in the MODEL statement to select a fitting algorithm.

Iteratively Reweighted Least Squares Algorithm (Fisher Scoring)

Let $\text{[math]}$ be the response variable that takes values $\text{[math]}$ $\text{[math]}$ . Let $\text{[math]}$ index all observations and $\text{[math]}$ be the value of response for the $\text{[math]}$ th observation. Consider the multinomial variable $\text{[math]}$ such that

$\text{[math]}$

and $\text{[math]}$ . With $\text{[math]}$ denoting the probability that the jth observation has response value i, the expected value of $\text{[math]}$ is $\text{[math]}$ , and $\text{[math]}$ . The covariance matrix of $\text{[math]}$ is $\text{[math]}$ , which is the covariance matrix of a multinomial random variable for one trial with parameter vector $\text{[math]}$ . Let $\text{[math]}$ be the vector of regression parameters—for example, $\text{[math]}$ for cumulative logit model. Let $\text{[math]}$ be the matrix of partial derivatives of $\text{[math]}$ with respect to $\text{[math]}$ . The estimating equation for the regression parameters is

$\text{[math]}$

where $\text{[math]}$ , and $\text{[math]}$ and $\text{[math]}$ are the WEIGHT and FREQ values of the $\text{[math]}$ th observation.

With a starting value of $\text{[math]}$ , the pseudo-estimate of $\text{[math]}$ is obtained iteratively as

$\text{[math]}$

where $\text{[math]}$ , $\text{[math]}$ , and $\text{[math]}$ are evaluated at the $\text{[math]}$ th iteration $\text{[math]}$ . The expression after the plus sign is the step size. If the log likelihood evaluated at $\text{[math]}$ is less than that evaluated at $\text{[math]}$ , then $\text{[math]}$ is recomputed by step-halving or ridging. The iterative scheme continues until convergence is obtained—that is, until $\text{[math]}$ is sufficiently close to $\text{[math]}$ . Then the maximum likelihood estimate of $\text{[math]}$ is $\text{[math]}$ .

By default, starting values are zero for the slope parameters, and starting values are the observed cumulative logits (that is, logits of the observed cumulative proportions of response) for the intercept parameters. Alternatively, the starting values can be specified with the INEST= option in the PROC SURVEYLOGISTIC statement.

Newton-Raphson Algorithm

Let

	$\text{[math]}$	$\text{[math]}$	$\text{[math]}$
	$\text{[math]}$	$\text{[math]}$	$\text{[math]}$

be the gradient vector and the Hessian matrix, where $\text{[math]}$ is the log likelihood for the $\text{[math]}$ th observation. With a starting value of $\text{[math]}$ , the pseudo-estimate $\text{[math]}$ of $\text{[math]}$ is obtained iteratively until convergence is obtained:

$\text{[math]}$

where $\text{[math]}$ and $\text{[math]}$ are evaluated at the $\text{[math]}$ th iteration $\text{[math]}$ . If the log likelihood evaluated at $\text{[math]}$ is less than that evaluated at $\text{[math]}$ , then $\text{[math]}$ is recomputed by step-halving or ridging. The iterative scheme continues until convergence is obtained—that is, until $\text{[math]}$ is sufficiently close to $\text{[math]}$ . Then the maximum likelihood estimate of $\text{[math]}$ is $\text{[math]}$ .

Top of Page