The NLP Procedure

Covariance Matrix

The COV= option must be specified to compute an approximate covariance matrix for the parameter estimates under asymptotic theory for least-squares, maximum-likelihood, or Bayesian estimation, with or without corrections for degrees of freedom as specified by the VARDEF= option.

Two groups of six different forms of covariance matrices (and therefore approximate standard errors) can be computed corresponding to the following two situations:

In either case, the following matrices are used:

g = \nabla^2 f(x)
j(f) = (\nabla f_1, ... ,\nabla f_m)    = ( \frac{\partial f_i}{\partial x_j} )
 jj(f) = j(f)^t j(f)
 v = j(f)^t {\rm diag}(f_i^2) j(f)
w = j(f)^t {\rm diag}(f_i^{\dagger}) j(f)
where
f_i^{\dagger} = \{ 0, & {if f_i=0} \    1 / f_i, & {otherwise}    .

For unconstrained minimization, or when none of the final parameter estimates are subjected to linear equality or active inequality constraints, the formulas of the six types of covariance matrices are as follows:

COV MIN or MAX Statement LSQ Statement
    
1M ({\rm \_nobs\_}/d) g^{-1} jj(f) g^{-1} ({\rm \_nobs\_}/d) g^{-1} v g^{-1}
2H ({\rm \_nobs\_}/d) g^{-1} \sigma^2 g^{-1}
3J (1/d) w^{-1} \sigma^2 jj(f)^{-1}
4B (1/d) g^{-1} w g^{-1} \sigma^2 g^{-1} jj(f) g^{-1}
5E ({\rm \_nobs\_}/d) jj(f)^{-1} (1/d) v^{-1}
6U ({\rm \_nobs\_}/d) w^{-1} jj(f) w^{-1} ({\rm \_nobs\_}/d) jj(f)^{-1} v jj(f)^{-1}

The value of  d depends on the VARDEF= option and on the value of the _NOBS_ variable:

d = \{ \max(1,{\_nobs\_} - {\_df\_}), &    {for vardef=df} \    {\_nobs\_}, & {for vardef=n}    .
where _DF_ is either set in the program statements or set by default to  n (the number of parameters) and _NOBS_ is either set in the program statements or set by default to nobs x mfun, where nobs is the number of observations in the data set and mfun is the number of functions listed in the LSQ, MIN, or MAX statement.

The value \sigma^2 depends on the specification of the SIGSQ= option and on the value of  d:

\sigma^2 = \{ sq x \_nobs\_ / d, & {if sigsq=sq\space is specified} \    2 f(x^*) / d, & {if sigsq= is not specified}    .
where  f(x^{*}) is the value of the objective function at the optimal parameter estimates  x^{*}.

The two groups of formulas distinguish between two situations:

For minimization, the inversion of the matrices in these formulas is done so that negative eigenvalues are considered zero, resulting always in a positive semidefinite covariance matrix.

In small samples, estimates of the covariance matrix based on asymptotic theory are often too small and should be used with caution.

If the final parameter estimates are subjected to  n_{act} \gt 0 linear equality or active linear inequality constraints, the formulas of the covariance matrices are modified similar to Gallant (1987) and Cramer (1986, p. 38) and additionally generalized for applications with singular matrices. In the constrained case, the value of  d used in the scalar factor \sigma^2 is defined by

d = \{ \max(1,{\_nobs\_} - {\_df\_} + n_{act}), &   {for vardef=df} \    {\_nobs\_}, & {for vardef=n}    .
where  n_{act} is the number of active constraints and _NOBS_ is set as in the unconstrained case.

For minimization, the covariance matrix should be positive definite; for maximization it should be negative definite. There are several options available to check for a rank deficiency of the covariance matrix:

For optimization techniques that do not use second-order derivatives, the covariance matrix is usually computed using finite-difference approximations of the derivatives. By specifying TECH=NONE, any of the covariance matrices can be computed using analytical derivatives. The covariance matrix specified by the COV= option can be displayed (using the PCOV option) and is written to the OUTEST= data set.

Previous Page | Next Page | Top of Page