PROC NLMIXED: Finite-Difference Approximations of Derivatives

The NLMIXED Procedure

Finite-Difference Approximations of Derivatives

The FD= and FDHESSIAN= options specify the use of finite-difference approximations of the derivatives. The FD= option specifies that all derivatives are approximated using function evaluations, and the FDHESSIAN= option specifies that second-order derivatives are approximated using gradient evaluations.

Computing derivatives by finite-difference approximations can be very time-consuming, especially for second-order derivatives based only on values of the objective function (FD= option). If analytical derivatives are difficult to obtain (for example, if a function is computed by an iterative process), you might consider one of the optimization techniques that use first-order derivatives only (QUANEW, DBLDOG, or CONGRA). In the expressions that follow, $\text{[math]}$ denotes the parameter vector, $\text{[math]}$ denotes the step size for the $\text{[math]}$ th parameter, and $\text{[math]}$ is a vector of zeros with a $\text{[math]}$ in the $\text{[math]}$ th position.

Forward-Difference Approximations

The forward-difference derivative approximations consume less computer time, but they are usually not as precise as approximations that use central-difference formulas.

For first-order derivatives, $\text{[math]}$ additional function calls are required:

$\text{[math]}$ $\text{[math]}$
For second-order derivatives based on function calls only (Dennis and Schnabel 1983, p. 80), $\text{[math]}$ additional function calls are required for dense Hessian:

$\text{[math]}$ $\text{[math]}$
For second-order derivatives based on gradient calls (Dennis and Schnabel 1983, p. 103), $\text{[math]}$ additional gradient calls are required:

$\text{[math]}$ $\text{[math]}$

Central-Difference Approximations

Central-difference approximations are usually more precise, but they consume more computer time than approximations that use forward-difference derivative formulas.

For first-order derivatives, $\text{[math]}$ additional function calls are required:

$\text{[math]}$ $\text{[math]}$
For second-order derivatives based on function calls only (Abramowitz and Stegun 1972, p. 884), $\text{[math]}$ additional function calls are required.

$\text{[math]}$ $\text{[math]}$

$\text{[math]}$ $\text{[math]}$
For second-order derivatives based on gradient calls, $\text{[math]}$ additional gradient calls are required:

$\text{[math]}$ $\text{[math]}$

You can use the FDIGITS= option to specify the number of accurate digits in the evaluation of the objective function. This specification is helpful in determining an appropriate interval size $\text{[math]}$ to be used in the finite-difference formulas.

The step sizes $\text{[math]}$ , $\text{[math]}$ are defined as follows:

For the forward-difference approximation of first-order derivatives that use function calls and second-order derivatives that use gradient calls, $\text{[math]}$ .
For the forward-difference approximation of second-order derivatives that use only function calls and all central-difference formulas, $\text{[math]}$ .

The value of $\text{[math]}$ is defined by the FDIGITS= option:

If you specify the number of accurate digits by using FDIGITS= $\text{[math]}$ , $\text{[math]}$ is set to $\text{[math]}$ .
If you do not specify the FDIGITS= option, $\text{[math]}$ is set to the machine precision $\text{[math]}$ .

Top of Page