Nonlinear Optimization Examples

Overview

The IML procedure offers a set of optimization subroutines for minimizing or maximizing a continuous nonlinear function f = f(x) of n parameters, where x = (x_1, ... ,x_n)^t. The parameters can be subject to boundary constraints and linear or nonlinear equality and inequality constraints. The following set of optimization subroutines is available:

NLPCGConjugate Gradient Method
NLPDDDouble Dogleg Method
NLPNMSNelder-Mead Simplex Method
NLPNRANewton-Raphson Method
NLPNRRNewton-Raphson Ridge Method
NLPQN(Dual) Quasi-Newton Method
NLPQUAQuadratic Optimization Method
NLPTRTrust-Region Method

The following subroutines are provided for solving nonlinear least squares problems:

NLPLMLevenberg-Marquardt Least Squares Method
NLPHQNHybrid Quasi-Newton Least Squares Methods

A least squares problem is a special form of minimization problem where the objective function is defined as a sum of squares of other (nonlinear) functions.

f(x) = \frac{1}2 \{ f_1^2(x) +  ...  + f_m^2(x) \}
Least squares problems can usually be solved more efficiently by the least squares subroutines than by the other optimization subroutines.

The following subroutines are provided for the related problems of computing finite difference approximations for first- and second-order derivatives and of determining a feasible point subject to boundary and linear constraints:

NLPFDDApproximate Derivatives by Finite Differences
NLPFEAFeasible Point Subject to Constraints

Each optimization subroutine works iteratively. If the parameters are subject only to linear constraints, all optimization and least squares techniques are feasible-point methods; that is, they move from feasible point x^{(k)} to a better feasible point x^{(k+1)} by a step in the search direction s^{(k)}, k=1,2,3, ...  . If you do not provide a feasible starting point x^{(0)}, the optimization methods call the algorithm used in the NLPFEA subroutine, which tries to compute a starting point that is feasible with respect to the boundary and linear constraints.

The NLPNMS and NLPQN subroutines permit nonlinear constraints on parameters. For problems with nonlinear constraints, these subroutines do not use a feasible-point method; instead, the algorithms begin with whatever starting point you specify, whether feasible or infeasible.

Each optimization technique requires a continuous objective function f = f(x), and all optimization subroutines except the NLPNMS subroutine require continuous first-order derivatives of the objective function f. If you do not provide the derivatives of f, they are approximated by finite-difference formulas. You can use the NLPFDD subroutine to check the correctness of analytical derivative specifications.

Most of the results obtained from the IML procedure optimization and least squares subroutines can also be obtained by using the OPTMODEL procedure or the NLP procedure in SAS/OR software.

The advantages of the IML procedure are as follows:

The advantages of the NLP procedure are as follows:

Previous Page | Next Page | Top of Page