The Likelihood Function and Maximum-Likelihood Estimation
The log-likelihood function
![l(\theta , \phi ; y) = \log f(y ; \theta, \phi) = \frac{y \theta-b(\theta)}{a(\phi)} + c(y , \phi)](images/fit_fiteq57.gif)
can be expressed in terms of the mean
![\mu](images/fit_fiteq18.gif)
and the dispersion parameter
![\phi](images/fit_fiteq23.gif)
:
- Normal
![{l(\mu , \phi ; y) = -\frac{1}2 \log(\phi) -\frac{1}{2\phi} (y-\mu)^2} \hfil {for -\infty\lt y\lt\infty}](images/fit_fiteq58.gif)
- Inverse Gaussian
![{l(\mu , \phi ; y) = - \log( y^3 \phi) -\frac{(y-\mu)^2}{2 y \mu^2 \phi} } {for y\gt}](images/fit_fiteq59.gif)
- Gamma
![{l(\mu , \phi ; y) = -\log(y {\Gamma}(\frac{1}{\phi})) + \frac{1}{\phi} \log(\frac{y}{\mu\phi}) -\frac{y}{\mu\phi}} {for y\gt}](images/fit_fiteq60.gif)
- Poisson
for y = 0, 1, 2, ...
- Binomial
![{l(\mu , \phi ; y) = r \log(\mu) + (m-r) \log(1-\mu) }](images/fit_fiteq62.gif)
for y=r/m, r=0, 1, 2,..., m
![](../../../../common/61925/HTML/default/images/note.gif)
Note |
Some terms in the density function have been dropped in the log-likelihood function since they do not affect the estimation of the mean and scale parameters. |
SAS/INSIGHT software uses a ridge stabilized Newton-Raphson algorithm to maximize the log-likelihood function
l(
![\mu](images/fit_fiteq18.gif)
,
![\phi](images/fit_fiteq23.gif)
; y) with respect to the regression parameters. On the
rth iteration, the algorithm updates the parameter vector
b by
- b(r) = b(r-1) - H-1(r-1) u(r-1)
where
H is the Hessian matrix and
u is the gradient vector, both evaluated at
![{{\beta}= b_{(r-1)}}](images/fit_fiteq63.gif)
.
![H = ( h_{jk} ) = ( \frac{\partial^2l}{\partial \beta_{j} \partial \beta_{k}} )](images/fit_fiteq64.gif)
![u = ( u_{j} ) = ( \frac{\partial l}{\partial \beta_{j} } ).](images/fit_fiteq65.gif)
The Hessian matrix
H can be expressed as
- H = - X' Wo X
where
X is the design matrix,
Wo is a diagonal matrix with
ith diagonal element
![w_{oi} = w_{ei} + ( y_{i}- \mu_{i}) \frac{V_{i}{g_{i}"} + {V_{i}'} {g_{i}'}}{V^2_{i} ({g_{i}'})^3 a_{i}(\phi) }](images/fit_fiteq66.gif)
![w_{ei} = E( w_{oi}) = \frac{1}{a_{i}(\phi) V_{i} ({g_{i}'})^2 }](images/fit_fiteq67.gif)
where
gi is the link function,
Vi is the variance function, and the primes denote derivatives of
g and
V with respect to
![\mu](images/fit_fiteq18.gif)
.All values are evaluated at the current mean estimate
![{ \mu_{i}}](images/fit_fiteq3.gif)
.
![{ a_{i}(\phi) = \phi / w_{i}}](images/fit_fiteq68.gif)
,where
wi is the prior weight for the
ith observation.
SAS/INSIGHT software uses either the full Hessian matrix
H = -
X'
Wo X or the Fisher's scoring method in the maximum-likelihood estimation. In the Fisher's scoring method,
Wo is replaced by its expected value
We with
ith element
wei.
- H = X' We X
The estimated variance-covariance matrix of the parameter estimates is
![\hat{{{\Sigma}}} = - H^{-1}](images/fit_fiteq69.gif)
where
H is the Hessian matrix evaluated at the model parameter estimates.
The estimated correlation matrix of the parameter estimates is derived by scaling the estimated variance-covariance matrix to 1 on the diagonal.
![](../../../../common/61925/HTML/default/images/note.gif)
Note |
A warning message appears when the specified model fails to converge. The output tables, graphs, and variables are based on the results from the last iteration. |
Copyright © 2007 by SAS Institute Inc., Cary, NC, USA. All rights reserved.