ULS, GLS, and ML Discrepancy functions

In each estimation method, the parameter vector is estimated iteratively by a nonlinear optimization algorithm that minimizes a discrepancy function , which is also known as the fit function in the literature. With denoting the number of manifest variables, the sample covariance matrix for a sample with size , the vector of sample means, the fitted covariance matrix, and the vector of fitted means, the discrepancy function for unweighted least squares (ULS) estimation is:

The discrepancy function for generalized least squares estimation (GLS) is:

By default, is assumed so that is the normal theory generalized least squares discrepancy function.

The discrepancy function for normal-theory maximum likelihood estimation (ML) is:

In each of the discrepancy functions, and are considered to be given and and are functions of model parameter vector . That is:

Estimating by using a particular estimation method amounts to choosing a vector that minimizes the corresponding discrepancy function .

When the mean structures are not modeled or when the mean model is saturated by parameters, the last term of each fit function vanishes. That is, they become:

If, instead of being a covariance matrix, is a correlation matrix in the discrepancy functions, would naturally be interpreted as the fitted correlation matrix. Although whether is a covariance or correlation matrix makes no difference in minimizing the discrepancy functions, correlational analyses that use these functions are problematic because of the following issues:

• The diagonal of the fitted correlation matrix might contain values other than ones, which violates the requirement of being a correlation matrix.

• Whenever available, standard errors computed for correlation analysis in PROC TCALIS are straightforward generalizations of those of covariance analysis. In very limited cases these standard errors are good approximations. However, in general they are not even asymptotically correct.

• The model fit chi-square statistic for correlation analysis might not follow the theoretical distribution, thus making model fit testing difficult.

Despite these issues in correlation analysis, if your primary interest is to obtain the estimates in the correlation models, you might still find PROC TCALIS results for correlation analysis useful.

PROC TCALIS is primarily developed for the analysis of covariance structures, and hence COVARIANCE is the default option in the procedure. Depending on the nature of research, you can add the mean structures in the analysis. However, for the analysis of correlation structures, you can not add the mean structures for modeling in PROC TCALIS.

WLS and ADF Discrepancy Functions

Another important discrepancy function to consider is the weighted least squares (WLS) function. Let be a vector containing all nonredundant elements in the sample covariance matrix and sample mean vector , with representing the vector of the lower triangle elements of the symmetric matrix , stacking row by row. Similarly, let be a vector containing all nonredundant elements in the fitted covariance matrix and the fitted mean vector , with representing the vector of the lower triangle elements of the symmetric matrix .

The WLS discrepancy function is:

where is a positive definite symmetric weight matrix with rows and columns. Because is a function of model parameter vector under the structural model, you can write the WLS function as:

Suppose that converges to with increasing sample size, where and denote the population covariance matrix and mean vector, respectively. By default, the WLS weight matrix in PROC TCALIS is computed from the raw data as a consistent estimate of the asymptotic covariance matrix of , with partitioned as

where denotes the asymptotic covariance matrix for , denotes the asymptotic covariance matrix for , and denotes the asymptotic covariance matrix between and .

To compute the default weight matrix as a consistent estimate of , define a similar partition of the weight matrix as:

Each of the submatrices in the partition can now be computed from the raw data. First, define the biased sample covariance for variables and as:

and the sample fourth-order central moment for variables , , , and as:

The submatrices in are computed by:

Assuming the existence of finite eighth-order moments, this default weight matrix is a consistent but biased estimator of the asymptotic covariance matrix .

By using the ASYCOV option, you can use Browne’s (1984, formula (3.8)) unbiased estimator of as:

There is no guarantee that computed this way is positive semidefinite. However, the second part is of order and does not destroy the positive semidefinite first part for sufficiently large . For a large number of independent observations, default settings of the weight matrix result in asymptotically distribution-free parameter estimates with unbiased standard errors and a correct test statistic (Browne 1982, 1984).

With the default weight matrix computed by PROC TCALIS, the WLS estimation is also called as the asymptotically distribution-free (ADF) method. In fact, as options in PROC TCALIS, METHOD=WLS and METHOD=ADF are totally equivalent, even though WLS in general might include cases with special weight matrices other than the default weight matrix.

When the mean structures are not modeled, the WLS discrepancy function is still the same quadratic form statistic. However, with only the elements in covariance matrix being modeled, the dimensions of and are both reduced to , and the dimension of the weight matrix is now . That is, the WLS discrepancy function for covariance structure models is:

If is a correlation rather than a covariance matrix, the default setting of the is a consistent estimator of the asymptotic covariance matrix of (Browne and Shapiro 1986; DeLeeuw 1983), with and representing vectors of sample and population correlations, respectively. Elementwise, is expressed as:

where

and

The asymptotic variances of the diagonal elements of a correlation matrix are 0. That is,

for all . Therefore, the weight matrix computed this way is always singular. In this case, the discrepancy function for weighted least squares estimation is modified to:

where is the penalty weight specified by the WPENALTY= option and the are the elements of the inverse of the reduced weight matrix that contains only the nonzero rows and columns of the full weight matrix .

The second term is a penalty term to fit the diagonal elements of the correlation matrix . The default value of can be decreased or increased by the WPENALTY= option. The often used value of seems to be too small in many cases to fit the diagonal elements of a correlation matrix properly.

Note that when you model correlation structures, no mean structures can be modeled simultaneously in the same model.

DWLS Discrepancy Functions

Storing and inverting the huge weight matrix in WLS estimation requires considerable computer resources. A compromise is found by implementing the diagonally weighted least squares (DWLS) method that uses only the diagonal of the weight matrix from the WLS estimation in the following discrepancy function:

When only the covariance structures are modeled, the discrepancy function becomes:

For correlation models, the discrepancy function is:

where is the penalty weight specified by the WPENALTY= option. Note that no mean structures can be modeled simultaneously with correlation structures when using the DWLS method.

As the statistical properties of DWLS estimates are still not known, standard errors for estimates are not computed for the DWLS method.

Input Weight Matrices

In GLS, WLS, or DWLS estimation you can change from the default settings of weight matrices by using an INWGT= data set. The TCALIS procedure requires a positive definite weight matrix that has positive diagonal elements.

Multiple-Group Discrepancy Function

Suppose that there are independent groups in the analysis and , , ..., are the sample sizes for the groups. The overall discrepancy function is expressed as a weighted sum of individual discrepancy functions ’s for the groups:

where

is the weight of the discrepancy function for group , and

is the total number of observations in all groups. In PROC TCALIS, all discrepancy function ’s in the overall discrepancy function must belong to the same estimation method. You cannot specify different estimation methods for the groups in a multiple-group analysis. However, you can fit covariance and mean structures to a group but fit covariance structures only to another group.