The IRT Procedure (Experimental)

Example 51.2 Multidimensional Exploratory and Confirmatory IRT Models

This example illustrates how to use the IRT procedure to fit multidimensional exploratory and confirmatory IRT models. The data set that is introduced in Example 51.1 is also used here. Two more items, item9 and item10, are added to the data set. These two items are designed to measure subjects’ satisfaction with their friendships and their family life, respectively.

data IrtMulti;
   input item1-item10 @@;
   datalines;
1 0 0 0 1 1 2 1 2 1 1 1 1 1 1 3 3 3 3 3 0 1 0 0 1 1 1 1 1 1 1 0 0 1 0
1 2 3 2 2 0 0 0 0 0 1 1 1 1 1 1 0 0 1 0 1 3 3 1 2 0 0 0 0 0 1 1 3 3 2
0 0 1 0 0 1 2 2 3 2 0 1 0 0 1 1 1 2 2 2 0 0 0 0 0 2 2 3 3 2 0 1 0 1 0
2 3 3 3 3 0 0 1 0 1 1 2 3 2 3 1 1 1 1 1 2 2 3 2 2 0 0 0 0 1 1 2 2 3 1
1 0 1 1 1 2 3 3 2 3 0 1 0 0 1 1 2 3 3 3 1 0 1 1 1 2 3 3 3 3 0 1 0 1 1
3 2 3 3 2 1 1 1 0 0 1 3 3 2 1 1 1 0 0 1 2 3 3 3 3 0 1 1 1 1 1 2 1 2 3
1 0 0 1 1 3 1 1 1 1 1 0 0 0 1 1 1 3 3 1 0 0 0 0 1 1 3 3 3 3 0 0 0 0 0
1 1 1 3 2 1 0 0 0 0 1 3 3 3 3 1 1 0 1 1 3 1 1 3 3 1 0 1 1 1 1 3 1 1 1

   ... more lines ...   

3 3 1 3 2 0 0 0 1 0 1 3 2 2 1 0 0 0 0 1 1 2 2 2 3 1 0 1 0 1 2 2 3 2 1
1 0 0 1 1 1 2 2 3 1 0 1 0 1 0 1 3 1 1 1 0 1 1 0 1 3 3 3 3 2 1 0 1 0 0
1 2 1 1 1 1 0 1 1 0 1 3 3 1 3 1 1 0 1 0 2 2 2 2 3 1 1 0 1 1 3 2 3 2 2
0 0 0 1 0 2 2 3 1 2 0 0 0 1 0 2 3 3 3 2 0 1 0 0 1 2 2 1 2 1 
;

Now, suppose that previous research results suggest that two latent factors underlie these 10 items. However, knowledge about the factor structure is very limited. The first step you can take is to fit an exploratory IRT model by using two factors. This can be accomplished easily by submitting the following statements:

ods graphics on;
proc irt data=IrtMulti nfactor=2 plots=scree;
   var item1-item10;
run;

The first table that you want to check is the Eigenvalue table, shown in Output 51.2.1. There are only two eigenvalues greater than 1 in this example. This result, to some extent, suggests that two factors might be enough in this example. Output 51.2.2 include the scree and variance explained plots.

Output 51.2.1: Eigenvalues of the Polychoric Correlation matrix

The IRT Procedure

Eigenvalues of the Polychoric Correlation Matrix
  Eigenvalue Difference Proportion Cumulative
1 3.65750431 2.47883117 0.3658 0.3658
2 1.17867314 0.23738137 0.1179 0.4836
3 0.94129177 0.04672399 0.0941 0.5777
4 0.89456778 0.07508308 0.0895 0.6672
5 0.81948471 0.14774320 0.0819 0.7492
6 0.67174151 0.12081203 0.0672 0.8163
7 0.55092948 0.01698300 0.0551 0.8714
8 0.53394648 0.08647230 0.0534 0.9248
9 0.44747418 0.14308753 0.0447 0.9696
10 0.30438664   0.0304 1.0000


Output 51.2.2: Scree and Variance Explained Plots


If the optimization algorithm converges successfully, the original and rotated slope matrices are produced. The default rotation method is varimax. You can use the ROTATE= option in the PROC IRT statement to specify a different rotation method.

Output 51.2.3: Slope Matrix

Slope Matrix Estimate/StdErr/p-value
  _Factor1 _Factor2
item1
1.97958
0.46669
0.00001
0.00000
.
.
item2
1.80876
0.40091
0.00000
-0.01809
0.36716
0.48035
item3
1.40127
0.24627
0.00000
0.09672
0.26426
0.35719
item4
0.82625
0.16489
0.00000
0.10368
0.16652
0.26675
item5
0.59336
0.14659
0.00003
0.47262
0.16699
0.00233
item6
0.54533
0.12300
0.00000
0.23692
0.14686
0.05335
item7
1.03088
0.18495
0.00000
0.90169
0.21392
0.00001
item8
1.33497
0.26840
0.00000
1.63089
0.34774
0.00000
item9
0.78616
0.19392
0.00003
1.06630
0.24535
0.00001
item10
0.78577
0.17303
0.00000
1.06969
0.20167
0.00000

Rotated Slope Matrix
  _Factor1 _Factor2
item1 1.87474 0.63566
item2 1.71878 0.56368
item3 1.29600 0.54156
item4 0.74920 0.36351
item5 0.41017 0.63813
item6 0.44038 0.39948
item7 0.68674 1.18496
item8 0.74058 1.97319
item9 0.40213 1.26228
item10 0.40067 1.26536


This example uses the default varimax rotation. Output 51.2.3 shows the original and rotated slope matrices. The original slope matrix is presented in the stack matrix format. You can see that each cell has three values: parameter estimate, standard error, and p-value. Because standard errors and p-values are not available after rotation, the rotated slope matrix is displayed in the standard matrix format. From the rotated slope matrix, you can see that the first factor is mainly reflected by item1 to item4 and item6, and the second factor is mainly reflected by the rest of the items. The exploratory results suggest a hypothesis about the factor structure of the items. In practice, you might want to confirm this structure by a confirmatory analysis of the new data. However, for illustration purposes, the same data set is used here to demonstrate the confirmatory model fitting by using the following statements:

ods graphics on;
proc irt data=IrtMulti;
   var item1-item4 item6 item5 item7-item10;
   factor  Factor1->item1-item4 item6,
           Factor2->item5 item7-item10;
run;

Output 51.2.5 and Output 51.2.4 show the model fit statistics for the confirmatory model and the exploratory model, respectively. Output 51.2.6 shows the slope matrix from this confirmatory analysis. Because the number of response patterns in the data set, 500, is much lower than the total number of possible response patterns, $2^5 \times 3^5=7,776$, you cannot use Pearson’s chi-square or the likelihood ratio statistic to test the overall fit of the confirmatory model.

If you have two nested models, you can still use the likelihood ratio statistic to test the difference between these two models. For illustration purposes, assume that the exploratory model and the confirmatory model are two independent models. Because the confirmatory model is nested within the exploratory model, you can use the likelihood ratio test to compare the two models. The likelihood ratio test statistic is 264.8. The degree of freedom is 9. The corresponding p-value is 0, which suggests that the difference between these two models is significant. There are many options that you can use to improve the model fit. For this example, you can try to free some of the fixed slope parameters or free the covariance between the factors.

Output 51.2.4: Model Fit Statistics for Exploratory Model

The IRT Procedure

Model Fit Statistics
Log Likelihood -3993.389529
AIC (Smaller is Better) 8054.7790586
BIC (Smaller is Better) 8198.075734
Likelihood Ratio 2042.4837278


Output 51.2.5: Model Fit Statistics for Confirmatory Model

The IRT Procedure

Model Fit Statistics
Log Likelihood -4125.792484
AIC (Smaller is Better) 8301.5849679
BIC (Smaller is Better) 8406.9501703
Likelihood Ratio 2307.2896372


Output 51.2.6: Slope Matrix

Slope Matrix Estimate/StdErr/p-value
  Factor1 Factor2
item1
2.04932
0.41637
0.00000
0.00000
.
.
item2
1.77313
0.32285
0.00000
0.00000
.
.
item3
1.36097
0.22842
0.00000
0.00000
.
.
item4
0.80650
0.15641
0.00000
0.00000
.
.
item6
0.55226
0.12128
0.00000
0.00000
.
.
item5
0.00000
.
.
0.75216
0.14089
0.00000
item7
0.00000
.
.
1.36470
0.17608
0.00000
item8
0.00000
.
.
2.29997
0.37170
0.00000
item9
0.00000
.
.
1.33480
0.17139
0.00000
item10
0.00000
.
.
1.34809
0.17566
0.00000