I am curious to know what kind of independence model is used to generate the CFI for the following model.
TYPE = TWOLEVEL EFA 3 UW UB;
Note in the model described above that the between-level model is unstructured (i.e., df = 0). According to Hox (2002) and Ryu and West (2009) the perfect fit of the unstructured between-level model may affect the value of the comparative fit index (CFI) if the baseline independence model is in fact a within and between level independence model. This problem can be addressed by estimating an alternative multilevel partial independence model that consists of a unstructured between-level model with a within-level independence model. The chi-square value from this new baseline partial independence model may then be used to manually calculate the CFI for the 3 factor within-level EFA shown above in a way that is not influenced by the perfect fit of the saturated between-level model.
Is this the method that has been used to calculate the CFI in Mplus for multilevel EFA in those instances where one level is specified as being unstructured, and if this is not the method used what method is?
Can I ask a follow-up question here? Is it possible in Mplus to estimate the "partially saturated independence model" that Ryu and West (2009) describes using for calculating the modified CFI for this type of model?
Hello, Dr. Muthen indicated that an analyst can directly estimate the partially-saturated independence model and manually carry out the level-specific fit calculation for CFI from Ryu & West. I have attempted this approach and encountered a result that was not intuitive. To perform the within-level fit calculation, I directly specify a partially-saturated within-level independence model.
%between% Item1 with Item2 - Item5 ; Item2 with Item3 - Item5 ; Item3 with Item4 - Item5 ; Item4 with Item5; Item1; Item2; Item3; Item4; Item5; %within%
Have I properly estimated this partially-saturated independence model in MPlus? With this code I obtain the following output:
Chi-Square Test of Model Fit Value 34008.044* Degrees of Freedom 10
Chi-Square Test of Model Fit for the Baseline Model Value 26623.515 Degrees of Freedom 20
My concern is that the Chi-Square value has increased in this partially-saturated model compared to the original baseline model. I would have expected the partially-saturated model to provide better fit to the H1 Unrestricted model than the original baseline model since the between-level is saturated. My initial thought is that because the items are specified as categorical, the within-level independence model is not being properly specified in MPlus.
I assume you are using 2-level WLSMV. One can’t count on WLSMV chi-square values being ordered in line with the restrictiveness, although this large difference is a bit strange. Try WLSM. If that doesn't change matters, send relevant files and license number to Support.
I wanted to follow-up on your response. I was trying to obtain the chi-square for the partial-independence model to perform the level-specific fit evaluation for CFI (Ryu & West, 2009). I am assuming I have specified the model appropriately in MPlus.
The two-level WLSMV estimator produces a partially-saturated model chi-square that is larger than the original baseline model test. This is my finding across three different data sets.
You indicated that there is nothing wrong with the results and that DIFFTESTing gives the right conclusions. However, DIFFTEST cannot be done in type=twolevel.
My question is, can we trust this partially-saturated model chi-square for performing the recalculations of the CFI?
I have found I can replicate the original MPlus fit indices using the original baseline model chi-square through the calculations in Ryu & West (2009). So the logic seems to stand in the regular way.
My feeling though is that I should not trust this partially-saturated chi-square result even though the original CFI calculations seem to be straightforward.