Independence model for TWOLEVEL EFA 3...
Message/Author
 Jonathon Little posted on Sunday, February 27, 2011 - 8:08 am
I am curious to know what kind of independence model is used to generate the CFI for the following model.

TYPE = TWOLEVEL EFA 3 UW UB;

Note in the model described above that the between-level model is unstructured (i.e., df = 0). According to Hox (2002) and Ryu and West (2009) the perfect fit of the unstructured between-level model may affect the value of the comparative fit index (CFI) if the baseline independence model is in fact a within and between level independence model. This problem can be addressed by estimating an alternative multilevel partial independence model that consists of a unstructured between-level model with a within-level independence model. The chi-square value from this new baseline partial independence model may then be used to manually calculate the CFI for the 3 factor within-level EFA shown above in a way that is not influenced by the perfect fit of the saturated between-level model.

Is this the method that has been used to calculate the CFI in Mplus for multilevel EFA in those instances where one level is specified as being unstructured, and if this is not the method used what method is?
 Linda K. Muthen posted on Monday, February 28, 2011 - 11:27 am
For TWOLEVEL EFA the baseline model is the model of means, between variances, and
within variances.
 Fredrik Falkenström posted on Thursday, March 13, 2014 - 5:50 am
Can I ask a follow-up question here? Is it possible in Mplus to estimate the "partially saturated independence model" that Ryu and West (2009) describes using for calculating the modified CFI for this type of model?

Thank you very much!

Fredrik Falkenström
 Linda K. Muthen posted on Thursday, March 13, 2014 - 11:25 am
You could specify that model and compute CFI by hand. There is not option to change the baseline model.
 Benjamin Brumley posted on Friday, October 16, 2015 - 10:42 am
Hello, Dr. Muthen indicated that an analyst can directly estimate the partially-saturated independence model and manually carry out the level-specific fit calculation for CFI from Ryu & West. I have attempted this approach and encountered a result that was not intuitive. To perform the within-level fit calculation, I directly specify a partially-saturated within-level independence model.

%between%
Item1 with Item2 - Item5 ;
Item2 with Item3 - Item5 ;
Item3 with Item4 - Item5 ;
Item4 with Item5;
Item1;
Item2;
Item3;
Item4;
Item5;
%within%

Have I properly estimated this partially-saturated independence model in MPlus?
With this code I obtain the following output:

Chi-Square Test of Model Fit
Value 34008.044*
Degrees of Freedom 10

Chi-Square Test of Model Fit for the Baseline Model
Value 26623.515
Degrees of Freedom 20

My concern is that the Chi-Square value has increased in this partially-saturated model compared to the original baseline model. I would have expected the partially-saturated model to provide better fit to the H1 Unrestricted model than the original baseline model since the between-level is saturated. My initial thought is that because the items are specified as categorical, the within-level independence model is not being properly specified in MPlus.

Thank you very much!

Benjamin Brumley
 Bengt O. Muthen posted on Friday, October 16, 2015 - 1:29 pm
I assume you are using 2-level WLSMV. One can’t count on WLSMV chi-square values being ordered in line with the restrictiveness, although this large difference is a bit strange. Try WLSM. If that doesn't change matters, send relevant files and license number to Support.
 Benjamin Brumley posted on Friday, October 16, 2015 - 2:57 pm
Hi Dr. Muthen,

Thank you for the suggestion. Yes, it is a two-level model with the WLSMV estimator.

I reran the model using WLSM and the results look more reasonable to me.

Chi-Square Test of Model Fit

Value 50799.673*
Degrees of Freedom 10

Chi-Square Test of Model Fit for the Baseline Model

Value 50934.805
Degrees of Freedom 20

If we use this solution, I am assuming we would also need to rerun our 'target' models using the WLSM estimator as well?

Thank you again for your help.
 Bengt O. Muthen posted on Friday, October 16, 2015 - 5:44 pm
Note that there is nothing wrong with using the WLSMV results. DIFFTESTing gives the right conclusions.
 Jonathon Little posted on Friday, October 16, 2015 - 9:59 pm
I notice that DIFFTEST is not available for TYPE = TWOLEVEL.

Something for the Mplus 7.4 wish list perhaps.
 Linda K. Muthen posted on Saturday, October 17, 2015 - 3:18 pm
Perhaps later. Not for 7.4.
 Benjamin Brumley posted on Thursday, October 29, 2015 - 11:37 am
Hi Dr. Muthen,

I wanted to follow-up on your response. I was trying to obtain the chi-square for the partial-independence model to perform the level-specific fit evaluation for CFI (Ryu & West, 2009). I am assuming I have specified the model appropriately in MPlus.

The two-level WLSMV estimator produces a partially-saturated model chi-square that is larger than the original baseline model test. This is my finding across three different data sets.

You indicated that there is nothing wrong with the results and that DIFFTESTing gives the right conclusions. However, DIFFTEST cannot be done in type=twolevel.

My question is, can we trust this partially-saturated model chi-square for performing the recalculations of the CFI?

I have found I can replicate the original MPlus fit indices using the original baseline model chi-square through the calculations in Ryu & West (2009). So the logic seems to stand in the regular way.

My feeling though is that I should not trust this partially-saturated chi-square result even though the original CFI calculations seem to be straightforward.

Thank you very much,

Benjamin Brumley
 Bengt O. Muthen posted on Thursday, October 29, 2015 - 5:33 pm
Why not use WLSM here?