Chi-square difference test using WLSMV PreviousNext
Mplus Discussion > Confirmatory Factor Analysis >
 Jennifer Hartwig posted on Tuesday, April 18, 2006 - 8:35 am
I am conducting a CFA using categorical (binary) variables (54 items) using WLSMV. I am attempting to evaluate the fit of 3 different models:

Model 1 consists of a general factor.

Model 2 is a two-factor model (based on item format: MC versus student-produced response items)

Model 3 is a four-factor model based on content areas.

My questions pertain to conducting a Chi-square difference test between Models 1 & 2 and Models 1 & 3:

1) I tried to use the command "DIFFTEST", but received an
error message indicating that my "models were not nested." Is there another method I can use to compare model fit?

2) I am unclear as to why my models are not nested. It seems as though both Model 2 and Model 3 would be nested in Model 1 (separately). I am using all 54 items in all three models.

Please advise.
 Linda K. Muthen posted on Tuesday, April 18, 2006 - 1:02 pm
Model 1 is nested in models 2 and 3. You may have your setup backwards. You should run model 1 last. If you are doing this and still having problems, send your input, data, output, and license number to
 Ruth Parslow posted on Sunday, January 21, 2007 - 7:56 pm
I am using DIFFTEST with WLMSV to compare the goodness of fit of 3 models based on 17 categorical items. (There are some theoretical reasons for comparing these different models.) In the first model the 17 all load on 1 factor; in the second model, 9 items are on 1 factor and 8 on a second. In the third model, also with 2 factors, 7 of the original 9 load on the first factor while the remaining 2 of Factor 1 now load on the second factor in addition to the 7 already specified in the second model. Is there any way I can compare the Goodness of fit between the second and third models since the message I get is that these are not nested. Can I use the difference between the two Chi-square difference measures, the first from comparing Models 1 and 2 and the second from comparing Models 1 and 3 as a further Chisquare difference measure?

 Linda K. Muthen posted on Monday, January 22, 2007 - 8:40 am
Your second and third models are not nested so using a difference test to compare their fit is not appropriate.
 Sanja Franic posted on Wednesday, November 18, 2009 - 5:26 am
I cannot use the "difftest" option in MPlus when also using the "model constraint" option. Is there any other way to calculate the chi-square difference, given that I am using the WLSMV estimator?
 Sanja Franic posted on Wednesday, November 18, 2009 - 6:05 am
I think I just found the answer to my own question. I use WLSM instead of WLSMV and use the scaling correction factors I get in the output to perform a shi-sq difference test.
 Alexander Tokarev posted on Thursday, August 31, 2017 - 11:52 am
Dear Professors Muthen,

I have a second-order latent variable in CFA, and I would like to see if the third-order latent factor is a better alternative of the construct. Both second-order factor and the third-order alternative have the same number of 34 ordinal indicators, so I am using WLSMV. The only difference is whether a 2nd or a 3rd-order construct is used. Hence I would like to ask if the DIFFTEST is appropriate in this situation? Or should I simply compare values of fit indices of both models and select a better fitting one?

Thank you

 Bengt O. Muthen posted on Thursday, August 31, 2017 - 4:54 pm
We would need to see the 2 models to be able to say - you can send to Support along with your license number. Or ask on SEMNET.
 Madison Aitken posted on Sunday, July 22, 2018 - 11:12 pm
Dear Drs. Muthen,
I am conducting a CFA testing the following models of ordinal data using WLSMV estimation:
1. 3 correlated factors
2. 3 correlated factors plus a general factor
3. 3 orthogonal factors plus a general factor

I am trying to compare models 2 and 3 with model 1 using difftest. The difftest for model 1 vs. model 2 runs successfully; however, for model 1 vs. model 3, I get the error:

I have conducted the difference test of model 1 vs. model 3 by running model 3 (orthogonal bifactor) with the save command:

difftest is orthbifacdiff.dat;

then running model 1 (correlated factors) with the following analysis command:


The orthogonal bifactor and 3 correlated factors models both ran without issues previously. I would be grateful for any suggestions you have about how I may be able to address this. Thank you in advance.
 Bengt O. Muthen posted on Monday, July 23, 2018 - 10:33 am
That message may appear due to non-nested models. First check that your models are nested. See the new option NESTED as described in the paper on our website under Papers, SEM, especially Section 4.2:

Asparouhov, T. & Muthén, B. (2018). Nesting and equivalence testing in Mplus. Technical Report. May 16, 2018. (Download scripts).
 Madison Aitken posted on Thursday, July 26, 2018 - 12:25 pm
Thank you, Dr. Muthen. I have now been able to resolve the issue. I appreciate your help!
Back to top
Add Your Message Here
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Options: Enable HTML code in message
Automatically activate URLs in message