Message/Author 

A. Dyrlund posted on Saturday, March 18, 2006  10:52 am



I am using the MLMV estimator for my CFA so I am trying to compute the chisquare difference test. I followed the manual by first running the analysis using the DIFFTEST savedata command and then running it a second time with the DIFFTEST analysis command. The file is correctly created by the savedata command but my output keeps telling me "THE CHISQUARE DIFFERENCE TEST COULD NOT BE COMPUTED BECAUSE THE H0 MODEL IS NOT NESTED IN THE H1 MODEL." Can anyone tell me what I'm doing wrong? 


The models must not be nested. If you send the inputs, outputs, data, and your license number to support@statodel.com, we can take a look at it. 

A. Dyrlund posted on Thursday, March 23, 2006  11:06 am



No, the models are not nested. I shouldn't be using a DIFFTEST. I was reading a warning in the output as an error. Thank you anyway. 


Hi I want to compare two nested models. my samples are large so I have used the WLS estimator. How do I do chi square difference testing? I cannot use the difftest option with the WLS estimator. 


Take the difference of the chisquare values and the difference of the degrees of freedom. 


Can I do that when my data is categorical? 


Yes. 


Dear Dr. Multhen, I'd like to run nested models with 1 factor and 17 categorical variables(WLSMV method). The commands are: First step: H1 (the less restrictive) DATA:FILE IS C:\Dokumentumok\Madat\RS_random_CFA_ASCII.dat; Format IS 20F3.0; VARIABLE: NAMES ARE y1y20; usevariables = y1y2 y5y12 y14y20; categorical is y1y2 y5y12 y14y20; missing is blank; ANALYSIS: Estimator = WLSMV; MODEL: F1 BY y1 y5 y6 y8 y10 y15 y18 y19 y2 y7 y9 y11 y12 y14 y16 y17 y20; savedata: difftest is C:\Dokumentumok\munkahelyi gep\1faktoralap.dat; Second step: H0 (the more restrictive) DATA: FILE IS C:\Dokumentumok\Madat\RS_random_CFA_ASCII.dat; Format IS 20F3.0; VARIABLE: NAMES ARE y1y20; usevariables = y1y2 y5y12 y14y20; categorical are y1y2 y5y12 y14y20; missing is blank; ANALYSIS: Estimator = WLSMV; DIFFTEST IS C:\Dokumentumok\munkahelyi gep\1faktoralap.dat; MODEL: F1 BY y1 y5 y6 y8 y10 y15 y18 y19 y2 y7 y9 y11 y12 y14 y16 y17 y20; y1 with y15; And the output is: the Chisquare difference test is could not be computed because the H0 model is not nested in the H1 model. I can't see what is wrong. Could you help me? Thank you for your help, Veronika 


You have the models reversed. Use the model in the second step to save DIFFTEST and the model in the first step to use it in the ANALYSIS command. 

Kaatje Kraft posted on Monday, September 23, 2013  11:13 am



I am attempting to run a difftest and I have received the following error message, "THE CHISQUARE DIFFERENCE TEST COULD NOT BE COMPUTED BECAUSE THE FILE CONTAINING INFORMATION ABOUT THE H1 MODEL HAS INSUFFICIENT DATA." I'm not sure how to interpret this error (in the first step I saved the data to a .dat file and used that as my reference file for the difftest analysis; when I ran the original analysis with the WSLMV estimator it terminated normally with no error messages). Help? Thank you 


The second model must have some other difference besides a restriction on a parameter in the first model. 

Kaatje Kraft posted on Tuesday, September 24, 2013  7:55 am



I apologize for not clarifying my 2nd modelI set all of the covariances to zero as per the MPlus manual instructions. Is that what you mean by a difference? 


Please send the two outputs and your license number to support@statmodel.com. 


Hi, I am running measurement invariance model with dichotomous items. I follow the steps suggested in the Topic 2 handout. When I test threshold invariance model, I get the following error message: THE MODEL ESTIMATION TERMINATED NORMALLY THE CHISQUARE DIFFERENCE TEST COULD NOT BE COMPUTED BECAUSE THE H0 MODEL MAY NOT BE NESTED IN THE H1 MODEL. DECREASING THE CONVERGENCE OPTION MAY RESOLVE THIS PROBLEM. DATA: FILE IS ALL.DAT; FORMAT IS 30F8.0; VARIABLE: NAMES ARE ID AGE DHH I1I27; USEVARIABLES ARE I1I27; CATEGORICAL ARE I1I27; MISSING ARE BLANK; GROUPING IS DHH (0=HEAR 1 = DHH); ANALYSIS: ESTIMATOR IS WLSMV; PARAMETERIZATION = THETA; DIFFTEST = METRIC.DAT; MODEL: ELIPIC BY I1* (1); . . BLEFREE BY I27* (27); [I1$1I27$1*]; I1I27@1; [ELIPICBLEFREE@0]; ELIPICBLEFREE@1; MODEL DHH: I1I27@1; [ELIPICBLEFREE*]; ELIPICBLEFREE*; SAVEDATA: DIFFTEST = SCALAR.DAT; 


Please send both full outputs and your license number to support@statmodel.com. 


Hi Dr. Muth¨¦n, I have tested MI (2 groups) for a fourfactor model with 21 binary indicators (WLMSV estimator). The total sample size is over 2,000. I have used DIFFTEST chisquare difference, CFI difference, RMSEA difference to evaluate the MI. The results are: chisquare difference is significant at ¦Á = .001, CFI difference = .008, RMSEA difference = .004. If only using CFI difference, RMSEA difference to interpret the results, it seems the model reaches the MI, but not according to the DIFFTEST result. Is the DIFFTEST chisquare difference influenced largely by the sample size? In my study, how much weight should I give DIFFTEST chisquare difference to interpret the MI, compare to CFI significance, RMSEA significance? Many thanks! Jiebing 


I don't believe there has been any work done on using differences in CFI and RMSEA for the weighted least squares estimator. I would look only at the DIFFTEST results in this case. 


Hi Dr. Muthen, Thank you very much for replying me back. I have more questions on this: (1) Is the adjusted chisquare and DIFFTEST using WLSMV estimator also influenced largely by the sample size? (2) Is it normally for researchers to report the adjusted chisquare and df in the result part of the study? (3) I have tested MGCFA (2 groups; N1 = 1,983, N2 = 395) for a fourfactor model with 21 binary indicators (WLMSV estimator). The results for configural model 1: chisquare (df = 193) = 702.249, p < .001; CFI = .943, TLI = .955, RMSEA = .047. The results for partial scalar model 2 (I constrained only the factor loading and threshold of items for one of the four factors): chisquare (df = 193) = 713.615, p < .001; CFI = .941, TLI = .954, RMSEA = .048. DIFFTEST results: chisquare difference (df difference = 3) = 10.829, p = .013. My question is that why the df for model 1 and model 2 are the same, also why in the DIFFTEST, the df difference is 3? For my model with large sample size, what is recommended alpha value for chisquare difference test? 


Which version of Mplus are you using? 


Dr. Muthen, Mplus 5.21 Many thanks! Jiebing 


In that version with WLSMV, you can interpret only the pvalue not the chisquare test statistics or the degrees of freedom. These are adjusted to obtain a correct pvalue. 


Many thanks! In what version with WLSMV, could I also refer to the chisquare test statistics and degrees of freedom? I think I have to report them in my study. 


You can give the values. But you must interpret only the pvalue. 


Thank you! 


Hi Dr. Muthen, I am attempting to run a difftest between a four bifactor model & four correlated factor model. I run into the following error: "THE MODEL ESTIMATION TERMINATED NORMALLY THE CHISQUARE COMPUTATION COULD NOT BE COMPLETED BECAUSE OF A SINGULAR MATRIX." My first model is: MODEL: Int by SRP7 SRP9 SRP10 SRP15 SRP19 SRP23 SRP26; Aff by SRP3 SRP8 SRP13 SRP16 SRP18 SRP24 SRP28; Life by SRP1 SRP4 SRP11 SRP14 SRP17 SRP21 SRP27; Ant by SRP20 SRP5 SRP6 SRP12 SRP22 SRP25 SRP29; G1 by SRP7 SRP9 SRP10 SRP15 SRP19 SRP23 SRP26 SRP3 SRP8 SRP13 SRP16 SRP18 SRP24 SRP28 SRP1 SRP4 SRP11 SRP14 SRP17 SRP21 SRP27 SRP20 SRP5 SRP6 SRP12 SRP22 SRP25 SRP29; G1 with Int@0; Int with Aff@0; Aff with Life@0; Life with Ant@0; G1 with Aff@0; G1 with Life@0; G1 with Ant@0;Int with Ant@0; Int with Life@0; Aff with Ant@0; My second model is: MODEL: F1 by SRP7 SRP9 SRP10 SRP15 SRP19 SRP23 SRP26; F2 by SRP3 SRP8 SRP13 SRP16 SRP18 SRP24 SRP28; F3 by SRP1 SRP4 SRP11 SRP14 SRP17 SRP21 SRP27; F4 by SRP20 SRP5 SRP6 SRP12 SRP22 SRP25 SRP29; Thank you in advance, Hailey 


Please send the output and your license number to support@statmodel.com. 

Jiebing Wang posted on Wednesday, January 13, 2016  10:49 am



Hello Dr. Muthen, I tested measurement invariance with WLSMV, it is binary data. The result of my two models is listed below: Ho, constraining all five items equally across two groups: chisquare = 1265.156 df = 442 p < .001 CFI = .939 TLI = .930 RMSEA = .040 [.037, .042] H1 (less restrictive), set free on one item while constraining equally on other 4 items across two groups: chisquare = 1269.287 df = 440 p < .001 CFI = .938 TLI = .929 RMSEA = .040 [.037, .042] DIFFTEST chisquare difference = 2.388 df difference = 2 p = .303 My questions: 1. since the Ho is nested in H1 (less restrictive), does it always that the chisquare of Ho is larger than that of H1? Is it possible Ho is smaller than that of H1, like what happened in my result? 2. which one do you prefer, for reporting the difftest, ∆¦Ö2 or ¦Ö2diff? Thank you very much! Jiebing 


1. The WLSMV chi2 functions differently than regular chi2: Q1 No. Q2 Yes. 2. You should focus on the difftest result. 

Jiebing Wang posted on Wednesday, January 13, 2016  1:45 pm



Many thanks! 

Back to top 