Message/Author 

Sally Czaja posted on Friday, August 18, 2006  11:09 am



I am using procedures for WLSMV to compare models that I feel sure are nested (exactly the same except one path is removed in the H0), but MPLUS is telling me that the H0 model is not nested in the H1 model. It reports the same degrees of freedom for both models, which I also find puzzling. Any advice? These are the models: H1 ANALYSIS: TYPE=general missing h1; MODEL: y4 ON y1 y2 y3 x1; y1 ON cont1 cont2 x1; y2 ON cont3 x1; y3 on cont1 cont3 x1; y1 WITH y2; y2 WITH y3; y1 WITH y3; H0 ANALYSIS: TYPE=general missing h1; MODEL: y4 ON y1 y2 y3 x1@0; y1 ON cont1 cont2 x1; y2 ON cont3 x1; y3 on cont1 cont3 x1; y1 WITH y2; y2 WITH y3; y1 WITH y3; Thank you! 


In using DIFFTEST, are you sure you are not putting the H0 model in the place that the H1 model should be? To check nesting, Mplus simply compares the fitting function value at the optimum (lower is better)  the model with a lower value cannot be nested within a model with a higher value. A model with one parameter fixed cannot have a lower (better) fitting function value than the corresponding model with that parameter free. The fitting function values can be seen in Tech5, left column. 

Sally Czaja posted on Monday, August 21, 2006  6:33 am



Thank you for your response. I feel certain that I am not switching the models but to clarify, I am saving the data file when I run the full model (which should be the better fit) and then running the DIFFTEST on the trimmed model with the fixed parameter. Is this correct? 


Sounds right  see ex 12.12 in the User's Guide. Also check TECH1 to see the parameters used. If that doesn't help, you need to send your input, output, data, and license number to support@statmodel.com. 

Sally Czaja posted on Monday, August 21, 2006  7:18 am



Thank you! Ex 12.12 solved the problem. I was using the FILE IS command for saving the data file (rather than DIFFTEST IS). 


I am using WLSMV to compare a model with dichotomous covariate (begenl) including direct effect (H1), with the same model without the covariate (H0). MPLUS does not report the Chisq comparison and it says that the H0 model is not nested in the H1 model. Any help on this would be very much appreciated. The models are: H1: MODEL: ROLEF by fd4 fd7 fd8 fd9; COGNIT by fd11a fd11b fd11c fd11d ; MOBILT by fd13a fd13b fd13c; SLFCARE by fd15a fd15b fd15c; SOCIAL by fd17a fd17b fd17c fd17d fd17e; PARTICI by fd18b fd18c fd18d fd18e fd20 fd21 fd22; ROLEF COGNIT MOBILT SLFCARE SOCIAL PARTICI ON begenl; fd20 ON begenl; SAVEDATA: DIFFTEST IS modelh1.dat; H0: MODEL: ROLEF by fd4 fd7 fd8 fd9; COGNIT by fd11a fd11b fd11c fd11d ; MOBILT by fd13a fd13b fd13c; SLFCARE by fd15a fd15b fd15c; SOCIAL by fd17a fd17b fd17c fd17d fd17e; PARTICI by fd18b fd18c fd18d fd18e fd20 fd21 fd22; ANALYSIS: DIFFTEST IS modelh1.dat; 


Nesting requires the same set of observed variables. You should add the following to the H0 model: ROLEF COGNIT MOBILT SLFCARE SOCIAL PARTICI ON begenl@0; fd20 ON begenl@0; 


Thanks Linda. I have tried your suggestion, but now the estimated parameters are not the same as those for the initial H0 model (whithout fixing the coefficients of begenl to 0). Thanks very much! Gemma 


You will need to send your inputs, data, outputs, and license number to support@statmodel.com. 


I am using WLSMV to test mediation and I wish to compare models with DIFFTEST. I am comparing a model with two IV, one mediator, ond (binary) DV: 1) MIN WITH SYM; alc ON MIN; alc ON SYM; alc ON EXP; EXP ON MIN; EXP ON SYM; MODEL INDIRECT: alc IND EXP MIN; alc IND EXP SYM; with a nested model without the IV MIN: 2) MIN WITH SYM@0; alc ON MIN@0; alc ON SYM; alc ON EXP; EXP ON SYM; EXP ON MIN@0; MODEL INDIRECT: alc IND EXP SYM; alc IND EXP@0 MIN@0; In order to show that adding MIN makes the model better. My problem is that the nested model 2) has really bad fit indices, compared with the same model calculated without adding MIN and then constraining coeffiecients to 0 (and thus nonnested with 1)): 3) EXP ON SYM; alc ON SYM; alc ON EXP; MODEL INDIRECT: alc IND EXP SYM; Nevertheless, when I am describing the fits of my models I suppose I have to take the fit indices from 3) because those in 2) are 'artificially worstened'. But then, why am I allowed to calculate the DIFFTEST on the base of 2), which is of course worst? Where I am doing something wrong? Thank you. 


You should not have MIN WITH SYM; in model 1) or model 2) because they are exogenous variables and should be correlated as the default. 


Thank you. Now the fit in 2)=nestedoneVI has become better but it is still not as good as in 3)=nonnestedoneVI. 2) alc ON MIN@0; alc ON SYM; alc ON EXP; EXP ON SYM; EXP ON MIN@0; MODEL INDIRECT: alc IND EXP SYM; alc IND EXP@0 MIN@0; ChiSquare Test of Model Fit Value 78.147* Degrees of Freedom 32 PValue 0.0000 CFI 0.938 TLI 0.913 3) EXP ON SYM; alc ON SYM; alc ON EXP; MODEL INDIRECT: alc IND EXP SYM; ChiSquare Test of Model Fit Value 19.108* Degrees of Freedom 12 PValue 0.0000 CFI 0.986 TLI 0.975 So I still have my previous doubt: 1. Is it right to describe fit of the model with only one VI using the indices from model 3)=nonnested and not from 2)=nested? 2. If so, then the question arise if it is right to compute the DIFFTEST between model 1)=twoVIs and 2), as this last has worst fit then 3) and thus the DIFFTEST is more likely to confirm my hypothesis that 1) is better. 


When you say "VI", I think you mean "IV". 1. Model fit with one IV should have only one IV on the USEV list, otherwise you are also testing the zero restrictions for the other IV. 2. DIFFTEST can only be used when the same USEV variables are used in both models  so model 2) is the correct comparison model to the model with MIN having effects because this tests whether MIN has effects. 


Dear Mplus team, Am I right in assuming that a CFA with two postulated factors would not strictly be nested in a model with one factor, even if they had the same indicators? 


A onefactor model can be nested within a 2factor model, not the other way around. 


Thanks Bengt, I assume that a two factor model with perfect correlation specified between the two factors is then equivalent to a one factor model and the difference between the models' fit can then be tested (using DIFFTEST for WLSMV). In that case what is the best way to specify perfect factor correlation? would it be, say: f1 ON f2@1; ? Your help is appreciated. Many thanks Paul 


First, you will have to set the metric in the 2factor model using factor variances @1. Then you say f1 with f2@1. See how that works  it gives a nonpos def factor covariance matrix. Note also that you can't have any crossloadings in the 2factor model. 


thanks that makes senses best wishes Paul 


Dear Mplus team, This approach did not seem to work in my case. However, there is some debate amongst methodologists whether models with varying numbers of factors are truly nested. Therefore it may be better to compare models using the BIC (ie.derived using MLR with montecarlo integration the indicators are ordinal). Is there any way of deriving a significance test for improvement in model fit using the BIC? Many thanks Paul 


Not that I know of. 


Dear Mplus team, I would like to compare the following models containing the same set of observed variables: 1. ERA by inq deg pla col irr peu tri joy des fie sur amu sou int; irr with col; des with tri; sou with pla; joy with fie; 2. POS by pla joy fie amu sou int ; NEG by inq deg col irr peu tri des; irr with col; des with tri; sou with pla; joy with fie; sur with POS; sur with NEG; Are these models nested? If not, why? In this case, how can I use the BIC to compare the models if there is no significance test for this index? Thank you very much! 


We do believe these models are nested. The lower BIC is the best BIC. You can do a statistical test by 2 times the loglikelihood difference which is distributed as chisquare. 


Hi, I am using WLSMV to fit a model with a binary dependent. Now i am trying to compare another three models with my research model. ALl models are based on the same indicators, and i increase some paths in one model, decrease some in another, and using a mediating in a third. I am trying to compare those three models with my research model. I have used Difftest but i got a message saying that difference can not be computed because models are not nested. So how can i compare these NON nested models? Thanks, Mohamed 


Using BIC may be a good idea. 


Thanks but BIC does not appear in my output when i use WLSMV? how to calculate it? Many thanks, 


BIC is for maximum likelihood not weighted least squares. I would think some of your models are nested. Perhaps you are using DIFFTEST incorrectly. You can send the relevant outputs and your license number to support@statmodel.com if you want to check this out. Otherwise, I would see which model seems to have the best overall fit taking all fit indices into account. 


Thanks Linda, if i am going to use the last option of yours (I would see which model seems to have the best overall fit taking all fit indices into account),Can i have a reference to support this point of view? Thanks indeed. 


I don't have a reference to support this point of view. It's simply the only alternative I can think of given you don't have BIC with WLSMV. You can probably get more opinions on SEMNET. 


Hi Linda, i am trying to test the following models:they base on the same indicators: Variable: names are x1x39 u1; usevariable are x1x39 u1; Categorical is u1; First Model: f1 by x1x4; f2 by x5x9; f3 by x10x15; f4 by x16x21; f5 by x22x25; f6 by x26x30; f7 by x31x36; f8 by x37x39; f9 by f1f3; f10 by f5f8; f9 on f4; u1 on f9 f10; Second Model: f1 by x1x4; f2 by x5x9; f3 by x10x15; f4 by x16x21; f5 by x22x25; f6 by x26x30; f7 by x31x36; f8 by x37x39; f9 by f1f3; f10 by f5f8; f10 on f4; u1 on f9 f10; Third model: f1 by x1x4; f2 by x5x9; f3 by x10x15; f4 by x16x21; f5 by x22x25; f6 by x26x30; f7 by x31x36; f8 by x37x39; f9 by f1f3; f10 by f5f8; f9 f10 on f4; u1 on f9 f10; Do you think these are nested models? what is puzzling me is that chisquare value for all the models is different while df is the same in all the three models? why? Thanks, Mohamed 


If the degrees of freedom are the same, the models are not nested. Having the same degrees of freedom does not mean that chisquare will be the same. You may be interested in the following article: Bentler, P.M. and Satorra, A. (2010). Testing model nesting and equivalence. Psychological Methods, Vol. 15, No. 2, 111123. 


Hello, I believe that my models are nested but I receive the message warning that they are not H1  MultiGrp (11grp) ANALYSIS: ESTIMATOR=WLSMV; PARAMETERIZATION=theta; MODEL: F BY w* (L1) s* (L2) wg* (L3) e* (L4); [F@0]; F@1; we@1; MODEL S: F BY w* (L1) s* (L2) wg* (L3) e* (L4); [F*]; F*; we@1; SAVEDATA: difftest IS scal.dat; H0  Free FL and threshold for S ANALYSIS: DIFFTEST=scal.dat; MODEL: F BY w* (L1) s* (L2) wg* (L3) e* (L4); [F@0]; F@1; we@1; MODEL S: F BY w* !free s* (L2) wg* (L3) e* (L4); [F*]; F*; [w$1*]; !free we@1; Any help with would be really appreciated. I was wondering if I was getting a negative difference? Cheers, Bellinda 


I believe you need to remove we@1; from the first model. 


Hello, I am having the same problem Gemma has detailed above (Gemma vilagut posted on Tuesday, May 08, 2007  10:21 am). My models differ by the removal of one path. When I simply remove the path from the input and try to run it, I get a message that the models are not nested. When I constrain the path to 0 as suggested in the response above (Linda K. Muthen posted on Tuesday, May 08, 2007  7:52 am), I get the diff test in my output but my fit indices and parameters estimates are slightly different than they would be if I ran a model that just had the path removed from the input. Could you help me clarify the source of this trouble? Thanks for your time. 


The difference is that the set of variables used in the analysis differs when you remove the path rather than fixing it at zero. 


Which parameters are the most appropriate to report; the ones that result from removing the path or the ones that result from constraining the path to zero? My interest is in the former, but I'm not sure it's appropriate to report those if the difftest is associated with the latter. Thank you. 


You should report the models used in DIFFTEST. 


Dear Dr. Muthen, I am trying to conduct a model comparison between two model with MLR estimation. My questions: Q1. Are these two models nested with each other?" Q2. If yes, is SatorraBentler Scaled ChiSquare applicable in this case? Model 1: MODEL: %Within% fw1 BY y1@1 y2 y3; fw2 BY y4@1 y5 y6; y1y6; fw1; fw2; fw1 WITH fw2; %Between% y1y6 with y1y6; Model 2: MODEL: %Within% y1y6 WITH y1y6 @0; %Between% y1y6 WITH y1y6; Thanks for your reply in advance. Best, HsienYuan 


Yes and yes. 

Back to top 