Comparison of Nested Models using pro... PreviousNext
Mplus Discussion > Structural Equation Modeling >
Message/Author
 Sally Czaja posted on Friday, August 18, 2006 - 11:09 am
I am using procedures for WLSMV to compare models that I feel sure are nested (exactly the same except one path is removed in the H0), but MPLUS is telling me that the H0 model is not nested in the H1 model. It reports the same degrees of freedom for both models, which I also find puzzling. Any advice?
These are the models:

H1
ANALYSIS:
TYPE=general missing h1;
MODEL:
y4 ON y1 y2 y3 x1;
y1 ON cont1 cont2 x1;
y2 ON cont3 x1;
y3 on cont1 cont3 x1;
y1 WITH y2;
y2 WITH y3;
y1 WITH y3;

H0
ANALYSIS:
TYPE=general missing h1;
MODEL:
y4 ON y1 y2 y3 x1@0;
y1 ON cont1 cont2 x1;
y2 ON cont3 x1;
y3 on cont1 cont3 x1;
y1 WITH y2;
y2 WITH y3;
y1 WITH y3;

Thank you!
 Bengt O. Muthen posted on Friday, August 18, 2006 - 4:59 pm
In using DIFFTEST, are you sure you are not putting the H0 model in the place that the H1 model should be? To check nesting, Mplus simply compares the fitting function value at the optimum (lower is better) - the model with a lower value cannot be nested within a model with a higher value. A model with one parameter fixed cannot have a lower (better) fitting function value than the corresponding model with that parameter free. The fitting function values can be seen in Tech5, left column.
 Sally Czaja posted on Monday, August 21, 2006 - 6:33 am
Thank you for your response. I feel certain that I am not switching the models but to clarify, I am saving the data file when I run the full model (which should be the better fit) and then running the DIFFTEST on the trimmed model with the fixed parameter. Is this correct?
 Bengt O. Muthen posted on Monday, August 21, 2006 - 6:41 am
Sounds right - see ex 12.12 in the User's Guide. Also check TECH1 to see the parameters used. If that doesn't help, you need to send your input, output, data, and license number to support@statmodel.com.
 Sally Czaja posted on Monday, August 21, 2006 - 7:18 am
Thank you! Ex 12.12 solved the problem. I was using the FILE IS command for saving the data file (rather than DIFFTEST IS).
 Gemma vilagut posted on Tuesday, May 08, 2007 - 6:22 am
I am using WLSMV to compare a model with dichotomous covariate (begenl) including direct effect (H1), with the same model without the covariate (H0). MPLUS does not report the Chi-sq comparison and it says that the H0 model is not nested in the H1 model.
Any help on this would be very much appreciated.
The models are:
H1:

MODEL:
ROLEF by fd4 fd7 fd8 fd9;
COGNIT by fd11a fd11b fd11c fd11d ;
MOBILT by fd13a fd13b fd13c;
SLFCARE by fd15a fd15b fd15c;
SOCIAL by fd17a fd17b fd17c fd17d fd17e;
PARTICI by fd18b fd18c fd18d fd18e fd20 fd21 fd22;

ROLEF COGNIT MOBILT SLFCARE SOCIAL PARTICI ON begenl;
fd20 ON begenl;

SAVEDATA: DIFFTEST IS modelh1.dat;

H0:
MODEL:
ROLEF by fd4 fd7 fd8 fd9;
COGNIT by fd11a fd11b fd11c fd11d ;
MOBILT by fd13a fd13b fd13c;
SLFCARE by fd15a fd15b fd15c;
SOCIAL by fd17a fd17b fd17c fd17d fd17e;
PARTICI by fd18b fd18c fd18d fd18e fd20 fd21 fd22;

ANALYSIS: DIFFTEST IS modelh1.dat;
 Linda K. Muthen posted on Tuesday, May 08, 2007 - 7:52 am
Nesting requires the same set of observed variables. You should add the following to the H0 model:

ROLEF COGNIT MOBILT SLFCARE SOCIAL PARTICI ON begenl@0;
fd20 ON begenl@0;
 Gemma vilagut posted on Tuesday, May 08, 2007 - 10:21 am
Thanks Linda.

I have tried your suggestion, but now the estimated parameters are not the same as those for the initial H0 model (whithout fixing the coefficients of begenl to 0).
Thanks very much! Gemma
 Linda K. Muthen posted on Tuesday, May 08, 2007 - 11:09 am
You will need to send your inputs, data, outputs, and license number to support@statmodel.com.
 ClaudiaBergomi posted on Wednesday, February 02, 2011 - 9:13 am
I am using WLSMV to test mediation and I wish to compare models with DIFFTEST. I am comparing a model with two IV, one mediator, ond (binary) DV:

1)
MIN WITH SYM;
alc ON MIN;
alc ON SYM;
alc ON EXP;
EXP ON MIN;
EXP ON SYM;
MODEL INDIRECT:
alc IND EXP MIN;
alc IND EXP SYM;

with a nested model without the IV MIN:
2)
MIN WITH SYM@0;
alc ON MIN@0;
alc ON SYM;
alc ON EXP;
EXP ON SYM;
EXP ON MIN@0;
MODEL INDIRECT:
alc IND EXP SYM;
alc IND EXP@0 MIN@0;

In order to show that adding MIN makes the model better.

My problem is that the nested model 2) has really bad fit indices, compared with the same model calculated without adding MIN and then constraining coeffiecients to 0 (and thus non-nested with 1)):

3)
EXP ON SYM;
alc ON SYM;
alc ON EXP;
MODEL INDIRECT:
alc IND EXP SYM;

Nevertheless, when I am describing the fits of my models I suppose I have to take the fit indices from 3) because those in 2) are 'artificially worstened'. But then, why am I allowed to calculate the DIFFTEST on the base of 2), which is of course worst?

Where I am doing something wrong?
Thank you.
 Bengt O. Muthen posted on Wednesday, February 02, 2011 - 4:58 pm
You should not have

MIN WITH SYM;

in model 1) or model 2) because they are exogenous variables and should be correlated as the default.
 ClaudiaBergomi posted on Thursday, February 03, 2011 - 5:16 am
Thank you. Now the fit in 2)=nested-one-VI has become better but it is still not as good as in 3)=non-nested-one-VI.

2)
alc ON MIN@0;
alc ON SYM;
alc ON EXP;
EXP ON SYM;
EXP ON MIN@0;
MODEL INDIRECT:
alc IND EXP SYM;
alc IND EXP@0 MIN@0;

Chi-Square Test of Model Fit
Value 78.147*
Degrees of Freedom 32
P-Value 0.0000
CFI 0.938
TLI 0.913

3)
EXP ON SYM;
alc ON SYM;
alc ON EXP;
MODEL INDIRECT:
alc IND EXP SYM;

Chi-Square Test of Model Fit
Value 19.108*
Degrees of Freedom 12
P-Value 0.0000
CFI 0.986
TLI 0.975

So I still have my previous doubt:
1. Is it right to describe fit of the model with only one VI using the indices from model 3)=non-nested and not from 2)=nested?
2. If so, then the question arise if it is right to compute the DIFFTEST between model 1)=two-VIs and 2), as this last has worst fit then 3) and thus the DIFFTEST is more likely to confirm my hypothesis that 1) is better.
 Bengt O. Muthen posted on Thursday, February 03, 2011 - 12:17 pm
When you say "VI", I think you mean "IV".

1. Model fit with one IV should have only one IV on the USEV list, otherwise you are also testing the zero restrictions for the other IV.

2. DIFFTEST can only be used when the same USEV variables are used in both models - so model 2) is the correct comparison model to the model with MIN having effects because this tests whether MIN has effects.
 Paul A.Tiffin posted on Wednesday, August 24, 2011 - 2:18 am
Dear Mplus team,

Am I right in assuming that a CFA with two postulated factors would not strictly be nested in a model with one factor, even if they had the same indicators?
 Bengt O. Muthen posted on Wednesday, August 24, 2011 - 12:26 pm
A one-factor model can be nested within a 2-factor model, not the other way around.
 Paul A.Tiffin posted on Thursday, August 25, 2011 - 5:22 am
Thanks Bengt,

I assume that a two factor model with perfect correlation specified between the two factors is then equivalent to a one factor model and the difference between the models' fit can then be tested (using DIFFTEST for WLSMV).

In that case what is the best way to specify perfect factor correlation?
would it be, say:
f1 ON f2@1; ?

Your help is appreciated.

Many thanks

Paul
 Bengt O. Muthen posted on Thursday, August 25, 2011 - 8:22 am
First, you will have to set the metric in the 2-factor model using factor variances @1. Then you say f1 with f2@1. See how that works - it gives a non-pos def factor covariance matrix. Note also that you can't have any cross-loadings in the 2-factor model.
 Paul A.Tiffin posted on Thursday, August 25, 2011 - 8:35 am
thanks- that makes senses

best wishes

Paul
 Paul A.Tiffin posted on Thursday, August 25, 2011 - 9:08 am
Dear Mplus team,

This approach did not seem to work in my case. However, there is some debate amongst methodologists whether models with varying numbers of factors are truly nested. Therefore it may be better to compare models using the BIC (ie.derived using MLR with montecarlo integration- the indicators are ordinal).

Is there any way of deriving a significance test for improvement in model fit using the BIC?

Many thanks

Paul
 Linda K. Muthen posted on Thursday, August 25, 2011 - 2:04 pm
Not that I know of.
 Katja Schlegel posted on Tuesday, October 04, 2011 - 3:33 am
Dear Mplus team,
I would like to compare the following models containing the same set of observed variables:
1. ERA by inq deg pla col irr peu tri joy
des fie sur amu sou int;
irr with col;
des with tri;
sou with pla;
joy with fie;

2. POS by pla joy fie amu sou int ;
NEG by inq deg col irr peu tri des;
irr with col;
des with tri;
sou with pla;
joy with fie;
sur with POS;
sur with NEG;

Are these models nested? If not, why? In this case, how can I use the BIC to compare the models if there is no significance test for this index?
Thank you very much!
 Linda K. Muthen posted on Tuesday, October 04, 2011 - 9:23 am
We do believe these models are nested. The lower BIC is the best BIC. You can do a statistical test by -2 times the loglikelihood difference which is distributed as chi-square.
 Mohamed Abdel-Raouf posted on Wednesday, October 05, 2011 - 12:21 pm
Hi,
I am using WLSMV to fit a model with a binary dependent. Now i am trying to compare another three models with my research model. ALl models are based on the same indicators, and i increase some paths in one model, decrease some in another, and using a mediating in a third. I am trying to compare those three models with my research model. I have used Difftest but i got a message saying that difference can not be computed because models are not nested. So how can i compare these NON nested models?
Thanks,
Mohamed
 Bengt O. Muthen posted on Wednesday, October 05, 2011 - 8:50 pm
Using BIC may be a good idea.
 Mohamed Abdel-Raouf posted on Thursday, October 06, 2011 - 4:43 am
Thanks but BIC does not appear in my output when i use WLSMV? how to calculate it?
Many thanks,
 Linda K. Muthen posted on Thursday, October 06, 2011 - 8:27 am
BIC is for maximum likelihood not weighted least squares. I would think some of your models are nested. Perhaps you are using DIFFTEST incorrectly. You can send the relevant outputs and your license number to support@statmodel.com if you want to check this out. Otherwise, I would see which model seems to have the best overall fit taking all fit indices into account.
 Mohamed Abdel-Raouf posted on Thursday, October 06, 2011 - 10:48 am
Thanks Linda, if i am going to use the last option of yours (I would see which model seems to have the best overall fit taking all fit indices into account),Can i have a reference to support this point of view?

Thanks indeed.
 Linda K. Muthen posted on Thursday, October 06, 2011 - 1:21 pm
I don't have a reference to support this point of view. It's simply the only alternative I can think of given you don't have BIC with WLSMV. You can probably get more opinions on SEMNET.
 Mohamed Abdel-Raouf posted on Friday, October 07, 2011 - 7:45 am
Hi Linda,
i am trying to test the following models:they base on the same indicators:

Variable: names are x1-x39 u1;
usevariable are x1-x39 u1;
Categorical is u1;

First Model:
f1 by x1-x4;
f2 by x5-x9;
f3 by x10-x15;
f4 by x16-x21;
f5 by x22-x25;
f6 by x26-x30;
f7 by x31-x36;
f8 by x37-x39;

f9 by f1-f3;
f10 by f5-f8;

f9 on f4;
u1 on f9 f10;

Second Model:
f1 by x1-x4;
f2 by x5-x9;
f3 by x10-x15;
f4 by x16-x21;
f5 by x22-x25;
f6 by x26-x30;
f7 by x31-x36;
f8 by x37-x39;

f9 by f1-f3;
f10 by f5-f8;

f10 on f4;
u1 on f9 f10;

Third model:
f1 by x1-x4;
f2 by x5-x9;
f3 by x10-x15;
f4 by x16-x21;
f5 by x22-x25;
f6 by x26-x30;
f7 by x31-x36;
f8 by x37-x39;

f9 by f1-f3;
f10 by f5-f8;

f9 f10 on f4;
u1 on f9 f10;

Do you think these are nested models? what is puzzling me is that chi-square value for all the models is different while df is the same in all the three models? why?

Thanks,
Mohamed
 Linda K. Muthen posted on Friday, October 07, 2011 - 11:01 am
If the degrees of freedom are the same, the models are not nested. Having the same degrees of freedom does not mean that chi-square will be the same. You may be interested in the following article:

Bentler, P.M. and Satorra, A. (2010). Testing model nesting and equivalence. Psychological Methods, Vol. 15, No. 2, 111-123.
 Bellinda King-Kallimanis posted on Tuesday, February 28, 2012 - 3:20 am
Hello,

I believe that my models are nested but I receive the message warning that they are not

H1 - MultiGrp (11grp)
ANALYSIS: ESTIMATOR=WLSMV;
PARAMETERIZATION=theta;
MODEL: F BY w* (L1)
s* (L2)
wg* (L3)
e* (L4);
[F@0];
F@1;
w-e@1;
MODEL S: F BY w* (L1)
s* (L2)
wg* (L3)
e* (L4);
[F*];
F*;
w-e@1;
SAVEDATA:
difftest IS scal.dat;

H0 - Free FL and threshold for S
ANALYSIS: DIFFTEST=scal.dat;
MODEL: F BY w* (L1)
s* (L2)
wg* (L3)
e* (L4);
[F@0];
F@1;
w-e@1;
MODEL S: F BY w* !free
s* (L2)
wg* (L3)
e* (L4);
[F*];
F*;
[w$1*]; !free
w-e@1;

Any help with would be really appreciated. I was wondering if I was getting a negative difference?
Cheers,
Bellinda
 Linda K. Muthen posted on Tuesday, February 28, 2012 - 10:16 am
I believe you need to remove w-e@1; from the first model.
 Yessenia Castro posted on Friday, May 11, 2012 - 1:42 pm
Hello,
I am having the same problem Gemma has detailed above (Gemma vilagut posted on Tuesday, May 08, 2007 - 10:21 am). My models differ by the removal of one path. When I simply remove the path from the input and try to run it, I get a message that the models are not nested. When I constrain the path to 0 as suggested in the response above (Linda K. Muthen posted on Tuesday, May 08, 2007 - 7:52 am), I get the diff test in my output but my fit indices and parameters estimates are slightly different than they would be if I ran a model that just had the path removed from the input. Could you help me clarify the source of this trouble? Thanks for your time.
 Linda K. Muthen posted on Saturday, May 12, 2012 - 2:42 pm
The difference is that the set of variables used in the analysis differs when you remove the path rather than fixing it at zero.
 Yessenia Castro posted on Monday, May 14, 2012 - 9:52 am
Which parameters are the most appropriate to report; the ones that result from removing the path or the ones that result from constraining the path to zero? My interest is in the former, but I'm not sure it's appropriate to report those if the difftest is associated with the latter. Thank you.
 Linda K. Muthen posted on Monday, May 14, 2012 - 10:11 am
You should report the models used in DIFFTEST.
 Hsien-Yuan Hsu posted on Wednesday, March 13, 2013 - 11:59 pm
Dear Dr. Muthen,

I am trying to conduct a model comparison between two model with MLR estimation.

My questions:
Q1. Are these two models nested with each other?"
Q2. If yes, is Satorra-Bentler Scaled Chi-Square applicable in this case?


Model 1:
MODEL:
%Within%
fw1 BY
y1@1
y2
y3;
fw2 BY
y4@1
y5
y6;
y1-y6;
fw1;
fw2;
fw1 WITH fw2;
%Between%
y1-y6 with y1-y6;

Model 2:
MODEL:
%Within%
y1-y6 WITH y1-y6 @0;
%Between%
y1-y6 WITH y1-y6;

Thanks for your reply in advance.

Best,
Hsien-Yuan
 Linda K. Muthen posted on Thursday, March 14, 2013 - 9:03 am
Yes and yes.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: