Model fit comparison mediation model
Message/Author
 Alexander Nobel posted on Friday, May 27, 2011 - 12:09 am
Hello,

I have a question about the following:
I have a path model with 4 IV's, 3 Mediators and one DV. Two of the four IV's are not significantly related to any of the mediators or the DV. Do I understand it corectly, that if a take another identical model except now without these two IV's, that these two models are not nested, since I have a different number of observed variables in these two models? And thus, that I can use the AIC to compare the two models, but not the Chi-sq. difference?

Kind regards, Alex
 Linda K. Muthen posted on Friday, May 27, 2011 - 5:42 am
It is correct that a minimum requirement for two models to be nested is that they share the same set of observed variables.
 Alexander Nobel posted on Friday, May 27, 2011 - 7:46 am
OK, clear, thanks a lot!
I've read that the AIC can be used to compare model fit when models are not nested? Can I then use the AIC to compare my two models, or should you still have the same set+number of variables to make a comparison between non-nested models?
 Bengt O. Muthen posted on Friday, May 27, 2011 - 4:30 pm
You can use AIC to compare models that are not nested but they have to have the same DVs - otherwise the AICs are not on the same scales.
 Jamin Day posted on Saturday, March 19, 2016 - 3:57 am
Regarding the use of AIC to compare non-nested models:

In our mediator analyses we are exploring one smaller model (2 IVs, 2 mediators, 1 DV), and a larger model which incorporates more of the constructs of theoretical interest (2 IVs, 4 mediators, 1 DV, with multiple pathways for indirect effects).

The AIC is 5063 for the smaller model and 5800 for the larger model. If were using AIC alone we would reject the larger model due to poorer fit. However other fit indices (RMSEA, CFI, SRMR) are poorer in the smaller model (e.g. CFI = 0.923, RMSEA = 0.085), and demonstrate good to adequate fit in the larger model (CFI = 0.952, RMSEA = 0.059).

I know AIC includes a penalty for model complexity, would it be a justifiable approach to retain the larger model as final due to the other fit indices and it being of more theoretical interest?
 Bengt O. Muthen posted on Saturday, March 19, 2016 - 4:13 pm
You cannot compare the smaller and larger models with respect to AIC because they have different dependent variables and AIC is therefore on different metrics. Note that the dependent variables include both mediators and outcomes.