
Message/Author 


I am validating a previously tested structural equation model on a new dataset. The outcome variable is binary (smoking or abstinent). I would like to determine if alternate theory based models are better than the previously developed and tested model. Models are not nested, but do include the same variables (the configuration of the models differ). I ran the initial model using WLSMV. However, I am now using ML to estimate the model so that I can obtain the BICs for each model. I have not been able to find the appropriate procedure for determining whether one BIC is better than another. Do I merely subtract one Mplus BIC from another? If the difference is greater than 10 is this strong evidence that the model with the lower BIC is superior? Can you send me some references for using BIC to determine superiority of nonnested SEM models? Thanks in advance. 


A couple of papers may be of interest: Wasserman (2000) in J of Math Psych gives a formula (27) which implies that a BICrelated difference between two models is logBij where B is the Bayes factor for choosing between model i and j. Wasserman's (27) says that logBij is approximately what Mplus calls minus 1/2 BIC. This means that 2log Bij is in the Mplus BIC scale apart from the ignorable sign difference. Kass and Raftery (1995) in J of the Am Stat Assoc gives rules of evidence on page 777 for 2log_e Bij which say that >10 is very strong evidence in favor of the model with largest value. So, to conclude, this says that an Mplus BIC difference > 10 is strong evidence against the model with the highest Mplus BIC value (I hope I got that right). Raftery has a Soc Meth chapter from around 1995 (?) that talks about Bij from a SEM perspective 

Rob Dvorak posted on Wednesday, July 14, 2010  6:45 pm



Hi Michael, Here's the Raftery cite: Raftery, A. E. (1995). Bayesian Model Selection in Social Research. Sociological Methodology, 25, 111163. There's also a good discussion about this here: http://www.statmodel.com/discussion/messages/23/2232.html?1209409498 


Thanks to you both for the quick responses. Just to make sure I am not misinterpreting you. My MPLUS BICs are: Model 1 BIC = 14414 Model 2 BIC = 14403 14414  14403 = 11 Which indicates that there is "strong" evidence that model 2 is superior to model 1. Please correct me if I am wrong. 


Yes, that's how I understand it  well, actually, Kass & Raftery (1995) in JASA use the term "Very Strong" for an Mplus BIC diff > 10. They view 610 as "Strong" evidence. They say that "From our own experiences, these categories seem to furnish appropriate guidelines." 

Back to top 

