Comparing models in bayesian framework PreviousNext
Mplus Discussion > Confirmatory Factor Analysis >
Message/Author
 Holmes Finch posted on Tuesday, October 18, 2011 - 10:21 am
Hi,

I am using the knownclass option to estimate what is essentially a multiple groups CFA model and would like to compare the fit with a model where groups are ignored (fully constrained). Everything is working fine, but I am unsure how to compare the fit for the two models. I would like to use the DIC or Bayes Factor, but the former doesn't seem to be supported with mixture models, and I cannot find the posterior model probability for the latter. I was wondering if you had any suggestions. Thanks so much for any thoughts.

Holmes
 Bengt O. Muthen posted on Tuesday, October 18, 2011 - 2:16 pm
Some questions. What do you mean by "a model where groups are ignored"? You mean like analyzing boys and girls together? And why mixture and Bayes? Why not multigroup and ML with BIC?
 Holmes Finch posted on Tuesday, October 18, 2011 - 5:14 pm
Thanks Bengt. I originally tried to fit the constrained and unconstrained models using ML, but couldn't get convergence, so I thought I would give the Bayesian approach a go. Both the fully constrained (Kenyan and American kids together) and separate groups analyses produce estimates, with nice looking trace plots for the parameters of interest. What I would like to do now is compare the fit of the models to see whether I have invariance or not.

Please let me know if you have more questions. Thanks.

Holmes
 Bengt O. Muthen posted on Thursday, October 20, 2011 - 9:50 am
So you want to do a 2-group Bayes analysis and check invariance. Why not compare the PPP for the no-invariance and invariance models? It doesn't test the difference specifically, but should be helpful. New tools are coming for such multi-group Bayes comparisons.

But you should be able to get ML to converge if Bayes converges. At least by using SVALUES to take the Bayes ending values into the ML run as start values.
 Holmes Finch posted on Thursday, October 20, 2011 - 11:07 am
Thanks very much, Bengt. I will take a look at the PPP values for sure. I can try the SVALUES with ML approach as well. That might well help with convergence. I look forward to the multi-group Bayes stuff when it comes out.

Holmes
 an-tsu chen posted on Wednesday, March 12, 2014 - 12:42 am
Hi,

I think my question should be classified under this thread. When conducting Bayesian CFA with categorical variables, the Mplus result seems not to show BIC value, making me unable to use this index to do model comparison. Does that mean it is not appropriate to consider BIC when data are categorical? Moreover, to the best of knowledge, PP p-value could also be used to do model comparison with categorical data, even though it is a goodness-of-fit index. Is that correct?

Andrew
 Linda K. Muthen posted on Wednesday, March 12, 2014 - 11:34 am
This is not yet available with categorical variables.
 Kyle Kent posted on Monday, July 06, 2020 - 9:37 am
Congrats on your recent paper on bayesian model fit evaluation http://www.statmodel.com/download/BayesFit.pdf and getting the bayes estimator to work with the MODEL TEST command.

I'm working with a BSEM which involves comparing factor structures for a specific measure. This measure is categorical so I'm resorting to a wald chi-square test of parameter constraint. I'd like to verify that I'm properly constraining the model.

My measure is 20 categorical questions (cariq1-cariq20). I'm comparing a 3 vs 4 factor CFA where all questions are used in both models. The difference comes in the first 10 questions where in the 3 factor cariq1-10 load together but in the 4 factor they're broken up. Here's my model constraint syntax:

MODEL:
neg by cariq1-cariq3 (f1);
psyc by cariq4-cariq10 (f2);
phys by cariq11-cariq15 (f3);
sex by cariq16-cariq20 (f4);

MODEL CONSTRAINT:
NEW (negpsych);
negpsych = f2*f1;

MODEL TEST:
0 = negpsych;

Will this produce a p-value that compares the 4 factor model to the 3 factor model where neg (f1) and psyc (f2) are combined? Does significance of the wald test mean that the parameter constraint (aka the 3 factor model) worsens model fit?
 Bengt O. Muthen posted on Monday, July 06, 2020 - 4:38 pm
What's your thinking behind using a product f2*f1?
 Kyle Kent posted on Monday, July 06, 2020 - 4:49 pm
That came from an example and since I'd like to combine the factors I did that rather than subtraction.

What would be the best way to constrain the model so that neg (f1) and psyc (f2) are essentially combined as if they were one factor?
 Tihomir Asparouhov posted on Monday, July 06, 2020 - 6:19 pm
I would recommend fixing the variance of the factors to 1 and then look at the correlation between f1 and f2. If the point estimates + 1.96*SE is smaller than 1 (ideally much smaller) then you can claim that the two factors are different. If point estimates + 1.96*SE >0.98 then you can claim the opposite.

You can use PPP to evaluate both models and if the three factor model fits well than you don't need a forth factor.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: