Validation of mixture model solution PreviousNext
Mplus Discussion > Latent Variable Mixture Modeling >
Message/Author
 Ian Kudel posted on Monday, April 14, 2008 - 9:37 am
Hello,
I ran a LCA in a large dataset. I will call it x. I am interested in testing whether the 5-class solution can be replicated in another dataset, which I will call y. It seems there are two possibilities.
Example 7.13 in the MPLUS v.5 manual, describes a confirmatory LCA. Of the three possibilities listed to constrain the model only 1 seems applicable. That is, I would take results from x to constrain the solution for y. For example I can constrain a threshold for item in the y dataset for a specific class by using a weight of -15. That will work some items. I assume +15 would work for low counts. I’m not sure about weights in between. Is it possible to identify appropriate weights?
The other possibility is a multigroup analysis with known groups. I think in this scenario I would run a 5-class solution for the y dataset. Then I can use the multigroup feature to run both the x and y solutions. I assume poor fit (I'm not sure how that would be assessed) would indicate the solution is not replicated in the new dataset
Which approach is better? Is there another possibility that I had not considered? Any suggestions for addressing the problems I indicated in these 2 approaches?

Thanks
 Linda K. Muthen posted on Monday, April 14, 2008 - 9:54 am
I would run the same analysis in the second data set as I ran in the first and see if I can recover the same 5-class solution.
 Ian Kudel posted on Wednesday, April 16, 2008 - 2:07 pm
Thanks. I am interested in a something analogous to a CFA. It seems running the analysis on another dataset without any constraints would be like running an EFA again. Are there any other possibilities?
 Linda K. Muthen posted on Wednesday, April 16, 2008 - 5:10 pm
I would fit the exploratory and confirmatory models in the first sample and then fit only the confirmatory model in the second sample.
 Ian Kudel posted on Wednesday, April 23, 2008 - 12:09 pm
Yes. Agreed. This goes back to my original question, which method is preferable for conducitng confirmatory latent class analysis.

It seems there are two possibilities.
Example 7.13 in the MPLUS v.5 manual, describes a confirmatory LCA. Of the three possibilities listed to constrain the model only 1 seems applicable. That is, I would take results from x to constrain the solution for y. For example I can constrain a threshold for item in the y dataset for a specific class by using a weight of -15. That will work some items. I assume +15 would work for low counts. I’m not sure about weights in between. Is it possible to identify appropriate weights?
The other possibility is a multigroup analysis with known groups. I think in this scenario I would run a 5-class solution for the y dataset. Then I can use the multigroup feature to run both the x and y solutions. I assume poor fit (I'm not sure how that would be assessed) would indicate the solution is not replicated in the new dataset
 Linda K. Muthen posted on Thursday, April 24, 2008 - 7:55 am
I would not fix values as in your first suggestion. You could do your second suggestion. It is a very strict test. It might be sufficient just to see if the results look the same in the two samples, for example, would you come to the same conclusions from both results?
 Ian Kudel posted on Friday, May 02, 2008 - 1:26 pm
Thank you, Linda,
As a validation test of my initial LCA a colleague suggested extracting data from the output and feeding it into a discriminant function analysis in SPSS. The estimates don’t seem like the correct values. Any thoughts?

Thanks again.
 Linda K. Muthen posted on Saturday, May 03, 2008 - 9:15 am
I would not expect this to match because you are using most likely class membership which is not what is used in model estimation of the LCA. Also, I don't think discriminant function analysis treats the variables as categorical.
 lisa Car posted on Monday, July 24, 2017 - 4:27 am
Hello,

I am interested in internal validation of my model and am considering doing a split half analysis of the original dataset. My questions are

1. Is this a suitable approach to take? If not do you have a different recommendation?

2. If so, do you have any recommendations for interpretation? For example, does one go through the same procedure as with the full dataset, re: deciding on the number of classes and
what criteria are used to decide whether each split half is valid?
thank you
thank you

Thank you
 Bengt O. Muthen posted on Monday, July 24, 2017 - 4:28 pm
This question is suitable for SEMNET.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: