Profs Muthen, because the LCA in LGMM is like a factor analysis, when attempting to find a model with the most parsimony (i.e., fewest classes) which has the lowest AIC/BIC values, would it make sense to plot AIC/BIC values and interpret them like a scree plot? In this case, one would be looking for a discrete "break" in the plot to determine where the models with differing numbers of classes become "significantly" different from one another.
This sounds like it would go along with Bengt's discussion in the Kaplan chapter on searching for parsimony in conjunction with low AIC/BIC values. Do you think it's a reasonable thing to say in a write-up with a LGMM?
I think that is reasonable and I would also include the loglikelihood in the scree plot. Although note that unlike a scree plot AIC and BIC (but not logL) can also increase, not only decrease.
Paul Norris posted on Monday, September 11, 2006 - 7:06 am
I am currently running a cross-sectional LCA analysis on 22 binary indicators. Unfortunatly, given the complex nature of my sample MPlus does not allow me to use Tech14 so instead I have compiled a table which give LogL, AIC, BIC and Tech 11 Values for a range of models. The Tech11 output indicates a solution with several fewer classes than the lowest BIC. The AIC continues to fall across all the models I have tested (up to 22 groups). However, both the AIC and BIC statistics fall sharply as the number of classes initially increases but only improve very marginally as additional classes are added above the number reccomened by Tech11. In effect, taking a scree-plot interpretation of AIC, BIC and LogL would appear to support the findings of Tech 11. Does using such an interpretation, combined with comparing the different models around this point for substantive relevance, seem a resonable strategy to use?
Also can anyone point me to any examples which have used a scree plot interpretation of AIC/BIC rather than simply taking the solution with the lowest BIC?
Thanks in advance for any guidance anyone can offer me.
What you do here sounds reasonable. The Nylund, Asparouhov, Muthen (2006) paper on deciding on the number of classes (see our web site under Recent Papers) uses a scree plot for the loglikelihood. Another useful statistic with categorical outcomes is the number of significant bivariate standardized residuals shown in Tech10 - when that number levels off, you are probably close to the best LCA model. These z scores are also influenced by complex sample featues, so the use of them is more descriptive than inferential.