I am carrying out a LCA with 6 categorical variables (n ~ 1,300). I am using BIC, entropy, LMR(tech 11), and BLRT(tech14). I've read the Nylund et al. (2007) simulation paper and reviewed tech11 and tech14 sections in the User's manual.
As others have noted in various Mplus discussion threads, the p-values for the LMR and BLRT diverge dramatically... 3 class - LMR .000, BLRT .000 4 class - LMR .8835, BLRT .000 5 class - LMR 1.00, BLRT .000 ... 7 class - LMR 1.00, BLRT .000
Also the LL's for the k and k-1 class models are replicated in the BLRT's. For all baseline models the lowest LL's were replicated 5+ times. By manipulating starting values the largest class is identified last in the BLRT and LMR tests. Also, for BLRT test bootstrap draws were set at 100 and LRT starts was set at 0 0 500 75 so that the best LL was replicated in each of the 100 bootstrap draws. So, these tests have been carried out carefully and follow guidelines as set out in the user's manual.
What might be accounting for the diverging p-values for the LMR and BLRT tests? Do you have suggestions for moving forward?
I assume that the 2-class model has 0 p values for both as well.
BLRT and LMR agree well in the Nylund et al simulations, but we have now seen many real-data examples where they don't and where BLRT seems to often reject even for a high number of classes. More research into this phenomenon is needed.
In these instances of disagreement between BLRT and LMR I would rely more on BIC which also performed well in the Nylund et al study unless sample size is small. I would also rely on interpretability. I usually work with BIC as a first step and then I may use BLRT and LMR to help choose between the key number of classes. I wonder if BIC shows a distinct minimum in your case. If not, this could be an indication that the model type is not suitable for the data - for example a factor model or a factor mixture model may be more suitable than LCA.
Randy Mowes posted on Tuesday, February 21, 2017 - 2:19 am
I have a similar problem. I am performing LPA with 27 continuous variables with data from ~1000 participants. The LMR suggests a 3 cluster solution with: 2 class p= .0000 3 class p= .0000 4 class p= .7395 5 class p= .3557 6 class p= .5026 7 class p= .2826
The BLRT is .0000 in all cases. Additionally, the BIC and adjustedBIC values continue to decrease with up to 15 clusters. So the LMR indicates a 3 class solution, but both the BLRT and the information criterion based model fit indices suggest a much higher number of clusters.
What might cause these differences in model fit indices and how to deal with this?
I would go by BIC in these cases. The problem of BIC not showing a minimum can be solved by allowing within-class covariance between some pairs of variables. Sometimes adding a single factor to soak up some such covariance can give an indication of which pairs of items are the culprits.
Randy Mowes posted on Tuesday, February 28, 2017 - 6:55 am
Dear Mr. Muthen,
thank you for your fast response! If I understand you correctly you say that the default LPA in Mplus does not allow within-class covariance and in order to solve that I should look at the correlation between variables to see which ones might have high levels of covariance and allow for in-class covariance for these variables. Did I understand that correctly? And if so: How do I adapt my syntax to do so? This is an excerpt of my syntax: