Hi. I have depression scores at 4 time points after an acute event (0, 1, 6 & 12 months) and am trying to fit GMMs. I've tried various growth curve parameterizations (linear, quadratic, cubic, freeing time points, etc.) and consistently find that cubic fits best. Also allowing only the intercept to vary within class seems sufficient. So my base model is:
A 4-class model fit best, with a BIC of 37045 and entropy of 0.878. I then explored varying the intercept variance and residual variances between classes. The BIC improved a lot (36873; 15 add'l parameters vs. the base model), but the entropy dropped to 0.526. Moreover, there was a class in the base model (that we did not anticipate a priori but) that had strong clinical justification: a distinct trajectory for patients undergoing a certain procedure (albeit only 30 such patients out of 2000). The relaxed-restrictions model had no such class.
I'm having trouble explaining to my colleagues how a better fitting model (1) does a markedly poorer job at classifying patients and (2) obscures a clearly meaningful class that was found in a poorer-fitting model. Thoughts? Thanks!
An in-between model is one where only i is allowed to have class-specific variances, not the residuals. Perhaps that has the best BIC.
(1) You can get very good classification for very poorly fitting models. You have the same situation in SEM in terms of R-square versus chi-square (R-square can be great in an ill-fitting model). Interpretations based on the poorly fitting model shouldn't be trusted.
(2) The meaningful class may (i) exist but be hard to find, (ii) not exist despite its tempting interpretation, (iii) be found in a better fitting model of yet another kind.
Thanks. Do you have suggestions on how to evaluate goodness-of-fit on an absolute scale? Unless I'm missing something I can't get anything like chi-square, residuals, TECH12 or TECH13 since I'm using TYPE=RANDOM and numerical integration.