

Entropy and interpretation 

Message/Author 


I know entropy has been discussed numerous times here but just want to check my interpretation. I have a 5class GMM of depression scores over 4 times points. The entropy is around 0.5 and the ondiagonal average probabilities of membership range from 0.6 to 0.8. To me this means that the classes do not "separate" the population into distinct subgroups of patients very well. I take it to mean that there are 5 fundamental patterns or archetypes of depression course, and a given individual's trajectory can be viewed as a weighted combination of these archetypes. Is that an accurate interpretation? A corollary to this question is that my clients (physician health services researchers) have an exceedingly hard time getting their heads around this issue. They (perhaps naively) expected a categorization of patients into distinct subgroups that could then be targeted/monitored/followed up appropriately. We're not sure how best to utilize these results that identify "patterns" that aren't necessarily tangibly and specifically associated with a single group of patients. The best I've been able to offer so far is that the population is pretty heterogeneous and is not easily classifiable into distinct groups. Any thoughts? 


You really want an entropy of 0.8 or higher for a model to be able to clearly identify individuals following different trajectory types (although lower entropies can still produce good parameter estimates). So in your case the GMM isn't very useful for classifying individuals. Although, the average diagonal values you give suggest that maybe 1 class (the one with 0.8) can be distinguished from the rest. You may get higher entropy by adding key covariates to the model. Or, key distal outcomes. I assume that getting 5 classes was based on BIC. If not, perhaps too many classes are extracted, which can result in a lower entropy. But for some data, a possible outcome is that a clear distinction of trajectory types is not evidenced in the data. 

Back to top 

