In doing some multilevel modeling, my ICC appears to be quite low (barely .01). My thought is that it would be foolish to run a multilevel model with such a low ICC. To still acccount for this very slight effect of clusters, I was thinking I should run the analysis in TYPE=Complex as conceptually and by design there are in fact clusters. Please tell me if that's logical enough.
It is not the size of the intraclass correlation alone that determines whether non-indpendence of observations needs to be accounted for. It is the design effect which is a function of the size of the intraclass correlation and average cluster size. TYPE=TWOLEVEL and TYPE=COMPLEX are two different ways to take non-independence of observations into account. With TWOLEVEL, non-independence of observations is modeled. With COMPLEX, it is taken into account. The choice depends on whether you want to know something about the cluster-level parameters or just control for non-independence of observations.
I would also calculate the Design Effect which is based on the ICC and the average cluster size, where DEFF = 1 + (Average cluster size - 1)*ICC. If you find that DEFF >= 2, then you should address the clustering.
Note that this formula holds for means with equal cluster sizes. It is probably a decent approximation in other cases though.
Lucy Morgan posted on Tuesday, February 03, 2015 - 3:44 am
I hope this hasn't been answered elsewhere (I did have a good search first!). My study was designed to be multilevel, measuring care assistant variables (Level 1) within nursing homes (manager variables - level 2). The ICCs for dependent variables are very low (.062 and .086), both DEFF values are < 2 and all manager (L2) variables are non-significant predictors so I have decided that multilevel analysis is not suitable. I have then run the analyses with TYPE = COMPLEX to account for the clustered data but this in fact makes the overall model fit worse. My questions are 1) am i correct to get rid of multilevel modelling given the low ICC AND low DEFF? And would it be better NOT using the TYPE = COMPLEX analysis since it makes model fit worse? (In another post you state that TYPE = COMPLEX is the right analysis to use with clustered data but in that instance the person posting found that model fit improved using TYPE = COMPLEX)
Hi, is there a citation for the need to address clustering when DEFF >=2 but ICCs are low? What about the reverse: high ICCs and low DEFF? Also, does non-independence mean that there is sufficient between group variation to say that the measure distinguishes among clusters?