Naomi Dyer posted on Wednesday, April 09, 2008 - 1:30 pm
We have data collected from individuals within departments within organizations. We ran Mplus multilevel analyses on the factor structure at the dept level and then again at the organization level.
The ICC's for the dept level are pretty high (there are many depts with relatively small N inside) though the design effects are around 2. The ICC's for the organizations are very low, though the design effects are pretty high (around 12).
Since we are reporting these numbers side by side - the reader will want to make some comparisons as to where there is the most variability. ICC would seem at the dept level. Design effect would seem at the org level. Obviously these differences are based on within cluster size.
QUESTION: 1. How should one interpret these design effects? Is there a way to compare design effects?
2. The factor structure holds at both the dept and org levels and the fit is good- so it seems it does hold at these levels - is there any other way to compare using Mplus?
Design effects are examined to decide whether or not clustering can be ignored. If a design effect is 2 or greater, the clustering should not be ignored. At this point, you have decided not to ignore clustering so you can just look at the ICC's.