| Message/Author |
|
|
|
|
Greetings - I am in the process of simulating power for a randomized control trial while account for the measurement error of the manifest variable. For a single level trial, I have been able to corroborate power estimates using both Mplus and Optimal Design. However, when moving to a multilevel illustration with a single manifest outcome, I'm running into difficulties. Assuming an ICC of 10% at the between level, the results suggest power of .45; however, Optimal Design suggests power of .36. I'm wondering if this, indeed, a true estimate or whether my code is not accounting for something else.
montecarlo:
names are y w;
cutpoints=w(0);
nobservations = 100;
ncsizes=1;
csizes = 10 (10);
seed = 58459;
nreps = 1000;
between = w;
ANALYSIS:
TYPE IS TWOLEVEL;
processor=8;
estimator=ml;
MODEL POPULATION:
%Within%
fw by y@1;
y@.01;
fw*1;
%Between%
w@1;
fb by y@1;
y@0;
fb on w*.5;
fb*.1;
MODEL:
%Within%
fw by y@1;
y@.01;
fw*1;
%Between%
fb by y@1;
y@0;
fb on w*.5;
fb*.1; |
|
|
|
|
2 quick thoughts:
- do you really have icc=0.10? Isn't the between variance 0.35 and the within variance 1, which means icc=0.35/(1+0.35) = 0.26.
- with only 10 clusters the SEs' will be poorly estimated and therefore the Monte Carlo estimated power will be poorly estimated. |
|
|
|
|
| Thank you for the response! When I ran the code here the ICC for Y came up as .096. So is it a reasonable hypothesis that the power for the Monte Carlo, though poorly estimated due to relatively few clusters, may more accurately reflect the power than Optimal Design which does not account for the residual variance? |
|
|
|
|
| If the population, icc=0.26 and you get icc=0.096 for your sample, then the estimate isn't very good - and that is probably due to few clusters. I would not trust the Monte Carlo power estimates in such a case. |
|
|
|
|
| Thank you, Bengt. One final issue I'm curious about is how you came up with the between-variance of .35. |
|
|
|
|
The fb on w regression seems to give between variance:
0.5*0.5*1 + 0.1 = 0.35
Within variance
1+ 0.01 = 1.01.
icc = 0.35/(0.35+1.01) = 0.26 |
|
|
|
|
Apologies for the novice question, but is there a reference you may be able to point me toward for those underlying equations? I had assumed that the total variance was the 1 + .01, so that the between variance was .01/(.01+1) and the within was 1/(.01+1).
In larger cluster size conditions (20, 30, 50, 100) I used this convention for the ICCs and effect sizes, and I was able to obtain converging estimates of power between Mplus and Optimal Design. |
|
|
|
|
Total variance = between + within.
The Raudenbush-Bryk multilevel book should have these formulas. |
|
|
|
|
Hi Bengt - One final follow-up as I've been working through this. In the code from this thread the between variance, as I understand, is .1 from the fb*.1 code, and the within variance is 1 from the fw*.1 code. From this I thought the ICC would be .10 (between) / .1 + 1 (between + within).
In your previous response saying that the beta weight is factoring into the estimation of the between variance? Is the general form then B^2*between variance+within variance? |
|
|
|
|
Your statements
fb on w*.5;
fb*.1;
do not imply that the fb variance is 0.1. It is the residual variance of fb that is 0.1. The variance of fb also consists of the variance due to the w influence, just like in regular regression. |
|
|
| Back to top |
