David Myers posted on Thursday, April 06, 2000 - 11:46 am
Imagine we have a growth model that is a quadratic, then the rate of learning can be expressed as S + 2Qt where S is the linear term, Q is the quadtratic term, t is time (this is just the partial derivate of outcome--test scores in this case, with respect to time). Now, I want to be able to compute the variance of learning. It would seem that I could take the predicted factor scores for S and Q, for each child i, form the quanity S+2Qt for each child, and then compute the variance of the new variable. Is this correct?
I want to be able to use this to compute effect sizes where the numererator of the effect size is the difference in learning rates for two groups of children. Thanks in advance.
That would be one way of getting the variance - through estimated factor scores. This variance, however, may have the usual biases due to estimating factor scores. How about the alternative of computing
V(S + 2Qt) = V(S) + 4tt V(Q) + 4t Cov(S,Q)
where those variances and covariance are the growth model estimates?
Julia Kim posted on Tuesday, April 11, 2000 - 2:11 pm
We've come across an instance where the equation above yields a negative variance. Specifically, a growth model for Math achievement produced the following:
V(S) = 322.341 V(Q) = 10.746 COV(S,Q) = -59.359
When t = 3, V(S + 2Qt) = -3.
What may be the implications for our model? Thanks.
b1uesman44 posted on Wednesday, November 23, 2005 - 10:10 am
This is an old post, but you might notice that for these parameter estimates the correlation between S and Q is out of range. I've computed
Such results are usually an indication of model misspecification, or a potential overspecification. It seems that the Q term is not needed (multicollinearity).
Estimated covariance matrices are not required to follow the rules associated with correlation matrices, and sometimes negative estimates of variance, out of range correlations, etc., can occur.
Perhaps you can use constraints, and/or boundaries so that your covariance parameters result in correlations that fall between -1 and +1.
Then this formula should be positive.
bmuthen posted on Wednesday, November 23, 2005 - 6:39 pm
Good point. And note that centering at the average time might make the s, q correlation go down.
Bill Dudley posted on Friday, October 03, 2008 - 2:02 pm
I have data from a study in which slopes differ for treatment and control - as estimated in a Latent growth curve analysis. The variables are similar to those in my next study. I would like to use the estimates from the successful model in a Monte Carlo and model different effects sizes and N’s (vary the slopes and NOBS)
In “How to use Monte Carlo …” Muthen & Muthen 2002, the effect size is defined as the group difference in slopes divided by the standard deviation of the slope growth factor.(page 7). In the growth model monte carlo set up on page 17 we see that the intercept and slope are set at 0 and .2 [i*0 s*.2]. Because S is regressed onto X we are provided with the residual variance of s is .09. s*.09;RIGHT??
Given my confusion on this, I am at a loss about how to use my current analyses to model power in my next study - ----
Q1 I assume that the standard deviation of the slope is computed from the parameters in this model OR perhaps derived from the preceding unconditional l model. Page 16. If the SD is computed can you point me to the formula for this so that I might see how to vary to effect sizes
Q2 Your Monte carlo examples use variables with means of zero. Can I just use the mean and SD’s from my real data or is there an advantage to outcomes with mean = 0 ? or should I convert my raw data to scores for this purpose?
where b is the slope in the regression of the slope on the covariate x (b^2) is the square of b). You get this in Tech4. You can also define effect size as the outcome mean difference (ctrl-tx)/SD at a certain time point - might be more down to earth.
Q2. Keep your variables as they are. The means of the growth factors are given as in the Topic 3 handout, but also in Tech4.
Bill Dudley posted on Wednesday, October 29, 2008 - 11:29 am
Please excuse the hiatus between postings, however I'd like to follow up my power estimation post of 10/03. Consider a test of differences in slopes between two groups as modeled in the growth2 example from Muthen & Muthen 2002. I think that this is comparable to the test of the time by group interaction in a Repeated measures ANOVA. If I use GPOWER to estimate the effect size for this interaction with 50 per group, alpha = .05 and correlations between measures a .3, I get an f ~.15. If I use Growth2 from Muthen and Muthen for a group difference in linear slopes I can detect an effect size of .63. It seems likely that the GPOWER estimate is far too liberal. I assume that this discrepancy is multiply determined – in the ANOVA we are dealing with Cohen’s f and in the LCM I assume that we are working Cohen’s d (??). Also I assume that in the Growth2 model that Y1 – y4 are not correlated whereas they are in the GPOWER estimation. And I assume that the Growth2 model is far more restrictive than the more general interaction test in GPOWER in that the Growth2 model is for a linear change in the positive direction etc. Can you comment on this?
I would think GPOWER (which I am not familiar with) considers effect size in terms of the means of the outcomes for a certain time point, not the random slope means. In the Mplus Monte Carlo setting you can also consider effect size for the outcome at a certain time point - expressing the y mean difference divided by the SD. The growth model does have correlated y's because the y's are all influenced by the growth factors. Tech4 says how strongly correlated they are. You are probably right that the linear growth model gets a different power estimate than the repeated measures ANOVA which doesn't impose linear growth.
socrates posted on Tuesday, June 30, 2009 - 7:22 am
I developed a GMM of treatment courses in which I regressed the variance of the slopes on type of treatment within each of the latent classes. Now, I am confused how to derive effect sizes from the slopes. According to Raudenbush & Xiao-Feng (2001), effect sizes can computed by the formula:
mean slope of treatment 1 minus mean slope of treatment 2 divided by the population standard deviation of the polynomial trend of interest (tau).
So, is tau the square root of the residual variance of the slope in the Mplus output?
I'm analyzing a randomized trial over 4 years (4 measurement points, the first is pre-test) via LGM and want to compute the effect size as "d" [estimated outcome mean difference (ctrl-tx)/SD at the last time point]. I've heard that one should use the pre-test SD to calculate d for the last time point in LGMs. That seems not very comprehensible to me and in my special case SDs for the outcome are heavily increasing over time for both control and intervention. So I prefer to use SDs pertaining to the last measurment point to get a smaller but more "realistic" d. What is your opinion?
I agree with you. If one is interested in the mean difference at the end of a study, it seems natural to consider the variation at the end of the study. One argument against that is that this variation is affected by treatment, so you should work with baseline variance. But even the control group can change variance from baseline to the end. My question would more be - if at the end the control and treatment group variances are very different, which should we use? In that case, perhaps the control group variance at the end is more meaningful because that refers to the normative development without treatment.
I was using the estimated outcome variance at the last measurement point because I was calculating "d" for estimated means and thought this would be more stringent. This estimated variance was equal across intervention/control because I modeled Intervention as a dummy-covariate. There were rather small variance residuals and observed variances seemed comparable across intervention and control. Is it o.k. to use the estimated variation in this case?
I too need to compute effect sizes for a tx effect (S regressed on dichotomous tx variable 0 = control, 1 = intervention) in LGMs with 5 unequal time points, continuous outcomes, and 2 additional covariates (age and gender). I think I am clear on how to do this, but have a few questions:
1. Do any of the procedures outlined in these posts change when the Tscores option is used (TYPE = random missing)? I ask because I am getting huge "d" values, 3+ in some cases when S on tx p values are in the .05 range. This is due in part because I have very small residual variances on S in the denominator.
2. What about when outcomes are count variables? (Tscores option and TYPE = random). I am getting excessively large effect sizes on these in particular.
3. Is there a way to look at overall effect size across 5 time points rather than difference in mean at the final time point? Isolating the change effect to the last time point seems to undermine the advantage of using repeated measures in the first place.
1. I assume "d" is the effect size? I think it is more understandable to look at effect size as a y mean difference (see also 3.), rather than an s difference. The variance of s can be small even when the y variance is large, which makes the s difference less important.
2. For counts you don't have a y variance parameter, so I am not sure what the effect size definition for y would be.
3. I agree that this should be done. It is really the area between the tx and ctrl curves that is important. Not sure off hand how to best quantify that, however.
I am stuck understanding the computation of effect sizes from the Muthen & Muthen (2002) paper (page 7). In your paper, you say that a slope effect of 0.2 translates into an effect size of 0.63 and that the effect size is computed as the ratio of the slope difference divided by the standard error. Here is my computation and I appreciate your advice, because I am stuck.
Slope| x=0 = 0.2 Slope| x=1 = 0.2 + 0.2*0.460 where 0.460 is the mean of x To compute the variance of the slope I used:
I am running a parallel process LGM. The correlation coefficients between the respective slopes (both linear and quadratic) of my two constructs were positive and significant. I would like to know how best to determine the magnitude of these significant correlations, in order to determine the effect sizes. Specifically, which parameter estimates should be used?
Just look at the TECH4 output, where you find the correlations (in contrast, the results section gives the covariances). I don't know that you can use the term "effect size" in the context of a correlation.
I've estimated a two-level linear growth model (not latent) to test the impact of an intervention on scores across 9 time points. I have one treatment group and one control group and need to calculate an effect size. Would I use the formula from Feingold (2009)?
Beta_11 / (square root of tau)
Where Beta_11 is the unstandardized coefficient of the regression of the slope on treatment (0/1). Would I take the square root of the slope residual variance for the denominator?
It isn't clear what one should use for effect size in growth models. You refer to the effect size on the slope in which case the denominator SD should be based on the slope variances, not the slope residual variance (so see TECH4). But one could also argue for the effect size on the outcome itself at a certain time point, so you need to express the outcome model-estimated mean at that time point and the model-estimated variance at that time point. Our Topic 3 handouts show how to compute those.
which implies that the s variance is easily computed as
V(s) = b^2 * V(dgroup) + res var (s),
where b is the model-estimate slope in s ON dgroup that you see in the output, V(group) is obtained as the sample variance from Sampstat, and res var (s) is the model-estimated residual variance of s in the output.
I think the most down to earth effect size is to compute the effect size in the outcome at the last time point as implied by the growth model. That is, using the difference in model-implied estimated mean for control and intervention groups divided by the SD for the outcome at the last time point. The SD is obtained directly in the output (use RESIDUAL) and the means are computed from the estimates in the output in line with slides 97-98 of our Topic 3 handout on our web site.