Testing for intervention effects in LGMs PreviousNext
Mplus Discussion > Growth Modeling of Longitudinal Data >
Message/Author
 Cindy Schaeffer posted on Friday, December 14, 2007 - 8:41 am
I'm trying to get clear on what the gold standard is in testing for intervention effects in longitudinal models. I would like some insights as to how to decide between simply treating intervention condition as a covariate (i.e., regressing the growth factors on the intervention dummy variable) or using a multiple group approach, in which a model that constrains the growth parameters to be equal between groups is compared to the model that allows them to vary, using a chi-square difference test. I am surprised how often these two approaches do not yield consistent results; in my experience, the multiple group model seems to detect intervention effects more often.

Also, what fit indices should be interpreted if using the multiple group approach?
Thank you,
Cindy Schaeffer
 Michael Spaeth posted on Friday, December 14, 2007 - 9:26 am
Hello cindy,

at the moment I'm also in this topic. Regarding multiple group modeling i would recommend you to read an article by "van Lier et. al, 2004". It seems to me, that multiple group modeling fits very well in the stepwise approach reccomended by muthen 2002 in his intervention related article. In other words, if you plan a mixture analyses of your intervention data, multiple group modeling is a nice and uncomplicated way to show overall effects, before doing mixture analyses with intevention as a covariate. if you expect no variances regarding the growth factors, i.e. you would like to do a "normal" latent growth model i would enter intervention status as a covariate in combination with some as covariates controlling for sex, SES and so on.
But i would prefer to rely on Bengt and Linda, this was just my opinion ;-)
 Bengt O. Muthen posted on Friday, December 14, 2007 - 9:40 am
It is interesting to hear your experience, Cindy, that multiple group analysis more often finds intervention effects. This probably indicates that in some cases the covariate approach is too restrictive, for example in specifying equal growth factor variances for intervention and control groups. The multiple group approach was discussed in the Muthen-Curran 1997 Psych Methods paper. That was a special, more effective multiple group approach than a conventional one, where the intervention effect was singled out in the model. The approach focused on 3 intervention effects: main effect (on the mean slope), interaction effect, and effect on the variance of the slope. I am not sure it could be called a gold standard, but I would take this approach.
 Cindy Schaeffer posted on Friday, December 14, 2007 - 10:18 am
Thanks to you both - this was helpful. I'll take a look again at the articles you both mentioned. Looking at the citation dates, I'm either 3 or 10 years behind on this issue!!
 Vanessa posted on Wednesday, June 30, 2010 - 7:26 pm
Hi,
I'm planning to test for an intervention effect using the approach depicted in the Muthen-Curran 1997 Psych Methods paper.
I'm new to LGCM (and Mplus) and have looked through your great notes on your site and various other papers, but still have a few queries regarding the order of the steps that need to be taken.

I have multiple indicators of my outcome construct, all assessed at 4 time points, in a control and treatment group.
When testing for measurement invariance, is the appropriate approach to

1) test for invariance of loadings and intercepts across time, separately in the control and treatment group a) in measurement models (not LGCM)?
b) and freeing up any eg. loadings that are not invariant (as indicated by modification indices)? then

2) test for invariance between groups, with all 4 timepoints specified a) but testing for invariance between groups, only for those parameters that showed invariance across time in the single group analyses? b) again in measurement models not LGCM?

3) run the two group LGCM but only constraining to equal those parameters that were invariant in the above

Apologies if the questions are basic, but there seems to be quite varying approaches to the preliminary steps in such models.
 Linda K. Muthen posted on Thursday, July 01, 2010 - 9:17 am
This seems to be a reasonable strategy.
 Vanessa posted on Thursday, July 01, 2010 - 10:57 pm
Thanks for the reply. Does it matter as to whether the tests for invariance are carried out in LGCM or in models that account for time simply by allowing all correlated residuals for same indicators at different time points and correlations between the factors?
 Linda K. Muthen posted on Friday, July 02, 2010 - 11:17 am
The first step is to test for measurement invariance across time and across groups. After measurement invariance is established, you can use the latent variables in whatever type of model you want.
 Vanessa posted on Tuesday, July 06, 2010 - 11:43 pm
Again thank you.
Should the final 'invariance model' then form the basis for the LGCM? eg. should any residual covariances needed in the measurement model then be modeled in the LGCM?

I ask because covarying residuals were needed in my measurement model (additional to correlated factors for the measurement occasions), between all of the same indicators, at the different time points, in order to obtain good fit.

So, is it problematic if covarying residuals are allowed between all of the same indicators, at the different time points, in a LGCM?
 Bengt O. Muthen posted on Wednesday, July 07, 2010 - 10:08 am
The answers to your three questions are: Yes, yes, and no.
 Vanessa posted on Monday, July 12, 2010 - 8:39 pm
Thanks Bengt. Back to my original question on testing invariance longitudinally and between groups (when planning to test an intervention following Muthen-Curran 1997 Psych Methods paper):

I am planning to test longitudinally (4 timepoints) in each group separately, and then, using these longitudinal models, test for invariance between the groups...

However, a paper I have read suggests testing for invariance between groups at each timepoint separately... and other papers suggest testing between groups first, and then longitudinally second.

Is there any correct or more logical order in which to test for invariance in a scenario with several timepoints and two groups? If so, what is it??
Thanks for your help
 Linda K. Muthen posted on Tuesday, July 13, 2010 - 8:03 am
I would start by testing what is most likely to hold. For example, if you think it is more likely that measurement invariance will hold across groups than time, I would start with groups.
 Vanessa posted on Tuesday, July 13, 2010 - 5:06 pm
Thanks Linda. If I have strict (longitudinal) invariance in one group, and only strong in the other group, when I combine the groups' models to test for group invariance, should my final model include strict invariance across time for the group in which this holds (and not the other)?
 Linda K. Muthen posted on Tuesday, July 13, 2010 - 5:59 pm
If by strict you mean that in addition to intercept and factor loading invariance you also have residual variance invariance, I would use strong for both groups.
 Mine Yildirim posted on Monday, February 07, 2011 - 1:04 pm
Hello, I have 2 questions;
1) My question is about the loadings of the treatment growth factor for two-group LGM (Muthen 1997). Should it be the same with the linear growth factor loadings (eg.0 1 1.5 2.5) or as just fixing the first loading to 0 and others to 1 to introduce intervention after the first time point (as int BY bmi0@0 bmi1@1 bmi2@1 bmi3@1)?

2)At next step, I applied the parallel process LGM to test mediation as;
MODEL;
ib sb| bmi0@0 bmi1@1 bmi2@1.5 bmi3@2.5;
is ss| ssb0lt@0 ssb1lt@1 ssb2lt@1.5 ssb3lt@2.5;
sb ON is ss;
ss ON ib;
ib-ss ON group;

With the two-group LGM i found significant intervention effects on growth trajectories but when i use the intervention as a dummy coded covariate and regress the growth factors on it then all disappeared for all outcomes.Even the direction of the effect changed for some of them.

Is it because of the way of testing the intervention effect?
Do you have any idea why this happens with the parallel process LGM? I appreciate a lot any idea/help on that problem.
 Bengt O. Muthen posted on Tuesday, February 08, 2011 - 11:17 am
1) That depends on your hypothesis for the treatment effect. Having the time scores 0, 1, 1, ... means that your treatment gives a jump right after it is introduced but the development has the slope (growth rate) as for the control group. Having the same say linear time scores as in the control group says that there is a gradual effect that alters the growth rate.

2) Seems like it should give the same results so it's hard to say without seeing your 2 runs - send input, output, data, and license number to support@statmodel.com.
 Vanessa posted on Thursday, January 12, 2012 - 10:28 pm
Hello,

I'm following the two-group LGM approach for testing for intervention effects (Muthen & Curran 1997), using multiple indicators, with latent outcome factors for 4 timepoints.

I have a couple of queries:

1) some of the residuals for the outcome factors are negative (non-sig. though). Depending on which I constrain to be zero, the value (and significance) of the treatment slope mean changes.

a) Is it ok to constrain the negative ones to zero or is it best to constrain them all to be equal in each group (but not across groups)?

2) in some models, the treatment slope mean is significant in the non-standardised solution (eg. p=.002) and not significant in the standardised solution (eg. p=.15). This is a considerable difference - is this an issue and which would you report?

Thanks in advance
 Linda K. Muthen posted on Friday, January 13, 2012 - 11:39 am
I would hold them all equal. This is what is typically done in multilevel modeling.

These differences occur because the raw and standardized coefficients have different sampling distributions. I would probably use the raw coefficient but I would also be conservative in that many tests are being performed so some type of Bonferroni correction should probably be made.
 Anna-Maria Fall posted on Wednesday, March 21, 2012 - 3:11 pm
I have a question related to measurement invariance testing in pretest-post test data assuming change from intervention. My ultimate goal is to estimate intervention effect in latent means at post test. The latent trait is measured by 46 binary items. As a first step I compared groups on pretest and established measurement invariance across group. At post test I wasn’t able to establish neither full nor partial measurement invariance across groups. Therefore I am wondering if measurement invariance across groups and time is an unrealistic goal in research involving subjects that are expected to change as a result of participation in the intervention. Treatment effects are evident at item level, for example items had very different difficulties across groups. To illustrate, for one item the difficulty in C group was 3.26 and in the treatment group was -1.44.
Therefore I am also wondering if is reasonable to limit measurement invariance testing to pretest data only?
Thank you,
Anna-Mari
 Bengt O. Muthen posted on Wednesday, March 21, 2012 - 9:05 pm
Perhaps the groups your refer to are treatment-control groups. Typically, with a factor model you want an intervention to affect the factor, not the items directly. It sounds like you have the latter situation, where direct effects from the treatment onto the post-test items makes for treatment-control group measurement non-invariance at that time point. That can also be modeled, however.
 Anna-Maria Fall posted on Thursday, March 22, 2012 - 6:47 am
Thank you, Dr. Muthen, for your quick response. Yes, my groups refer to treatment and control group in a randomized control trial. Will you please elaborate on how the intervention affecting the items directly can be modeled or point me to references that can help me to understand this situation?


I need to add that there are significant differences between groups in factor means, if I hold item parameters equal across groups and across time. But as you pointed out I have measurement invariance at pretest only.
 Bengt O. Muthen posted on Thursday, March 22, 2012 - 8:07 am
One approach is to impose post-test measurement invariance across treatment-control as a first step and then check Modification indices to see which items should relax the invariance. Those items experience specific treatment effects.
 Jing Zhang posted on Tuesday, September 10, 2013 - 8:27 am
I am trying to fit a two-group growth model following the approach discussed in Muthen, 1997.

Would anyone share the relevant syntax for this model?

Thank you very much.
 Linda K. Muthen posted on Wednesday, September 11, 2013 - 9:33 am
What is the title of the paper you refer to?
 Jing Zhang posted on Friday, September 13, 2013 - 2:00 pm
General longitudinal modeling of individual differences in experimental designs: a latent variable framework for analysis and power estimation.

Thank you very much,
Jing
 Bengt O. Muthen posted on Friday, September 13, 2013 - 5:42 pm
You'll find that on slide 70- of Topic 4 of our handout on our website.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: