Testing for intervention effects in LGMs PreviousNext
Mplus Discussion > Growth Modeling of Longitudinal Data >
 Cindy Schaeffer posted on Friday, December 14, 2007 - 8:41 am
I'm trying to get clear on what the gold standard is in testing for intervention effects in longitudinal models. I would like some insights as to how to decide between simply treating intervention condition as a covariate (i.e., regressing the growth factors on the intervention dummy variable) or using a multiple group approach, in which a model that constrains the growth parameters to be equal between groups is compared to the model that allows them to vary, using a chi-square difference test. I am surprised how often these two approaches do not yield consistent results; in my experience, the multiple group model seems to detect intervention effects more often.

Also, what fit indices should be interpreted if using the multiple group approach?
Thank you,
Cindy Schaeffer
 Michael Spaeth posted on Friday, December 14, 2007 - 9:26 am
Hello cindy,

at the moment I'm also in this topic. Regarding multiple group modeling i would recommend you to read an article by "van Lier et. al, 2004". It seems to me, that multiple group modeling fits very well in the stepwise approach reccomended by muthen 2002 in his intervention related article. In other words, if you plan a mixture analyses of your intervention data, multiple group modeling is a nice and uncomplicated way to show overall effects, before doing mixture analyses with intevention as a covariate. if you expect no variances regarding the growth factors, i.e. you would like to do a "normal" latent growth model i would enter intervention status as a covariate in combination with some as covariates controlling for sex, SES and so on.
But i would prefer to rely on Bengt and Linda, this was just my opinion ;-)
 Bengt O. Muthen posted on Friday, December 14, 2007 - 9:40 am
It is interesting to hear your experience, Cindy, that multiple group analysis more often finds intervention effects. This probably indicates that in some cases the covariate approach is too restrictive, for example in specifying equal growth factor variances for intervention and control groups. The multiple group approach was discussed in the Muthen-Curran 1997 Psych Methods paper. That was a special, more effective multiple group approach than a conventional one, where the intervention effect was singled out in the model. The approach focused on 3 intervention effects: main effect (on the mean slope), interaction effect, and effect on the variance of the slope. I am not sure it could be called a gold standard, but I would take this approach.
 Cindy Schaeffer posted on Friday, December 14, 2007 - 10:18 am
Thanks to you both - this was helpful. I'll take a look again at the articles you both mentioned. Looking at the citation dates, I'm either 3 or 10 years behind on this issue!!
 Vanessa posted on Wednesday, June 30, 2010 - 7:26 pm
I'm planning to test for an intervention effect using the approach depicted in the Muthen-Curran 1997 Psych Methods paper.
I'm new to LGCM (and Mplus) and have looked through your great notes on your site and various other papers, but still have a few queries regarding the order of the steps that need to be taken.

I have multiple indicators of my outcome construct, all assessed at 4 time points, in a control and treatment group.
When testing for measurement invariance, is the appropriate approach to

1) test for invariance of loadings and intercepts across time, separately in the control and treatment group a) in measurement models (not LGCM)?
b) and freeing up any eg. loadings that are not invariant (as indicated by modification indices)? then

2) test for invariance between groups, with all 4 timepoints specified a) but testing for invariance between groups, only for those parameters that showed invariance across time in the single group analyses? b) again in measurement models not LGCM?

3) run the two group LGCM but only constraining to equal those parameters that were invariant in the above

Apologies if the questions are basic, but there seems to be quite varying approaches to the preliminary steps in such models.
 Linda K. Muthen posted on Thursday, July 01, 2010 - 9:17 am
This seems to be a reasonable strategy.
 Vanessa posted on Thursday, July 01, 2010 - 10:57 pm
Thanks for the reply. Does it matter as to whether the tests for invariance are carried out in LGCM or in models that account for time simply by allowing all correlated residuals for same indicators at different time points and correlations between the factors?
 Linda K. Muthen posted on Friday, July 02, 2010 - 11:17 am
The first step is to test for measurement invariance across time and across groups. After measurement invariance is established, you can use the latent variables in whatever type of model you want.
 Vanessa posted on Tuesday, July 06, 2010 - 11:43 pm
Again thank you.
Should the final 'invariance model' then form the basis for the LGCM? eg. should any residual covariances needed in the measurement model then be modeled in the LGCM?

I ask because covarying residuals were needed in my measurement model (additional to correlated factors for the measurement occasions), between all of the same indicators, at the different time points, in order to obtain good fit.

So, is it problematic if covarying residuals are allowed between all of the same indicators, at the different time points, in a LGCM?
 Bengt O. Muthen posted on Wednesday, July 07, 2010 - 10:08 am
The answers to your three questions are: Yes, yes, and no.
 Vanessa posted on Monday, July 12, 2010 - 8:39 pm
Thanks Bengt. Back to my original question on testing invariance longitudinally and between groups (when planning to test an intervention following Muthen-Curran 1997 Psych Methods paper):

I am planning to test longitudinally (4 timepoints) in each group separately, and then, using these longitudinal models, test for invariance between the groups...

However, a paper I have read suggests testing for invariance between groups at each timepoint separately... and other papers suggest testing between groups first, and then longitudinally second.

Is there any correct or more logical order in which to test for invariance in a scenario with several timepoints and two groups? If so, what is it??
Thanks for your help
 Linda K. Muthen posted on Tuesday, July 13, 2010 - 8:03 am
I would start by testing what is most likely to hold. For example, if you think it is more likely that measurement invariance will hold across groups than time, I would start with groups.
 Vanessa posted on Tuesday, July 13, 2010 - 5:06 pm
Thanks Linda. If I have strict (longitudinal) invariance in one group, and only strong in the other group, when I combine the groups' models to test for group invariance, should my final model include strict invariance across time for the group in which this holds (and not the other)?
 Linda K. Muthen posted on Tuesday, July 13, 2010 - 5:59 pm
If by strict you mean that in addition to intercept and factor loading invariance you also have residual variance invariance, I would use strong for both groups.
 Mine Yildirim posted on Monday, February 07, 2011 - 1:04 pm
Hello, I have 2 questions;
1) My question is about the loadings of the treatment growth factor for two-group LGM (Muthen 1997). Should it be the same with the linear growth factor loadings (eg.0 1 1.5 2.5) or as just fixing the first loading to 0 and others to 1 to introduce intervention after the first time point (as int BY bmi0@0 bmi1@1 bmi2@1 bmi3@1)?

2)At next step, I applied the parallel process LGM to test mediation as;
ib sb| bmi0@0 bmi1@1 bmi2@1.5 bmi3@2.5;
is ss| ssb0lt@0 ssb1lt@1 ssb2lt@1.5 ssb3lt@2.5;
sb ON is ss;
ss ON ib;
ib-ss ON group;

With the two-group LGM i found significant intervention effects on growth trajectories but when i use the intervention as a dummy coded covariate and regress the growth factors on it then all disappeared for all outcomes.Even the direction of the effect changed for some of them.

Is it because of the way of testing the intervention effect?
Do you have any idea why this happens with the parallel process LGM? I appreciate a lot any idea/help on that problem.
 Bengt O. Muthen posted on Tuesday, February 08, 2011 - 11:17 am
1) That depends on your hypothesis for the treatment effect. Having the time scores 0, 1, 1, ... means that your treatment gives a jump right after it is introduced but the development has the slope (growth rate) as for the control group. Having the same say linear time scores as in the control group says that there is a gradual effect that alters the growth rate.

2) Seems like it should give the same results so it's hard to say without seeing your 2 runs - send input, output, data, and license number to support@statmodel.com.
 Vanessa posted on Thursday, January 12, 2012 - 10:28 pm

I'm following the two-group LGM approach for testing for intervention effects (Muthen & Curran 1997), using multiple indicators, with latent outcome factors for 4 timepoints.

I have a couple of queries:

1) some of the residuals for the outcome factors are negative (non-sig. though). Depending on which I constrain to be zero, the value (and significance) of the treatment slope mean changes.

a) Is it ok to constrain the negative ones to zero or is it best to constrain them all to be equal in each group (but not across groups)?

2) in some models, the treatment slope mean is significant in the non-standardised solution (eg. p=.002) and not significant in the standardised solution (eg. p=.15). This is a considerable difference - is this an issue and which would you report?

Thanks in advance
 Linda K. Muthen posted on Friday, January 13, 2012 - 11:39 am
I would hold them all equal. This is what is typically done in multilevel modeling.

These differences occur because the raw and standardized coefficients have different sampling distributions. I would probably use the raw coefficient but I would also be conservative in that many tests are being performed so some type of Bonferroni correction should probably be made.
 Anna-Maria Fall posted on Wednesday, March 21, 2012 - 3:11 pm
I have a question related to measurement invariance testing in pretest-post test data assuming change from intervention. My ultimate goal is to estimate intervention effect in latent means at post test. The latent trait is measured by 46 binary items. As a first step I compared groups on pretest and established measurement invariance across group. At post test I wasnít able to establish neither full nor partial measurement invariance across groups. Therefore I am wondering if measurement invariance across groups and time is an unrealistic goal in research involving subjects that are expected to change as a result of participation in the intervention. Treatment effects are evident at item level, for example items had very different difficulties across groups. To illustrate, for one item the difficulty in C group was 3.26 and in the treatment group was -1.44.
Therefore I am also wondering if is reasonable to limit measurement invariance testing to pretest data only?
Thank you,
 Bengt O. Muthen posted on Wednesday, March 21, 2012 - 9:05 pm
Perhaps the groups your refer to are treatment-control groups. Typically, with a factor model you want an intervention to affect the factor, not the items directly. It sounds like you have the latter situation, where direct effects from the treatment onto the post-test items makes for treatment-control group measurement non-invariance at that time point. That can also be modeled, however.
 Anna-Maria Fall posted on Thursday, March 22, 2012 - 6:47 am
Thank you, Dr. Muthen, for your quick response. Yes, my groups refer to treatment and control group in a randomized control trial. Will you please elaborate on how the intervention affecting the items directly can be modeled or point me to references that can help me to understand this situation?

I need to add that there are significant differences between groups in factor means, if I hold item parameters equal across groups and across time. But as you pointed out I have measurement invariance at pretest only.
 Bengt O. Muthen posted on Thursday, March 22, 2012 - 8:07 am
One approach is to impose post-test measurement invariance across treatment-control as a first step and then check Modification indices to see which items should relax the invariance. Those items experience specific treatment effects.
 Jing Zhang posted on Tuesday, September 10, 2013 - 8:27 am
I am trying to fit a two-group growth model following the approach discussed in Muthen, 1997.

Would anyone share the relevant syntax for this model?

Thank you very much.
 Linda K. Muthen posted on Wednesday, September 11, 2013 - 9:33 am
What is the title of the paper you refer to?
 Jing Zhang posted on Friday, September 13, 2013 - 2:00 pm
General longitudinal modeling of individual differences in experimental designs: a latent variable framework for analysis and power estimation.

Thank you very much,
 Bengt O. Muthen posted on Friday, September 13, 2013 - 5:42 pm
You'll find that on slide 70- of Topic 4 of our handout on our website.
 Mathew Marques posted on Saturday, January 16, 2016 - 5:49 pm
we're analysing some data, whereby we used a school intervention (2 interventions and control) across 4 schools, over 3 timepoints. My understanding is that one approach to test whether the interventions are having an effect on the outcome, is to model this as an LGM.

Given all 3 conditions (2 interventions & 1 control) were collected at all 4 schools, I believe it would be best to consider whether there are any between effects at the school level.

So I was thinking of considering a model whereby the school level is the cluster, and checking to see whether the intraclass correlations are small, and if so then just considering the model without the schools as cluster.

Could you please let me know if this approach is correct, and whether this input (below) is appropriate to answer this question?

USEVARIABLES ARE t0 t1 t2 x school;
USEV = x;
CLUSTER = school;
iw sw | t0@0 t1@1 t2@2;
t0 t1 t2 (1);
iw sw ON x;
ib sb | t0@0 t1@1 t2@2;
t0@0 t1@0 t2@0;
ib sb ON x;

 Bengt O. Muthen posted on Sunday, January 17, 2016 - 5:42 pm
You can't really do 2-level analysis with only 4 clusters (schools). The recommended number is at least 20 and preferably many more than that.

You can create dummy variables for the 3 intervention categories, but it sounds like you don't have several schools per category so I don't see how intervention type and school can be disentangled. So the growth modeling is possible but assessing the intervention effect - assuming again that it is on the school level - may be hard.
 Mathew Marques posted on Sunday, January 17, 2016 - 6:08 pm
Thank you for your response.

So given the low number of schools, what would an appropriate approach be to establish homogeneity across the cluster level? My initial explorations suggest that the ICCs are very low, and therefore it may be feasible to only consider individuals over time (ignoring there are four schools). Note, that all three intervention conditions were present across each school.

In that instance, if you started with an unconditional model and established that the ICCs were low, then would it be possible to continue with the conditional analyses to test the effect of the intervention disregarding cluster?

 Bengt O. Muthen posted on Monday, January 18, 2016 - 2:50 pm
Sounds like you have all 3 intervention conditions given in each of the 4 schools, so the intervention is perhaps best viewed as classroom-based.

So you can compare the 3 interventions within each of the 4 schools. And you could do a multiple-group analysis to see which parameters are the same across school. This means that school is viewed as a fixed, not random mode.
 Mathew Marques posted on Wednesday, February 10, 2016 - 6:53 pm
Thank you kindly for your responses. They have been most useful. I was hoping to ask, what would hopefully be the last questions relating to this topic as I learn more about it.

I have read the Muthen & Curran (1997) paper on using a two-group formulation, the added growth factor, for modeling intervention effects in LGMs and this seems more appropriate for what we are doing. In a two-group model for intervention studies (Figure 5), I understand that the added growth factor indicates the in/decrease beyond that of the control population.

My questions are in terms of the interpretation of the added growth factor, in particular the path from the treatment condition to the timepoints.

Is the correct interpretation that these are the mean differences at tx between control and intervention, after controlling for differences at t0 ?
Put another way, does this model (Topic 4, pp 70-71) covary differences between the control and intervention at t0 for all subsequent comparisons?

Thanks, Mathew
 Bengt O. Muthen posted on Friday, February 12, 2016 - 5:36 pm
The growth factor "t" is an added growth slope factor that influences the outcome means in the tx group after the first time point. So it represents an increasing or decreasing difference between the groups after the first time point.
 Mathew Marques posted on Tuesday, February 16, 2016 - 4:16 pm
Dear Bengt,

thanks once again for your response. Given that the growth factor 't' is an added growth factor slopes, representing an in/decreasing difference between groups after the first time point, what approach would you advise to test mean differences at each timepoint between groups?

In addition to modelling growth change in the intervention group, as psychologists we are interested in whether the control and intervention group differ at each timepoint (t0, t1, t2). My understanding now, is that it is not as simple as interpreting the added growth factor at each timepoint, but was hoping there was a method to test this.


 Bengt O. Muthen posted on Tuesday, February 16, 2016 - 6:57 pm
You would have to express the model-implied mean at each time point and for each group, as well as their differences, using Model Constraint. To express growth model means, see the handout for Topic 3 in our short courses online.
 Mathew Marques posted on Wednesday, February 17, 2016 - 7:01 pm
Thanks again,

so if I have it correct it would be some additional syntax like this in the context of the added growth model syntax (I have starred where the new syntax would be)

I S | x0@0 x1@1 x2@2;
I ADDED | x0@0 x1@1 x2@2;
[x1](a1); ********

MODEL control:
[x1](c1); ********


NEW(t1compar); ********

t1compar = a1 - c1; ********


 Bengt O. Muthen posted on Thursday, February 18, 2016 - 5:03 pm
No, see the formulas for calculating outcomes means from a growth model shown in the Topic 3 handout.
 Anna MacKinnon posted on Thursday, May 16, 2019 - 12:40 pm
Dear Drs Mutthen,

What code can be used to model the effect of the intercept on the slope within an MLM analysis for an RCT, in order to test whether baseline values of the outcome moderate the treatment effect.

I've seen S with Y under %between%, can we also do S ON Y? But wouldn't Y not represent the intercept if treatment group and other covariates are entered in the model?

Thank you kindly,

 Bengt O. Muthen posted on Thursday, May 16, 2019 - 4:47 pm
Right, you would say S ON Y but if you have covariates on Between you want to center them so the Y intercept would refer to the control group (TX=0) and evaluated at the covariate means.
 Anna MacKinnon posted on Tuesday, May 21, 2019 - 2:56 pm
Thank you for your reply. In the following code, would Depress (my Y) represent the intercept (baseline level of Depression at Time = 0) for the control group (TxGroup = 0) while controlling for average Age and testing for the treatment effect?

Center Age (grandmean);

Missing are all (-9999);
Usevariable = ID TxGroup Age Time Depress;
Cluster = ID;
Within = Time;
Between = TxGroup Age;

Type = Twolevel Random;

S | Depress ON Time;
S ON TxGroup Age;
S ON Depress;
 Bengt O. Muthen posted on Tuesday, May 21, 2019 - 5:30 pm
Yes. Make sure that the model estimates a Depress variance at Between. If it doesn't by default, just say


I think you also want

Depress on Txgroup Age;
 Anna MacKinnon posted on Tuesday, May 21, 2019 - 5:58 pm
Okay great, thank you!

What does Depress on Txgroup Age estimate? Predictors of the intercept for depression, or as a dependent variable would it represent average level of depression across Time?
 Bengt O. Muthen posted on Wednesday, May 22, 2019 - 4:06 pm
You want Depress to not be unrelated to Txgroup and Age.
 Anna MacKinnon posted on Monday, August 26, 2019 - 11:55 pm
Dear Drs Muthen,

I have a few follow up questions about running a twolevel random model for an RCT, compared to SPSS mixed models.

1) Does Mplus provide estimates for the random intercept and random slope? When I run the code below, the output under Between includes Intercepts and Variances for S and Y, but I'm not sure what these represent.
2) In SPSS I can specify that a scaled identity covariance structure will be used for the random and repeated effects Ė how do you know or change which structure is being used in Mplus?
3) Should I include Y WITH S, and what would this indicate? Given there are covariates in the model does Y represent the average across time here?
4) The estimates for the covariates are the same in SPSS, but the estimate for Y ON TxGroup has the opposite sign (becomes negative). In Mplus, does this estimate represent the slope for the group coded as 0 (control) or 1 (treatment)?

Slinear | GAD ON Time;
Slinear ON TxGroup; !Treatment*Time interaction
GAD ON Age Ethnicity TxGroup; !fixed effects
 Bengt O. Muthen posted on Wednesday, August 28, 2019 - 2:41 pm
1) Yes, ask for FSCORES (see UG index). Every random effect has a mean and a variance because they are variables.

2)You can use WITH to specify any variance-covariance structure.

3) Y on Between represents the random intercept of Y.

4) 1.
 Anna MacKinnon posted on Monday, September 09, 2019 - 12:10 pm
Thank your for your responses! To clarify I am understanding correctly, based on my output pasted below can I say that:

1) there is a significant random intercept for GAD and a significant random slope (output under Intercepts)

2) there was a significant effect of treatment (coded a 1) on the average level of GAD (my Y) but not on change over time/slope (Slinear)

3) covariance between repeated measurements is assumed to be 0, since there are no WITH statements in the output? Given my data is in long format (so I canít use y1 WITH y2), what code would specify a variance components/identity structure?

Between Level

Slinear ON
TxGroup b = .279 p = .681

Age b =-.015 p = .881
Ethnicity b =-.446 p = .705
TxGroup b =-2.097 p = .039

GAD b = 9.061 p = .002
Slinear b = -1.180 p= .017
 Bengt O. Muthen posted on Monday, September 09, 2019 - 3:59 pm
1) The intercept for GAD is of interest only if the covariates have mean zero, that is, standardized for continuous covs and referring to the zero category for binary covs. Also, don't say "significant random intercept" - you should say "significant mean of the random intercept" because the random intercept is a variable, not a parameter.

2) Yes, if you add "controlling for age and ethnicity.

3) The covariance between repeated measurements is not zero because the random intercept and random slope makes them correlate. And identity type structure is obtained if you have a random intercept only.

Also, why don't you regress Slinear on Age and ethnicity as well?
 Anna MacKinnon posted on Monday, September 09, 2019 - 7:20 pm
Thanks again! To follow-up:

I'm still a bit confused, because what I'm looking for is the equivalent variance and p-value for the random intercept and slope that the SPSS mixed models procedure provides in the output under estimates of covariance parameters
1) In Mplus, if GAD is significant under the Intercepts output, does this indicate there is variation between individuals in baseline values?

2) And if Slinear is significant under the Intercepts output, does this indicate that the slope varies between individuals?

3) what is the default covariance structure?
 Bengt O. Muthen posted on Wednesday, September 11, 2019 - 11:55 am
You say:
"the SPSS mixed models procedure provides in the output under estimates of covariance parameters"

I assume that you are talking about a run where you have covariates (conditional run as opposed to unconditional run). If so, what you are referring to is the variances and covariances among the residuals on level 2 (between) - so look for estimates labeled as such in the Mplus output.

If you send Support the SPSS output and the Mplus output for the same model, we can help you find the corresponding values.
 Anna MacKinnon posted on Friday, February 07, 2020 - 1:39 pm
Dear Drs. Muthen,

I have an additional follow-up question about running MLM for an RCT.

If I am interested in a 3-way interaction between Time, Treatment, and a fixed covariate, should I model a two-way interaction between Treatment and the covariate on SLinear? Or do I also need to model it on Y?

Example code:

Center T1Sleep (grandmean);
INT = TxGroup*T1Sleep;

Missing are all (-9999);
Usevariable = ID TxGroup White T1Sleep Time Depress INT;
Cluster = ID;
Within = Time;
Between = TxGroup White T1Sleep INT;

Type = Twolevel Random;

Slinear | Depress ON Time;
Slinear ON White TxGroup T1Sleep INT;
Depress ON White TxGroup T1Sleep INT;
 Bengt O. Muthen posted on Saturday, February 08, 2020 - 2:00 pm
Q1: Yes.

Q2: No, typically you would not also have direct effects on Y.
 Anna MacKinnon posted on Monday, February 10, 2020 - 11:18 am
Thanks for your response! I have a few additional questions:

1)Do you mean not coding Y ON INT, or not coding any Y ON effects? Previously you mentioned that I should also model Y ON TxGroup and covariates, because we want Y to not be unrelated to Txgroup and covariates..

2) When I remove the line of code for Y ON covariates, group and INT, the interaction is no longer significant on Slinear - do you know why this may be?

3) Is this the correct code for simple slopes to probe the 3-way interaction?

Slinear | Depress ON Time;
[Slinear] (gam0);
Slinear ON T1Sleep (gam1);
Slinear ON White TxGroup INT;

Model Constraint:
Plot (low med high);
Loop (level1, -2, 2, 0.01);
Low = (gam0+(gam1*(-1))*level1;
Med = (gam0+(gam1*(0))*level1;
High = (gam0+(gam1*(1))*level1;
Type = Plot2;
 Bengt O. Muthen posted on Tuesday, February 11, 2020 - 5:01 pm
1) I mean not include Y on your 4 x'sY is not unrelated to those x's cause the x's influence Slinear which influences Y.

2) An interaction means something else when the main effects are excluded.

3) You give a label (gam1) for T1Sleep but you seem to be missing a label for INT given that T1Sleep interacts with TxGroup. Maybe I am misunderstanding what -1, 0, 1 and level1 refer to.
 Anna MacKinnon posted on Thursday, February 27, 2020 - 9:44 am
Hi Dr. Muthen,

Thank you for your answers!

To clarify 1) do you mean the x's influence Slinear on the Between level? Because the only x I model on the Within level is Time.

A related question; you mentioned Y on Between represents the random intercept of Y. Is this only when Time is coded as starting from 0? If I recode time to be 1,2,3 would Y represent the mean across time?

Thank you kindly,

 Bengt O. Muthen posted on Thursday, February 27, 2020 - 12:15 pm
Q1: Right.

Q2-Q3: Y is a random intercept in the regression of Depress on Time. This intercept represents the mean of Y at the time point where the time variable is zero.
Back to top
Add Your Message Here
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Options: Enable HTML code in message
Automatically activate URLs in message