Inferrring improvement in performance PreviousNext
Mplus Discussion > Structural Equation Modeling >
Message/Author
 KDA posted on Thursday, February 09, 2006 - 1:11 pm
Hi Bengt and Linda:

I have data on 424 subjects' task performance BEFORE and AFTER an intervention (computer mediated instruction - CMI) and would like to assess the improvement,if any, that is specifically attributable to the intervention. Is it appropriate to infer improvement by regressing Post-intervention perf on Pre-intervention task perf, Extent of CMI, and relevant controls?

My colleagues are of the opinion that the CMI-Perf path, if significant and positive, would only allow me to infer that CMI has a positive association with post-intervention sales, and says nothing about improvement in performance. How would I be able to demonstrate an IMPROVEMENT relative to pre-intervention performance? Am I better off creating a difference score (post- less pre-intervention performance) and then regressing this difference score on CMI?

I would greatly appreciate your insights.
 bmuthen posted on Saturday, February 11, 2006 - 12:50 pm
Regressing post-perf on CMI and pre-perf (and other pre-intervention control variables) is a standard ANCOVA approach so that certainly is a common way to assess intervention effects. The estimated regression model can then be used to calculate what post-perf value is obtained from a pre-perf value so that you can see if there is a pre-post perf improvement.

There is a large literature on using difference scores - it would probably be covered in for example

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.
 KDA posted on Wednesday, February 15, 2006 - 11:04 am
Many thanks, Bengt.

This is very helpful.
 Ross Larsen posted on Tuesday, June 09, 2009 - 1:00 am
Bengt,

Your advice makes sense, but how do you know whether the difference is statistically significant from the regression output? Is there anyway to do a post-hoc test in MPLUS to answer that question?

Thank you
 Bengt O. Muthen posted on Tuesday, June 09, 2009 - 6:10 pm
The typical ANCOVA application is that you have parallel regression lines (equal slopes in treatment and control group), so that the intercept represents the post-performance group difference, controlling for pre-performance. In this case, the significance of the intercept is what you are after.

Generally speaking, post hoc tests can be done in Mplus using Wald testing in MODEL TEST.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: