KDA posted on Thursday, February 09, 2006 - 7:11 am
Hi Bengt and Linda:
I have data on 424 subjects' task performance BEFORE and AFTER an intervention (computer mediated instruction - CMI) and would like to assess the improvement,if any, that is specifically attributable to the intervention. Is it appropriate to infer improvement by regressing Post-intervention perf on Pre-intervention task perf, Extent of CMI, and relevant controls?
My colleagues are of the opinion that the CMI-Perf path, if significant and positive, would only allow me to infer that CMI has a positive association with post-intervention sales, and says nothing about improvement in performance. How would I be able to demonstrate an IMPROVEMENT relative to pre-intervention performance? Am I better off creating a difference score (post- less pre-intervention performance) and then regressing this difference score on CMI?
I would greatly appreciate your insights.
bmuthen posted on Saturday, February 11, 2006 - 6:50 am
Regressing post-perf on CMI and pre-perf (and other pre-intervention control variables) is a standard ANCOVA approach so that certainly is a common way to assess intervention effects. The estimated regression model can then be used to calculate what post-perf value is obtained from a pre-perf value so that you can see if there is a pre-post perf improvement.
There is a large literature on using difference scores - it would probably be covered in for example
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.
KDA posted on Wednesday, February 15, 2006 - 5:04 am
The typical ANCOVA application is that you have parallel regression lines (equal slopes in treatment and control group), so that the intercept represents the post-performance group difference, controlling for pre-performance. In this case, the significance of the intercept is what you are after.
Generally speaking, post hoc tests can be done in Mplus using Wald testing in MODEL TEST.