|
|
Inferrring improvement in performance |
|
Message/Author |
|
KDA posted on Thursday, February 09, 2006 - 7:11 am
|
|
|
Hi Bengt and Linda: I have data on 424 subjects' task performance BEFORE and AFTER an intervention (computer mediated instruction - CMI) and would like to assess the improvement,if any, that is specifically attributable to the intervention. Is it appropriate to infer improvement by regressing Post-intervention perf on Pre-intervention task perf, Extent of CMI, and relevant controls? My colleagues are of the opinion that the CMI-Perf path, if significant and positive, would only allow me to infer that CMI has a positive association with post-intervention sales, and says nothing about improvement in performance. How would I be able to demonstrate an IMPROVEMENT relative to pre-intervention performance? Am I better off creating a difference score (post- less pre-intervention performance) and then regressing this difference score on CMI? I would greatly appreciate your insights. |
|
bmuthen posted on Saturday, February 11, 2006 - 6:50 am
|
|
|
Regressing post-perf on CMI and pre-perf (and other pre-intervention control variables) is a standard ANCOVA approach so that certainly is a common way to assess intervention effects. The estimated regression model can then be used to calculate what post-perf value is obtained from a pre-perf value so that you can see if there is a pre-post perf improvement. There is a large literature on using difference scores - it would probably be covered in for example Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin. |
|
KDA posted on Wednesday, February 15, 2006 - 5:04 am
|
|
|
Many thanks, Bengt. This is very helpful. |
|
|
Bengt, Your advice makes sense, but how do you know whether the difference is statistically significant from the regression output? Is there anyway to do a post-hoc test in MPLUS to answer that question? Thank you |
|
|
The typical ANCOVA application is that you have parallel regression lines (equal slopes in treatment and control group), so that the intercept represents the post-performance group difference, controlling for pre-performance. In this case, the significance of the intercept is what you are after. Generally speaking, post hoc tests can be done in Mplus using Wald testing in MODEL TEST. |
|
Back to top |
|
|