Effect Size interpretation under type...
Message/Author
 Qi posted on Monday, June 02, 2008 - 3:01 pm
We did a multiple regression to test interaction between A and B on outcome C, using type=complex (MLR) for multilevel data. We got a significant interaction, and plotted and tested whether the slopes at -1SD, mean, and +1SD are different from zero, and found interaction does make sense.

However, we were stuck when describing the magnitude of the interaction effect. I tried 2 ways (model1 and model2 are displayed for later description):

Model 1: C on A B;
Model 2: C on A B A*B

1. compute F to test R2 change between model 1 and 2 (this test is given by SPSS when doing hierarchical regression), but result was not significant.

However, because the s.e. was adjusted to be smaller under type=complex, I tried method2:

2. “tricked" Mplus to do Chi-square test for adding A*B (i.e., run Model-1A by fixing the residual variance of C to be same as in Model-2). again, test was not significant.

So, under type=complex,

1)is either above method applicable to test whether A*B explained significant amount of variance? Because, if I get a significant interaction effect, I should get a significant test result for adding A*B, right?

2)is the R2 difference between model1 and model2 the effect size for A*B?

3) What is the correct way of calculating effect size and interpreting magnitude of interaction?

Thanks a lot! Qi Chen
 Linda K. Muthen posted on Tuesday, June 03, 2008 - 10:27 am
I would suggest looking at the book on interactions in regression by Aiken and West and see how they would approach your problem.
 Qi posted on Tuesday, June 03, 2008 - 12:52 pm
Linda,

Thanks for your suggestion. I already referred to Aiken & West's book in the past. I guess my question here is essentially whether the interpretation (or method of interpretation) of interaction effect would be any different when using Mplus's "type=complex" feature vs. OLS estimation using other common software such as SPSS or SAS. Can I use the R2, s.e. etc from type=complex analysis to calculate the same statistics that is usually obtained under OLS estimation.

Thanks!
Qi
 Linda K. Muthen posted on Tuesday, June 03, 2008 - 2:44 pm
The interpretation of the interaction effect would be the same. I am not sure whether the R-square value can be used in the same way.