I have a question concerning the report of a (missing) cross-level interaction in the result section. So far I only found posts regarding the Syntax(input).
I first calculated a 1-1-1 model with two mediators with random intercepts and fixed slopes (following preacher et al. 2010/2011) to look at the indirect effects. In the second step I tested for cross-level interaction. I did not look at a moderated mediation, but only at moderation on the two a paths (1-1), predictor to mediators, and at a level-2 moderator's influence. I set the two slopes as random (s1 and s2) and level-2 variable was Cfam. Consequently, I get in the output in the between part:
S1 ON CFAM -0.066 0.063 -1.055 0.291
S2 ON CFAM -0.033 0.060 -0.548 0.584
Thus, none of both is significant. However, I am wondering how this is well reported (e.g.in a table) and if anything of the remaining output information needs to be reported as well!?
Maybe there is a good example paper, but all I found till now rather used SPSS and wrote the classical interaction term in a table. That doesnt seem to fit here I guess since the random slope is modeled as a variable itself...!?
The question is if the random slopes s1 and s2 are needed or if they can be fixed. You are seeing that they are not significantly related to Cfam, but what' their intercepts in these regression and what's the residual variances? You could use BIC to choose between making s1 and s2 random or just keep them as fixed (no cross-level interaction).
I see the logic in that random slopes need to be meaningful (sign.) for testing cross-level interaction. However, I have difficulties with how to check that in my case. I used to work with SPSS before, but for that complex model MPlus was more appropriate.
In my cross-level moderation model, the intercepts are DJS(DV) 4.277 (p<o.1) S1 0.394, n.s.(0.107) S2 0.139, n.s.(0.532) residual variances S1 0.061, marginally sign (0.098) S2 0.118, sign. (p<0.05) Can I argument with this information only (without model fit)?
I assume I cannot compare the BIC of the interaction model with the BIC of the "1-1-1 fixed slopes model" (6676.447 compared to 7871.244) since the between part is different.., so Cfam (level2) would need to be removed to compare models!?
Would it help to model the mediation only at the within-person level and then in a second model change the two a-paths into random slopes and compare the BICs then? I tried that, but Mplus told me that Monte Carlo simulation was needed for the operation.
It is well known that z-score testing of variances is problematic given that you test against the inadmissible value of zero. I would use BIC. BIC is on a comparable metric as long as the DVs are the same so having Cfam in the model or not should not matter I think. But why not keep it in for both models, letting it also influence the intercept.
so I can argue that the BIC decreased (from 7871.244 to 6676.447), thus indicating that model fit for the one with random slopes is better, right? And then report the non-sign. cross-level moderation!? Is it necessary/normally done to give the information about intercepts and residual variances as well? I did not find a paper reporting a similar case using mplus...
Thanks for the suggestion. My theoretical argumentation rather suggests the cross-level interaction only I guess....