How to report a (missing) cross-level... PreviousNext
Mplus Discussion > Multilevel Data/Complex Sample >
 Judith Lucas posted on Tuesday, July 26, 2016 - 7:51 am

I have a question concerning the report of a (missing) cross-level interaction in the result section. So far I only found posts regarding the Syntax(input).

I first calculated a 1-1-1 model with two mediators with random intercepts and fixed slopes (following preacher et al. 2010/2011) to look at the indirect effects. In the second step I tested for cross-level interaction. I did not look at a moderated mediation, but only at moderation on the two a paths (1-1), predictor to mediators, and at a level-2 moderator's influence. I set the two slopes as random (s1 and s2) and level-2 variable was Cfam. Consequently, I get in the output in the between part:

Between Level

S1 ON CFAM -0.066 0.063 -1.055 0.291

S2 ON CFAM -0.033 0.060 -0.548 0.584

Thus, none of both is significant. However, I am wondering how this is well reported ( a table) and if anything of the remaining output information needs to be reported as well!?

Maybe there is a good example paper, but all I found till now rather used SPSS and wrote the classical interaction term in a table. That doesnt seem to fit here I guess since the random slope is modeled as a variable itself...!?

Thank you very much for any advice you may have.
 Bengt O. Muthen posted on Tuesday, July 26, 2016 - 1:32 pm
The question is if the random slopes s1 and s2 are needed or if they can be fixed. You are seeing that they are not significantly related to Cfam, but what' their intercepts in these regression and what's the residual variances? You could use BIC to choose between making s1 and s2 random or just keep them as fixed (no cross-level interaction).
 Judith Lucas posted on Wednesday, July 27, 2016 - 5:26 am
Thank you very much for this quick answer.

I see the logic in that random slopes need to be meaningful (sign.) for testing cross-level interaction.
However, I have difficulties with how to check that in my case. I used to work with SPSS before, but for that complex model MPlus was more appropriate.

In my cross-level moderation model, the intercepts are
DJS(DV) 4.277 (p<o.1)
S1 0.394, n.s.(0.107)
S2 0.139, n.s.(0.532)
residual variances
S1 0.061, marginally sign (0.098)
S2 0.118, sign. (p<0.05)
Can I argument with this information only (without model fit)?

I assume I cannot compare the BIC of the interaction model with the BIC of the "1-1-1 fixed slopes model" (6676.447 compared to 7871.244) since the between part is different.., so Cfam (level2) would need to be removed to compare models!?

Would it help to model the mediation only at the within-person level and then in a second model change the two a-paths into random slopes and compare the BICs then? I tried that, but Mplus told me that Monte Carlo simulation was needed for the operation.

I would apreciate any help/suggestions.

 Bengt O. Muthen posted on Wednesday, July 27, 2016 - 12:21 pm
It is well known that z-score testing of variances is problematic given that you test against the inadmissible value of zero. I would use BIC. BIC is on a comparable metric as long as the DVs are the same so having Cfam in the model or not should not matter I think. But why not keep it in for both models, letting it also influence the intercept.
 Judith Lucas posted on Thursday, July 28, 2016 - 4:01 am
Dear Mr. Muthen,

so I can argue that the BIC decreased (from 7871.244 to 6676.447), thus indicating that model fit for the one with random slopes is better, right? And then report the non-sign. cross-level moderation!?
Is it necessary/normally done to give the information about intercepts and residual variances as well? I did not find a paper reporting a similar case using mplus...

Thanks for the suggestion. My theoretical argumentation rather suggests the cross-level interaction only I guess....
 Bengt O. Muthen posted on Thursday, July 28, 2016 - 9:11 am
Q1. Right

Q2. Right

Q3. Not necessary but sometimes done - see papers under Multilevel on our website.
Back to top
Add Your Message Here
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Options: Enable HTML code in message
Automatically activate URLs in message