I ran a multilevel SEM and found a significant interaction between two between-level variables and would like to graph their relationship with my outcome variable. Specifically, I would like to generate a graph that shows the relationship between office referrals (IV - continuous at between-level) and ratings of order & discipline (DV - factor at within-level) separately for younger vs. older teachers (IV - binary at between-level). However, I'm not sure which plot command/option to use. Thank you for your help!
I have another question for you - I found in my SEM output that one of my path estimates was nonsignificant in the unstandardized section of output, but was significant in the standardized output. I found that the p-values were very similar - .06 for the nonstandardized estimate and .04 for the standardized, so I'm not sure if I should report this estimate as being significant or not? If the two sets of estimates are in conflict, which one do you report?
I would go with the raw coefficient although I know of no study that has investigated differences in significance between raw and standardized coefficients. I would also remember that I am doing many tests and be conservative in what I call significant.
I ran a multilevel (students in schools) latent covariance model which included latent estimation of both the outcome and one of the predictors at the school level. Additionally, I included interaction terms for a couple of the within level observed variables, and I estimated a few random slopes that I am trying to predict with the latent predictor.
I found significant interactions both within and cross-level, and now I would like to plot the simple slopes. I saw in other discussion posts that Mplus does not currently have this capability. However, I was wondering... (1) Is it possible to use Preacher's HLM slopes calculator with Mplus estimates? Are the two compatible? (http://www.people.ku.edu/~preacher/interact/hlm2.htm)
(2) If they are compatible, is it possible to get Mplus to print the asymptotic covariance matrix?
A correction to the above: I did NOT end up modeling the predictor as a latent variable in the final model that converged. I used a group aggregated observation to predict the random slopes. However, my questions about plotting the interactions still apply.
My syntax is as follows, if that is helpful.
USEVARIABLES ARE unit gender grade RaceThrt lep sped lunch plan_m centS CGM_S centGPA ReCGMPop CGMMob CGMLep CGMIep CGMFrl CGMRdi CGMFem x1x2 x1x3; MISSING= all (-9999); WITHIN = gender grade RaceThrt lep sped lunch centGPA centS x1x2 x1x3; BETWEEN = ReCGMPop CGMMob CGMLep CGMIep CGMFrl CGMRdi CGMFem CGM_S; Cluster = unit;
ANALYSIS: TYPE IS TWOLEVEL random;
Define: x1x2=gender*centS; x1x3=gender*RaceThrt;
MODEL: %WITHIN% s1 | plan_m ON gender; s2 | plan_m ON centS; s3 | plan_m ON RaceThrt; plan_m ON grade lep sped lunch centGPA; centGPA centS ON lunch lep sped gender racethrt; plan_m ON x1x2 x1x3;
%BETWEEN% s1 s2 s3 plan_m; plan_m ON ReCGMPop CGMMob CGMLep CGMIep CGMFrl CGMRdi CGMFem CGM_S; s1 ON CGM_S; s2 ON CGM_S; s3 ON CGMRDI;
(1) I haven't looked at Preacher's calculator but you say that it refers to HLM slopes, which means that you consider random slopes for observed covariates. That is, not using the general Mplus approach of also allowing latent covariates (or observed y variables). It sounds like your Mplus model is of this special kind and therefore the calculator would be applicable.
(2)-(3) You get the asymptotic covariance matrix by requesting TECH3 in the OUTPUT command. You can also save this using the SAVEDATA command.
Dave posted on Monday, November 07, 2011 - 4:37 pm
I ran a model to examine a three-way interaction between a within and two between-level variables predicting a within-level outcome. I am trying to use Topic 3 slides 164-171 to interpret these findings. Is this appropriate for this type of interaction? If so:
S1 = 0.149 + 0.096*IV1 - 0.074*IV2 - 0.104*IV1*IV2 + 0.276*Control1 + e and S1 = 0.149 + 0.096*IV1 + (-1)(0.074 + 0.104*IV1)*IV2 + 0.276*Control1 + e
Are these the changes in the slope between the within-level IV and DV depending on IV1 and IV2?
ANALYSIS: TYPE IS TWOLEVEL RANDOM; MODEL: %WITHIN% s1 | DV on IV; %BETWEEN% DV s1 ON IV1 IV2 INT Control1; DV with s1;
RESULTS Between Level S1 ON IV1 0.096 0.066 1.456 IV2 -0.074 0.054 -1.366 INT -0.104 0.029 -3.572 Control1 0.276 0.174 1.588 DV ON IV1 0.039 0.044 0.887 IV2 -0.048 0.044 -1.087 INT -0.033 0.031 -1.068 Control1 0.235 0.118 1.999 DV WITH S1 0.081 0.070 1.164 Intercepts DV -0.179 0.085 -2.092 S1 0.149 0.102 1.465
is not correct because your model doesn't have an interaction bewteen IV1 and IV2.
Dave posted on Wednesday, November 09, 2011 - 4:05 pm
Thank you for your response to my question and for pointing me towards Topic 7 slide 45. This helped clarify what I am looking for in the model and output. I would like to follow-up on on your response and more details regaring my model might be needed.
Similar to a post under the Multilevel Data/Complex Sample/Cross-level and between-group interac... around 2010/09/10 I would like to model an interaction between IV, IV1 and IV2. IV, IV1 and IV2 are observed variables. The post I saw for a similar model (IV at within level, two latent variables and their interaction at the between level) could be modeled using XWITH by including the interaction in the between level of the model. Given the observed variables, I created an interaction term (INT in my model, IV1*IV2, I did this in the data but could also use the DEFINE statement). I then added the interaction term to the between level of the model. Is this the appropriate way to model an interaction between IV1 and IV2 in the model from my last post? I think this would yield the following based on the output:
If so, can I simply replace INT with IV1*IV2 to rearrange the terms? I would appreciate any guidance you can provide in extending Topic 7 slide 45 to the three-way (two-way interaction at the between level) case.
Dave posted on Wednesday, July 11, 2012 - 11:10 am
I have a question drawing on the example above (2011/11/07) based on the syntax:
ANALYSIS: TYPE IS TWOLEVEL RANDOM; MODEL: %WITHIN% s1 | DV on IV; %BETWEEN% DV s1 ON IV1 IV2 INT Control1; DV with s1;
I have noticed that in examples 9.2 and 9.5 in the manual and in the NLES Math Achievement example in the Topic 7 slides, starting about slide 33, that when between level covariates are added they are included in the ON statement for both the random intercept and random slope. I am wondering if it is appropriate to have different variables in these statements like altering the example to:
%BETWEEN% S1 ON IV1 IV2 INT; DV ON Control1; DV with s1;
This works when I estimate a similar model. Are there any concerns regarding this asymmetrical approach? If this is appropriate, is the interpretation that IV1, IV2, and the interaction affect the slope of DV on IV, not the intercept - these are cross level interactions. Control1, alternatively only affects the random intercept - and is not a cross level interaction since it does not alter the slope. Thanks in advance for your input.
All of that is perfectly ok - the choice depends on your substantive theory.
Dave posted on Monday, February 11, 2013 - 4:04 pm
I am using a model similar to the one outlined in the post from July 11, 2012. A reviewer has asked that I demonstrate that there is sufficient variance in the slope representing the relationship between IV and DV, specifically that there is significant variance in relationship between IV and DV at level 2 prior to testing level 2 moderators that may explain this variance.
I would appreciate your insight on how I could provide this information. Is it simply the matter of including the BETWEEN residual variances for DV and S1 with IV1 IV2 and INT constrained to zero where the estimate of the residual variances are statistically significant? Thanks in advance for your input.
I think you are considering a random slope. As a first step you may just want to estimate the variance of that slope on Between, that is, not including any level-2 covariates predicting the random slope. Perhaps that is what the reviewer refers to.
I ran the model you suggested - outlined below - and got an estimate for the variance of the slope at the between level. I had missed that when you do not have the covariates at the between level that Mplus returns variances rather than residual variances.
I am reporting the variance estimate and CI in the text of a paper and am wondering if there is a standard greek letter (like beta for regression weights) that I should use for this estimate. The estimate in the Mplus output is in the psi matrix, should I use psi, tau_1 which I have seen used for the variance of a random slope, or something else? Thank you in advance for your input.
Model: %Within% s1 | DV on IV; %Between% DV; s1; DV with s1;
There doesn't seem to be a common multilevel notational standard. HLM uses tau, MLwiN uses sigma_u I think, and Mplus uses psi because it associates it with factors. Sigma is a typical Greek letter for variances, so a fairly middle-of-the road approach could be to use sigma_b (reserving sigma_w for the within-level residual variance) - thereby not favoring any software...
I have constructed a multilevel three way interaction similar to the one from July 11, 2012. I have used model constraint to calculate the simple slopes and their significance to further understand a significant three way interaction.
I am now wondering if I can use model constraint to test whether two of the slopes are significantly different. The equation I have used in the following code is similar to the one above from November 9, 2011. The code I have used is the following and it has worked (meaning I get an estimate for DD and its significance): Model Constraint: NEW(LL LH HL HH DD); LL = equation...; LH = equation...; HL = equation...; HH = equation...; DD = HH - LH;
I am wondering if I can interpret DD as the estimate and significance of the difference between the slopes estimated in HH and LH? For clarification, L and H are estimates of the slopes for combinations of + and - 1 SD of the variables in the interaction. Thanks in advance for your assistance.
I am trying to create an interaction term, MVD, by multiplying two observed variables, DepVig and Ms. When I use the following syntax, I receive an error statement saying that MVD is an unknown variable in an on statement. How do I make it known? I thought that using the define statement was how to make it known. Thank you for your assistance! Johnna
USEVARIABLES ARE ID Zvig Ms Cs Depvig; Within are Ms Cs Zvig DepVig;
It's stated at the bottom of page 574 and the top of page 575 in the user's guide.
Dave posted on Wednesday, October 30, 2013 - 4:45 pm
I am interested in testing an interaction between two level 1 variables (x and m) predicting a level 1 (y) variable accounting for clustering in the variables. I would appreciate your insight regarding setting up the code to run the models. As an example, we can think of x as a rating of customer service quality, m as interaction duration, and y as purchase size. These variables are observed for sales people with multiple ratings for each sales person. Assume that I created the interaction (int) term using the define statement such that int = x * m. I am wondering if the following code is how I would set up this model when there is clustering at level 2 for both x and m. %model% s1| y on x; s2| y on m; s3| y on int; %between% y on; s1 on; s2 on; s3 on;
Alternatively, is the following code correct when x has level 2 clustering but m does not? As an example, m could be customer gender, which is random across employees. %model% s1| y on x; y on m; s3| y on int; %between% y on; s1 on; s3 on;
Dave posted on Thursday, April 24, 2014 - 12:12 pm
I have clustered customer service data (similar to my post 2013/10/30) with measures of multiple service interactions for each employee. The customer changes in each interaction. I would like to use customer behavior (CB) at the start of the interaction (time 1) to predict employee behavior (EB) at the end of the interaction (time 2), accounting for employee behavior at time 1. I also have between level controls for each employee (e.g., employee tenure) that I would like to include in the model. I have built, and ran the following models (M1 and M2):
Usevariables are ID CB1 EB1 EB2 Control1;
M1 Within = ; Between = Control1; Cluster is ID; Model: %Within% EB2 on CB1 EB1; %Between% CB1 EB1 EB2; EB2 on Control1;
M2 Within = CB1 EB1; Between = Control1; Cluster is ID; Model: %Within% EB2 on CB1 EB1; %Between% EB2; EB2 on Control1;
When I run M1 I get ICCs(aprox value) for EB1(>.2) EB2(>.2) and CB1(<.1). For M2 I get an ICC for EB2(>.2).
I am wondering: 1 - Conceptually, what is the difference between M1 and M2 when I move CB1 and EB1 to the within level? 2 - Is M2 enough to account for the clustering? Should I include CB1 and EB1 on the between level since they are also clustered by the employee? Thanks in advance for your assistance.
Hi! I am running a COMPLEX type model with clustered data. I am using the asymptotic covariance matrix generated by TECH3 output to find variances/covariances of the coefficients. (I am using Kris Preacher's online calculator to determine simple slopes.) I have a simple question. Where do I find the variance and covariance of the constant/intercept? Do I need to specially request this?