Test indirect effect using bootstrap PreviousNext
Mplus Discussion > Structural Equation Modeling >
Message/Author
 Qi posted on Wednesday, October 08, 2008 - 4:06 pm
I want to use MacKinnon & Lockwood (2001) proposed asymmetric distribution of products method to test the significance of the mediation effect, do I just specify "bootstrap=1000;" under analysis, and then specify "CINTERVAL (BCBOOTSTRAP);" in output? There are several options for CINTERVAL, I am not sure about which one I should use, BOOTSTRAP or BCbootstrap? I shouldn't use the default symmetric, should I? And when interpreting the output, I just check whether 0 is in the 95% CI in "
CONFIDENCE INTERVALS OF STANDARDIZED TOTAL, TOTAL INDIRECT, SPECIFIC INDIRECT,
AND DIRECT EFFECTS for STDYX Standardization", right? Thanks for your guidance!
 Linda K. Muthen posted on Wednesday, October 08, 2008 - 4:15 pm
BCBOOTSTRAP is what MacKinnon and Lockwood do. This does not require the BOOTSTRAP option unless you want to use bootstrapped standard errors with the bootstrapped confidence interval.
 Qi posted on Wednesday, October 08, 2008 - 8:01 pm
Hi Linda,

thanks for your prompt reply. Did you mean that I don't need to specify "bootstrap=1000" under analysis? I tried with only "CINTERVAL (BCBOOTSTRAP)" in the output command, however, I got the following warning and didn't get any CI for indirect effect, am I missing sth? Thanks again!

*** WARNING in OUTPUT command
BOOTSTRAP and BCBOOTSTRAP confidence intervals require the specification
of BOOTSTRAP in the ANALYSIS command. Request for CINTERVAL is ignored.
 Qi posted on Wednesday, October 08, 2008 - 8:09 pm
I am sorry, I guess misunderstood your previous instruction. I think you were talking about the difference between CI(BCBOOTSTRAP) and CI(BOOTSTRAP).

What's the difference between CI(BCBOOTSTRAP) and CI(BOOTSTRAP)? Is CI(BCBOOTSRRAP) not using bootstrapped SE with bootstrapped CI? What's the purpose of each option? Thanks!
 Linda K. Muthen posted on Thursday, October 09, 2008 - 11:03 am
You are correct. You do need the BOOTSTRAP option of the ANALYSIS command for both CINTERVAL (BOOTSTRAP) and CINTERVAL (BCBOOTSTRAP). Both of these options are described in the MacKinnon and Lockwood paper. CINTERVAL (BOOTSTRAP) uses 2.5% and 97.5% percentiles of the sample distribution while CINTERVAL (BCBOOTSTRAP) these percentiles values are estimated from the sample distribution. The bias-corrected confidence intervals are more accurate.
 Brianna H posted on Sunday, August 18, 2013 - 4:39 pm
Hello,

The Mplus v.6 manual on p.647 indicates that "with frequentist estimation, only SYMMETRIC confidence intervals are available for standardized parameter estimates." I am using BCBOOTSTRAP to obtain bias-correct bootstrap confidence intervals. Does this mean I have to use the unstandardized output? I.e. I cannot use the confidence intervals of standardized indirect effects, if those are available only for SYMMTERIC?

Also, in the CINTERVAL output, the fourth column labeled Estimate contains the parameter estimates. If the standardized estimate of an indirect effect is 0.255, is it correct to say that 25.5% of the variance in DV is explained by the indirect effect of the IV on the DV? Thank you.
 Linda K. Muthen posted on Monday, August 19, 2013 - 12:17 pm
If you want non-symmetric confidence intervals, you must use unstandardized estimates.

Your indirect effect is interpreted as an increase of one standard deviation is x results in an increase of 25.5% of a standard deviation increase in y.
 Taejin Hwang posted on Sunday, September 22, 2013 - 5:02 pm
Hello,
I'm trying to calculate indirect effects in my model.
There are two mediators (m1-m2), linking independent variables(IV, x1-x4) and dependent variables (DV, y1-y2).
My model is specified as below. Since the dependent variables are nested within a group, I used multi-level modeling to allow for the group-level random effect.

Q1. It says that bootstrapping is not allowed for multi-level modeling. How should I test indirect effects? can i just skip bootstrapping? (Mplus tested indirect effects although i didn't type bootstrapping)

Q2. I want to test main effects from IV to DV in this model. The thing is that from IV to DV, there are direct paths, as well as indirect paths. Can I sum both direct paths and indirect paths and test total effects (not just sum of two indirect paths)?

Thank you very much!

data: file is data.txt;
variable: names are lid y1-y2 x1-x4 m1-m2;
cluster is lid;
within= x1-x4 m1-m2;
analysis: type=twolevel;
model:
%within%
m1 on x1-x4;
m2 on x1-x4;
y1 on m1-m2;
y2 on m1-m2;
y1 on x1-x4;
y2 on x1-x4;
model indirect:
y1 ind m1 x1;
y2 ind m2 x2;
 Taejin Hwang posted on Sunday, September 22, 2013 - 6:32 pm
I also tried another model drawing on the Mplus syntax for testing indirect effects in a multi-level model as recommended by Preacher, Zyphur, Zhang (2010) & Preacher, Zhang, Zyphur (2011),

data: file is data.txt;
variable: names are lid y1-y2 x1-x4 m1-m2 c1-c5;
usevariables are lid y1 x1 m1;
cluster is lid;
within= x1 m1;
analysis: type=twolevel;
model:
%within%
m1 on x1(a);
y1 on m1(b);
model constraint: new(indm);
indm=a*b;
output: tech1 tech8 cinterval;

Q1. Do you think this modeling is correct?

Q2. in the model results, there is
New/Additional Parameters INDM Estimate S.E. Est./S.E. P-value
0.055 0.022 2.457 0.014

can i refer to estimate and p-value to judge significance of indirect effect?
(I can't understand which part of tech1 and tech8 output I should look for...)

Thank you!
 Bengt O. Muthen posted on Monday, September 23, 2013 - 3:41 pm
Please don't post in more than one window - this is our rule.

Answers to your first message:

Q1 Indirect effect estimation and testing can be done without doing bootstrapping. In my experience it is the exception rather than the rule that you need bootstrapping - you may need it with very small samples or when considering ratio estimates.

Q2. Yes.

Answers to your second message:

Q1. Yes and this should give the same results as in your Model Indirect run.

Q2. Yes, the p-value says that INDM is significant on the 5% level. You don't need to look at tech1 or tech8. You may want to listen to our Topic 1 teachings on mediational modeling.
 Taejin Hwang posted on Monday, September 23, 2013 - 5:01 pm
Thank you very much for your prompt and kind reply!
And sorry for multiple-posting. i'll keep that in mind. Have a lovely week!
 Stefanie Morgenroth posted on Thursday, September 26, 2013 - 1:56 am
Dear Sir and Madame,

I did a bootstrapping - but I don't know how I could interpret the results of the confindence intervals.

Could you help me?

Thank you very much,
Steffi M.
 Linda K. Muthen posted on Thursday, September 26, 2013 - 8:23 am
Please send your output and license number to support@statmodel.com so I can see which confidence intervals you are using.
 Julia Schorlemmer posted on Thursday, July 02, 2015 - 3:32 am
Dear Sir and Madame,

regarding bootstraping in MLM with Mplus: One question above was: "It says that bootstrapping is not allowed for multi-level modeling. How should I test indirect effects? can i just skip bootstrapping? (Mplus tested indirect effects although i didn't type bootstrapping)".

I wonder why it is not allowed? I could find any reasons... And if you have any literatur recomendation where I can read about the topic Mediation in MLM and bootstrap?
Thank you very much, Julia
 Linda K. Muthen posted on Thursday, July 02, 2015 - 11:50 am
Multilevel bootstrapping is a research area.

Use the regular standard errors or Bayes if you are concerned that the indirect effect has a non-normal distribution.

Google for literature.
 Julia Schorlemmer posted on Friday, July 03, 2015 - 7:45 am
Dear Linda,

thank you for your advice!
Best
Julia
 Jodie Stearns posted on Thursday, October 01, 2015 - 10:40 am
I am running a multilevel mediational path analysis and have 1 IV (dichotomous), 2 mediators (ordinal), 2 DVs (continuous). I am using the REPSE=Bootstrap to obtain the bootstrapped confidence intervals for the mediated effect because CINTERVAL (BOOTSTRAP) is not available for TYPE=COMPLEX. However, when I use REPSE=Bootstrap I do not get fit statistics and I also notice that the regression coefficients change a bit from when bootstrap is not included in the syntax. Therefore, it appears that the bootstrapping is applied to the whole model. I would just like to use the bootstrapped confidence intervals to indicate the significance of the mediated effect. Do you suggest running the model without bootstrap in the syntax to obtain fit statistics and path coefficients for the model and then running the model again with bootstrap in the syntax to test the significance of the mediated effect? OR do you recommend choosing between obtaining fit statistics or bootstrap confidence intervals and using the results from one analysis only.
I very much appreciate any advice you have for me!
 Linda K. Muthen posted on Thursday, October 01, 2015 - 12:16 pm
The REPSE option is for replicate weights. Do you have replicate weights?
 Jodie Stearns posted on Thursday, October 01, 2015 - 12:57 pm
I do not have replicate weights so I just set weight=1. This is the only way I could run the bootstrap with TYPE=COMPLEX. I got this idea from another thread on this forum.
 Christian Sinner posted on Thursday, May 05, 2016 - 2:32 am
Hello,

i calculated a simple mediation with Bootstrap. The estimate of total indirect effect is .095 p=.112, but the confidence interval lower 2.5% is .006 and upper 2.5% is .233. How does this fit? I cannot decide whether there exist a mediation!

Is the total indirect effect the indirect effect?

Greetings
Christian
 Bengt O. Muthen posted on Thursday, May 05, 2016 - 9:56 am
I think you are talking about a bootstrap confidence interval, that is, a non-symmetric interval. The p-value is based on a symmetric interval. Go with the non-symmetric bs interval.
 Christian Sinner posted on Thursday, May 05, 2016 - 3:13 pm
Thank you very much for your answer.
I used output: cinterval (BCBOOTSTRAP);

Is it correct?
 Linda K. Muthen posted on Thursday, May 05, 2016 - 6:22 pm
We recommend CINTERVAL (BOOTSTRAP).
 Christian Sinner posted on Friday, May 06, 2016 - 2:55 am
I have tested CINTERVAL (BOOTSTRAP) before and it makes no difference. Indirect effect is not significant but in my opinion Bootstrap CI tells something different.

with CINTERVAL (BOOTSTRAP):

Effects from X to Y:
The estimate of the indirect effect is .095 and two-tailed p-value .112 -> I infer that there is no mediation.


Confidence Intervals of total indirect

Lower 2.5%: .006
Estimate: .095
Upper 2.5%: .233

-> I infer that there is a mediation.


Another computation with CINTERVAL (BCBOOTSTRAP):

The same estimate of the indirect effect (.095; p=.112) -> no mediation

Confidence Intervals of total indirect

Lower 2.5%: .011
Estimate: .095
Upper 2.5%: .247

-> exist a Mediation

Am I interpret the CI wrong? Shell I interpret that .006 contains the 0 because of the low value?

Is the first indirect effect (Effects from X to Y) not tested via a 95% CI?
If so how can I change that?


I am sorry if i interpret something complete wrong.
 Linda K. Muthen posted on Friday, May 06, 2016 - 6:45 am
Both types of bootstrap confidence intervals are non-symmetric. In your case, they yield the same results. You can use either. Neither contain zero.
 DavidBoyda posted on Thursday, January 19, 2017 - 12:18 pm
Dear Dr Muthen,

when reporting confidence intervals, should I report the model results or the logistic regression ones? This is mediated model with binary outcomes if that makes a difference.

BW,

D
 Bengt O. Muthen posted on Thursday, January 19, 2017 - 5:53 pm
Could you send the output to Support so we can see exactly what you refer to. And the license number.
 DavidBoyda posted on Monday, January 23, 2017 - 12:04 pm
yes no problem.

thanks again.
 Skyler Hawk posted on Friday, February 17, 2017 - 12:31 am
Hello,
Is there a resource I can use to determine the number of bootstrapped resamples I should be using to test my indirect effects? I'm examining two simultaneous mediators. The choice of how many resamples seems to be rather arbitrary, I haven't been able to find clear guidelines. Thanks for your advice!
 Bengt O. Muthen posted on Friday, February 17, 2017 - 9:53 am
We describe this in detail in Chapter 3 of our new book.
 Lars Dietrich posted on Tuesday, December 05, 2017 - 8:22 am
My problem is similar to Jodie Stearns'. I have an SEM model and use type=complex. How do i get indirect & total effects using bootstrapping as recommended by MacKinnon?

So far I specified:

Analysis:
Type = complex ;
bootstrap = 1000;
REPSE=Bootstrap ;

Define:
weight=1 ;

Can I trust my results? Or am I missing something?

Thanks,
Lars
 Tihomir Asparouhov posted on Tuesday, December 05, 2017 - 5:07 pm
Yes.
 dummyvariable123 posted on Tuesday, February 13, 2018 - 3:06 am
Dear Dr. Muthen,

I am testing a multilevel mediation (2-2-1) in a 3-level model:

%within%
Y;
Y ON wave;

%BETWEEN id%
Y;
M ON X(a);
Y ON X(c);
Y ON M(b);

%BETWEEN class%
Y;

MODEL CONSTRAINT:
new (direct, indirect, total);
indirect = a*b;
direct = c;
total = c+a*b;

Should I compare this model fit with a model where the path A "M ON X" is constrained to zero to show a relative model fit improvement in terms if -2LL (deviance)? Or with a model where path A is not specified at all?
 Bengt O. Muthen posted on Tuesday, February 13, 2018 - 6:05 pm
I'd say don't do either of those analyses - but SEMNET might have a different opinion.
 Ab Firoozabadi posted on Friday, June 01, 2018 - 6:15 am
Dear Linda,
I have tested a Multilevel mediation model.The results of within-person level show a significant indirect mediation effect (Indirect effect = .006, p-value=0.04). However, when I apply CINTERVAL, the results show that 95% confidence interval [CI] = .000 to .012. Is then the indirect hypothesis rejected?
 Bengt O. Muthen posted on Friday, June 01, 2018 - 5:36 pm
Use Bootstrap with Cinterval(Bootstrap) to get the best available CIs.
 Ab Firoozabadi posted on Saturday, June 02, 2018 - 5:54 am
Is bootstrapping allowed for multi-level modeling?
 Bengt O. Muthen posted on Monday, June 04, 2018 - 12:59 pm
No. Bayes can be an alternative here.
 Binay Adhikari posted on Tuesday, June 05, 2018 - 3:49 pm
Hi Linda,

Is the "CINTERVAL (BOOTSTRAP)", which is recommended in Mplus, biascorrected?

Thanks,
Binay
 Bengt O. Muthen posted on Tuesday, June 05, 2018 - 4:47 pm
No. We recommend this version which is not bias corrected.
 Binay Adhikari posted on Wednesday, June 06, 2018 - 9:14 am
Thanks.
 Ningning Zhou posted on Friday, August 24, 2018 - 6:09 am
I can't obtain the standardized indirect effect. The warnings are below.

*** WARNING in OUTPUT command
BOOTSTRAP and BCBOOTSTRAP confidence intervals are only available for the regular
results of the model and MODEL INDIRECT. No confidence intervals are printed for the standardized results of the model and MODEL INDIRECT.

What's the problem? Thank you.
 Ningning Zhou posted on Friday, August 24, 2018 - 7:57 am
Sorry for bothering. I found where is the problem. It is the version of the Mplus.
 Ningning Zhou posted on Friday, August 24, 2018 - 9:13 am
After I changed my Mplus version into Mplus 7.4 and 8.0, I still can't obtain the confidence intervals of standardized model and indirect effect.

The following is the warning.

*** WARNING in OUTPUT command
BOOTSTRAP and BCBOOTSTRAP confidence intervals are only available for the regular
results of the model and MODEL INDIRECT. No confidence intervals are printed for the standardized results of the model and MODEL INDIRECT.

My syntax is listed below:

USEVARIABLES = ocihoa ociche ociord ociadd ocicle iusneg iusavo iusdis rrsobs rrsref
isi1 isi2 isi3 isi4 isi5 isi6 isi7 psasom psacog;
ANALYSIS: bootstrap = 1000;
model: oci by ocihoa ociche ociord ociadd ocicle;
ius by iusneg iusavo iusdis;
rrs by rrsobs rrsref;
psa by psasom psacog;
isi by isi1 isi2 isi3 isi4 isi5 isi6 isi7;
ius rrs on oci;
psa on ius rrs;
isi on psa oci;
model indirect:
isi ind oci;
OUTPUT: sampstat stdyx cinterval(bootstrap);
 Bengt O. Muthen posted on Friday, August 24, 2018 - 5:51 pm
Send your input and data to Support along with your license number.
 Dana McCoy posted on Friday, January 25, 2019 - 1:15 pm
Dear Mplus team,

My colleagues and I are running an indirect effects model with 1 IV (binary), 2 DVs (continuous), and 6 mediators (continuous). We are using MLR with TYPE = COMPLEX to account for clustering in our data (~500 individuals within ~20 clusters).

We have been asked to use a bootstrap method. After reviewing the discussion board, our understanding is that Mplus only supports bootstrapping with TYPE = COMPLEX if weights are used. Following suggestions from the discussion board, we created a weight variable with a value of 1 and estimated the model with REPSE=BOOTSTRAP and CINTERVAL(BCBOOTSTRAP).

When we compare the results of the model from ML with bootstrapping vs. MLR, the standard errors of the indirect effects are larger with bootstrap than with MLR. Some paths were significant with MLR but non-significant with bootstrapping. We are not sure what is leading to the differences in results between the two methods, and are not sure which method is appropriate. We would appreciate any input on this.

Thank you in advance for your help!
 Tihomir Asparouhov posted on Monday, January 28, 2019 - 9:01 am
I assume you are comparing confidence intervals rather than SE.

Since you have only 20 clusters, asymptotic based inference could be questionable and a good simulation study (as close as you can make it) would be the best guide.
 Josefina Banales posted on Sunday, February 17, 2019 - 12:31 pm
Hello,

I ran indirect effects with the command "cinterval (BOOTSTRAP)" in the OUTPUT command. I noticed that the standard errors of the indirect effects nearly doubled, as compared to the standard errors for the indirect effects that did not include bootstrapping. Any idea why this change occurred? My dependent variables are binary, and the independent and mediators are continuous.
 Andre Maharaj posted on Monday, February 18, 2019 - 3:33 pm
Hello everyone,

I don't think I've quite found an answer to my specific question about using the bootstrap for indirect effects.

If the dependents are not multivariate normal, and I would like to test the indirect effects using the confidence intervals, which estimator should I use, since MLR does not allow bootstrapping?

Thanks in advance,
Andre.
 Bengt O. Muthen posted on Monday, February 18, 2019 - 4:21 pm
Answer to Banales:

Such big differences in SEs are usually not observed. You can send your two outputs to Support along with your license number.
 Bengt O. Muthen posted on Monday, February 18, 2019 - 4:22 pm
Answer to Maharaj:

Use ML with bootstrapping and bootstrapped CIs.
 Andre Maharaj posted on Monday, February 18, 2019 - 6:53 pm
Thank you very much. Just needed to confirm.
 Peter Rehder posted on Monday, August 10, 2020 - 1:42 pm
Drs. Muthen,

I'm conducting a latent growth curve model with multiple covariates and distal outcomes. The model also includes indirect effects examining the growth parameters as mediators of the covariates and distal outcomes. It's a large model, but it terminates normally including the indirect effects. However, when I add bootstrapping to improve the validity of the indirect effects, the model takes an extremely long time to run. I let the model run for about 12 hours and it didn't even complete 16 of the 1000 requested bootstrap draws. The computer I'm using has a 9th generation Intel 6-core CPU and 32 GB of RAM, so it's not a computing power issue.

1. Do you have any recommendations for speeding up the analysis when using bootstrapping with several indirect effects?

2. I understand that using Bayesian estimation may be an alternative that runs more quickly than ML estimation with bootstrapping and provides almost identical estimates. If I went that direction, would you recommend that I conduct step 1 of the analysis (the unconditional LGCM) using Bayes, or would it be acceptable to use Bayes in step 3 only? My suspicion is that it would be most appropriate to use Bayes from the beginning, if going that route, but your input would be helpful.

I greatly appreciate any guidance you can provide!

Thanks,
Pete
 Bengt O. Muthen posted on Monday, August 10, 2020 - 3:21 pm
Long computing time with bootstrapping may be due a large sample, many dimensions of integration, and many DVs. Bayes can help avoiding bootstrapping but if the sample is very large, it too can be slow.

I would use Bayes in both steps to be consistent.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: