95% Bootstrap Confidence Interval PreviousNext
Mplus Discussion > Structural Equation Modeling >
Message/Author
 Estee posted on Wednesday, October 16, 2013 - 6:22 am
Hi,

I tested a mediation model. The indirect effect was bootstrapped. The 95% confidence limit was [.000, .038]. According to literature, an indirect effect is considered sig. if the CI does not include/cross zero. Is the .000 showed in the results really equal to zero or it may be for example.0001 ? so, how to confirm whether the 95% confidence limit [.000, .038] is sig or not?

Thanks very much.
 Linda K. Muthen posted on Wednesday, October 16, 2013 - 3:57 pm
I would count that as non significant even if the value was .0001. With so many tests being performed, I would be conservative.
 Joao Garcez posted on Friday, March 13, 2015 - 10:27 am
Hi,

I was wondering something similar. I have a non-significant indirect effect (b = -.019, p>.05) with the following bc bootstrap 95% CI: ( -.047 - .002 ).

Is a value of .002 considered to be equal to zero?

Also, when values in the lower and upper bound have opposite signs like the one above, does it mean something? The path above is negative with a positive value in the CI. Does it mean is non-significant?

Thank you very much.
 Bengt O. Muthen posted on Friday, March 13, 2015 - 12:22 pm
If the CI covers zero, the coefficient is deemed insignificant. Your lower limit is negative and your upper limit positive, so your coefficient is non-significant.
 Joao Garcez posted on Tuesday, March 17, 2015 - 2:34 am
Thank you for the clarification.

Best,
JA
 Rosie Essery posted on Thursday, April 07, 2016 - 8:42 am
Hi there,

I too have a similar query - I have a non-significant indirect effect (B = 0.153, p = .104), but then the 95% BCa CI is [0.032 , 0.417].

I'm a but confused as to whether this is a significant mediation or not - the CI would suggest yes, but the p value no?

Would appreciate any advice - many thanks!

RE
 Linda K. Muthen posted on Thursday, April 07, 2016 - 8:52 am
If the confidence interval and p-value do not agree, you must not have a symmetric confidence interval.
 Rosie Essery posted on Thursday, April 07, 2016 - 9:26 am
Thank you for the very quick response!

I would agree that the confidence interval is asymmetric - looking at my data, the point estimate doesn't sit in the centre of the BCa confidence interval.

Does this mean that it is not meaningful to interpret this in the normal way, i.e. 0 not being between the lower and upper bounds of the confidence interval as a sign of significant mediation?

Many thanks for your helpful advice.

RE
 Bengt O. Muthen posted on Thursday, April 07, 2016 - 10:02 am
The non-symmetric CI is interpreted the same way: If 0 is not within the interval then the effect is significant.
 Kristin Hildenbrand posted on Thursday, May 25, 2017 - 2:02 am
Hello,

Related to the above: I run a single-level mediation model as specified below.The indirect effect (a1d1b2) based on z-test does not become sign (p = .112), but the 95% CI does not include 0 (.008-.394). If I run the respective process model in SPSS, the CI does also not include 0, showing significance. Bootstrapping is a more powerful test, but any idea how I should report these findings? My n is 146 and the CIs asymmetric.

Many thanks, Kristin

USEVARIABLES = WFBS Health ALQ FSSB sexI1 ageI1 child;
ANALYSIS:
TYPE = GENERAL;
ESTIMATOR = ML;
BOOTSTRAP = 5000;
MODEL:
Health ON FSSB (b1);
Health ON WFBS (b2);
Health ON ALQ (cdash); ! direct effect of X on Y
FSSB ON ALQ (a1);
WFBS ON ALQ (a2);
WFBS ON FSSB (d1);
Health on sexI1 ageI1 child;
MODEL CONSTRAINT:

NEW(a1b1 a2b2 a1d1b2 TOTALIND TOTAL);
a1b1 = a1*b1;
a2b2 = a2*b2;
a1d1b2 = a1*d1*b2;
TOTALIND = a1*b1 + a2*b2 + a1*d1*b2;
TOTAL = a1*b1 + a2*b2 + a1*d1*b2 + dash;
OUTPUT:
STAND CINT(bcbootstrap);
 Bengt O. Muthen posted on Thursday, May 25, 2017 - 6:36 pm
Perhaps you made a typo when you said (.008-.394) because this includes zero. In any case, a good approach is to use bootstrapping and the non-symmetric CI you get from that.
 Kristin Hildenbrand posted on Thursday, June 01, 2017 - 5:20 am
Thank you for this Bengt and sorry for the delay in responding. Yes, I meant (.008,.394) and just used the "-" to indicate the boundaries of the CI (I am aware that this is not normally done so, so sorry). Would you mind advising me on the best way to do bootstrapping in multi-level analysis as I think MPlus does not (yet) offer this feature and hint me towards a relevant paper/article etc. where I can read up on this? I heard various approaches but unsure which to pursue.
 Bengt O. Muthen posted on Thursday, June 01, 2017 - 6:54 pm
multilevel bootstrap is not available in Mplus. You can use twolevel Bayes and get the same kinds of non-symmetric CIs.
 Kristin Hildenbrand posted on Friday, June 02, 2017 - 2:57 am
Great, thanks!
 Mohammed Almosaiwi posted on Friday, September 22, 2017 - 11:18 am
Hello, I have a bootstrapped multilevel mixed effect model where the CI's contain 0 [-.0013, .04162] but the p < .05?

Are asymmetrical CI's the only explanation for this anomaly? Do you know of a reference I could cite to justify this?
 Bengt O. Muthen posted on Friday, September 22, 2017 - 4:44 pm
Q1: Yes.

Q2: You can cite our new RMA book if you like but it is a basic fact.
 Ceren Gunsoy posted on Friday, September 29, 2017 - 10:41 am
Hi,
When the CI and p value do not match for an indirect effect; for example, when the p value is > .05 but the CI does not include zero, do I need to report both values or is the CI more informative than the p value?
Thank you
 Bengt O. Muthen posted on Friday, September 29, 2017 - 1:39 pm
Use the CI only - it is more informative and takes into account possible non-symmetry in the distribution of the estimate.
 Ceren Gunsoy posted on Sunday, October 01, 2017 - 9:28 am
Thank you!
 Alice Graham posted on Thursday, October 05, 2017 - 1:51 pm
Hello,
You recently commented that you no longer recommend using the bias corrected CI (BCBOOTSTRAP)to test indirect effects. Could you provide a brief explanation of why you are no longer recommending this? Any information would be very much appreciated as we have consistently used this method in the past to test indirect effects.
Thanks very much
 Bengt O. Muthen posted on Friday, October 06, 2017 - 11:47 am
We discuss this in connection with simulations studies in our 2016 book Regression and Mediation Analysis using Mplus. The Type I error is found to be somewhat high for bias-corrected bootstrap which is in line with Biesanz et al., 2010 in Multiv Behav Res. The bias-corrected bootstrap CIs will most likely not be substantially different, however. You can also compare with what you get using Bayes which automatically gives non-symmetric CIs.
 Alice Graham posted on Friday, October 06, 2017 - 5:05 pm
Thanks very much for your response. I'll take a look at the references to learn more. I believe that the impetus for using the bias corrected CIs was because of concern that the non-bias corrected CIs for an indirect effect were too conservative. Is there something about the new bootstrap method in mplus that addresses this concern? Or do you recommend just using the Bayes to address this concern as it is still likely that the non-bias corrected CIs may be too conservative?
 Bengt O. Muthen posted on Saturday, October 07, 2017 - 8:51 am
The regular bootstrap method with percentiles creating the CI is what Mplus does (in addition to the bias-corrected version). Biesanz et al showed that it did better than the bias-corrected and they also showed Bayes to be good - as we have found.
 Sevgi Özdemir posted on Friday, November 17, 2017 - 12:27 am
When the CI and p value do not match for DIRECT effects in a mediation model (e.g., p value is larger than .05, but CI does not include 0), which one should I rely on to make a conclusion?
 Bengt O. Muthen posted on Friday, November 17, 2017 - 12:27 pm
The CI and p-values should agree unless the CI is obtained by bootstrapping. If this doesn't help, send your output along with your license number and mark where you see a disagreement.
 Sevgi Özdemir posted on Friday, November 17, 2017 - 1:30 pm
Thank you for your response. The CI is obtained by bootstrapping. In this case, which estimate should I rely on? p value versus CI for the DIRECT effects?
 Bengt O. Muthen posted on Friday, November 17, 2017 - 1:45 pm
The bootstrap CI is the better choice because it takes into account any non-normality in the sampling distribution for the direct effect.
 Amanda Flores posted on Friday, December 15, 2017 - 10:20 am
Hi. We have in our article a relationship between A and B, b = 0.009 [.001, .016], and between C and D, b = 0.0002 [.00004, .0003]. In a review we have been asked to use the same number of decimals for both values within the last CI.
How we should do this?? b = 0.0002 [.000, .000]??
Thank you very much!!
 Bengt O. Muthen posted on Friday, December 15, 2017 - 5:09 pm
This general question is suitable for SEMNET.
 Angeliki Argyriou posted on Monday, March 12, 2018 - 4:51 am
Hi, quick question: I'm using bootstrapped CIs, is there any way that I can also request for bootstrapped p values in my output command?

Many thanks.
 Bengt O. Muthen posted on Monday, March 12, 2018 - 3:38 pm
No, but bootstrapped SEs are used in computing the p-values.
 Angeliki Argyriou posted on Monday, March 12, 2018 - 4:49 pm
I see. Thank you for your quick response!
 Sara Namazi posted on Friday, May 25, 2018 - 12:55 pm
Hi Dr. Muthen,

I have a question regarding mediation analysis in mplus:

I am looking at whether the relationship between overtime work (OVERHRS) and depressive symptoms (BSISD) is mediated by work-to-family conflict (WFC).

I found the following results:

STDYX Standardization

Total indirect:

Lower .5% (-0.042)
Lower 2.5% (0.001)
Lower 5% (0.018)
Estimate (0.111)

Upper 5% (0.326)
Upper 2.5% (0.416)
Upper .5% (0.743)

Is it safe to interpret the lower & upper 2.5% and say that my results do show that work-to-family conflict mediates the relationship between overtime work and depression?
 Bengt O. Muthen posted on Friday, May 25, 2018 - 1:07 pm
That's what the 95% CI says, yes. I assume you use bootstrapped CIs. To get more information, you could use more bootstraps and you can also use Bayes and see what it says.
 Sara Namazi posted on Friday, May 25, 2018 - 1:15 pm
Hi Dr. Muthen,

Thank you for your quick reply and confirmation of my interpretation. Yes, I did use bootstrapped CIs. I will look at Bayesian analysis.
 Sonya Xinyue Xiao posted on Tuesday, July 17, 2018 - 11:01 am
Hello, Dr.Muthen,

I am running a cross-lagged model with two time points, and I want to test two mediation paths. 1) Predicting T2Y from T1X, mediated by M. 2) Predicting T2Y from M, mediated by T1X. M was a variable assessed at T1. First, is there a better way to test mediation paths given my time point constraints? That is, the predictor and mediator were both at T1 which is not ideal. I do have information of T2X, although I am not sure that's a better solution.
Second, I used Bootstrap and Model indirect command, however, the estimates were all zeros in the "confidence interval of total, total indirect, specific indirect, and direct effects" section as well as the sections below (STDYX, STDY, etc). I suspect there is a problem with those estimates because I dont think they should all be zeros...Fit indices, descriptives, as well as parameter estimates of path models all seemed ok. Do you have any suggestions for troubleshooting?

e.g.,

CONFIDENCE INTERVALS OF TOTAL, TOTAL INDIRECT, SPECIFIC INDIRECT, AND DIRECT EFFECTS


Lower .5% Lower 2.5% Lower 5% Estimate Upper 5% Upper 2.5% Upper .5%

Effects from T2Y to T1X via M

Sum of indirect 0.000 0.000 0.000 0.000 0.000 0.000 0.000
 Bengt O. Muthen posted on Wednesday, July 18, 2018 - 6:57 am
Your first question is more suited for SEMNET.

Regarding your second question - send your output to Support along with your license number.
 Sara Namazi posted on Tuesday, July 24, 2018 - 7:39 pm
Hi Dr. Muthen,

I recently ran a mediation analysis, and I am having a hard time interpreting my findings. I have been trying to look for examples, but have not been successful.

Lower (.5%) = -0.045
Lower (2.5%) = -0.001
Lower (5%) = 0.016
Estimate = 0.110
Upper (.5%) = 0.318
Upper (2.5%) = 0.410
Upper (5%) = 0.781

From this finding, is there an indication of a significant indirect effect?
 Sara Namazi posted on Wednesday, July 25, 2018 - 7:45 am
Hi Dr. Muthen,

I re-ran my, and I am having a hard time interpreting my findings. I have been trying to look for examples, but have not been successful.

Lower (.5%) = -0.042
Lower (2.5%) = 0.001
Lower (5%) = 0.018
Estimate = 0.111
Upper (.5%) = 0.326
Upper (2.5%) = 0.416
Upper (5%) = 0.743

From this finding, is there an indication of a significant indirect effect? I am interpreting it as there being an indirect effect at the 95% confidence interval beta= 0.11[0.001, 0.42].

Thanks,
Sara
 Bengt O. Muthen posted on Wednesday, July 25, 2018 - 4:08 pm
The first example has an insignificant effect and the second a significant one.

You can increase the number of bootstrap draws to better estimate the lower end of the CI.
 Sara Namazi posted on Wednesday, July 25, 2018 - 4:45 pm
Thank you, Dr. Muthen. What is your recommendation in term of increasing the number? I am currently drawing 10,000 random bootstrapped samples.
 Bengt O. Muthen posted on Wednesday, July 25, 2018 - 4:50 pm
That should be enough. You can also try Bayes which also gives non-symmetric CIs.
 Sara Namazi posted on Wednesday, July 25, 2018 - 5:31 pm
Thank you, Dr. Muthen. I am not familiar with Bayes. Do you have guidance on this topic and Mplus code?

Thanks
Sara
 Bengt O. Muthen posted on Thursday, July 26, 2018 - 6:06 pm
See our Short Course videos and handouts for Topic 9 and the shorter Bayes version within Topic 11.

See also my intro under Papers, Bayesian Analysis:

Muthén, B. (2010). Bayesian analysis in Mplus: A brief introduction. Technical Report. Version 3. Click here to view Mplus inputs, data, and outputs used in this paper.
download paper contact author show abstract
 Lu posted on Wednesday, September 26, 2018 - 1:49 pm
Hi Dr. Muthen,

I ran 18 models testing 6 mediators and 3 outcomes in Mplus using SEM. The mediators are correlated, so are outcomes. And I reported 95% bootstrap CIs.

Is there a way to correct for multiple testing in this case (a reviewer asked for this), esp given that mplus does give bootstrap p-values?

Thanks!
 Bengt O. Muthen posted on Wednesday, September 26, 2018 - 6:25 pm
You can be more stringent in your testing by using a 98% instead of a 95% interval in the bootstrap output.
 Snigdha Dutta posted on Thursday, September 19, 2019 - 1:24 pm
Having looked at all the queries and responses here, can I clarify as what counts as non-sig?

Would (lower 0.009) (estimate 0.035) (upper 0.061) be non-sig?

This is for lower/upper 5%.
If the second decimal place is also a 0, then that's close enough for it to be 0 and therefore be non-sig?
 Bengt O. Muthen posted on Saturday, September 21, 2019 - 11:32 am
Q1: yes.

Q2: This is a judgement call. I would report the CI.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: