Bias-corrected bootstrap
Message/Author
 Viktoriya Magid posted on Monday, April 18, 2005 - 7:47 am
Hi all,

Just wanted to find out what formula is being used by Mplus to calculate the bias in bias-corrected bootstrap confidence intervals. Is it the formula that Efron had established or is the bias being calculated some other way?

Thank you very much.
Yours,
Viktoriya Magid
SUNY at Buffalo
 bmuthen posted on Tuesday, April 19, 2005 - 12:04 pm
Please see the KcKinnon et al 2004 reference in the Mplus Version 3 User's Guide. I believe that is the Efron version.
 bmuthen posted on Tuesday, April 19, 2005 - 12:07 pm
P.S. I meant to say MacKinnon.
 Viktoriya Magid posted on Wednesday, April 20, 2005 - 1:09 pm
Dear Dr. Muthen,

Thank you for getting back to me. From what I know, the Efron formula considers both bias and acceleration, however MacKinnon (2004) considered only the bias in his simulation (he found that considering the acceleration in addition to the bias did not improve the accuracy of confidence intervals). Hence, to elaborate on my earlier question, does Mplus consider both bias and acceleration in computing the bias-corrected bootstrap confidence intervals (as has been suggested by Efron in his formula) or does it consider the bias only (as has been done by MacKinnon, 2004)?

Yours,
Viktoriya
 Linda K. Muthen posted on Wednesday, April 20, 2005 - 2:48 pm
Mplus considers bias only as in MacKinnon.
 Viktoriya Magid posted on Saturday, April 23, 2005 - 7:00 am
Thank you very much.
 Andrew posted on Wednesday, June 08, 2005 - 12:33 pm
Does Mplus have a way to change the seed of the random bootstrapping algorithm? All the seed options in ANALYSIS: don't seem to do anything, and the bootstrap results don't appear to vary across runs. The only thing that I can do to get variation across bootstrap results is to change the number of bootstraps.
 Linda K. Muthen posted on Wednesday, June 08, 2005 - 5:17 pm
No, there is no option to change to seed in bootstrap.
 Minnik Findik posted on Thursday, January 17, 2013 - 7:16 am
I am using a subset of a longitudinal data set (as not everyone attended a focus group). The ones who dropped out were mostly disadvantaged individuals with less education, from lower SES level etc. I want to run a SEM model (with 4 latent variables) and 3 observed variables (all binary). I use the WLSMV estimator.
So my question is:
1) is it possible to run a SEM model with Heckman correction for the missing data?
2) if I run the model for the full sample (regardless of whether they have attended the focus group or not), Mplus will do some kind of imputation, would be similar to Heckman correction or not?

Thank you very much in advance.
 Bengt O. Muthen posted on Thursday, January 17, 2013 - 8:10 am
Missing data is properly handled by ML or Bayes assuming MAR, which is the Mplus default setting. WLSMV is less suited to handling missing data (see UG). Heckman modeling can be done in Mplus, but I would simply use ML or Bayes and our default settings. For more on missing data modeling, see

Muthén, B., Asparouhov, T., Hunter, A. & Leuchter, A. (2011). Growth modeling with non-ignorable dropout: Alternative analyses of the STAR*D antidepressant trial. Psychological Methods, 16, 17-33.
 Minnik Findik posted on Thursday, January 17, 2013 - 8:48 am
But I also want to see indirect effects and as far as I know ML does not allow estimating indirect analysis.

I am not aware of Bayes. Will it allow for estimating indirect results?

Best wishes.
 Bengt O. Muthen posted on Thursday, January 17, 2013 - 8:54 am
You can use Model Constraint to form your own indirect effects and get their est's and SEs.
 Minnik Findik posted on Thursday, January 17, 2013 - 9:06 am
Thank you very much. I think it would have been great and much easier (and maybe better) than applying Heckman correction but unfortunately I do not have the computer capacity to run the ML estimator. It gives an error "*** FATAL ERROR THERE IS NOT ENOUGH MEMORY SPACE TO RUN THE PROGRAM ON THE CURRENT INPUT FILE. THE ANALYSIS REQUIRES 4 DIMENSIONS OF INTEGRATION RESULTING IN A TOTAL OF 0.50625E+05 INTEGRATION POINTS. THIS MAY BE THE CAUSE OF THE MEMORY SHORTAGE. YOU CAN TRY TO FREE UP SOME MEMORY BY CLOSING OTHER APPLICATIONS THAT ARE CURRENTLY RUNNING. NOTE THAT THE MODEL MAY REQUIRE MORE MEMORY THAN ALLOWED BY THE OPERATING SYSTEM.***"
And a reviewer suggested that I should use Heckman correction so I would appreciate if you can direct me to somewhere where I can learn more about how to do that.
 Bengt O. Muthen posted on Thursday, January 17, 2013 - 10:30 am
You can use ML with

integration=montecarlo(5000);
 Minnik Findik posted on Thursday, January 17, 2013 - 11:32 am
unfortunately it still gives the same error.
 Bengt O. Muthen posted on Thursday, January 17, 2013 - 12:31 pm
 Minnik Findik posted on Thursday, January 17, 2013 - 1:29 pm
I will send the output as soon as possible but I managed the run my model using estimator = Bayes (it took 2 hours but I have it now! so thank you very much for the suggestion).

I have 3 questions:
1) Is it similar to WLSMV estimator, so if the endogenous variable is categorical it is probit coefficients and if it is a latent variable or continuous variable it is a linear regression coefficient?

2) Can I use model constraint to look for indirect effects?

3) when I run my model with WLSMV, I get some fit indices (CFI & TLI) but with Bayes I did not get any of that so how would I know if my model fits?

Thank you very much!
 Bengt O. Muthen posted on Thursday, January 17, 2013 - 3:12 pm
1. Yes. WLSMV and Bayes only differ in Bayes being a full-information estimator, therefore handling missing data better.

2. Yes.

3. Bayes in Mplus gives PPP (posterior predictive p-values) - see our Bayes reports under Papers, Bayesian Analysis. That is, Asparouhov-Muthen (2010) papers.
 Samuel McAbee posted on Sunday, July 06, 2014 - 1:51 pm
Dear Dr. Muthen,

I am conducting a number of bias corrected bootstrap mediation models involving one predictor variable (each), two mediating variables, and three outcome variables.

I used 1000 bootstrap replications and the CINTERVAL(BCBOOTSTRAP) option to run my models.

I have run several of these models, but for one of my models I obtain 2.5% and 97.5% confidence interval estimates for the indirect effects that are beyond -1 and 1. I recognize that my models do not imply mediation, however, I am concerned that I might have missed something... These out-of-range values do not occur for any of my other models. Could you please point me to a few references that might help me understand why this might have occured?

 Bengt O. Muthen posted on Sunday, July 06, 2014 - 4:15 pm
The size of an indirect effect depends on the scales of the variables and is not restricted to be less than 1.
 Samuel McAbee posted on Sunday, July 06, 2014 - 6:14 pm
Thank you for your quick response.
 Daniel Kopala-Sibley posted on Thursday, October 23, 2014 - 5:30 pm
Hello, I'm running a mediation model with bias corrected bootstrap.

I'm confused though, as the output from the model indirect command gives me a p-value of .10 for the indirect effect, but the cinterval command says that my indirect effect ranges from .02 (lower 2.5%) to .052 (upper 2.5%). Give that these are bounded within a 95% CI, shouldn't the p-value for indirect effect be <.05?

 Daniel Kopala-Sibley posted on Thursday, October 23, 2014 - 5:36 pm
And, to follow up, which of the two should I report when reporting te significance of the bias-corrected bootstapped indirect effect? (I used cinterval(bcbootstrap) ).

Thanks again
 Bengt O. Muthen posted on Friday, October 24, 2014 - 8:06 am
The z-value should be compared to a symmetric bootstrap. The bias-corrected bootstrap is not symmetric. It may be more appropriate for an indirect effect.
 Daniel Kopala-Sibley posted on Friday, October 24, 2014 - 8:25 am
Thank you for the suggestion. I reran the model with CINTERVAL(BOOTSTRAP) rather than BCBOOSTRAP.
The 95% CI for the indirect effect is still .001 to .05, implying a significant indirect effect, but the p-value of the indirect output is .09.

Which would you suggest interpreting in terms of concluding whether or not the indirect effect is significant? Also, do you know why the two would be different?

Thanks
 Bengt O. Muthen posted on Friday, October 24, 2014 - 10:28 am
We recommend BCBOOTSTRAP in line with the mediation book by MacKinnon. We recommend reporting the 95% CI.
 Timothy Allen posted on Friday, September 08, 2017 - 11:59 am
Hello,

I'm running a latent growth curve mediation model where the growth factors are my mediators.

When I use BCBOOTSTRAP for bootstrapping, the results suggest the indirect effects for both the intercept and slope term are significant. However, the "A path" from slope to my IV is not significant according to the CIs (-.10, .02).

When I use the BOOTSTRAP option, only the indirect effect for the intercept is significant, and both the a and b paths are also significant, as one would expect.

I'm wondering how to make sense of this discrepancy, and which bootstrap option to use? I know you usually recommend BCBOOTSTRAP, but that seems to be providing some odd results in this case.

Thanks!
 Bengt O. Muthen posted on Saturday, September 09, 2017 - 5:56 pm
We now recommend BOOTSTRAP per recent research that we discuss in our book:

http://www.statmodel.com/Mplus_Book.shtml

You may also want to try Estimator = Bayes to get another assessment.
 sarah M posted on Monday, April 08, 2019 - 11:59 am
Dear Dr. Muthen,

I wanted to set a seed number when obtaining a bootstrap confidence interval. Is there a way to specify seed number in order to replicate the results?

Thanks.
 Tihomir Asparouhov posted on Tuesday, April 09, 2019 - 10:01 am
We currently do not have such a command. You can try to double the number of bootstrap draws and make sure that the confidence intervals do not change substantially.
 Scott R. Colwell posted on Sunday, May 05, 2019 - 7:22 pm
I note in a post above that, as of 2005, there wasn't an option to set the seed number for Type = Bootstrap. Is this possible now?

Thank-you,
 Tihomir Asparouhov posted on Monday, May 06, 2019 - 11:19 am
Still not available. Choosing a large enough bootstrap replications should eliminate the need to study the variability of the estimates.
 Sarah Edmunds posted on Friday, February 07, 2020 - 5:37 pm
Hello,

Is there a way to have Mplus generate p-values that directly correspond to the bias-corrected confidence intervals obtained via bootstrapping (i.e., BCBOOTSTRAP)?

For a recent manuscript, we ran several serial mediation analyses based off of a priori hypotheses. We have received reviewer requests that we make a familywise error adjustment. For this, we would need p-values directly equivalent to the bias-corrected CIs. Is this possible to obtain?

Thank you!

Sarah
 Tihomir Asparouhov posted on Monday, February 10, 2020 - 10:17 am
It is possible but not very easily available. Here is what you would need to do. Make sure you use a large amount of bootstrap draws, at least 1000. Add
plot:type is plot3;
Run the analysis and click on the plot icon. Select "Bootstrap distribution". Select the parameter you want and use 1000 bins for this exercise as you will need the extra precision. This is the raw (uncorrected) bootstrap distribution. You need two things from this bootstrap distribution. The percentile of draws p1 below zero and the percentile of draws p2 below the point estimate. To get these, right button click on the plot and select save plot data. The save data file has two columns. The first column is the bin value and the second column is the count (the actual bootstrap values are not currently available). In excel or google spreadsheet add the count values up to the bin value closes to 0 and closest to the point estimate (from the Mplus output). If Phi is the standard normal distribution function the bias corrected one-sided P-value will be computed by
Phi(2*Phi^-1(p2)+Phi^-1(p1))
Here 2*Phi^-1(p2) represents the bias correction and p1 is the standard bootstrap one sided p-value. This will work for positive point estimate. For negative point estimate you would compute p1 as the proportions of bootstrap draws above zero. If p2=0.5, the bias correction is zero and the bias corrected P-value is the same as the bootstrap P-value.
 Tihomir Asparouhov posted on Thursday, May 14, 2020 - 9:00 am
The one sided P-value formula given in the previous message has an error. There is a missing negative sign. It should be

Phi(-2*Phi^-1(p2)+Phi^-1(p1))

This is now described in more details in this FAQ