BOOTSTRAP cannot be used with COMPLEX. BOOTSTRAP does not take into account lack of independence of observations. You need COMPLEX for that.
Sarah Ryan posted on Thursday, September 15, 2011 - 11:01 am
I am using data from the ELS:2002 data set (an NCES data set). Is it possible to use "Resampling Methods in Mplus for Complex Survey Data" (Tihomir Asparouhov and Bengt Muthen, May 4, 2010) as a guide in obtaining bootstrap standard errors? If so, would bootstrap SE's also then be provided for the MODEL INDIRECT output?
Like the example given with ECLS data in this paper, the sampling structure is available in the ELS:2002 data, i.e., the strata and PSU (cluster) variable are available in the sample.
I try to do a mediation and compute indirect/direct effects using SEM with latent variables and a complex sample design (Cluster, Stratum and weight). Since I cannot use bootstrap with COMPLEX, I tried to use the "Resampling Methods in Mplus for Complex Survey Data". Since the p values changed slightly, I would like to be able to justify this choice.
So: 1-Is the “Resampling Methods [bootstrap in my case] in Mplus for Complex Survey Data" give SE that are more “accurate”, or does the idea of the resampling methods was implemented to answer specific problems. In other words, when we have complex sample design, is it generally better to use this method, or only in certain (precise) cases?
2-Will it still be possible/adequate to use model constraints to compare strength of my direct and indirect effects?
Just for clarification, is it or is it not incorrect to use the "repe=bootstrap" command if you do not have replicate weights?
Said another way, if I have weights, strata, and clustering design effects but not replicate weights, it is appropriate to use "repse=bootstrap" to obtain bootstapped confidence intervals for indirect effects?
what would you recommend for 2 mediatian analyses on 2 levels with TYPE IS TWOLEVEL COMPLEX that is comparable to bootstrapping? I tried to calculate the indirect effects via Estimator = bayes and MODEL CONSTRAINT. However Estimator BAYES is not allowed with TYPE=COMPLEX. Also the resampling methods does not seem to work.
You can use Bayes with type=threelevel and model both levels that are on the cluster command. At this point this is the only way to obtain asymmetric distributions for the indirect effects in your situation.
I have data from children nested in classes so I use type = complex. Model is with indirect effects (latent and observed variables) . Models without bootstrap and with bootstrap (type =complex with WEIGHT IS weight; and weight =1) give substantially different results not only in estimation of INDIRECT effects but also DIRECT effects. For example: part of result from model WITHOUT bootstrap DEPR ON LIKE -0.247 0.208 -1.189 0.234 DISLIKE -0.359 0.359 -1.001 0.317 And the same WITH bootstrap DEPR ON LIKE -0.275 0.140 -1.966 0.049 DISLIKE -0.421 0.224 -1.877 0.061 Which result should I believe? What kind of analysis do you recommend to obtain trustworthy results?
Bootstrap does not affect the point estimates. There must be another difference between the files (such as different weight variable which can affect the point estimates). Since this is a mixture model it maybe that one of the runs is at incomplete convergence/local maximum (but that is unlikely if you have all other estimation settings the same). To avoid that problem you can use the run with the higher likelihood to obtain good starting values for the other run (using output:svalaues in the higher likelihood run and then use these starting values for the other run with starts=0). If you can't figure it out send your example and data to firstname.lastname@example.org
Thank you, I add wieght to model without bootstrap (the same as in model with) but it doesn't change it's point estimates (the differences are still there). Now I'll try with svalues.
shonnslc posted on Wednesday, March 06, 2019 - 4:51 pm
I try to compute indirect effect with type = complex (I have nested data structure). However, bootstrapping is not choice in this context. Also, I don't have information about replicate weight. In this situation, should I simply go with type = complex option and accept the possible bias with the sobel test of the indirect effects? Thanks.
If the complex sample issue is only that of clustering, you can use a two-level model with Bayes which gives you the desired non-symmetric confidence intervals for indirect effects.
shonnslc posted on Wednesday, March 06, 2019 - 5:38 pm
Dear Dr. Muthen,
Thank you! Since two-level with Bayes is still new to me, especially I am working on structural equation modeling (but only interested in first-level relationship), could you point out some references I can read by myself in order to use the approach you suggest. I really appreciate your help.
An intro to Mplus Bayes is given in the paper on our website
Muthén, B. (2010). Bayesian analysis in Mplus: A brief introduction. Technical Report. Version 3. Click here to view Mplus inputs, data, and outputs used in this paper. download paper contact author show abstract
See also our Short Course Topics 9 and 11 as well as Chapter 9 of our book:
BRR replicate weights are constructed for standard error estimation, not for chi-square testing or fit index computations or non-symmetric confidence interval construction.
You can get TLI and CFI when you remove the REPWEIGHTS command - that is the same model and has the same model fit. To get the confidence intervals use CINT option, which gives the symmetric interval based on the variance estimated with the replicate weights.
If you want to get bootstrap or bcbootstrap CINT you have to use the bootstrap command, not BRR replicate weights.