Anonymous posted on Thursday, November 21, 2002 - 2:07 pm
The RUNALL Utility allows one to run Mplus with a set of external data files and appends the model results from each run into one file for other studies. I guess, the external data files should be generated by bootstrapping from the original data. If so, this approach should be called bootstrapping simulation rather than Monte Carlo simulation as the latter has no raw data to start with. Please correct me, if I am wrong.
bmuthen posted on Thursday, November 21, 2002 - 2:59 pm
You are right that the RUNALL Utility runs a set of external data files. How these files have been generated is up to the user. RUNALL was intended for Monte Carlo studies where data were generated. However, it's fine if the data files have been generated by bootstrapping a real data set.
I have created an input file which I put in the directory C:\testdir, along with the four files runall.bat, runstart.bat, runone.bat, and runend.bat. I modified runstart.bat
I created 50 datafiles in ASCII using SPLUS. I put these files in the directory C:\testdata.
When I type RUNALL at the MS-DOS window, I get a message in Dutch that can be translated as follows: RUNONE.BAT is not recognized as an internal of external task, programm or batch file. This message is given 50 times (once for each data file it seems).
What may have gone wrong? I am using Windows 2000 professional on an IBM compatible computer.
I believe that you have Version 3. In Version 3, RUNALL has been incorporated in what we refer to as external Monte Carlo. It is much easier to use. See Step 2 of Example 11.6. See the FILE and TYPE options of the DATA command for more information.
Kent posted on Wednesday, August 18, 2004 - 12:37 pm
From reading the posts and the MPLUS manual, it seems that version 3.01 has bootstrapping capabilities. Specifically, it appears that MPLUS produces naive bootstrap standard errors. Is this correct? Can it produce Bollen-Stine bootstrap standard errors or Bollen-Stine bootstrap chi-square probability values (Bollen and Stine, 1992)? Thanks
bmuthen posted on Wednesday, August 18, 2004 - 12:56 pm
Mplus produces the bootstrap and bias-corrected bootstrap SEs described in the MacKinnon et al (2004) article in MBR. No (Bollen-Stine) chi-square is currently given.
Kent posted on Thursday, August 19, 2004 - 9:45 am
Thanks for the clarification. Two more quick questions: It's my understanding that the following syntax specifies 500 bootstrapped samples being drawn from the original data. Is that correct?
BOOTSTRAP = 500
What is the size of the bootstrap sample? Is the default the sample size of the original observed data (i.e. if original sample included 1000 cases each bootstrapped sample would consist of 1000 cases)? Is there a way to specify the size of the bootstrap sample? Thanks
bmuthen posted on Thursday, August 19, 2004 - 9:56 am
Yes, Bootstrap = 500; is the correct syntax. The size of the sample, drawn with replacement, is the same as the original sample size. Other sizes cannot be specified.
Emily Blood posted on Saturday, July 28, 2007 - 7:10 am
Using the RUNALL utitlity I am trying to output both the parameter estimates and standard errors of the parameter estimates. In my input file SAVEDATA: statment, if I specify ESTIMATES = then only the parameter estimates appear. If I specify RESULTS ARE then only the iterations from the estimation occur. In the Mplus manual it states that the RESULTS saved when specifying RESULTS ARE include parameter estimates, standard errors of the parameter estimates and fit statistics. I do not find this to be the case. I also find that outside of the RUNALL utility, the same is true in terms of saved output. I cannot get a saved output file that contains the parameter estimates and their standard errors. Any help with this would be much appreciated. Thank you. (I am working with Mplus version 4.2)
Mplus external Monte Carlo facility supersedes RUNALL. I think you will find it more convenient to use. It also summarizes the results in a Monte Carlo format as described in Chapter 11. See Example 11.6 Step 2.
Dr. Bengt Muthen mentioned in Mpus Discussion, bootstrap should be used when sample size is small. Mplus provides bootstrap and bias-corrected bootstrap SEs using MacKinnon’s method (2004). With the bias-corrected bootstrap, how robust the Mplus results could be when sample size is small (e.g., <200)? How small is small? What about N<100? Your help will be appreciated.
Hi Linda, Thanks for answering my question. By your statement ˇ°I don't think bootstrap makes much difference for sample sizes greater than 50,ˇ± do you mean bootstrap would work better when N<=50? I just want to confirm that bootstrap can deal with small samples (N<100) for less complex SEM. For example, for a single factor CFA with 10 indicators, can we trust bootstrap results for model fit and parameter testing when sample size is small ( e.g., N=50)? Thanks a lot for your help.
The reason why I keep asking this question is that findings in different studies are inconsistent. For example, in their study (2001), Nevitt & Hancock conclude: "use of the bootstrap with samples of size n = 100 is unwise." "bootstrapping may fail with samples of sizes n = 200 (or even larger) with more complicated models (p.374)." Sorry to take you too much time.
It may not be that the findings are inconsistent but that the studies aren't looking at the exact same model, sample size, etc. I am not an expert on the bootstrap literature so cannot really say more than I have said and point you to the articles that I did.
Just to add a comment to this, I think the bootstrap approach to SEs was revitalized in the mediation context, where it was of interest to study the SE for a product of 2 slope estimates. ML gives asymptotically normal estimates, but the product of 2 normally distributed estimates is non-normal for finite-sized samples, motivating bootstrapping. But with n=200, I doubt that the non-normality is something one has to worry about and the conventional Delta method could be used.
Tobias Koch posted on Monday, September 21, 2009 - 6:07 am
I am trying to use the RUNALL utility. Therefore, I generated 10 external files, and specified the INPUT file, and changed the RUNSTART file. After running the RUNALL command, I received an error message, saying the FILE that needed to be specified in the INPUT file cannot be found. This sounds awkward, because this file is suppose to be a generic file, which is only specified in the INPUT file and the RUNSTART file. Is should not exist.
What may have gone wrong? I cannot find my mistake.
Try using external Monte Carlo. This has replaced RUNALL. See Example 11.6 Step 2.
Jenny Chang posted on Tuesday, September 28, 2010 - 4:45 am
Hi, Linda. I learn from above that Mplus external Monte Carlo facility supersedes RUNALL.
If I generate 10 datasets from the same latent class model, and want to analyze the accuracy of BIC and AIC for each dataset. But the output of external Monte Carlo only contains the means of these criteria. And it is the same with the output of ex11.6 step2, which contains only the mean of chi-square rather than 100 respective chi-square.
How can I output all the BIC and AIC for each data respetively and orderly?