How might one go about calculating the expected value of chi-square (for both WLS and WLSMV separately) when a model is known to be misspecified in a Monte Carlo study?
(Sorry if I have missed this elsewhere in the discussion. I have searched a few times recently but haven’t found it.)
Curran, West & Finch described a method in 1996 that they used to approximate the expected value of both the ADF and S-B chi-square. I was wondering if it could be adapted to both WLS and WLSMV, or if perhaps there is a better way. (Or if there is no way at all!)
As I understand the Curran, West, & Finch method:
1) Fit the misspecified model to an extremely large data set (e.g., 100,000 cases) that was generated in accordance with the true model.
2) Get the minimum of the fit function.
3) Scale this fit function according to the sample size to be used in the misspecified MC runs (e.g., 200, 1000, etc.)
4) Add this value to the misspecified model’s degrees of freedom.
5) The resulting value is a reasonable estimate of the expected chi-square given that particular model misspecification, that sample size, that estimation method, and the particular distributional characteristics of the variables involved.
Of course this would be somewhat complicated in the case of WLSMV, since the df vary across repetitions.
Do you think this is a reasonable way to proceed? Or is there a better way?
If this is a good way to go, would handling WLSMV be as simple as just adjusting the expected chi-square value according to the df estimated for that particular repetition? (Accommodating #4 above)
Anything you could say would be greatly appreciated.
I don't know the answer to that. With WLSMV we only use the p value which might complicate considering the expected mean. I would ask if it is necessary to get the expected value, or if the problem could be approached differently.
Hi Dr. Muthén, thank you for your comments. My aim in trying to figure it out is to compare the two methods for bias in chi-square when a model is misspecified.
I need to play with it some, but I suppose this can basically be done about as well with the p-values instead of the actual chi-square values. I could start by using the technique above for WLS. Then I guess that information could be used to provide a point estimate of what the p-value should be in any particular cell if WLS were performing in an unbiased fashion.
I could also then compare the empirical WLSMV p-values to both the empirical WLS p-values and the “ideal” WLS p-values, but then this means that I am using WLS standards of unbiasedness for WLSMV p-values. This seems reasonable, but what do you think?
I’m not quite sure what to think of this. Applied in this context, the Curran, West & Finch method basically seems to assume that an estimation method asymptotes to the correct place. From what I have read about WLS, that seems reasonable. And if WLS is in fact asymptotically unbiased, then it seems fine to use its ideal p-values as a yardstick for WLSMV also. This makes sense to me because the two methods are both designed for use in the same situations.