I am testing for measurement invariance, and I had a question about testing for strict factorial invariance, where the item residual variances should be equal across groups (men and women for example). I have categorical items and am using WLSMV with the PARAMETERIZATION = DELTA option. If I do the following under this setting in the multiple groups (i.e. equalize the scalar factors across groups):
Does that have the same effect as equalizing the residual variances? Is this a correct way of testing strict factorial invariance (given that the constraints for configural, metric and strong factorial invariance are also incorporated in the model)? If no, what would be the correct way to test whether the residual item variances are equal between groups? How can I equalize the residual variances?
If you want to test for residual variance invariance, you should use the THETA parametrization where the residual variances are parameters in the model instead of the scale factors of the DELTA parametrization. The models that test this are a model with all residual variances fixed to one in all groups versus a model with residual variances fixed to one in one group and free in the other groups.
I want to do exactly the test you specified above: I want to compare a model which tests strong factorial invariance (intercepts and factor loadings free; residuals fixed to one in one group and free in the other 2 groups) with a model of strict factorial invariance with intercepts and factor loadings free and residuals fixed to 1 in each group.
Obs.Data is categorical; WLSMV estimator and Parameterization = THETA is used.
syntax differences: in the more restrictive model observed residuals (variable names "kd1";"kd2";etc.) are equal e.g., kd (1); kd (2); etc.
Result: The Chi square value is much lower for the strict model (and has less dfs) than for the less strict model in which residuals are free.
As i understand, a more restrictive model should generally have higher dfs and a higher Chi-square value, because there are less parameters to estimate which should yield in worse fit - or in the best case equal fit, indicating that the model is preferable (more parsimonious).
Questions: (1)May you help me with the problem described above?
(2)In Guide 6, you do not recommend testing strict factorial invariance with categorical data (p. 434). Why not?
(3)If i want to compare different models concerning invariance, i have to use DIFFTEST instead of comparing chi-values, right?
When you say intercepts and factor loadings are "fixed", I think you mean "held equal across groups". The stricter model with equal residual variances should not hold them equal and free across groups, but fixed at 1 in all groups.
A more restrictive model has more df's. It also has a worse, i.e. higher, fitting function value, where this value is found using Tech5 at the end of the iterations in the left-most column. With very ill-fitting models, chi-square is a less good gauge than the fitting function. DIFFTEST checks nesting using the fitting function values.
Testing of residual variance equality for categorical outcomes can be done, but since you often don't find equality and since it is not necessary for group comparisons of the factor means and variances, why do it?
Yes, DIFFTEST is the right approach.
Sascha posted on Monday, September 19, 2011 - 3:36 am
Thank you very much for your advice!
I am still wondering, if my results are correct, because the more restrictive model showed better fit indices (WLSMV and Theta Parameterization used in both models).
Question: I thought, that restricting a model should yield in an equal or worse fit ... Are CFI and RMSEA better because of parsimony or do you think there are mistakes in the syntax?
Data: Model 1: Configural invariance; factor loadings and intercepts free, latent means fixed in all groups; Result: Chi2=161.045, df=104;free par.=116;CFI=.975,RMSEA=.051, last Function-value in Tech5 output: 0.18299438D+00.
Model 2: Intercept Invariance; factor loadings and intercepts equal across groups; latent means fixed in one group. Chi2=170.101, df=115;CFI=.976,RMSEA=.047. Function-value in Tech5 output: 0.19829076D+00.
Model 3: Strict factorial invariance; factor loadings and intercepts equal across groups; latent means fixed in one group; equal residual variances (fixed to 1 in each group). Chi2=165.826, df=115;CFI=.978,RMSEA=.045. Function-value in Tech5 output: 0.25836231D+00.
Latent variances free in all models.
Difftest 1 vs. 2 is not significant: Chi2=22.294, df=23, p=.50 Difftest 1 vs. 3 is not significant: Chi2=53.159, df=43, p=.13
I have a run a CFA to test for strict factorial invariance of a 12 item list where all items have 3 ordinal categories. I have chosen the theta parametrization and used the WLSMV estimation method. Most fit indices point towards adequate fit of the strict factorial invariance model (Chi-Square/df = 1.6, RMSEA = 0.054, CFI = 0.981, TLI = 0.985). Moreover, the model fit is not worse when compared to the partial factorial invariance model where residual variances are not fixed (p = 0.0526 using DIFFTEST), albeit it borderline. My only concern is the large WRMR for the models: WRMR = 3.125 for the strict invariance model and WRMR = 2.813 for the partial invariance model. Ideally these should not exceed 1. Should I worry about these values? All modification indices are below 20 so not big improvements are to be expected by freeing some parameters.
I am trying to run a strict invariance with dichotomous variables and am having some difficulties. It SEEMS like I have everything right, but I'm getting the following error message: *** ERROR The following MODEL statements are ignored: and then it just lists the variables. I've played around and found I do not get the error message if I remove the rotation line from the syntax (cf-quartimax(oblique), I do not get the error message because Mplus gives no output whatsoever. I'm using the theta parameterization, wlsmv estimator but this odd error message; What is going on?