I am evaluating measurement invariance with categorical indicators. I am following procedures in chapter 14 of the user manual in terms of model specification. Looking at partial invariance I am using modification indices and EPCs using the WLSMV estimator and TECH2 output using the MLR estimator. I wonder why, given the same model (aside from constraints for identification), only difference being the estimator, suggested parameter changes seem to disagree markedly. I had assumed results would be somewhat similar. Is my assumption incorrect?
Thank you for including that request. I will explore both options.
Martina Gere posted on Thursday, February 09, 2012 - 4:10 am
Dear Prof. Muthen,
I get different parameters when using WLSMV and MLR too, but in a different context. I am running a model with two independent latent variables f1 and f2 (using ordinal item indicators), and dependent latent variable f3 (all continuous indicators). I am testing f3 ON f1 f2.
I used WLSMV for model fit. Regression coefficients were as expected (f3 ON f1 sign, f3 ON f2 not sign). In addition I ran MLR to get the loglikelihood estimate (because in the next step I want to compare loglikelihood with a moderator model, where I add f3 ON f1xf2 using XWITH). However, in MLR, regression coefficients are the opposite (f3 ON f1 not sign., but f3 ON f2 sign.)
Q1: What is the most appropriate estimator for my model, WLSMV, MLR, or something else? Q2: Can I use ML instead of MLR as estimator in the models with and without the interaction (to compare loglikelihood)? ML leads to regression coefficients similar to WLSMV and as expected. Q3: Why does MLR lead to different regression coefficients than ML or WLSMV?
If you have a log of missing data and less than four factors, I would recommend maximum likelihood. If you don't have missing data and have many factors, I would recommend weighted least squares. Weighted least squares uses probit regression. Maximum likelihood uses logistic regression as the default but can also use probit regression.
ML and MLR produce the same parameter estimates. The parameter estimates for ML and MLR should differ from WLSMV unless you are using the probit link with ML and MLR. I would need to see the outputs and your license number at firstname.lastname@example.org to explain the differences.
I did find a reference in Beauducel and Herzberg (2006,p. 201) and Dolan (1994). It appears that:
- WLSMV outperforms ML on sample size - WLSMV outperforms ML with variables that had 2 or 3 categories - Other than chi-square, there appears to be no difference in model fit statistics - ML underestimates size of loadings when variables had only two or three categories - Factor loadings and standard errors not affected by number of factors - Even with small sample, large model and moderate loadings, WLMVS is better - 5 categories is the minimum for 5 categories - "It is clear that a method like WLSMV, which was designed to deal with categorical variables, cannot outperform ML estimation when the number of categories is very large."
Thus it seems to me that at least with binary indicators or 3 category categorical indicators, WLSMV is the recommended approach unless you have missing data, in which case, ML is likely better as it handles MAR.
However, this seems contrary to Dr. Muthen's Feb 9, 2012 posting that WLSMV should be used if you don't have missing data and have many factors, I would recommend weighted least squares. Weighted least squares uses probit regression
You can also use multiple imputation to generate data and analyze it with WLSMV.
You can have as many factors as you want with categorical indicators and maximum likelihood estimation. It may not be feasible to wait for more than four.
It sounds to me like the paper compares weighted least squares treating the variables as categorical with maximum likelihood treating the variables as continuous. This does not compare estimators but compares treating variables as categorical versus continuous. The important and valid comparison would be treating the variables as categorical with both weighted least squares and maximum likelihood.
We got the article. When he says ML, he means treat the variables as continuous. That is not maximum likelihood estimation with categorical variables. It is simply using the continuous variable maximum likelihood fitting function whereas the true categorical ML would use another fitting function. Many people are under the false impression that ML cannot be used with categorical dependent variables. This is a common mistake. The clue here is that he presents chi-square values which are not available when ML treats variables as categorical.
Irene Dias posted on Saturday, March 11, 2017 - 12:29 pm
Dear Prof. Muthen. I tested measurement invariance of a three-factor model with ordered categorical indicators (5 point likert) using ML. However, one of the reviewers is insisting that we should use WLSMV, and even directed us to your 2004 paper in the journal Structural Equation Modeling.But the problem is that I have some missing data and have no MAR. This impairs the use of multiple imputation, right? In this case, would you recomend using MLR? This paper indicates that in CFA, ML is adequate if we have more than 5 categories: http://psych.colorado.edu/~willcutt/pdfs/Rhemtulla_2012.pdf But is says nothing about measurement invariance. But it says nothing about invariance.