Anonymous posted on Monday, August 15, 2005 - 3:13 am
I'm running a CFA using the WLSMV-estimator, because some of my indicators are censored, the others are continous. Now my problems: Using ONLY the censored indicators, the model is estimated normally, but there are 999.00's as standardized output (StdXY). How can I interprete the size of the factor loadings? Using the censored and the continous indicators in the same analysis, the estimation doesn's converge. I've tried it very often with several datasets, but I get no result. How can I resolve that problems? (I'm using MPlus version 3.11) Thank you!
bmuthen posted on Monday, August 15, 2005 - 9:40 am
999's indicate that the standardized value could not be computed, typically because of a non-positive variance (see your output). If that is not the case, please email email@example.com.
With a censored indicator, a factor loading can be interpreted in terms of how strongly a factor influences the (uncensored) latent response variable underlying the censored indicator. The relationship between the factor and the indicator itself is a non-linear function - see the literature on censored outcomes and "Tobit" regression.
Dustin posted on Tuesday, October 04, 2005 - 10:41 am
We are running a CFA across nine waves of longitudinal data to test for metric invariance using a WLSMV estimator (n=500). In conducting the two-step chi-square difference test in Mplus 3, we are finding that the model with factor loadings constrained to equivalence across the nine time points produces a worse fitting model in comparison to the model with freed loading; however the magnitude of the difference seems relatively small, chi square =61.413, df = 41, p=.021. In addition, the CFI, TLI, and RMSEA all improve for the constrained model, in comparison to the unconstrained model (although not by much).
1) Is it possible to have a worse fitting model in terms of chi-square difference testing with WLSMV, but "better" fit in terms of CFI, TLI, RMSEA. It seems counter intuitive and I am worried about publishing the finding (reference in this regard?).
2) In terms of interpretation, with a large sample it seems that the chi-square difference test for metric invariance over time may be overly sensitive. Has anyone written on the meaningfulness of using a chi-squared difference test to as an indicator of the invariance of a construct over time? Should other factors be taken into account with large samples (e.g., change in other fit statistics).
Anonymous posted on Thursday, October 06, 2005 - 3:04 pm
Hi, I have a related question. I have data on several hundred respondents on several questionnaires at multiple time points. I would like to examine the longitudinal invariance across time. For multiple group CFA, I would check configural invariance, then constrain factor loadings, etc. For the longitudinal data I would first look at the structure at each time point separately. Then would I just constrain factor loadings across the time points? Or should I be doing something else due to the data being dependent (same respondents across all time points)?
bmuthen posted on Saturday, October 08, 2005 - 12:03 pm
Regarding Dustin's question,
1)This is possible. For example, CFI tests against an uncorrelated variables model whereas chi square tests against an unrestricted model. I have no references on this.
2) Yes, chi-square difference testing can point out significant differences that are not of practical importance. I think one just has to decide what is practically important.
bmuthen posted on Saturday, October 08, 2005 - 12:06 pm
After having examined each time point separately, I would analyze all time points in 2 ways: with measurement invariance across all time points and without it. We discuss this topic in our Nov courses.
Hi Sorry, if this is a ridiculously silly question. I need to run a completely unconstrained MGCFA (so without the default factor loadings being specified as equal). What code do I put in the model command to specify this.
We are using Mplus 3.0 to run a CFA on ordinal data using the WLSMV estimator with an approximate sample size of 600. Our variables are 4-category Likert scale items and some have a very small number of affirmative responses (anything other than “no”). We are concerned that these low base-rate items may be causing problems and/or assumption violations within our analysis. So...
a. Is there a rule of thumb for the % of affirmative responses (across all categories) to retain an item in the analysis? For example, would you keep or remove an item if 96% of the sample says “no” and the remaining 4% all choose some form of affirmative response?
b. Similarly, is there a minimum percentage (i.e, 1%, 5%, etc) of response per category for item retention? We have some variables where about 1% or less falls into a particular category, but each of the remaining categories seem to have a sufficient number of responses.
c. In any of these cases, would it make more sense to dichotomize or truncate categories to get a higher response level?
The major issue here is the bivariate tables. If there are zero cells, this can be problematic. So you should look at your bivariate tables and collapse categories if needed to avoid zero cells.
By the way, you should download Version 3.13.
Mark posted on Friday, December 16, 2005 - 5:57 am
Thanks for you reply. Regarding your suggestion to look for zero cells in the bivariate tables, we are conducting this analysis on approximately 40 variables (some with different scales). Clearly we will encounter some (I dare guess a lot) zero cells in these tables. If these are encountered, how should they be handeled. I am assuming that categories need to be truncated to eliminate all zero cells.
I am particularly concerned given that some of our items are very low base rate (conduct disorder symptoms), while others have a full range across categories (ADHD symptoms). As a result, I am worried we will have to truncate more evenly distributed items causing a significant lose of information.
Lastly, how bad is the problem of zeros cells. Are a few o.K. or is the presence of any completely unacceptable, producing significant bias in parameter estimation?
Thanks for your help. I just have one final round of questions (I hope)... We have not received any errors about zero cells and are finding that our low base-rate items (i.e., less than 5% anything other than "never") either have counter-intuitive loadings, no loading, or cross-loading. This seems somewhat logical since the items have little or no variability. Based on this, would there be any justification in removing such items since these items seem to produce unstable results? We are somewhat concerned that these patterns are an artifact of the low base-rate items and not a "true" representation of the factor structure.
Rather than looking at the percent, you should be looking at the number of observations that percent corresponds to. If you have a small sample, 5% could be a problem. With a large sample, it may not be. You may find it useful to read the following article which discusses these issues:
Muthén, B. (1989). Dichotomous factor analysis of symptom data. In Eaton & Bohrnstedt (Eds.), Latent Variable Models for Dichotomous Outcomes: Analysis of Data from the Epidemiological Catchment Area Program (pp. 19-65), a special issue of Sociological Methods & Research, 18, 19-65. (#21)
If you can't find the paper, you can request it from firstname.lastname@example.org. It is paper 21.
Derek Kosty posted on Monday, October 06, 2008 - 10:34 am
It is my understanding that the WLSMV estimator simultaneously uses the tetrachoric correlation and asymptotic covariance matrices (ACM). Examining the tetrachoric correlations alone, I was struck by the pattern of associations that appear to support one model over the other which wasn't consistent with the generated fit statistics for the models. I was hoping to clear this up by looking at the ACM. Is it possible to request the ACM to be produced in the output?
Greetings, I would like to know which of the Mplus estimators corresponds to the DWLS estimator described (and studied) in the recent Forero, Maydeu-Olivares & Gallardo-Pujol paper (that just came out in Structural Equation Modeling (2009, 16, 4, 625-641). From what I get, it should be either the WLSMV or the WLSM
Greetings, Please, dont forget this question in the huge amount you are receiving... "I would like to know which of the Mplus estimators corresponds to the DWLS estimator described (and studied) in the recent Forero, Maydeu-Olivares & Gallardo-Pujol paper (that just came out in Structural Equation Modeling (2009, 16, 4, 625-641). From what I get, it should be either the WLSMV or the WLSM" Thanks
WLSM and WLSMV have the same parameter estimates and standard errors. It is chi-square that differs. In the article, only parameter estimates and chi-square are examined so it could be either WLSM or WLSMV.
Hi Linda, Not sure I follow you. You say that the chi-square differs from WLSM to WLSMV. Then you say that the article examine parameters estimates AND chi square so it could apply to both... In fact, what you are saying is that the paper conclusions apply to both estimators ? Is that it?
In a related way, what is the "practical" difference between WLSM (mean-adjusted) and WLSMV (mean and variance adjusted). In other words, are there context in which it would be preferable to use WLSM rather than WLSMV ? Thanks again.
I misspoke. The article examines parameter estimates and standard errors. So it could be WLSM or WLSMV.
I don't think it is clear when WLSM is preferable to WLSMV.
Erika Wolf posted on Wednesday, March 03, 2010 - 7:24 am
As I understand it, when using WLSMV with both continuous and dichotomous variables, the regression coefficients in the model involving dichotomous dependent variables are probit estimates and the coefficients with continuous dependent variables are linear regression coefficients--is this correct?
So then, when I look at the sample statistics reported on the output, are the sample correlations involving a dichotomous variable tetrachoric correlations while those involving 2 continuous variables are simple Pearson correlations?
Hi, I guess the run of questions for version 6 is on. The WLSMV estimator has been changed in version 6, yielding (as it is noted on the website) df that are more "traditional". Which is good. However, I tried running some models (for wich I already have outputs from previous versions) and found out that the fit indices also changed with the new version. So I am wondering: 1) What to make of this ? 2) What changed ? 3) How will this affect the suggested cut off scores for invariance testing with WLSMV (Yu 2002) ?
In an unrelated way: how do I make Mplus the default "opener" for .out and .inp files ?
A new chi-2 approximation is used. A technical description will be posted shortly under Technical Appendices (several others will also appear shortly). Because the other fit indices are based on chi-2, they are also affected. Our simulation studies with the new chi-2 indicate, however, that the chi-2 results are very similar between this new approach and the earlier one. I would therefore use the same cutoffs as before.
The editor can't be changed to have out show up instead of inp.
Jon Elhai posted on Tuesday, April 27, 2010 - 6:33 pm
Alexandre: Are you asking how to arrange for a .inp file to automatically open in Mplus (as opposed to Windows not knowing what application to use)? Locate a .inp file, right click on it, select "open with" and then "choose program". Then select "Mplus" and click "Always use selected program" in the checkbox at the bottom.
Hi all, thanks for the answers, Jon: Yes and thanks (I needed to restart).
Bengt: when you say that "chi-2 results are very similar between this new approach and the earlier one", you mean that cut off scores on the "fit indices" (RMSEA, CFI) when doing invariance testing are similar with the new aproach ? Or you are just assuming that since chi-2 works the same way fit indices also should ? In the first case, this is perfect, in the second one, hopefully these posts will motivate someone to test it :-)
I am running a CFA analysis on items. The response option is 1-5. However, I have noticed the serious deviation from normal distribution in items (floor effect), and decided to treat them as ordinal indicators. Is that correct?
Therefore I have performed WLSMV estimation in two independent samples. However the df value is different in both analyses although the model is the same. I know that calculation of df is not straightforward in WLSMV, however can the same model have different df value in different samples? If yes, can you give me a reference for it? Thanks, Robert
Yes, it is correct to treat 5-category variables with a floor effect as categorical rather than continuous.
You must not be using Version 6. Prior to Version 6 the same model could have different degrees of freedom. I suggest downloading Version 6. See Simple second order chi-square correction under Technical Appendices on the website for further information.
Just to clarify, it was not a problem. We just use a different method in Version 6 than in earlier versions. In Version 6, the chi-square value and degrees of freedom are interpretable in relation to the p-value. In earlier versions, the chi-square and degrees of freedom were adjusted to obtain a correct p-value. In both cases, difference testing must be done using the DIFFTEST option.
Jon Elhai posted on Monday, June 21, 2010 - 12:35 pm
Linda - you said in your previous email "In Version 6, the chi-square value and degrees of freedom are interpretable in relation to the p-value." Is this the case for difference testing using DIFFTEST in Version 6?
I'm not sure I understand your question. The DIFFTEST option must still be used to test nested models when using WLSMV.
Jon Elhai posted on Monday, June 21, 2010 - 4:14 pm
To clarify my question... Prior to Version 6, only the p value was interpretable in the difference test printed in DIFFTEST's output. Now, we can report the chi-square and df that is in DIFFTEST's output?
Yes. See the Technical Appendix entitled Simple second order chi-square correction which is available on the website.
Sara posted on Thursday, December 09, 2010 - 11:59 am
Hi, I want to check my understanding of the reasons for better performance of WLSM/WLSMV estimator over WLS estimator.
The greater bias in WLS parameter estimates, test statistics, and standard errors compared to the WLSM/WLSMV paramaters, test statistics and standard errors, especially with decreasing sample size, is attributed to differences in weight matrices. WLS employs the elements from the full asymptotic covariance matrix of the polychorics & thresholds which become inaccurate due to sampling error as sample size decreases and model size increases. Inversion of the full asymptotic matrix compounds these errors (inversion involves multiplying these error-prone elements), resulting in inaccurate elements in the inverted full matrix. Robust DWLS (WLSM/WLSMV), however, only inverts the asymptotic variances, resulting in an inverted weight matrix less susceptible to sampling error and inaccuracies. Hence the WLSM/WLSMV parameter estimates, standard errors, and test statistics are not influenced by the same inaccuracies associated with employing the inverted full asymptotic covariance matrix, unlike the corresponding WLS estimates. This then implies that the off-diagonal elements of the INVERTED full asymptotic covariance matrix are not only not useful, but are potentially harmful, resulting in convergence problems and biased parameter estimates, standard errors, and fit indices with small samples.
This is a reasonable description except that it could be stressed that there are two matrices involved, a weight matrix and a Gamma matrix. The weight matrix in WLSMV is easy to invert as you say and Gamma doesn't need to be inverted - see middle of page 4 of
Muthén, B., du Toit, S.H.C. & Spisic, D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Unpublished technical report.
Cecily Na posted on Sunday, December 12, 2010 - 8:26 am
Dear Linda I ran an SEM for non-normal categorical data with WLSMV. You mentioned the path coefficients are actually probit. I am wondering if the R square (explained variances) for each outcome variable can be interpreted in the same way as in an OLS regression, such as x% of variance is explained by its predictors? It seems probit regression generates pseudo R-square statistic, which should not be interpreted in the regular way? Thanks a lot!
The R-square is the variance explained by the covariates in the latent response variable underlying the categorical variable.
Sara posted on Monday, December 13, 2010 - 11:58 am
Thanks for the response regarding WLSMV. With respect to employing WLSMV, is the following true: Mplus has capability to employ two different procedures when modeling categorical indicators in CFA model, which is dictated by presence of exogenous variable influencing the factor. When there is no exogenous predictor, CVM is conceptualized as LRV formulation. It is assumed continuous normally distributed latent response variables (y*) underlie ordinal variables (y). Thresholds & latent correlations are estimated. Asymptotic covariance matrix is formed using both polychorics & thresholds. Latent correlations & thresholds & asymptotic covariance matrix, are employed with WLSM(V). When there is an exogenous predictor, CP curve formulation is employed. Instead of estimating linear relations between y* & the factor, non-linear relations between y & the factor may be modeled. A probit model can be used to estimate probability a specific category is selected or exceeded by modeling the non-linear relation between y and the factor. When CP formulation is employed, the first step involves computing sample statistics: probit thresholds, probit regression coefficients, and probit residual correlations. In the second step, the asymptotic covariance matrix of these sample statistics is constructed. In the final step, estimates of model parameters, standard errors, & model fit information can be computed using WLSM(V).
With x's, I wouldn't say that the CP curve formulation is employed instead of the LRV formulation. For the regressions (slopes, thresholds) you can view it both ways because the two ways are equivalent. So I don't see it as going from linear to non-linear regression. For the residual correlations, the LRV formulation is essential because those correlations are for latent response variables; CP doesn't have those correlations.
Sara posted on Wednesday, December 15, 2010 - 12:04 pm
Thanks!I believe I understand LRV and CP formulations are equivalent, in that one can derive CP parameters of alpha & beta from the LRV parameters of tau & lamba (and vice versa; Webnote 4). Where I am struggling is what exactly is in the s vector when employing a WLS estimator and modeling a single-factor CFA model with no predictors. Given your previous comment, I think the s vector contains thresholds and polychorics. However, when I examine Appendix 4 in Mplus manual (p. 356 1998 – 2001) it says “s represent probit thresholds, slopes, and residual correlations”. Is this the case with no X covariate? Or is this only the case with an X covariate? Example 5.2 (1998 – 2007) indicates robust weight least squares uses latent correlations (as I describe above) but then says probit regression for the factor indicator regressed on the factors are estimated. So, does this imply that s vector contains the probit thresholds, slopes, and residuals when X (covariate) is not in the model? If so, doesn’t this imply that latent correlations (e.g., polychorics) aren’t modeled? I guess I am confused as to if the CP or LRV formulation is being employed, because this would influence what is in the s vector and also what parameters are produced (intercept & slope from CP vs. tau & lambda from LRV), right?
These things are explained in the original article
Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115-132.
which is on our web site.
You should make a distinction between the sample statistics used in the analysis and what the model says. Without covariates, there are no slope sample statistics. But the model portrays the relationships between the indicators and the factors using probit regressions (they cannot have sample slopes, because the factors are not observed). So slope sample statistics in s are available only with x's.
Hope that helps.
Theodor Tes posted on Sunday, January 08, 2012 - 4:15 pm
I am relatively new to FA. I would like to perform CFA with 45 items, 9 factors measured on an ordinal scale (just 4 options on a Likert kind of scale). Each factor has 5 items. Even though previous researchers used ML estimation, after reading some books on FA, WLS (or WLSMV) seems to be the right option to do. However, my sample size is only about 500, while for my CFA i need at least 1080 (if I am counting right, ((45*46)/2)+45=1080).
My point is. In this unhappy situation I decided to use ML, but to back my results, I am considering performing a set of model estimations (say 30) of CFA/WLSMV with just randomly selected 5 factors and their corresponding items. My sample size should be enough. After this I would like to take a look at factor loadings, perhaps calculate the average, not sure, it all depends on the outcomes. Iam hoping, that the results will be similar to that of the ML estimation of the whole CFA model with 45 items and 9 factors.
I have two questions: 1) is it a complete nonsense to make this "little" simulation? 2) what would anyone of you do in my place?
This analysis is not possible using maximum likelihood because you would have 9 dimensions of integration, one for each factor with categorical indicators. I would try using WLSMV. As long as the number of parameters in your H0 model is not larger than the sample, you should be okay. If there is a problem, send the output and your license number to email@example.com.
Theodor Tes posted on Monday, January 09, 2012 - 11:10 am
I am running CFA on a large sample N=27000). As it appears from the model fits below the RMSEA is fine but the CFI/ TLIs are not too god (nor is the Chi square but I am not too concerned with that as my sample size is as large as it is). Should I just be happy with the RMSEA and don't modify my model or should I try to aim for high CFI/TLIs as well?
Number of Free Parameters 60
Chi-Square Test of Model Fit Value 6545.749 Degrees of Freedom 265 P-Value 0.0000 RMSEA Estimate 0.029 90% CI 0.029 0.030 CFI 0.883 TLI 0.868 Chi-Square Test of Model Fit for the Baseline Model 54103.671 Degrees of Freedom 300 P-Value 0.0000
I am running CFA on eight seperate samples - two large (N=27000) and 6 'small' (N=3000). I get the best fits when I use WLS for the large samples and WLSMV for the small samples (my data is categorical). My question is whether it is all right to use two different estimators in the same article and whether I can compare the different groups in multigroup analysis when I have applied different estimators. Thanks a lot...
I'm trying to calculate Bentler's (2009) model-based internal consistency reliability using Mplus version 7. I have a six-item test where the items are dichotomously scored. According to the Bentler article, a model-based internal consistency estimate can be calculated by:
1 - (1'psi_hat1/1'sigma_hat1) (1)
As far as I understand it, psi_hat is the diagonal matrix of item error variances and sigma_hat is the full matrix of model-implied item variances/covariances plus the item error variances.
This can also be calculated for binary items by substituting the matrices in (1) for their polychloric counterparts. This is where I get lost.
I'm not sure how to use the output I get from the "residual" output command in the Bentler formula. I see what I thought was the model-implied and residual item covariance matrix, but are these actually polychloric correlation matrices for the binary items?
If so, would the numerator of (1) just be the sum of the residual polychloric correlations and the denominator the model-implied polychloric correlations?
I appreciate any help you can provide. Thank you.
Bentler, P. M. (2009) Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137-143.
If you analyze binary variables and use tetrachoric correlation as with WLSMV, you get the residual variances under the standardized solution. I am not convinced that the formulas are applicable to binary items, however. You may want to consult the 2011 book by Raykov & Marcoulides: Introduction to Psychometric Theory.
Thank you Dr. Muthen. I will be sure to take a look at the Raykov book.
Thank you again.
Tracy Zhao posted on Monday, February 03, 2014 - 12:21 pm
Can you please elaborate or point to the references regarding this? I am having the same question.
Eric Teman posted on Tuesday, April 24, 2012 - 2:13 pm Why are the degrees of freedom from WLSMV the same as when ML is used. I thought df in WLSMV were estimated and were never the same.
Linda K. Muthen posted on Tuesday, April 24, 2012 - 2:47 pm This changed with Version 6 where we implemented a new method for second-order chi-square adjustment for WLSMV, ULSMV, and MLMV resulting in the usual degrees of freedom.
Tracy Zhao posted on Monday, February 03, 2014 - 1:13 pm
Hi, I have a question: I was told that the WLSMV is equivalent to the Satorra-Bentler Adjusted Chi-square statistic (C4) in Lisrel 9, and the WLSM is equivalent to the Satorra-Bentler Scaled Chi-square statistic (C3) in Lisrel 9. Is this a correct comparison? Thanks a lot!
Variances/residual variances of categorical dependent variables are not model parameters in cross-sectional models. They are fixed at one. In growth models or multiple group models, they can be model parameters with PARAMETERIZATION=THETA. They must be fixed at one at one time point or one group but can be free in the others.
WLSMV can be used only when there is one or more categorical dependent variables. Continuous variables can also be included and linear regression coefficients are estimated for them.
hazel liao posted on Tuesday, June 09, 2015 - 8:24 am
However,I still have a question.
Why variances/residual variances are fixed at one?
See the Topic 2 course video and handout on the website.
hazel liao posted on Tuesday, June 09, 2015 - 11:20 am
Lewina Lee posted on Tuesday, September 22, 2015 - 1:07 pm
I'd like to do a CFA on categorical variables using WLSMV (fit indices were worse with MLR). Because we have quite a bit of missing data, I'd like to use multiple imputation first, and then analyze the imputed datasets with WLSMV. However, I found that I cannot do DIFFTEST for model comparisons when I have WLSMV and imputed datasets - is there any way around this?
Lewina Lee posted on Tuesday, September 22, 2015 - 1:49 pm
In thinking about this more, I tried to get around the problem I described above by using WLSM rather than WLSMV on the imputed datasets, but it looks like the scaling correction factor isn't produced as part of the output. How can chi-square difference tests be done on imputed datasets using WLSMV or WLSM? The manifest indicators on my CFA are categorical.
MLR has better missing data handling. So if you have quite a bit of missing data, I would use MLR. You can use multiple imputation but you run into the issue that no fit statistics have been developed for imputation and WLSMV. I would not choose an estimator based on which gives the best fit statistics.
Lewina Lee posted on Tuesday, September 22, 2015 - 2:21 pm
Thanks, Linda. When I used MLR, I received the warning below. Is this a sign of model misspecification, or is it due to low covariance coverage from too much missing data? If the latter, will including auxiliary variables (not as part of the model, but just to facilitate maximum likelihood estimation) help?
THE STANDARD ERRORS FOR H1 ESTIMATED SAMPLE STATISTICS COULD NOT BE COMPUTED. THIS MAY BE DUE TO LOW COVARIANCE COVERAGE. THE ROBUST CHI-SQUARE COULD NOT BE COMPUTED.
A similar question was asked earlier but I would like to probe a little further. I am conducting a multigroup analysis with some Likert-style response options. I am using the WLSVM estimator in MPlus to test a configural and subsequently a metric invariance model. The chi-square value for the metric invariance model is much smaller with (1432.28, df = 80) than the chi-square value for the configural invariance model (2822.00, df = 70). Traditionally the metric invariance model would have a higher value if ML was used (unless the two models were "equivalent"), but I have not seen this dramatic of difference. The differences in the df suggest the invariance constraints have been imposed correctly but the chi-square values suggest otherwise. Is this an issue with the WLSVM estimator? Thanks!