Calculating df when using the WLSM es...
Message/Author
 Mike Willoughby posted on Sunday, July 09, 2000 - 1:56 pm
I'm running a series of CFA models using dichotomous indicators. I've noticed what appears to be a discrepancy in the computation of df & wondered if someone might spot an error in my reasoning.

Consider a cross-sectional 2 factor model with 13 dichotomous indicators. The tetrachoric corr matrix has 78 unique pieces of info [=13(12)/2]. If I estimate 11 factor loadings (2 items have factor loadings fixed to 1.0 for identification of the 2 latent variables), 2 latent variances & their covariance - 14 total params are estimated leaving model df = 64. This is exactly what I get in my output file (note: Mplus doesn't explicity estimate params for residuals, that's why they're not considered when counting dfs).

Now consider a longitudinal CFA of the same 13 items (2 factors at 2 points in time). I now have 26 indicators which yield 325 unique pieces of info [=26(25)/2]. 22 factor loadings are estimated. Furthermore with 4 latent variables a total of 10 (co)variances are estimated (all factors are inter-correlated). In total 32 params are esimated which should leave 293 df (325-32). However my output file evaluates the chi sq @ 306 df.

Finally when I specify another model that imposes equality constraints for each factor loading at 2 points in time (that is 11 factor loadings are estimated instead of 22), the resulting df is 318 instead of 306. Had 306 df been correct, shouldn't the resulting model be eval at 317 df (306+11)?

I'm assuming that WLSM doesn't adjust df (unlike WLSMV). Furthermore, I'm assuming that my imposing equal thresholds for items involved in the longitudinal model is in no way associated with computing df.

Does anyone see an error in my rationale? Further, has anyone else doing longitudinal CFA on dichtomous items noted a problem in computing df?

-mikew@unc.edu
 Linda K. Muthen posted on Sunday, July 09, 2000 - 6:24 pm
The difference in the degrees of freedom you expect and the degrees of freedom that you get is 293 versus 306 or a difference of 13. You imply that you have thresholds in the model and that you hold them equal across time. In such a model, thresholds do influence the degrees of freedom. There are 26 thresholds and 13 thresholds parameters estimated which I think explains the difference of 13.
 Mesfin Mulatu posted on Monday, September 09, 2002 - 8:24 am
Hi,

I am doing a series of CFAs with ordinal variables using Mplus 2.12. Everything looks fine except I am not sure I understand clearly the warning "* The chi-square value for MLM, MLR, MLMV, WLSM and WLSMV cannot be used for chi-square difference tests. MLM and MLR chi-square difference testing is described on page 360 in the Mplus User's Guide."

I read the chapter decribing these features but I found it too sophisticated for me. So here are my questions:

1) Is it possible to compare model fit indices (chi. sqs) involving categorical outcome variables estimated with WLSMV?

2) If yes, which statistic can be used or how are these statistic obtained?

 Linda K. Muthen posted on Monday, September 09, 2002 - 8:43 am
It is not possible to do nested model difference testing using WLSMV. There is no correction factor yet available. We recommend using WLS for difference testing and then using WLSMV for the final model.
 Stephane Vautier posted on Saturday, January 22, 2005 - 12:03 pm
Hello,
I would like to understand how dfs are obtained when analyzing a CFA model with categorical indicators when using WLSMV. For instance, with 2 correlated factors each predicting 3 variables, I would expect 6*7/2-13=8 dfs, but the output gives 7 dfs.
Apparently, the reference to p. 360 of the Mplus manual is obsolete. I have the manual for version 3. I was also unable to find something relevant in the technical appendices online.
 Linda K. Muthen posted on Saturday, January 22, 2005 - 3:29 pm
The technical appendices can be found on the website. The Version 3 Mplus User's Guide does not contain the technical appendices. You must be using an old version of the program. Degrees of freedom are calculated differently using WLSMV. If you want to see degrees of freedom the way you expect, use WLS or WLSM.
 Stephane Vautier posted on Sunday, January 23, 2005 - 3:33 am
Thank you Linda. May I insist? The output given by Mplus 3.11 is "The degrees of freedom for MLMV and WLSMV are estimated according to a formula given in the Mplus Technical Appendices at www.statmodel.com. See degrees of freedom in the index of the Mplus User's Guide".
Could you give me the number of the equation in the Mplus Tech App on line?
I have Version 3 of the manual, and there is no entry "Degree of freedom" in the index.
 bmuthen posted on Sunday, January 23, 2005 - 3:36 pm
The Version 3 User's Guide does not have Technical appendices. Technical appendices are on the Mplus web site. The WLSMV df discussion is in the appendix section on estimators and tests.
 Anonymous posted on Thursday, April 28, 2005 - 7:24 am
Hello

I just wanted to ask if the technical appendices for version 3 of M-Plus are out. In case they are, where are them? In the technical appendices section of the web I just can find the version 2 ones.

Thank you
 Linda K. Muthen posted on Thursday, April 28, 2005 - 7:36 am
No, they are not yet available. If you have a question about a particular techinical issue, post it and I will see if I can get you some references.
 Anonymous posted on Friday, April 29, 2005 - 3:58 am
I'm the anonymous on April 28 7:24am.
I just wanted to know what is the theory behind the WLSMV difference test that is implemented in M-Plus 3. Any references?

Thank you
 Linda K. Muthen posted on Friday, April 29, 2005 - 8:52 am
You may want to look at the literature by Satorra and Bentler on robust chi-square difference testing with continuouos nonnormal outcomes. The issues are the same.

One article is "A scaled difference chi-square test statistic for moment structure analysis", Psychometrika, 66 ,507-514, 2001(A. Satora and P.M. Bentler).
 anonymous posted on Friday, January 20, 2006 - 12:09 pm
Would anyone mind translating the notation from the technical manual into simpler terms to explain how dfs are calculated when using wlsmv?
 Linda K. Muthen posted on Friday, January 20, 2006 - 2:03 pm
The chi-square and degrees of freedom are adjusted until a correct p-value is found. The only number you should be concerned with with WLSMV is the p-value.
 Adrienne Tin posted on Tuesday, December 12, 2006 - 9:02 am
Hi, I have a very simple question in using the DIFFTEST option.
I followed Example 12.12 in the user's guide of version 3.

Where is the result of the test? From the previous posting, it seems I should be looking for a p-value. Do I need to add options to save it.

 Linda K. Muthen posted on Tuesday, December 12, 2006 - 10:39 am
It should be in the Test of Model Fit section. If you can't find it, please send your input, data, output and license number to support@statmodel.com.
 Jeremiah Schumm posted on Monday, February 05, 2007 - 10:27 am
Linda,
Your response on Jan 20 2006 was helpful in translating how df is estimated using the wlsmv estimator. Could you elaborate on what criterion is used to define a "correct p-value"? I am referencing this quotation from your posting on Jan 20 2006 2:03pm. I have been asked by a journal editor to explain the df calculation in non-technical terms for my manuscript submission involving wlsmv estimator analyses. Although I am not the anonymous posting author that you responded to on Jan 20 2006, this additional clarification would be very helpful to my explanation to the editor.
 Linda K. Muthen posted on Tuesday, February 06, 2007 - 11:22 am
There are some things in life that do not lend themselves well to non-technical terms. Let me try one more approach. A test statistic and its degrees of freedom are estimated as shown in formulas (108)-(110) in our technical appendices. The test statistic and the degrees of freedom refer to a chi-square distribution for which a p value is obtained. The p-value is the
probability that the null hypothesis is true. With WLSMV, the only quantity interpretable in the usual way is this p-value.
 Kathy posted on Friday, April 25, 2008 - 10:34 am
can someone tell me how to calculate the df for the models below, when i am using wlsm estimator?

mgfa with 2 groups 3 factor model with 35 dichotomous indicators and the factor loadings and thresholds are free to be estimated across groups and the residual variances @1 for both groups and factor means @0 for both groups.

using the calculations from a previous discussion above:
35(34)/2 = 595 X 2 groups = 1190 (i know i only need to do 35(34))

35-3=32 X 2 groups = 64

3 latent variances + 3 their covariance = 6 X 2 groups = 12

= 1114 but the output has 1113 df.

using the same model but this time the factors and thresholds are constrained to equal across the 2 groups and the residual variances @1 in both groups but the factor means @0 for one group but freed in the other. -- i have no idea how to calculate the df for this one. the output indicates df= 1174.

can anyone help with the calculations?
 Linda K. Muthen posted on Friday, April 25, 2008 - 12:21 pm
Please send the full output and your license number to support@statmodel.com if you would like help with this.
 Allison Holmes Tarkow posted on Sunday, November 23, 2008 - 8:35 am
Hi-
How do you calculate the df for LGC models estimated with WLSMV when multiple imputation has been used ?

Thanks!
 Linda K. Muthen posted on Monday, November 24, 2008 - 8:43 am
This is a research question that has not yet been answered. I would use WLSM and use the average chi-square value and the degrees of freedom from WLSM. This should work pretty well as long as you don't have a huge amount of missing data.
 Nate Breznau posted on Wednesday, October 05, 2016 - 8:02 am
Thank you for pointing out that we cannot use the value and df scores for chi-square, only the p-statistic when using WLSMV.

Does this also apply to the DIFFTEST for nested WLSMV models? Do we only rely on the p-statistic of this test as well? Or can we interpret the df and chi-square value as accurately capturing the difference between nested models? (i.e., is the p statistic calculated in some other way than a chi-square table using the given df and value difference?)
 Bengt O. Muthen posted on Wednesday, October 05, 2016 - 4:19 pm
The DIFFTEST-produced df and chi-square can be interpreted in the usual way.