In a CFA of 85 items with sample size of 454. Authors used WLSMV, model has 349 parameters with a ratio of 1.30 subjects per parameter. I used MLR which resulted in 247 parameters a ratio of 1.83. The items use a 4 point scale with 3 thresholds per item. I want to use MLR rather than WLSMV because the number of parameters is too large in relationship to sample size.I am concerned the WLSMV is seriously over-fitting to a particular sample data. The authors are resisting this change because the CFI and TLI for the WLSMV estimation is about .92, However under MLR the CFI and TLI only reach a level of about .80 which would need additional model modifications. The RMSEA and SRMR fit indices are very similar RMSEA = .05, SMSR = .06 I am concerned the CFI TLI are over-fitting with the WLSMC solution due to the large number of model parameters in relation to sample size. Are my concerns real ? My opinion is MLR has less parameters and less likely to be over-fitting the data. WLSMV has 102 more parameters than MLR which might explain the improved CFI and TLI but possibly is not replicable in a new sample. I realize the data is small for either model but I am leaning toward recommending MLR. Has any good work been done with CFI and TLI index when using WLSMV, the same cutoffs have been validated ?
A first question would be why the model used with ML has fewer parameters than the model used with WLSMV. Seems like you can get the same number of parameters with both estimators. Also, how many factors do you have?
Sorry, when using mlr items defined as continous, shen using wlsmv items defined as categorical, 4 point likert 3 thresholds estimated per item. Solution cfa has 5 factors, wlsmv modeled 85 items. It is a large scale of executive functions developed by Barkley.
That sounds like comparing apples and oranges. Note that ML, or MLR, does not imply treating item as categorical (for examples of ML with categorical items, see the IRT literature), although that is a common misunderstanding. For a recent, clear article on comparisons of these kinds of approaches, see the paper on our website under Papers, Categorical Factor Analysis:
Barendse, M.T., Oort, F.J., & Timmerman, M.E. (2014). Using exploratory factor analysis to determine the dimensionality of discrete responses. Structural Equation Modeling: A Multidisciplinary Journal, 00: 1-15.