would you say, it's possible to use the RMSR fit index to make a decision on how many factors one should include in a model?
I'm conducting an EFA (estim. ULS) and according to Eigenvalue criteria I should assume 2 factors but the RMSR is much (seems to be significant) better when assuming 3 factors. So I'm thinking about comparing the fit indices with something like a X² difference test - of course without X²s but perhaps based on RMSR. Would you say that's a feasible way?
that sounds good to me. I'm (still) following Grilli and Rampichini (2004).
Step 2 would be an EFA on the polychoric correlation matrix. -> WLSMV instead of ULS?
Step 3 requires seperate analysis of the between and pooled-within matrices. My indicators are actually ordinal, but the seperated matrices are only available when I treat them as being continuous ... So I've to conduct the EFA on the two matrices based on continuous data. -> ML instead of ULS?
Can I use the obtained RMSEA directly or would you recommend doing two CFAs with DIFFTEST?
To make a decision in favour of a certain number of factors I want take the fit into account as well as an Eigenvalue criterion. So I'd have to compare two factor solutions. The EFA outputs provides fit indices as well so I thought about comparing them. I just talked to my mentor and we decided to model two CFAs according to the EFA outcome and comparing the CFA fit indices afterwards.
So the question is answered in a way. I'll get fit indices for the CFA models and only take those from the EFA with a one factor-solution.
Thank you very much - for further comments on our model selection strategy, too.