I am trying to run a multilevel CFA (with 1-factor on each level) of 59 test items, with students nested within schools. I have 111 schools. When I tried to run a multilevel CFA, I got a message that the model is not identified because I have more paramters than clusters. However, when I run the same model as a multilevel EFA (with 1 factor on each level), I do not get a similar error message. Does the number of clusters not pose an issue to identification in multilevel EFA? Or is there some other reason why a multilevel EFA would be identified when the same 1-factor model in multilevel CFA would not be?
The difference is most likely simply due to a warning message being printed in one but not the other case (EFA vs CFA). I don't recall offhand if the standard errors have different defaults, but that can also affect the message appearing or not. I would not worry about this message because you clearly have fewer between-level parameters than your number of clusters. You should not interpret the message as saying the model is not identified due to too few clusters.
There are only 59 factor loadings being estimated at the between level, but there are also thresholds because the test data are mostly dichotomous (right or wrong) with a handful of extended response scores (0-3). So there really are more parameters being estimated at the between level than the number of clusters. The thresholds do not appear to be estimated in the multilevel EFA. Is this correct?
I see, you have categorical outcomes. The thresholds are unrestricted in both your EFA and CFA. You don't see them in EFA because that model focuses on correlation structure alone. Given that the threshold part is just-identified (unrestricted), I would not worry about about these threshold parameters bringing you over the top of the number of clusters. If you are concerned, the only way to know for sure if SEs are ok is to do a Monte Carlo simulation study in Mplus for this situation.
Thank you very much for your response. When you say, "I would not worry about about these threshold parameters bringing you over the top of the number of clusters," do you mean not to worry in the CFA or EFA or both? Also, does any potential problem only affect the SEs or could it affect the paramteter estimates as well? I am sorry to belabor this point, but this decision is crucial to my analyses so I want to make sure I understand any potential implications of what I am doing.
With reduced sample size - in this case for the Between level - SE quality is typically more affected than parameter estimate quality. You have 59 factor loading parameters on Between and 111 schools. This is not a large sample relative to the number of parameters. Your Within-level parameters and SEs will have better quality. To better know the quality of the Between-level parameters you would have to consult chapter 11 of the UG and do an Mplus Monte Carlo run for your model.