I'm analysing a questionnaire that measures altered states of consciousness. In order to have enough data, I pooled data from various experimental studies, in which hallucionogenic drugs were given to healthy volunteers. In some studies the hallucinogenic drugs was given just once, but in a few studies the drugs were administered within a repeated measurement design with three different doses on 3 separate days. The questionnaire was filled out on each experimental day. Some subjects also took part on more than one study. Therefore, some subjects in my dataset have up to 6 or 7 observations. About half of my dataset consists of single independent observations though.
Can I use type=complex with the subjects as clusters to correct for this partial non-indipendence of observations or do I have to take the repeated measures out of my dataset? The problem is, that I can hardly estimate my model without the repeated measures, since my dataset is already on the small side.
Since most of my clusters are rather small in size and the questionnaires where not filled out exactly under the same condition, I suspect, that the intraclass correlation is not too big and the bias in Chi-Square and standard errors would be rather small, wouldn't it? I also read somewhere, that the MLM-estimator corrects for non-independence. Can you elaborate on this?
Thank you so much! Compared to regular MLR estimation, chi square and fit indices get slightly better when I take clustering into account. I have 485 observations, 197 free parameters and 264 clusters.
Fit indices with MLR: Chi-Square: 1771.56 df 978 CFI: 0.916 TLI: 0.908 RMSEA: 0.041 SRMR: 0.055
Fit indices MLR & type=complex: Chi-Square: 1701.597 df 978 CFI: 0.920 TLI: 0.911 RMSEA: 0.039 SRMR: 0.055
I also calculated the difference in standard error estimation. The highest difference was 19.4%. The difference on average was 7.8%. Would you consider this difference as worth bothering? I'm asking because I have also tried to do some analysis with a subset of my dataset and I got the error message that I have more free parameters than clusters. I guess, type=complex is only appropiate with more clusters than free parameters. Is there a recommended ratio between clusters and free parameters?
If I take the regular MLM-estimator, the CFI and TLI are still better than with MLR and type=complex. Here is what I get with the MLM-estimator: Chi-Square: 1742.877 df 978 CFI: 0.921 TLI: 0.912 RMSEA: 0.040 SRMR: 0.055
The standard errors with the MLM-estimator are on average about 20% smaller than with MLR and type=complex.
I still don't understand the difference between MLM and MLR-estimaton and why they behave so differently in my case.
Thanks! I really tried hard to improve the model fit of my measurement model. This is about the best solution I can get, unless I free lots of parameters (crossloadings and correlated errors) that don't make sense from a theoretical point of view or unless I throw out more items and factors. I thought, the model fit wasn't too bad, since RMSEA and SRMR are well below recommend cut-off values and only CFI and TLI are too low. I suspected that this is because of my rather low sample size.
Would you consider the fit of my measurement model as unsufficient for proceeding into mimic-models?
I understand, that it's probably the best solution to take MLR with type=complex, but what would you suggest, if I have more parameters than clusters?
I would like to conduct a regression analysis with eight samples from different contexts. Alas, I cannot run a multilevel analsis, as eight is not a sufficient amount for most purposes. (For the same reason, just conducting a Type=complex cluster=clus analysis is not possible.) If I would like to correct standard errors for non-interdependence of observations, are there any other options, please?