I am running a MIMIC to test for the invariance of a measurement model for depression across 7 countries. I have created 6 dummy variables to represent the country groups. My question is: after adding all the direct paths, can I calculate factor scores for each of the 7 countries (to be used in further longitudinal analysis) or am I only able to calculate an overall factor score that is adjusted for country-differences?
Also, is there a limit to how many direct paths one should add? I have added 10 direct paths, as suggested by the modification indices and I have conducted a DIFF test at every stage; this resulted in a constant improvement in model fit. But I wonder when I should stop adding paths. I have no theoretical reason to guide my decision and I rely solely on the empirical findings. Should I just keep adding direct paths until the improvement in model fit is no longer significant? Also, I assume that my large sample size (12.000 participants) may be in part responsible for the statistically significant improvement in model fit. Thanks in advance.
The factor score estimation uses the information from the dummy variables as well and therefore gives you the factor score for each country. In other words, two countries with the same depression response would get different factor scores.
At some point, adding further direct effects does not change the factor scores in important ways. That is, you correlate the factor scores from a run with x direct effects with those from a run with x+c direct effects and at some point find a correlation of almost one.
Given that you have many groups, you may also want to consider "alignment" analysis as a more convenient alternative that also allows loading non-invariance (direct effects do not). See the paper on our website:
Asparouhov, T. & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21, 1-14. DOI: 10.1080/10705511.2014.919210. An earlier version of this paper was posted as web note 18.
My second question is related to your recommendation about establishing the right number of direct paths in the MIMIC model by correlating the factor scores from a run with x direct effects with those from a run with x+c direct effects. I did that and encountered a correlation of 1 when adding the very first direct path (while the DIFF test was significant for the first direct path and remained significant for the next dozen of direct paths added). I understand that a correlation of 1 between the factor scores has trivial practical implications. On this basis, would you recommend that I should add no direct paths at all to my model, thus concluding that my model is invariant across groups? How should I compromise between the results of the DIFF test and the factor scores correlations? Thank you in advance.
Thank you for your advice. I have 2 follow up questions.
The first questions is whether it is necessary to fit my CFA model not only in the pooled sample from different countries, but also in each country separately, before running a MIMIC model in the pooled sample. My CFA model fitted the data well in the pooled sample from 8 countries, but I encountered NPD issues in China when separately running the model in each country This was due to extreme collinearity among items (reflected in empty cells in the bivariate tables of 8 item pairs). When I ran the MIMIC model in the pooled sample (including China and the other 7 countries) I did not encounter NPD issues. Would you recommend that I should exclude China from my MIMIC analysis on the basis that no meaningful measurement model could be established in this country alone?
Q1. My advice gives an ad hoc, practical aspproach, but I don't know to which extent journals would find that sufficient to conclude essential invariance. Also, the MIMIC model doesn't study loading invariance as would for instance the Alignment method.
Q2. I am not sure, but I would also consider the size of the change in the measurement intercept for an item when allowing a direct effect to it (that is, the size of the direct effect relative to the intercept).
Second post - and please limit posts to one window:
I would first do a configural analysis (which is a special option in Mplus) of all countries without covariates. This way you get a convenient summary of how each country separately behaves. It sounds like you have done this manually by analyzing each country separately. If the country-specific problem is important enough I report that and exclude the country from further joint analyses.