I have compared the factor structure invariance of PIRLS 2001 data for 35 countries. And i didn't find any difference between 35 countries.
I decided to create subgroups of these 35 countries based on language (languages the tests are applied, such as countries that took the tests in English, Spanish etc), socioeconomic level (oecd countries, undeveloped and developed countries), EU member countries and others, etc.
I found significant differences between subgroups of these 35 countries based on RMSEA differences.
I cannot explain why i don't find any difference between 35 countries but find significant difference between subgroups.
Note: A professor i have asked advice, suggested that, this could be caused by DIF of items. What do you think about this.
Any suggestions and academical references will be appreciated. Thanks in advance.
It sounds like the more specific variables describing the countries are more precise and powerful in describing response differences than the broad classification of country itself. In that sense, it doesn't have to involve DIF.
Thank you very much for your reply. I explained the difference just like you said. And now i am testing country based variables and common variables and dif based on PIRLS data. Your advice has been very helpful, thanks again.
Dan Costa posted on Sunday, June 02, 2013 - 9:48 pm
Is there a need to control for Type I error rate when there are so many comparisons in a multigroup CFA? It seems to make sense that an adjustment is needed when using the chi-squared test to assess invariance, but what about if subjective fit indices or modification indices are used? Several co-authors and reviewers have suggested this possibility to me but none of them have been sure if it's necessary, and I've not found any guidance from published literature.
Dear Drs. Muthen & Muthen, I am practicing applications of the Convenience Feature and ALIGNMENT option using Mplus 7.11 to assess measurement invariance. I am testing a single factor CFA with 9 binary indicators for two groups. I have a couple of questions: 1. Is there any Convenience Feature to test invariance of factor variance and residual variance? 1) Invariance testing metric and scalar against configural show P<0.0001, indicating that the measurement invariance does not hold across the two populations. However, the results of Alignment using Bayes show that only one intercept and one factor loadings are noninvariant. How do I explain the inconsistent results? 2) In the results of Alignment, some unstandardized loadings seem quite different (p<0.05) across groups, but the factor loadings are not considered non-invariant. Is this because of adjustment for multiple comparisons? 3) Alignment results only show comparisons of unstandardized loadings across groups. Do the results apply to standardized loadings? 4) With binary indicators, can DIFF priors be specified to thresholds? Your help will be highly appreciated.
Hi Bengt, Thank you so much for your quick response. I tested a single CFA with 16 binary indicators using ALIGNMENT option and Convenience Feature. Two intercepts are noninvariant, while all factor loadings are invariant, across groups (only 2 groups). I expect that invariance testing for comparing metric against configural models would have insignificant p-value; but actually the p-value is 0.0047. How should I interpret the results? Many thanks for your help.
The metric model is somewhat stricter in that it requires exact loading invariance. Alignment requires only approximate loading invariance. The test statistics used are also somewhat different. If you like, you can send your 2 outputs to support for further diagnosis.
Questions about measurement invariance analysis for situations with many time points: 1) In Web Note 17, measurement invariance is assessed with a multiple indicator LGM. It would be hard to specify a LGM when there are many time points (e.g., 50). 2) Then, can we treat time points as multiple groups? If yes, how to handle autocorrelations? 3) When no item is invariant across time point with respect to loadings and intercepts/thresholds, does this mean that the corresponding instrument does not measure the same construct over time?
where slides 69-84 discuss this and point to Version 7 UG ex 9.27. This is taking a cross-classified approach to growth modeling where you can work with approximate measurement invariance over time. And you can have many timepoints.
A videotaped version of this discussion is in the Utrecht course on our website.
hi reading the Asparouhov & Muthén (2013). Multiple group factor analysis alignment. Web note 18. i realized is possible to use a more stricter p-value than the default 5% for the (alignment estimation)"factor mean comparison". how do i change the default 5% to 1% (stricter comparison).Interested in how to input the Mplus command.
I use the Convenience Feature and ALIGNMENT option to assess measurement invariance. my question is: 1) Invariance testing metric (CFI=.921, RMSEA=.038) against scalar (CFI= .906, RMSEA=.041) show P<0.00001,>.05. How do I explain the inconsistent result?
HI THE inconsistency is based on the fact that the alignment(fixed) method indicated that all the intercept are invariance. why the Convenience method indicated a worse model for the scalar. any possible explanation?
The chi-square testing (what you refer to as the "Convenience method") has more power because it considers all parameters jointly, whereas the Alignment method looks at each item's measurement parameter separately. This power difference is not essential because the focus of the Alignment method is not so much to detect/test non-invariance as it is to allow factor means to be compared, allowing for only approximate measurement invariance.
hi why is it that, with the alignment method if i specify a reference group e.g align =FIXED(2); the group rankings are different from if i specify the group specify the grouping variable name e.g KNOWNCLASS = C(GROUPS). I think if you specify the reference group it should override the default. copy of result to questions below: 1. specifying grouping values in the knownclass: Results for Factor F1 Ranking Group Value Groups With Significantly Smaller Factor Mean 1 3 0.029 2 2 0.000 3 1 -0.106 4 4 -0.208 2. specifying grouping name in the knon class: Ranking Group Value Groups With Significantly Smaller Factor Mean 1 2 0.000 1 2 4 -0.020 3 3 -0.133 4 1 -0.234 4 4 -0.208