Multigroup cfa for a lot of groups an... PreviousNext
Mplus Discussion > Confirmatory Factor Analysis >
 Murat Akyol posted on Tuesday, May 05, 2009 - 7:22 am

I have compared the factor structure invariance of PIRLS 2001 data for 35 countries. And i didn't find any difference between 35 countries.

I decided to create subgroups of these 35 countries based on language (languages the tests are applied, such as countries that took the tests in English, Spanish etc), socioeconomic level (oecd countries, undeveloped and developed countries), EU member countries and others, etc.

I found significant differences between subgroups of these 35 countries based on RMSEA differences.

I cannot explain why i don't find any difference between 35 countries but find significant difference between subgroups.

Note: A professor i have asked advice, suggested that, this could be caused by DIF of items. What do you think about this.

Any suggestions and academical references will be appreciated. Thanks in advance.
 Bengt O. Muthen posted on Tuesday, May 05, 2009 - 12:51 pm
It sounds like the more specific variables describing the countries are more precise and powerful in describing response differences than the broad classification of country itself. In that sense, it doesn't have to involve DIF.
 Murat Akyildiz posted on Sunday, May 17, 2009 - 11:26 am
Thank you very much for your reply. I explained the difference just like you said. And now i am testing country based variables and common variables and dif based on PIRLS data. Your advice has been very helpful, thanks again.
 Dan Costa posted on Sunday, June 02, 2013 - 3:48 pm
Is there a need to control for Type I error rate when there are so many comparisons in a multigroup CFA? It seems to make sense that an adjustment is needed when using the chi-squared test to assess invariance, but what about if subjective fit indices or modification indices are used? Several co-authors and reviewers have suggested this possibility to me but none of them have been sure if it's necessary, and I've not found any guidance from published literature.
 Bengt O. Muthen posted on Monday, June 03, 2013 - 8:53 am
I think it is a good idea to somehow take into account you make many comparisons. E.g. by making the p-value assessment stricter. See also the new multiple-group method presented in:

Asparouhov & Muthén (2013). Multiple group factor analysis alignment. Web note 18.

which you find on our website.
 Patrick Malone posted on Monday, June 03, 2013 - 11:30 am
Bengt, I'm working with FA alignment now and having some success.

However, I can't find in the web note, in the language addendum, or in your presentation slides how to read the noninvariance output. For example, with 9 groups and five items:

Q1 1 2 3 4 5 6 7 8 9
Q2 1 (2) (3) (4) (5) 6 7 (8) 9
Q3 1 2 3 4 5 6 7 8 9
Q5 1 2 3 4 5 6 7 8 9
Q6 1 2 3 4 5 6 7 8 9

If I haven't missed something, could you remind me how to read this?

Is the Q2 intercept the one with the noninvariance problem in several groups? I'm looking for the interpretation of the parentheses.

 Bengt O. Muthen posted on Monday, June 03, 2013 - 11:43 am
Ah, maybe the heading is too subtle:


- the parenthetical part is meant to show that groups in parentheses are noninvariant.

So, yes, item Q2 shows several groups with non-invariant intercepts.
 Patrick Malone posted on Monday, June 03, 2013 - 1:32 pm
 J.W. posted on Wednesday, July 03, 2013 - 2:40 pm
Dear Drs. Muthen & Muthen,
I am practicing applications of the Convenience Feature and ALIGNMENT option using Mplus 7.11 to assess measurement invariance. I am testing a single factor CFA with 9 binary indicators for two groups. I have a couple of questions:
1. Is there any Convenience Feature to test invariance of factor variance and residual variance?
1) Invariance testing metric and scalar against configural show P<0.0001, indicating that the measurement invariance does not hold across the two populations. However, the results of Alignment using Bayes show that only one intercept and one factor loadings are noninvariant. How do I explain the inconsistent results?
2) In the results of Alignment, some unstandardized loadings seem quite different (p<0.05) across groups, but the factor loadings are not considered non-invariant. Is this because of adjustment for multiple comparisons?
3) Alignment results only show comparisons of unstandardized loadings across groups. Do the results apply to standardized loadings?
4) With binary indicators, can DIFF priors be specified to thresholds?
Your help will be highly appreciated.
 Bengt O. Muthen posted on Wednesday, July 03, 2013 - 3:00 pm
1. No.

1) That's not inconsistent: Those 2 non-invariant parameters can be the cause of the rejection.

2) Yes - see Web Note 15.

3) No. When you compare across groups you want to work with unstandardized coefficients.

4) Yes.
 J.W. posted on Sunday, July 07, 2013 - 6:02 pm
Hi Bengt,
Thank you so much for your quick response.
I tested a single CFA with 16 binary indicators using ALIGNMENT option and Convenience Feature. Two intercepts are noninvariant, while all factor loadings are invariant, across groups (only 2 groups). I expect that invariance testing for comparing metric against configural models would have insignificant p-value; but actually the p-value is 0.0047. How should I interpret the results?
Many thanks for your help.
 Bengt O. Muthen posted on Monday, July 08, 2013 - 6:45 am
The metric model is somewhat stricter in that it requires exact loading invariance. Alignment requires only approximate loading invariance. The test statistics used are also somewhat different. If you like, you can send your 2 outputs to support for further diagnosis.
 J.W. posted on Thursday, July 11, 2013 - 1:00 pm
Questions about measurement invariance analysis for situations with many time points:
1) In Web Note 17, measurement invariance is assessed with a multiple indicator LGM. It would be hard to specify a LGM when there are many time points (e.g., 50).
2) Then, can we treat time points as multiple groups? If yes, how to handle autocorrelations?
3) When no item is invariant across time point with respect to loadings and intercepts/thresholds, does this mean that the corresponding instrument does not measure the same construct over time?
 Bengt O. Muthen posted on Thursday, July 11, 2013 - 1:20 pm
You may want to take a look at the workshop handout for Ljubljana at

where slides 69-84 discuss this and point to Version 7 UG ex 9.27. This is taking a cross-classified approach to growth modeling where you can work with approximate measurement invariance over time. And you can have many timepoints.

A videotaped version of this discussion is in the Utrecht course on our website.
 emmanuel bofah posted on Wednesday, August 07, 2013 - 7:58 am
reading the Asparouhov & Muthén (2013). Multiple group factor analysis alignment. Web note 18. i realized is possible to use a more stricter p-value than the default 5% for the (alignment estimation)"factor mean comparison". how do i change the default 5% to 1% (stricter comparison).Interested in how to input the Mplus command.
 Linda K. Muthen posted on Wednesday, August 07, 2013 - 8:44 am
There is no option to change to p-value.
 emmanuel bofah posted on Wednesday, August 07, 2013 - 4:28 pm
I use the Convenience Feature and ALIGNMENT option to assess measurement invariance. my question is:
1) Invariance testing metric (CFI=.921, RMSEA=.038) against scalar (CFI= .906, RMSEA=.041) show P<0.00001,>.05. How do I explain the inconsistent result?
 Linda K. Muthen posted on Thursday, August 08, 2013 - 10:26 am
I don't see the inconsistency. The scalar model has a worse CFI and RMSEA and the chi-square p-value is small when testing the scalar against the metric (not the other way around).
 emmanuel bofah posted on Thursday, August 08, 2013 - 12:59 pm
THE inconsistency is based on the fact that the alignment(fixed) method indicated that all the intercept are invariance.
why the Convenience method indicated a worse model for the scalar. any possible explanation?
 Bengt O. Muthen posted on Thursday, August 08, 2013 - 1:19 pm
The chi-square testing (what you refer to as the "Convenience method") has more power because it considers all parameters jointly, whereas the Alignment method looks at each item's measurement parameter separately. This power difference is not essential because the focus of the Alignment method is not so much to detect/test non-invariance as it is to allow factor means to be compared, allowing for only approximate measurement invariance.
 emmanuel bofah posted on Wednesday, August 14, 2013 - 11:05 am
why is it that, with the alignment method if i specify a reference group e.g align =FIXED(2); the group rankings are different from if i specify the group specify the grouping variable name e.g KNOWNCLASS = C(GROUPS). I think if you specify the reference group it should override the default.
copy of result to questions below:
1. specifying grouping values in the knownclass:
Results for Factor F1
Ranking Group Value Groups With Significantly Smaller Factor Mean
1 3 0.029
2 2 0.000
3 1 -0.106
4 4 -0.208
2. specifying grouping name in the knon class:
Ranking Group Value Groups With Significantly Smaller Factor Mean
1 2 0.000 1
2 4 -0.020
3 3 -0.133
4 1 -0.234
4 4 -0.208
 Bengt O. Muthen posted on Wednesday, August 14, 2013 - 11:54 am
Please send the output and data for your 2 runs to support so we can see exactly what you are doing.
 emmanuel bofah posted on Wednesday, August 14, 2013 - 2:51 pm
output and data sent
Back to top
Add Your Message Here
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Options: Enable HTML code in message
Automatically activate URLs in message