Measurement invariance and multilevel...
Message/Author
 student07 posted on Tuesday, October 30, 2007 - 8:22 am
Dear All

I would be grateful for any suggestions regarding the role of measurement invariance (between groups, not levels) when using multilevel sem?

Assume you have 30 groups (samples) from cross-cultural research. For a given constructs (say one-factor with six indicators), it is common practice first to establish measurement invariance of this construct between these groups using the individual data, e.g. via multigroup comparisons.

However, is it correct to argue that

(1)in multilevel SEM, all differences between individuals which are due to group-differences are zeroed out (as one uses a pooled within-matrix) and

(2) such group-differences
are then treated on the group-level (by using the between-matrix)?

Hence, would there still be any need to establish meassurement invariance between groups?

Many thanks for any comments or literature hints!
 Bengt O. Muthen posted on Tuesday, October 30, 2007 - 12:43 pm
Multilevel, multi group analysis needs more methodological investigation. There is a Muthen-Khoo-Gustafsson paper on 2-level factor analysis of Catholic and Public schools that might be useful. The key issue is if the grouping variable is a within- or a between-level variable. In that paper the group was school type and so a between-level variable. Measurement invariance is then needed for the between-level measurement parameters (such as between-level loadings) in order to compare school types on school factor mean and variance differences.

If group is an individual-level variable such as gender, the analysis is more complicated. Gender differences are not zeroed out as suggested in (1). This case needs further methodological study.
 student07 posted on Thursday, November 01, 2007 - 9:09 am
Dear Dr. Muthen,

many thanks for pointing me to this paper which I found on your hoempage. However, I now realized that I have problems with understanding a much more general issue:

why must Sigma-within be specified for both Spw AND S*b ?

Or:

Why does the between-groups covariance matrix S*b comprise of both (scaled) Sigma-between AND Sigma-within?

I can't figure out why the Sigma-within-component - which is already used for the pooled within-groups matrix Spw - reappears in S*B?

I would be really grateful for an answer to this basic question?
 Bengt O. Muthen posted on Thursday, November 01, 2007 - 2:19 pm
See Technical Appendix 10 on the web site, eqn (199). This shows that the expected value of the SB sample cov matrix (given in 197) is a function of both SigmaW and SigmaB. The fact that SigmaW is also part of this is because SB ultimately is created from individual values on y via the means.
 Jan Fax posted on Wednesday, March 05, 2008 - 3:23 pm
Hello all

I have a similar question with regard to the decomposition of the total variance in individual- and group-level variances in multilevel factor analysis. I tried to understand the technical appendix and Muthen 1989. but i really have problems in understanding. sorry, but what does it mean that "SigmaB is ultimately created from individual values on y via the means"? I thought that as soon as you take the group-means than you are then dealing with group-variance only? Ok, the group means are calculated from individual scores. but still -- why sigma-within and sigma-between are needed for the SB sample covariance matrix ?(sorry, I begin only ML factor analysis and really have problems with finding an answer to this basic question. Most other ML formulas are clear for me, but this not)

Many thanks to all of you
 Bengt O. Muthen posted on Thursday, March 06, 2008 - 1:03 am
Since you sound technically inclined, you might be interested in my paper #32 at my UCLA web site http://www.gseis.ucla.edu/faculty/muthen/full_paper_list.htm

which refers to Muirhead (1982) in eqn (44) for E(S_B) = Sigma_W + c*Sigma_B.

You see in (43) that S_B involves products of y-bar expressions which in turn involve products of sums of y_i observations. The expectation of producs of y_i observations correspond to covariances between individual observations, that is Sigma_W. So not only Sigma_B is involved.
 Inga BEck posted on Saturday, November 15, 2008 - 11:58 am
Hello All

I just realize that this is a more general question, but I came across the following problem when doing a twolevel-factor analyses:

In the multigroup literature, it is often argued that any meaningful comparisons of means and covariances between units (say, countries) require scalar invariance.

I now wonder if and to what extent this requirement applies to twolevel factor analyses as well?

Obviously, configural invariance is assumed anyway in twolevel FA(otherwise I have misunderstood Muthen's 1994 formulas).

Note that my request does not relate to the question of between-level invariance for a 2-level FA (which is surely another interesting, if not underresearched topic).

I would be very grateful for any hints from Mplus users on recent research discussing the relation of multigroup (fixed) vs. multilevel (random) approaches?

Best wishes !
Inga
 Bengt O. Muthen posted on Sunday, November 16, 2008 - 4:21 pm
Not quite sure what you are asking, but let me state a few things. Multiple-group, 2-level factor analysis involves group invariance of the between-level loadings as in the paper on our web site (Papers, Multilevel SEM):

Muthén, B., Khoo, S.T., & Gustafsson, J.E. (1997). Multilevel latent variable modeling in multiple populations. Unpublished technical report.

Invariance of loadings across levels (for a given group) is typically not realistic.

Multiple groups vs multiple levels concerns the choice between treating group as fixed or random just like the choice in anova. If you don't want to generalize to a population of groups, you take the fixed approach (multi-group). This is also needed with few groups (say < 20).
 Inga BEck posted on Monday, November 17, 2008 - 7:41 am
Dearr Dr. Muthen,
thanks - but let me try to ask my question more precisely, as I am not concerned with multigroup ML SEM and not invariance between levels (though both topics are very interesting as well)

"In contrast to multiple group [..] models, ML SEM [...] assumes rather than tests for measurement invariance"

(Selig, Card & Todd Little in: van de Vijver et al (2008): Multilevel analyses of individuals and cultures, p. 115).

This was the starting point for my earlier question #1 - namely, if and to what extent the requirement of measurement invariance must be accomplished for ML SEM/ twolevel factor analyses or if invariance need not be established for ML SEM.

The few papers I know which apply ML SEM almost never discuss this issue - perhaps for good reasons I am not aware of.

In short my question is:

Would it be appropriate to apply ML SEM/factor analyses (treating groups as random )even when a multigroup analyses (treating groups as fixed) would show that factor loadings and intercepts of the indicators are NOT invariant between groups - even if the number of groups/between-level units is "large" > 30?

Many thanks
 Bengt O. Muthen posted on Monday, November 17, 2008 - 12:15 pm
Perhaps those authors refer to the fact that in multilevel modeling in general there is the assumption that all members of a cluster come from a single population. Which implies that they also follow the same measurement model.

But note that multilevel modeling does allow for random effects in level 1 relationships where the random effects are quantities that vary across level 2 units (clusters). So if one cluster (one group) has a higher factor indicator intercept than another, this is captured by a higher random intercept value for that group. Mplus can also handle random slopes for group-varying loadings. As in regular multilevel modeling, the random effects are assumed to come from one and the same population (follow the same level 2 parameters). So for example, you estimate the mean and variance of the random intercept for the factor indicator.

Thoughts from others?
 Inga BEck posted on Monday, November 17, 2008 - 1:54 pm
Dear Dr. Muthen

1. Which syntax would you try to examine whether a factor loading varies between groups ("clusters")?

2. If a factor loading carries random variance, then, in turn, this between-group variance might be modelled as well - is that right?

Example: Assume you consider the strength of a factor loading (= the size of the parameter) to indicate the "salience" of the content of a certain indicator of an underlying dimension for respondents (larger size = content of the indicator is more salient).

In turn, this salience might differ across groups - say, different countries.

Now assume social science theories would offer us a reasonable idea for which reason this salience differs across countries (they don't, I guess).

Then one might operationalize these country-level reason (= variables)and regress the random slope of the factor loading on this country-level variable.

3. Would this, in principle, make sense to you?

Best wishes + thanks
 Bengt O. Muthen posted on Monday, November 17, 2008 - 11:35 pm
1. You use the random slope approach, where on level 1 (within) you say:

sj | yj on f;

recognizing that "yj on f" is the same as f by yj but with a bar statement, so that the slope is random.

2. Yes, the latent variable sj can then be used as any continuous variable on level 2 (between). For example, you can estimate its mean and variance and its regression relations with other variables.

3. Yes. - And, it would be good to see applications of this.

The only caveat is that each random slope will cost you a dimension of numerical integration, so that you want to let very few slopes go random. One useful case is where slopes can be thought of as equal across items, so that only one random slope is used.
 Hermann Duelmer posted on Wednesday, May 27, 2009 - 2:04 pm
Dear All,

a) “ML SEM necessarily assumes weak measurement invariance given that only one set of loadings is estimated for the within group portion of the model” (Selig et al. 2008, 105 in: van de Vijver et al.: Multilevel Analysis; “weak invariance” means “metric invariance”).

I’m right in assuming that metric invariance is assumed/required in ML SEM for both, the within part of the CFA and the between part of the CFA? Scalar invariance is not required since it is modelled by the residual variances of the indicator’s intercept (by the e-terms of level 1 and the u-terms of level 2).

b) “the factor structure at the individual level of ML SEM is assumed to be equal across cultures, but is not assessed empirically … Therefore, it is possible that the pooled within structure in ML SEM fits well at the individual level but that structural equivalence does not hold for some cultural groups.” (Cheung et al. 2006, 526, Journal of Cross-Cultural Psychology 37).

Assumed the SRMR within/between for a 2-Level CFA are both below 0.05 (CFI, TLI, RMSEA fit well). Can I infer then that metric invariance is reached for both levels across the vast majority of countries (level 2 units) even if there might be a few outlier countries (or how far gives it at least a hint on it)? Testing invariance via multi-group CFA in a first step before ML SEM is unwieldy for n>30 countries.

Many thanks!
 Linda K. Muthen posted on Wednesday, May 27, 2009 - 5:21 pm
These issues are discussed in the following paper which can be downloaded from the website under Papers:

Muthén, B., Khoo, S.T., & Gustafsson, J.E. (1997). Multilevel latent variable modeling in multiple populations. Unpublished technical report.
 Mike Stoolmiller posted on Tuesday, June 19, 2012 - 2:58 pm
In a 2007 post, Bengt replied, "If group is an individual-level variable such as gender, the analysis is more complicated. Gender differences are not zeroed out as suggested in (1). This case needs further methodological study." in response to multi-level, multi-group models. I am interested in this kind of model and wonder if further methodological study has happened since 2007 and if there is a way to fit such a model in Mplus (e.g., a simple, 3 variable path model for male and female students nested within schools).
 Bengt O. Muthen posted on Tuesday, June 19, 2012 - 5:28 pm
Yes, it can now be done properly. We'll send you a writeup on this shortly, Mike.
 Mike Stoolmiller posted on Wednesday, June 20, 2012 - 1:17 am
Thanks Bengt.
 Michael Schurig posted on Tuesday, September 04, 2012 - 11:45 pm
Dear Dr. Muthén,

Is the Paper already in press or available online?

I´m dealing with partial scalar invariance (2 factor, 3-4 ordinal indicators) on the individual level (for all indicators thresholds of one factor) in a clustered setting and looking for the best way to incorporate this (propably a group specific social desirabilty response bias) for further longitudinal analysis (LGM). Therefore i would hope that it could help me to illuminate the issue for me.
 Bengt O. Muthen posted on Tuesday, September 04, 2012 - 11:56 pm
No, we've been busy with V7 and haven't gotten to it yet. But will soon be able to.