Multigroup ESEM PreviousNext
Mplus Discussion > Structural Equation Modeling >
Message/Author
 Tony LI posted on Thursday, June 17, 2010 - 3:40 am
Hi Linda and Bengt,
I am testing measurement invariance on a 14-item instrument across 3 groups using multigroup ESEM framework (example 5.27).

The No-invariance-model (Model 1) fits the data well (CFA=0.965; RMSEA: 0.042). Constraining factor loadings to be equal resulted in worse fit (CFA:0.915; RMSEA=0.066).

For the next step I'd like to test partial matric invariance bying freeing some factor loadings (as suggested by
modification indices). My question is how to do this properly in a ESEM framework? I tried to use the usual CFA sytax (Model 2: Model A) but this did not seem to work and I got an error message.
Would you be able to shed some light on this ?Thanks in advance.


!Model 1 no invariance
MODEL:
f1-f3 by i1-i14 (*1);
[f1-f3@0];
MOdel A: f1-f3 by i1-i14 (*1); [i1-i14];
Model B: f1-f3 by i1-i14 (*1); [i1-i14];
Model C: f1-f3 by i1-i14 (*1); [i1-i14];

! Model 2 partial loading invariance
MODEL:
f1-f3 by i1-i14 (*1); [f1-f3@0];

MOdel A: f1-f3 by i2 i3 i5 i10 i14 (*1);
[i1-i14];
Model B: [i1-i14];
Model C: [i1-i14];

*** FATAL ERROR
IMPROPER PARAMETER CONSTRAINT FOR EFA MEASUREMENT SPECIFICATION.
(Error Code: 1020)
 Linda K. Muthen posted on Thursday, June 17, 2010 - 9:01 am
Partial measurement invariance is not allowed with ESEM.
 Tony LI posted on Friday, June 18, 2010 - 2:57 am
Hi Linda,

Thanks for the reply.

Are you mean partial metric invariance is not allowed? If so, are there any thoretical reason for this?

Apparently I can run partial scalar (intecept) invariance test on my another dataset. Also it was used in Marsh et al's ESEM big five paper.

Back to my case above, if metric invariance is not tenable. Where would you suggest to go? Back to a MGCFA framwork to test partial invariance?


Thanks
 Tihomir Asparouhov posted on Friday, June 18, 2010 - 2:15 pm
Partial intercept invariance is allowed in ESEM but partial loadings invariance is currently not allowed. This is because during the rotation a set of unequal parameters will result in a different set of unequal parameters (and the rotation is either the same in the groups or different). One possible approach is to take a particular indicator say i1, exclude it from the definition of the factors (f1-f3 by i2-i14) and use i1 on f1-f3 as varying across the groups. This will however exclude i1 from the rotation mechanism. Another approach is to modify the model with group specific residual correlations such as say i1 with i3.
 Tony LI posted on Monday, June 21, 2010 - 2:42 pm
Hi Tihomir,
Thanks for the helpful insight.
Once further question regarding ESEM: is "missing by design" allowed?

I wanted to do multiple group analysis to test for the invariance for a model(three groups took two versions of a test linked by common items). When I tried to run the following syntax, I got an error saying that the grouping/pattern variable had multiple uses.

NAMES ARE country gender i1-i10 a1 a2;
USEVARIABLES ARE country i1-i20 a1 a2;
MISSING is ALL(999);
GROUPING is country (0=DE 1=GB 2=SP);
PATTERN is country(0=i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 1=i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 2=i1 i2 i3 i4 i5 i6 i7 i8 a1 a2);

MODEL: !No invariance
f1-f3 by i1-ia2 (*1); [f1-f3@0];
MOdel GB: f1-f3 by i1-ia2(*1); [i1-ia2];
MOdel SP: f1-f3 by i1-ia2(*1); [i1-ia2];
OUTPUT: MOD; TECH1;

I figured it was because the same variable was used both as a grouping and a pattern variable. So I tried two things:

first I removed the pattern syntax, but I kept getting errors that said variables had no non-missing values. I then only used the pettern command. This time Mplus appeared to be running but the OUT file didn't pop out. I opened the OUT file, there was no result apart from my sytax plus a sentence of "INPUT READING TERMINATED NORMALLY".

I would greatly appreciate any suggestions
 Linda K. Muthen posted on Monday, June 21, 2010 - 2:51 pm
Please send the files and your license number to support@statmodel.com.
 Tony LI posted on Monday, June 21, 2010 - 3:11 pm
Thanks Linda, i've just sent the files and license info. Looking forward to hearing from you.
 Stacey Farber posted on Saturday, May 14, 2011 - 9:15 am
Hello.

I would like to test measurement invariance for multiple groups and over time. ESEM seems best since items have non-zero loadings on non-primary factors. However, data are also nested (child within school).

Thanks to your reference lists, I see evidence of multilevel ESEM and multigroup ESEM. But, I have not seen multigroup, multilevel ESEM nor can I seem to find reference in the User Guide. All could be oversights on my part, but is it possible to do multilevel, multigroup ESEM?

If so, can you point me in the right direction for syntax?

Thank you!
 Bengt O. Muthen posted on Sunday, May 15, 2011 - 10:02 am
Mplus currently does not do multilevel ESEM. Multiple-group ESEM is however available - see UG ex5.27
 Al Grimm posted on Wednesday, June 01, 2011 - 9:22 am
Referring back to Tihomir's note above that partial intercept invariance is allowed in ESEM but partial loadings invariance is not, I take it that partial factor mean invariance is also not an option? Or is there a way around the Mplus message "EFA factors in the same set as FACTOR must have all fixed or free means. Problem with: [ FACTOR ]". Cheers.
 Tihomir Asparouhov posted on Wednesday, June 01, 2011 - 10:47 am
Partial factor mean invariance is also not available but you can test for it using model test.
 Al Grimm posted on Thursday, June 02, 2011 - 5:26 am
Tried that by assigning parameter labels to the factor means (plus fixing them to zero under MODEL TEST), but Mplus does not appear to like that: "Parameters involving EFA factors cannot be constrained with equality labels or assigned a parameter label." If it is not permitted to assign parameter labels to factor means, can you still fix them under MODEL TEST?
 Tihomir Asparouhov posted on Thursday, June 02, 2011 - 4:08 pm
Al

Model test doesn't fix or constrain anything - it just performs Wald test on the estimated parameters. But you are correct the current version will not allow the parameters that you are interested in Model Test. You can ask for the tech3 output, and using the estimated variance covariance matrix you can compute the Wald test by hand.

Tihomir
 David Bard posted on Friday, June 08, 2012 - 5:57 am
I've been tinkering with ESEM invariance testing and have a question about Table 1 in the latest two Marsh papers posted in your ESEM special topic site. Those tables indicate that the ESEM invariance models are fully nested as you move from strict to strong to weak to configural invariance (models 1,2,5, and 7 in the table). I trust that this is true, but it's difficult to envision when the factor variances and means are constrained for group 2 (or non-referent group) in the config model but one or both are not constrainted in the models nested within it. Are there starting unrotated solutions that make this progression of nested structure more obvious (much like the starting oblique rotated solution offered up in Asparouhov & Muthen's 2009 SEM paper, p. 420, helps to explain how CFA is nested within ESEM)?
 Tihomir Asparouhov posted on Friday, June 08, 2012 - 1:57 pm
David - this is exactly correct all the invariance and non-invariance is traced back to the unrotated solution.

First intercepts are not affected by the rotation (rotation affects the factor means but not the intercepts) - thus invariance or non-invariance for the intercepts is designed at the unrotated solution. The same applies to item uniquenesses.

Factor variances– covariances invariance - to achieve that we estimate unrorated model with the identity matrix as the factor variance covariance. If you want non-invariance then only in the first group you have I and in the second etc it is free for the unrotated solution. When the loadings are invariant - the loadings are held equal for the unrotated solution and are rotated with the same rotation across group so you get the same rotated loadings. The latent factor means - if you want invariance they are held to 0 for the unrotated model. If you want them non-invariant they are held to 0 in group 1 and free in the other groups (same logic as for the variance covariance).

Hope this helps.
 David Bard posted on Saturday, June 09, 2012 - 10:45 am
Maybe it would help if I just focus on the config and weak invariance models. In config model, we have 2 separate EFAs, so factor means and variances are constrained to 0 and 1, respectively, in both groups. This identification constraint forces factor means and variances to be constrained equal across groups. Everything else is unconstrained. In weak inv model, factor loadings are constrained equal, and by default, factor vars of non-referent group are free to vary. If we ignore the factor var-cov matrix, it's easy to see the weak inv model is nested within the config model b/c only the loading constraint in the former differs across the two. But, consider the factor var-cov matrix and notice factor vars are also different across models. For this particular matrix, the config specification appears nested within the weak spec.

In CFA context, Rensvold & Cheung (1998; 1999; 2001) discuss this as a "standardization problem" and argue for "Type-2 standardization" that constrains 1 invariant loading per factor so factor vars are free to vary in each model. A Type-2 standardized config model is not possible currently in M+ ESEM, but I'm wondering if this type of constraint might be equivalent (in terms of fit) to the default config where all factor vars equal 1.In CFA, it matters which "referent" items are selected for constraint, but perhaps this wouldn't matter in an ESEM context?
 David Bard posted on Saturday, June 09, 2012 - 10:46 am
A similar identification and nesting issue occurs once ready to test intercept constraints b/c the factor means can now be freed up in the strong and strict FI models. Meredith & Horn (2001) describe possible solutions to this identification and nested constraints problem. Again I'm wondering how those constrained solutions relate to the type of nesting specified in M+ ESEM invariance models.
 David Bard posted on Sunday, June 10, 2012 - 10:54 am
I realized I could actually run the unrotated config model proposed above using EFA within CFA. The code below does indeed reproduce the same fit as the default ESEM config model. The nesting of weak within config is more obvious (to me, at least) this way. I'm sure by respecifying factor means and intercept constraints the same could be done to make the nesting of strong and strict models within weak and config models more obvious.

MODEL:
f1 by t1*1 (1)
t2-t10*1
t12-t18*0
t20-t28*0;

f2 by t11*1 (2)
t2-t10*0
t12*t18*1
t20-t28*0;

f3 by t19*1 (3)
t2-t10*0
t12-t18*0
t20-t28*1;

[f1-f3@0];
f1-f3@1;

[t1-t28*];
t1-t28*;
MODEL GRP2:
f1 by t1*1 (1)
t2-t10*1
t12-t18*0
t20-t28*0;

f2 by t11*1 (2)
t2-t10*0
t12*t18*1
t20-t28*0;

f3 by t19*1 (3)
t2-t10*0
t12-t18*0
t20-t28*1;

[f1-f3@0];
f1-f3*1; !can free up factor vars this way;

[t1-t28*];
t1-t28*;
 Bengt O. Muthen posted on Sunday, June 10, 2012 - 3:37 pm
It seems that you answered your own question and convinced yourself that weak nested within config is correctly tested in ESEM.
 EFried posted on Wednesday, July 04, 2012 - 9:38 am
It is stated in this thread (2010 and 2011) that partial measurement invariance is not allowed with ESEM.

However, it appears the Marsh et al 2010 paper on the Big 5 does test exactly that. Am I mistaken? If not, what would be the parameterization for e.g. factor loading invariance?

f1-f2 x1 x2 x3 (*1) (1-3);
f3-f4 x4 x5 x6 (*2) (1-3);

does not work.

Thank you
 Linda K. Muthen posted on Thursday, July 05, 2012 - 10:40 am
Which page of the article are you looking at?
 EFried posted on Thursday, July 05, 2012 - 12:21 pm
"To address these questions, we applied our taxonomy of 13 ESEM models (see Table 1). The basic strategy is to apply the set of 13 models designed to test different levels of factorial and measurement invariance, …"

Text on p. 478 bottom left, table on p. 476 top left.

The supplemental material of the paper (http://dx.doi.org/10.1037/a0019227.supp) does not include MPLUS syntax for these tests.

Thank you
 Bengt O. Muthen posted on Saturday, July 07, 2012 - 6:39 pm
I only have the version of the paper at our web site

http://www.statmodel.com/download/Marsh%20Nagengast%20Morin%202012%20Big5%20Inv%20ESEM%20la%20dolce%20viita%20effect%20DevPsych.pdf

which has different page numbers. But I suspect that they are talking about using EFA/ESEM in a CFA framework, so not regular ESEM.
 EFried posted on Sunday, July 08, 2012 - 7:54 am
Bengt, I am talking about page five in the pdf you linked to, table 1.

"Marsh et al. (2009) introduced a taxonomy of 13 ESEM models (see Table 1) designed to measurement invariance […] . Importantly, ESEM allows applied FFA researchers to pursue appropriate tests of measurement invariance when CFA models are not appropriate."

They repeatedly state testing for measurement invariance in ESEM models. Maybe you could help me make sense of that.
 Bengt O. Muthen posted on Sunday, July 08, 2012 - 12:04 pm
Bottom of page 8, right column, last paragraph says:


"ESEM-within-CFA model (see the Supplemental Materials
for further discussion). Despite the flexibility of the ESEM
approach, we note that there are some aspects and extensions of
traditional SEM models that cannot readily be implemented with
ESEM as currently operationalized in Mplus"

This section describes what ESEM cannot do and what needs "ESEM-within-CFA".

The Mplus UG example 5.27 describes what "pure" ESEM can do in terms of invariance testing.
 EFried posted on Sunday, July 08, 2012 - 1:36 pm
Bengt, the misunderstanding was that I was referring to the 2010 Marsh et al paper "A New Look at the Big Five Factor Structure Through Exploratory Structural Equation Modeling", which you co-authored.

We are very interested to understand how the measurement invariance tests (including factor loading invariance) for ESEM models were performed in that paper (there is no syntax in the User's Guide or the supplemental materials that shows how to do this).

What we don't understand is that it is stated above in this thread that MPLUS cannot do this. However, Marsh et al talk about exactly this at various points in the paper and the supplemental materials. Two examples:

"Weak factorial/measurement invariance tests the invariance of factor loadings over time […] it is not surprising that the CFI is marginally better for LIM1E (.912) than for LIM2E (.907; see Table 5)."

"With ESEM models it is possible to constrain the loadings to be equal across two or more sets of EFA blocks in which the different blocks represent multiple discrete groups or multiple occasions for the same group."

The problem with the UG example 5.27 is that - as far as we understand - the "grouping" procedure assumes independence between groups, which does not hold when testing measurement invariance over time (factors are correlated over time).

Thanks
 Bengt O. Muthen posted on Sunday, July 08, 2012 - 1:51 pm
The longitudinal case is covered in UG ex 5.26. The "1" in parenthesis holds the factor loading matrix invariant across time. That 1 can be removed when you don't want loading invariance.
 EFried posted on Monday, July 09, 2012 - 11:58 am
Thank you Bengt!!
 Hsien-Yuan Hsu posted on Tuesday, October 30, 2012 - 4:42 am
Dear Dr. Muthen,

I try to replicate Marsh et al (2009) Model 13 (complete factorial invariance model)in SEM journal with Mplus 6.1. However, I got the following message. Can you figure out anything wrong?

Thanks.
Mark

*** ERROR in MODEL command
The variance of EFA factors in the reference block cannot be modified
in the first group.
Problem with: F1
*** ERROR in MODEL command
The variance of EFA factors in the reference block cannot be modified
in the first group.
Problem with: F2
*** ERROR in MODEL command
The variance of EFA factors in the reference block cannot be modified
in the first group.
Problem with: F3
*** ERROR in MODEL command
The variance of EFA factors in the reference block cannot be modified
in the first group.
Problem with: F4
*** ERROR in MODEL command
The variance of EFA factors in the reference block cannot be modified
in the first group.
Problem with: F5
*** ERROR in MODEL command
The variance of EFA factors in the reference block cannot be modified
in the first group.
Problem with: F6
*** ERROR
The following MODEL statements are ignored:
* Statements in Group G2:
F1
F2
F3
F4
F5
F6
 Tihomir Asparouhov posted on Thursday, November 01, 2012 - 9:49 am
There is no discussion on complete factorial invariance model in that paper. I think you are referring to a 2012 paper. In any case, I do not have access to the input files that were used in that paper. Maybe you can contact Marsh directly.
 PB posted on Tuesday, November 27, 2012 - 12:07 pm
Hi,

referring to the post on May 14, 2011 - 9:15 am, I wanted to ask, whether in the new version of MPlus multilevel multiple group ESEM is available?

Thanks in advance and best regards
Pablo
 Linda K. Muthen posted on Tuesday, November 27, 2012 - 12:20 pm
Multilevel ESEM is not available. ESEM is available with TYPE=GENERAL and TYPE=COMPLEX.
 Michelle Little posted on Sunday, January 27, 2013 - 7:11 am
Hello,

I have two questions about using multi-group ESEM.
1. Published examples use very large samples. IS there any literature on use in small samples (n=100-400)?

2. I find that when using the syntax suggested in the MPLUS manual and in the Marsh supplementary materials online, that my 2 groups show a similar pattern of factor loadings for a 4-factor measure. HOWEVER, the factors don't match across groups.. Factor 1 in group 1 matches factor 2 in group 2. Further attempts to use target loadings makes no difference in the output. I find the same pattern. This pattern doesn't match available MPLUS online examples, but the examples use only 2 factors. Follow-up use of target loadings per factor did not fix the issue. I am not clear what this discrepancy means and why it is happening.. i Any help would be appreciated.

Thank you.
 Linda K. Muthen posted on Monday, January 28, 2013 - 1:20 pm
1. You may find some information in the following paper which is available on the website:

Asparouhov, T. & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397-438.

Or you can do your own simulation study based on your data. See Example 12.5 in the Mplus User's Guide.

2. I don't think this matter in the configural case because nothing is being held equal.
 Michelle Little posted on Monday, January 28, 2013 - 1:31 pm
Thanks this is helpful.

I have a related question. I am having some problems getting one of the nested models to provide me a diftest using WLSMV.

I was able to do it by gender, but by race I get the suggestion to "decrease the convergence" criteria. I tried this and it did not work. But, I'm not sure which convergence is being referred to...

Thanks,

ML
 Linda K. Muthen posted on Monday, January 28, 2013 - 1:36 pm
Please send the output and your license number to support@@statmodel.com.
 Michelle Little posted on Monday, January 28, 2013 - 1:50 pm
Linda,
I think I may have figured it out.
I originally tested a model using separate loading and threshold tests. Since I'm using categorical data, I just tried to test difference by constraining both loadings and thresholds. I then got a difftest output.

I have a small sample of minorities, which may explain why I could get a gender test but not a race test.

Should we be testing weak and strong (load and threshold) together for the ESEM tests?

Thanks
 Linda K. Muthen posted on Monday, January 28, 2013 - 3:21 pm
With binary items, the models to be tested are the unrestricted model and the model with free thresholds and loadings. Freeing the thresholds and loadings separately is not identified. For the details on how the models should be specified, see multiple group analysis in the Topic 2 course handout on the website and pages 485-486 of the user's guide/
 Michelle Little posted on Monday, January 28, 2013 - 9:41 pm
Thanks

I typically use the theta parameterization and Millsap and Tein's (2004) approach (for 3-point) scales. It calls for separate loading, threshold tests. For ESEM, I tried both delta and theta models. The theta tests ran well and allowed me to test the thresholds separately. Results are similar.

Thanks again for your help.
 Linda K. Muthen posted on Tuesday, January 29, 2013 - 11:38 am
Yes, the Millsap and Tein's approach is useful for polytomous items.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: