Tony LI posted on Thursday, June 17, 2010 - 3:40 am
Hi Linda and Bengt, I am testing measurement invariance on a 14-item instrument across 3 groups using multigroup ESEM framework (example 5.27).
The No-invariance-model (Model 1) fits the data well (CFA=0.965; RMSEA: 0.042). Constraining factor loadings to be equal resulted in worse fit (CFA:0.915; RMSEA=0.066).
For the next step I'd like to test partial matric invariance bying freeing some factor loadings (as suggested by modification indices). My question is how to do this properly in a ESEM framework? I tried to use the usual CFA sytax (Model 2: Model A) but this did not seem to work and I got an error message. Would you be able to shed some light on this ?Thanks in advance.
!Model 1 no invariance MODEL: f1-f3 by i1-i14 (*1); [f1-f3@0]; MOdel A: f1-f3 by i1-i14 (*1); [i1-i14]; Model B: f1-f3 by i1-i14 (*1); [i1-i14]; Model C: f1-f3 by i1-i14 (*1); [i1-i14];
! Model 2 partial loading invariance MODEL: f1-f3 by i1-i14 (*1); [f1-f3@0];
MOdel A: f1-f3 by i2 i3 i5 i10 i14 (*1); [i1-i14]; Model B: [i1-i14]; Model C: [i1-i14];
Partial intercept invariance is allowed in ESEM but partial loadings invariance is currently not allowed. This is because during the rotation a set of unequal parameters will result in a different set of unequal parameters (and the rotation is either the same in the groups or different). One possible approach is to take a particular indicator say i1, exclude it from the definition of the factors (f1-f3 by i2-i14) and use i1 on f1-f3 as varying across the groups. This will however exclude i1 from the rotation mechanism. Another approach is to modify the model with group specific residual correlations such as say i1 with i3.
Hi Tihomir, Thanks for the helpful insight. Once further question regarding ESEM: is "missing by design" allowed?
I wanted to do multiple group analysis to test for the invariance for a model(three groups took two versions of a test linked by common items). When I tried to run the following syntax, I got an error saying that the grouping/pattern variable had multiple uses.
NAMES ARE country gender i1-i10 a1 a2; USEVARIABLES ARE country i1-i20 a1 a2; MISSING is ALL(999); GROUPING is country (0=DE 1=GB 2=SP); PATTERN is country(0=i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 1=i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 2=i1 i2 i3 i4 i5 i6 i7 i8 a1 a2);
MODEL: !No invariance f1-f3 by i1-ia2 (*1); [f1-f3@0]; MOdel GB: f1-f3 by i1-ia2(*1); [i1-ia2]; MOdel SP: f1-f3 by i1-ia2(*1); [i1-ia2]; OUTPUT: MOD; TECH1;
I figured it was because the same variable was used both as a grouping and a pattern variable. So I tried two things:
first I removed the pattern syntax, but I kept getting errors that said variables had no non-missing values. I then only used the pettern command. This time Mplus appeared to be running but the OUT file didn't pop out. I opened the OUT file, there was no result apart from my sytax plus a sentence of "INPUT READING TERMINATED NORMALLY".
I would like to test measurement invariance for multiple groups and over time. ESEM seems best since items have non-zero loadings on non-primary factors. However, data are also nested (child within school).
Thanks to your reference lists, I see evidence of multilevel ESEM and multigroup ESEM. But, I have not seen multigroup, multilevel ESEM nor can I seem to find reference in the User Guide. All could be oversights on my part, but is it possible to do multilevel, multigroup ESEM?
If so, can you point me in the right direction for syntax?
Mplus currently does not do multilevel ESEM. Multiple-group ESEM is however available - see UG ex5.27
Al Grimm posted on Wednesday, June 01, 2011 - 9:22 am
Referring back to Tihomir's note above that partial intercept invariance is allowed in ESEM but partial loadings invariance is not, I take it that partial factor mean invariance is also not an option? Or is there a way around the Mplus message "EFA factors in the same set as FACTOR must have all fixed or free means. Problem with: [ FACTOR ]". Cheers.
Partial factor mean invariance is also not available but you can test for it using model test.
Al Grimm posted on Thursday, June 02, 2011 - 5:26 am
Tried that by assigning parameter labels to the factor means (plus fixing them to zero under MODEL TEST), but Mplus does not appear to like that: "Parameters involving EFA factors cannot be constrained with equality labels or assigned a parameter label." If it is not permitted to assign parameter labels to factor means, can you still fix them under MODEL TEST?
Model test doesn't fix or constrain anything - it just performs Wald test on the estimated parameters. But you are correct the current version will not allow the parameters that you are interested in Model Test. You can ask for the tech3 output, and using the estimated variance covariance matrix you can compute the Wald test by hand.
David Bard posted on Friday, June 08, 2012 - 5:57 am
I've been tinkering with ESEM invariance testing and have a question about Table 1 in the latest two Marsh papers posted in your ESEM special topic site. Those tables indicate that the ESEM invariance models are fully nested as you move from strict to strong to weak to configural invariance (models 1,2,5, and 7 in the table). I trust that this is true, but it's difficult to envision when the factor variances and means are constrained for group 2 (or non-referent group) in the config model but one or both are not constrainted in the models nested within it. Are there starting unrotated solutions that make this progression of nested structure more obvious (much like the starting oblique rotated solution offered up in Asparouhov & Muthen's 2009 SEM paper, p. 420, helps to explain how CFA is nested within ESEM)?
David - this is exactly correct all the invariance and non-invariance is traced back to the unrotated solution.
First intercepts are not affected by the rotation (rotation affects the factor means but not the intercepts) - thus invariance or non-invariance for the intercepts is designed at the unrotated solution. The same applies to item uniquenesses.
Factor variances– covariances invariance - to achieve that we estimate unrorated model with the identity matrix as the factor variance covariance. If you want non-invariance then only in the first group you have I and in the second etc it is free for the unrotated solution. When the loadings are invariant - the loadings are held equal for the unrotated solution and are rotated with the same rotation across group so you get the same rotated loadings. The latent factor means - if you want invariance they are held to 0 for the unrotated model. If you want them non-invariant they are held to 0 in group 1 and free in the other groups (same logic as for the variance covariance).
Hope this helps.
David Bard posted on Saturday, June 09, 2012 - 10:45 am
Maybe it would help if I just focus on the config and weak invariance models. In config model, we have 2 separate EFAs, so factor means and variances are constrained to 0 and 1, respectively, in both groups. This identification constraint forces factor means and variances to be constrained equal across groups. Everything else is unconstrained. In weak inv model, factor loadings are constrained equal, and by default, factor vars of non-referent group are free to vary. If we ignore the factor var-cov matrix, it's easy to see the weak inv model is nested within the config model b/c only the loading constraint in the former differs across the two. But, consider the factor var-cov matrix and notice factor vars are also different across models. For this particular matrix, the config specification appears nested within the weak spec.
In CFA context, Rensvold & Cheung (1998; 1999; 2001) discuss this as a "standardization problem" and argue for "Type-2 standardization" that constrains 1 invariant loading per factor so factor vars are free to vary in each model. A Type-2 standardized config model is not possible currently in M+ ESEM, but I'm wondering if this type of constraint might be equivalent (in terms of fit) to the default config where all factor vars equal 1.In CFA, it matters which "referent" items are selected for constraint, but perhaps this wouldn't matter in an ESEM context?
David Bard posted on Saturday, June 09, 2012 - 10:46 am
A similar identification and nesting issue occurs once ready to test intercept constraints b/c the factor means can now be freed up in the strong and strict FI models. Meredith & Horn (2001) describe possible solutions to this identification and nested constraints problem. Again I'm wondering how those constrained solutions relate to the type of nesting specified in M+ ESEM invariance models.
David Bard posted on Sunday, June 10, 2012 - 10:54 am
I realized I could actually run the unrotated config model proposed above using EFA within CFA. The code below does indeed reproduce the same fit as the default ESEM config model. The nesting of weak within config is more obvious (to me, at least) this way. I'm sure by respecifying factor means and intercept constraints the same could be done to make the nesting of strong and strict models within weak and config models more obvious.
MODEL: f1 by t1*1 (1) t2-t10*1 t12-t18*0 t20-t28*0;
EFried posted on Thursday, July 05, 2012 - 12:21 pm
"To address these questions, we applied our taxonomy of 13 ESEM models (see Table 1). The basic strategy is to apply the set of 13 models designed to test different levels of factorial and measurement invariance, …"
Text on p. 478 bottom left, table on p. 476 top left.
Bengt, I am talking about page five in the pdf you linked to, table 1.
"Marsh et al. (2009) introduced a taxonomy of 13 ESEM models (see Table 1) designed to measurement invariance […] . Importantly, ESEM allows applied FFA researchers to pursue appropriate tests of measurement invariance when CFA models are not appropriate."
They repeatedly state testing for measurement invariance in ESEM models. Maybe you could help me make sense of that.
Bottom of page 8, right column, last paragraph says:
"ESEM-within-CFA model (see the Supplemental Materials for further discussion). Despite the flexibility of the ESEM approach, we note that there are some aspects and extensions of traditional SEM models that cannot readily be implemented with ESEM as currently operationalized in Mplus"
This section describes what ESEM cannot do and what needs "ESEM-within-CFA".
The Mplus UG example 5.27 describes what "pure" ESEM can do in terms of invariance testing.
Bengt, the misunderstanding was that I was referring to the 2010 Marsh et al paper "A New Look at the Big Five Factor Structure Through Exploratory Structural Equation Modeling", which you co-authored.
We are very interested to understand how the measurement invariance tests (including factor loading invariance) for ESEM models were performed in that paper (there is no syntax in the User's Guide or the supplemental materials that shows how to do this).
What we don't understand is that it is stated above in this thread that MPLUS cannot do this. However, Marsh et al talk about exactly this at various points in the paper and the supplemental materials. Two examples:
"Weak factorial/measurement invariance tests the invariance of factor loadings over time […] it is not surprising that the CFI is marginally better for LIM1E (.912) than for LIM2E (.907; see Table 5)."
"With ESEM models it is possible to constrain the loadings to be equal across two or more sets of EFA blocks in which the different blocks represent multiple discrete groups or multiple occasions for the same group."
The problem with the UG example 5.27 is that - as far as we understand - the "grouping" procedure assumes independence between groups, which does not hold when testing measurement invariance over time (factors are correlated over time).
I try to replicate Marsh et al (2009) Model 13 (complete factorial invariance model)in SEM journal with Mplus 6.1. However, I got the following message. Can you figure out anything wrong?
*** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F1 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F2 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F3 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F4 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F5 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F6 *** ERROR The following MODEL statements are ignored: * Statements in Group G2: F1 F2 F3 F4 F5 F6
There is no discussion on complete factorial invariance model in that paper. I think you are referring to a 2012 paper. In any case, I do not have access to the input files that were used in that paper. Maybe you can contact Marsh directly.
PB posted on Tuesday, November 27, 2012 - 12:07 pm
referring to the post on May 14, 2011 - 9:15 am, I wanted to ask, whether in the new version of MPlus multilevel multiple group ESEM is available?
I have two questions about using multi-group ESEM. 1. Published examples use very large samples. IS there any literature on use in small samples (n=100-400)?
2. I find that when using the syntax suggested in the MPLUS manual and in the Marsh supplementary materials online, that my 2 groups show a similar pattern of factor loadings for a 4-factor measure. HOWEVER, the factors don't match across groups.. Factor 1 in group 1 matches factor 2 in group 2. Further attempts to use target loadings makes no difference in the output. I find the same pattern. This pattern doesn't match available MPLUS online examples, but the examples use only 2 factors. Follow-up use of target loadings per factor did not fix the issue. I am not clear what this discrepancy means and why it is happening.. i Any help would be appreciated.
Linda, I think I may have figured it out. I originally tested a model using separate loading and threshold tests. Since I'm using categorical data, I just tried to test difference by constraining both loadings and thresholds. I then got a difftest output.
I have a small sample of minorities, which may explain why I could get a gender test but not a race test.
Should we be testing weak and strong (load and threshold) together for the ESEM tests?
With binary items, the models to be tested are the unrestricted model and the model with free thresholds and loadings. Freeing the thresholds and loadings separately is not identified. For the details on how the models should be specified, see multiple group analysis in the Topic 2 course handout on the website and pages 485-486 of the user's guide/
I typically use the theta parameterization and Millsap and Tein's (2004) approach (for 3-point) scales. It calls for separate loading, threshold tests. For ESEM, I tried both delta and theta models. The theta tests ran well and allowed me to test the thresholds separately. Results are similar.