Message/Author 

Tony LI posted on Thursday, June 17, 2010  3:40 am



Hi Linda and Bengt, I am testing measurement invariance on a 14item instrument across 3 groups using multigroup ESEM framework (example 5.27). The Noinvariancemodel (Model 1) fits the data well (CFA=0.965; RMSEA: 0.042). Constraining factor loadings to be equal resulted in worse fit (CFA:0.915; RMSEA=0.066). For the next step I'd like to test partial matric invariance bying freeing some factor loadings (as suggested by modification indices). My question is how to do this properly in a ESEM framework? I tried to use the usual CFA sytax (Model 2: Model A) but this did not seem to work and I got an error message. Would you be able to shed some light on this ?Thanks in advance. !Model 1 no invariance MODEL: f1f3 by i1i14 (*1); [f1f3@0]; MOdel A: f1f3 by i1i14 (*1); [i1i14]; Model B: f1f3 by i1i14 (*1); [i1i14]; Model C: f1f3 by i1i14 (*1); [i1i14]; ! Model 2 partial loading invariance MODEL: f1f3 by i1i14 (*1); [f1f3@0]; MOdel A: f1f3 by i2 i3 i5 i10 i14 (*1); [i1i14]; Model B: [i1i14]; Model C: [i1i14]; *** FATAL ERROR IMPROPER PARAMETER CONSTRAINT FOR EFA MEASUREMENT SPECIFICATION. (Error Code: 1020) 


Partial measurement invariance is not allowed with ESEM. 

Tony LI posted on Friday, June 18, 2010  2:57 am



Hi Linda, Thanks for the reply. Are you mean partial metric invariance is not allowed? If so, are there any thoretical reason for this? Apparently I can run partial scalar (intecept) invariance test on my another dataset. Also it was used in Marsh et al's ESEM big five paper. Back to my case above, if metric invariance is not tenable. Where would you suggest to go? Back to a MGCFA framwork to test partial invariance? Thanks 


Partial intercept invariance is allowed in ESEM but partial loadings invariance is currently not allowed. This is because during the rotation a set of unequal parameters will result in a different set of unequal parameters (and the rotation is either the same in the groups or different). One possible approach is to take a particular indicator say i1, exclude it from the definition of the factors (f1f3 by i2i14) and use i1 on f1f3 as varying across the groups. This will however exclude i1 from the rotation mechanism. Another approach is to modify the model with group specific residual correlations such as say i1 with i3. 

Tony LI posted on Monday, June 21, 2010  2:42 pm



Hi Tihomir, Thanks for the helpful insight. Once further question regarding ESEM: is "missing by design" allowed? I wanted to do multiple group analysis to test for the invariance for a model(three groups took two versions of a test linked by common items). When I tried to run the following syntax, I got an error saying that the grouping/pattern variable had multiple uses. NAMES ARE country gender i1i10 a1 a2; USEVARIABLES ARE country i1i20 a1 a2; MISSING is ALL(999); GROUPING is country (0=DE 1=GB 2=SP); PATTERN is country(0=i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 1=i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 2=i1 i2 i3 i4 i5 i6 i7 i8 a1 a2); MODEL: !No invariance f1f3 by i1ia2 (*1); [f1f3@0]; MOdel GB: f1f3 by i1ia2(*1); [i1ia2]; MOdel SP: f1f3 by i1ia2(*1); [i1ia2]; OUTPUT: MOD; TECH1; I figured it was because the same variable was used both as a grouping and a pattern variable. So I tried two things: first I removed the pattern syntax, but I kept getting errors that said variables had no nonmissing values. I then only used the pettern command. This time Mplus appeared to be running but the OUT file didn't pop out. I opened the OUT file, there was no result apart from my sytax plus a sentence of "INPUT READING TERMINATED NORMALLY". I would greatly appreciate any suggestions 


Please send the files and your license number to support@statmodel.com. 

Tony LI posted on Monday, June 21, 2010  3:11 pm



Thanks Linda, i've just sent the files and license info. Looking forward to hearing from you. 


Hello. I would like to test measurement invariance for multiple groups and over time. ESEM seems best since items have nonzero loadings on nonprimary factors. However, data are also nested (child within school). Thanks to your reference lists, I see evidence of multilevel ESEM and multigroup ESEM. But, I have not seen multigroup, multilevel ESEM nor can I seem to find reference in the User Guide. All could be oversights on my part, but is it possible to do multilevel, multigroup ESEM? If so, can you point me in the right direction for syntax? Thank you! 


Mplus currently does not do multilevel ESEM. Multiplegroup ESEM is however available  see UG ex5.27 

Al Grimm posted on Wednesday, June 01, 2011  9:22 am



Referring back to Tihomir's note above that partial intercept invariance is allowed in ESEM but partial loadings invariance is not, I take it that partial factor mean invariance is also not an option? Or is there a way around the Mplus message "EFA factors in the same set as FACTOR must have all fixed or free means. Problem with: [ FACTOR ]". Cheers. 


Partial factor mean invariance is also not available but you can test for it using model test. 

Al Grimm posted on Thursday, June 02, 2011  5:26 am



Tried that by assigning parameter labels to the factor means (plus fixing them to zero under MODEL TEST), but Mplus does not appear to like that: "Parameters involving EFA factors cannot be constrained with equality labels or assigned a parameter label." If it is not permitted to assign parameter labels to factor means, can you still fix them under MODEL TEST? 


Al Model test doesn't fix or constrain anything  it just performs Wald test on the estimated parameters. But you are correct the current version will not allow the parameters that you are interested in Model Test. You can ask for the tech3 output, and using the estimated variance covariance matrix you can compute the Wald test by hand. Tihomir 

David Bard posted on Friday, June 08, 2012  5:57 am



I've been tinkering with ESEM invariance testing and have a question about Table 1 in the latest two Marsh papers posted in your ESEM special topic site. Those tables indicate that the ESEM invariance models are fully nested as you move from strict to strong to weak to configural invariance (models 1,2,5, and 7 in the table). I trust that this is true, but it's difficult to envision when the factor variances and means are constrained for group 2 (or nonreferent group) in the config model but one or both are not constrainted in the models nested within it. Are there starting unrotated solutions that make this progression of nested structure more obvious (much like the starting oblique rotated solution offered up in Asparouhov & Muthen's 2009 SEM paper, p. 420, helps to explain how CFA is nested within ESEM)? 


David  this is exactly correct all the invariance and noninvariance is traced back to the unrotated solution. First intercepts are not affected by the rotation (rotation affects the factor means but not the intercepts)  thus invariance or noninvariance for the intercepts is designed at the unrotated solution. The same applies to item uniquenesses. Factor variances– covariances invariance  to achieve that we estimate unrorated model with the identity matrix as the factor variance covariance. If you want noninvariance then only in the first group you have I and in the second etc it is free for the unrotated solution. When the loadings are invariant  the loadings are held equal for the unrotated solution and are rotated with the same rotation across group so you get the same rotated loadings. The latent factor means  if you want invariance they are held to 0 for the unrotated model. If you want them noninvariant they are held to 0 in group 1 and free in the other groups (same logic as for the variance covariance). Hope this helps. 

David Bard posted on Saturday, June 09, 2012  10:45 am



Maybe it would help if I just focus on the config and weak invariance models. In config model, we have 2 separate EFAs, so factor means and variances are constrained to 0 and 1, respectively, in both groups. This identification constraint forces factor means and variances to be constrained equal across groups. Everything else is unconstrained. In weak inv model, factor loadings are constrained equal, and by default, factor vars of nonreferent group are free to vary. If we ignore the factor varcov matrix, it's easy to see the weak inv model is nested within the config model b/c only the loading constraint in the former differs across the two. But, consider the factor varcov matrix and notice factor vars are also different across models. For this particular matrix, the config specification appears nested within the weak spec. In CFA context, Rensvold & Cheung (1998; 1999; 2001) discuss this as a "standardization problem" and argue for "Type2 standardization" that constrains 1 invariant loading per factor so factor vars are free to vary in each model. A Type2 standardized config model is not possible currently in M+ ESEM, but I'm wondering if this type of constraint might be equivalent (in terms of fit) to the default config where all factor vars equal 1.In CFA, it matters which "referent" items are selected for constraint, but perhaps this wouldn't matter in an ESEM context? 

David Bard posted on Saturday, June 09, 2012  10:46 am



A similar identification and nesting issue occurs once ready to test intercept constraints b/c the factor means can now be freed up in the strong and strict FI models. Meredith & Horn (2001) describe possible solutions to this identification and nested constraints problem. Again I'm wondering how those constrained solutions relate to the type of nesting specified in M+ ESEM invariance models. 

David Bard posted on Sunday, June 10, 2012  10:54 am



I realized I could actually run the unrotated config model proposed above using EFA within CFA. The code below does indeed reproduce the same fit as the default ESEM config model. The nesting of weak within config is more obvious (to me, at least) this way. I'm sure by respecifying factor means and intercept constraints the same could be done to make the nesting of strong and strict models within weak and config models more obvious. MODEL: f1 by t1*1 (1) t2t10*1 t12t18*0 t20t28*0; f2 by t11*1 (2) t2t10*0 t12*t18*1 t20t28*0; f3 by t19*1 (3) t2t10*0 t12t18*0 t20t28*1; [f1f3@0]; f1f3@1; [t1t28*]; t1t28*; MODEL GRP2: f1 by t1*1 (1) t2t10*1 t12t18*0 t20t28*0; f2 by t11*1 (2) t2t10*0 t12*t18*1 t20t28*0; f3 by t19*1 (3) t2t10*0 t12t18*0 t20t28*1; [f1f3@0]; f1f3*1; !can free up factor vars this way; [t1t28*]; t1t28*; 


It seems that you answered your own question and convinced yourself that weak nested within config is correctly tested in ESEM. 

EFried posted on Wednesday, July 04, 2012  9:38 am



It is stated in this thread (2010 and 2011) that partial measurement invariance is not allowed with ESEM. However, it appears the Marsh et al 2010 paper on the Big 5 does test exactly that. Am I mistaken? If not, what would be the parameterization for e.g. factor loading invariance? f1f2 x1 x2 x3 (*1) (13); f3f4 x4 x5 x6 (*2) (13); does not work. Thank you 


Which page of the article are you looking at? 

EFried posted on Thursday, July 05, 2012  12:21 pm



"To address these questions, we applied our taxonomy of 13 ESEM models (see Table 1). The basic strategy is to apply the set of 13 models designed to test different levels of factorial and measurement invariance, …" Text on p. 478 bottom left, table on p. 476 top left. The supplemental material of the paper (http://dx.doi.org/10.1037/a0019227.supp) does not include MPLUS syntax for these tests. Thank you 


I only have the version of the paper at our web site http://www.statmodel.com/download/Marsh%20Nagengast%20Morin%202012%20Big5%20Inv%20ESEM%20la%20dolce%20viita%20effect%20DevPsych.pdf which has different page numbers. But I suspect that they are talking about using EFA/ESEM in a CFA framework, so not regular ESEM. 

EFried posted on Sunday, July 08, 2012  7:54 am



Bengt, I am talking about page five in the pdf you linked to, table 1. "Marsh et al. (2009) introduced a taxonomy of 13 ESEM models (see Table 1) designed to measurement invariance […] . Importantly, ESEM allows applied FFA researchers to pursue appropriate tests of measurement invariance when CFA models are not appropriate." They repeatedly state testing for measurement invariance in ESEM models. Maybe you could help me make sense of that. 


Bottom of page 8, right column, last paragraph says: "ESEMwithinCFA model (see the Supplemental Materials for further discussion). Despite the flexibility of the ESEM approach, we note that there are some aspects and extensions of traditional SEM models that cannot readily be implemented with ESEM as currently operationalized in Mplus" This section describes what ESEM cannot do and what needs "ESEMwithinCFA". The Mplus UG example 5.27 describes what "pure" ESEM can do in terms of invariance testing. 

EFried posted on Sunday, July 08, 2012  1:36 pm



Bengt, the misunderstanding was that I was referring to the 2010 Marsh et al paper "A New Look at the Big Five Factor Structure Through Exploratory Structural Equation Modeling", which you coauthored. We are very interested to understand how the measurement invariance tests (including factor loading invariance) for ESEM models were performed in that paper (there is no syntax in the User's Guide or the supplemental materials that shows how to do this). What we don't understand is that it is stated above in this thread that MPLUS cannot do this. However, Marsh et al talk about exactly this at various points in the paper and the supplemental materials. Two examples: "Weak factorial/measurement invariance tests the invariance of factor loadings over time […] it is not surprising that the CFI is marginally better for LIM1E (.912) than for LIM2E (.907; see Table 5)." "With ESEM models it is possible to constrain the loadings to be equal across two or more sets of EFA blocks in which the different blocks represent multiple discrete groups or multiple occasions for the same group." The problem with the UG example 5.27 is that  as far as we understand  the "grouping" procedure assumes independence between groups, which does not hold when testing measurement invariance over time (factors are correlated over time). Thanks 


The longitudinal case is covered in UG ex 5.26. The "1" in parenthesis holds the factor loading matrix invariant across time. That 1 can be removed when you don't want loading invariance. 

EFried posted on Monday, July 09, 2012  11:58 am



Thank you Bengt!! 


Dear Dr. Muthen, I try to replicate Marsh et al (2009) Model 13 (complete factorial invariance model)in SEM journal with Mplus 6.1. However, I got the following message. Can you figure out anything wrong? Thanks. Mark *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F1 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F2 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F3 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F4 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F5 *** ERROR in MODEL command The variance of EFA factors in the reference block cannot be modified in the first group. Problem with: F6 *** ERROR The following MODEL statements are ignored: * Statements in Group G2: F1 F2 F3 F4 F5 F6 


There is no discussion on complete factorial invariance model in that paper. I think you are referring to a 2012 paper. In any case, I do not have access to the input files that were used in that paper. Maybe you can contact Marsh directly. 

PB posted on Tuesday, November 27, 2012  12:07 pm



Hi, referring to the post on May 14, 2011  9:15 am, I wanted to ask, whether in the new version of MPlus multilevel multiple group ESEM is available? Thanks in advance and best regards Pablo 


Multilevel ESEM is not available. ESEM is available with TYPE=GENERAL and TYPE=COMPLEX. 


Hello, I have two questions about using multigroup ESEM. 1. Published examples use very large samples. IS there any literature on use in small samples (n=100400)? 2. I find that when using the syntax suggested in the MPLUS manual and in the Marsh supplementary materials online, that my 2 groups show a similar pattern of factor loadings for a 4factor measure. HOWEVER, the factors don't match across groups.. Factor 1 in group 1 matches factor 2 in group 2. Further attempts to use target loadings makes no difference in the output. I find the same pattern. This pattern doesn't match available MPLUS online examples, but the examples use only 2 factors. Followup use of target loadings per factor did not fix the issue. I am not clear what this discrepancy means and why it is happening.. i Any help would be appreciated. Thank you. 


1. You may find some information in the following paper which is available on the website: Asparouhov, T. & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397438. Or you can do your own simulation study based on your data. See Example 12.5 in the Mplus User's Guide. 2. I don't think this matter in the configural case because nothing is being held equal. 


Thanks this is helpful. I have a related question. I am having some problems getting one of the nested models to provide me a diftest using WLSMV. I was able to do it by gender, but by race I get the suggestion to "decrease the convergence" criteria. I tried this and it did not work. But, I'm not sure which convergence is being referred to... Thanks, ML 


Please send the output and your license number to support@@statmodel.com. 


Linda, I think I may have figured it out. I originally tested a model using separate loading and threshold tests. Since I'm using categorical data, I just tried to test difference by constraining both loadings and thresholds. I then got a difftest output. I have a small sample of minorities, which may explain why I could get a gender test but not a race test. Should we be testing weak and strong (load and threshold) together for the ESEM tests? Thanks 


With binary items, the models to be tested are the unrestricted model and the model with free thresholds and loadings. Freeing the thresholds and loadings separately is not identified. For the details on how the models should be specified, see multiple group analysis in the Topic 2 course handout on the website and pages 485486 of the user's guide/ 


Thanks I typically use the theta parameterization and Millsap and Tein's (2004) approach (for 3point) scales. It calls for separate loading, threshold tests. For ESEM, I tried both delta and theta models. The theta tests ran well and allowed me to test the thresholds separately. Results are similar. Thanks again for your help. 


Yes, the Millsap and Tein's approach is useful for polytomous items. 

Back to top 