Multiple group factor invariance PreviousNext
Mplus Discussion > Confirmatory Factor Analysis >
Message/Author
 Jennifer Rose posted on Friday, December 23, 2005 - 8:07 am
Hi,

I've conducted a multiple group analysis testing factor loading invariance across 2-3 groups using a 2 factor CFA with MLM estimation. The corrected chi-square difference test comparing this constrained model to an unconstrained model was nonsignificant, suggesting that the factor loadings are invariant across groups. My question is about reporting results. I would like to include a figure showing the standardized factor loadings and factor correlations. But, the standardized loadings differ across groups despite the fact that I imposed cross group equality constraints on the loadings (the parameter specification indicates that the constraints were correctly imposed and the estimates for the unstandardized loadings are the same across groups). I understand that the standardized loadings vary because of the variances used to calculate them, but I'm afraid that stating that I imposed equality constraints and then showing in the figure that the loadings are still different across groups will confuse readers, especially in the case where the standardized loadings are different by what would appear to be a nontrivial amount (e.g., difference of 0.07). I'm wondering if anyone has a suggestion for how to handle this apparent inconsistency.

Thanks,
Jen
 Linda K. Muthen posted on Friday, December 23, 2005 - 9:31 am
I prefer to work with raw coefficients and would definitley not report standardized coefficients for a multiple group analysis for the reasons you state. If you must report them, then I would include the explanation you have given. Maybe someone else has an opinion on this.
 kberon posted on Saturday, December 31, 2005 - 12:29 pm
I've also been interested in standardized coefficients across multiple groups. Lisrel has a feature that allows you to weight each group covariance matrix so that you end up having a common scale for all groups. This allows reporting a single "beta." I was wondering if Mplus had this facility? It sounds, from Linda's comment, that it doesn't but I'd like to confirm that. Thanks....Kurt
 Linda K. Muthen posted on Saturday, December 31, 2005 - 2:07 pm
No, Mplus does not have this facility.
 finnigan posted on Friday, March 23, 2007 - 9:27 am
Linda/Bengt

I have five factor model which I'm testing across two groups. I have a suspicion that the the five factors will not replicate across groups,i.e in one group 4 factors emerge and in the second group 5 factors may emerge. If this is the case am I correct in saying that within group comparisions can be made ,but between group comparisons on factor means cannot?

Or is it more appropriate to estimate one model for both groups and then test the factor structure? Any refs you may have would be appreciated.

Thanks
 Linda K. Muthen posted on Friday, March 23, 2007 - 2:47 pm
You should look at the factor structure in each group separately as a first step. If they don't have the same number of factors, then going on to look at them together is not appropriate unless four of the five factors are the same which would be fairly unusual I think.
 finnigan posted on Friday, March 23, 2007 - 3:34 pm
If the same number of factors is not present across groups, is it reasonable to carry out a within multi group CFA for each separate group and make within group comparisons on latent means once measurement invariance is present?

thanks
 Linda K. Muthen posted on Friday, March 23, 2007 - 6:09 pm
If you don't have the same number of factors in each group, you can look at the groups separately. I'm not sure what you mean by doing a multiple group CFA for each separate group since you would then be looking at a single group.
 Brian Hall posted on Monday, July 27, 2009 - 3:01 pm
Dear list,
I am testing a two-group CFA model testing for metric and configural invariance. I am extending this model to establish longitudinal invariance over three waves of data collection. I am testing several correlated models, and several hierarchical models.

Does anyone happen to have example programming syntax that can help with this model? I am not sure how to fix the paths to be equal in the case of metric invariance, and to compare the models in the case of configural invariance. Any assistance would be much appreciated.

Brian
 Linda K. Muthen posted on Monday, July 27, 2009 - 4:27 pm
See the Topic 1 course handout where all of the steps for testing for measurement invariance using multiple group analysis are given including inputs.

See the Topic 4 course handout where the first steps in the multiple indicator growth model show the steps for testing for longitudinal measurement invariance.
 Joe posted on Thursday, March 18, 2010 - 1:33 pm
Hi,
I have 3 latent factors, each BY 16 dichotomous observed items.

I would like to run an invariance analysis with the same factors across groups, but factor loadings, unique error variances, and item thresholds are freely estimated. I am getting an error that reads, "THE MODEL ESTIMATION TERMINATED NORMALLY THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE COMPUTED. THE MODEL MAY NOT BE IDENTIFIED."

Here is part of my syntax:

GROUPING IS disabil (0=RegEd, 1=SpEd);

ANALYSIS:
PARAMETERIZATION=THETA;

MODEL:
numop BY fp11@1 fp12 fp13 fp14 fp15 fp16 fp17 fp18 fp19 fp110 fp111 fp112 fp113 fp114 fp115 fp116;
geo BY fp21@1 fp22 fp23 fp24 fp25 fp26 fp27 fp28 fp29 fp210 fp211 fp212 fp213 fp214 fp215 fp216;
numopalg BY fp31@1 fp32 fp33 fp34 fp35 fp36 fp37 fp38 fp39 fp310 fp311 fp312 fp313 fp314 fp315 fp316;

MODEL SpEd:
numop BY fp11@1 fp12 fp13 fp14 fp15 fp16 fp17 fp18 fp19 fp110 fp111 fp112 fp113 fp114 fp115 fp116;
geo BY fp21@1 fp22 fp23 fp24 fp25 fp26 fp27 fp28 fp29 fp210 fp211 fp212 fp213 fp214 fp215 fp216;
numopalg BY fp31@1 fp32 fp33 fp34 fp35 fp36 fp37 fp38 fp39 fp310 fp311 fp312 fp313 fp314 fp315 fp316;

Can you tell how my model is misspecified?
 Linda K. Muthen posted on Thursday, March 18, 2010 - 1:59 pm
You should not be freeing the factor loading for the first indicator in the group-specific MODEL command. This causes the model not to be identified.

See pages 398-401 for the models to use for testing measurement invariance for categorical outcomes. See also the Topic 2 course handout.
 Joe posted on Thursday, March 18, 2010 - 3:00 pm
Thank you, Dr. Muthen. I thought the factor loading for the first indicator in the group-specific MODEL command was fixed with @1 (e.g., numop BY fp11@1). How do I then appropriately fix this factor loading in the syntax? Could you please give an example?

Thank you in advance for your time and assistance.
 Linda K. Muthen posted on Thursday, March 18, 2010 - 3:03 pm
I misread your input. Then I don't know what the problem is. Please send the full output and your license number to support@statmodel.com.
 andrea j wong posted on Thursday, June 17, 2010 - 5:42 am
hi,

i'm testing configural invariance for a 4 group, 2 latent factor, and 15 item (4 level likert scales) model. i'm getting the error: "THE MODEL MAY NOT BE IDENTIFIED." could you help me fix my input file? (the model stmts for the last 2 groups are the same as for month4.)

thanks!

MODEL:
dsclCncn BY S5Q1@1 S5Q2*0.5 S5Q3*0.5 S5Q5*0.5 S5Q7*0.5 S5Q9*0.5 S5Q14*0.5;
persStig BY S5Q4@1 S5Q6*0.5 S5Q8*0.5 S5Q10*0.5-S5Q13*0.5 S5Q15*0.5;
dsclCncn WITH persStig*0.5;
[S5Q2$2* S5Q3$2* S5Q5$2* S5Q7$2* S5Q9$2* S5Q14$2* S5Q1$3*
S5Q2$3* S5Q3$3* S5Q5$3* S5Q7$3* S5Q9$3* S5Q14$3*];
[S5Q6$2* S5Q8$2* S5Q10$2*-S5Q13$2* S5Q15$2* S5Q4$3* S5Q6$3*
S5Q8$3* S5Q10$3*-S5Q13$3* S5Q15$3*];

MODEL month4:
dsclCncn BY S5Q2* S5Q3* S5Q5* S5Q7* S5Q9* S5Q14*;
persStig BY S5Q6* S5Q8* S5Q10*-S5Q13* S5Q15*;
dsclCncn WITH persStig*0.5;
[S5Q2$2* S5Q3$2* S5Q5$2* S5Q7$2* S5Q9$2* S5Q14$2* S5Q1$3*
S5Q2$3* S5Q3$3* S5Q5$3* S5Q7$3* S5Q9$3* S5Q14$3*];
[S5Q6$2* S5Q8$2* S5Q10$2*-S5Q13$2* S5Q15$2* S5Q4$3* S5Q6$3*
S5Q8$3* S5Q10$3*-S5Q13$3* S5Q15$3*];
 Linda K. Muthen posted on Thursday, June 17, 2010 - 7:49 am
When you free the thresholds, factor means must be fixed to zero in all groups and, if you are using the default estimator WLSMV, scale factors must be fixed to one in all groups. See the models for testing for measurement invariance in Chapter 14 of the Version 6 user's guide after the multiple group discussion.
 andrea j wong posted on Thursday, June 17, 2010 - 10:09 am
thanks for your quick response!

i added the following model stmt for all the groups:

[dsclCncn@1 persStig@1];

and am getting a message about no convergence.

here's the analysis block:

TYPE = MGROUP;
PARAMETERIZATION = THETA;
ITERATIONS = 9999;
ESTIMATOR = WLS;

when i test for invariant loadings and/or thresholds, i am not having this problem. it's only occuring w/ the configural invariance test.

do you have any suggestions? any help would be appreciated!

andrea
 Linda K. Muthen posted on Thursday, June 17, 2010 - 10:26 am
The factor means should be fixed to zero not one. With the Theta parameterization, residual variances of the factor indicators should be fixed to one in all groups.
 andrea j wong posted on Friday, June 18, 2010 - 5:26 am
yea! that did the trick. :-)

i wanted to make sure i was doing the battery of tests correctly:

(1) baseline model, free thresholds (outlined above) - factor means set to zero in all groups, residual variances of factor indicators fixed to one in all groups

(2) invariant loadings, free thresholds - factor means set to zero in all groups, residual variances of factor indicators fixed to one in first group and free in other groups

(3) invariant loadings and thresholds - factor means set to zero in first group and free in other groups, residual variances of factor indicators fixed to one in the first group and free in other groups

(4) invariant loadings, thresholds, and uniqueness - factor means set to zero in first group and free in other groups, residual variances of factor indicators fixed to one in all groups.

thanks again for all your help!
andrea
 Linda K. Muthen posted on Friday, June 18, 2010 - 9:50 am
We recommend models 1, 3, and 4 for categorical outcomes.
 andrea j wong posted on Friday, June 18, 2010 - 12:48 pm
hi,

i'm working with a different set of variables now; however, everything is set up as above.

there are 4 groups: month0, month4, month8, month12

i am getting the error message: THE WEIGHT MATRIX FOR GROUP MONTH12 IS NOT POSITIVE DEFINITE

can you explain what this error message means and how i might go about fixing it? also, why does it happen for one group and not all groups?

i don't know if this matters, but the data is very skewed for all groups.

i rescaled an 11 pt likert scale into a 3 pt likert scale where:

0-3 -> 1
4-6 -> 2
7-10 -> 3

should i set the scale differently so that the data is not so skewed?

thanks again for your guidance!
andrea
 Linda K. Muthen posted on Friday, June 18, 2010 - 2:27 pm
Please send the full output and your license number to support@statmodel.com.
 Hans Leto posted on Tuesday, April 10, 2012 - 10:33 am
Hello,

I am performing a single group analysis with the same syntax of slide 209 of Mplus' handout no. 1:

USEOBSERVATIONS ARE (gender EQ 1); !change 1 to !0 for females

MODEL:
f1 BY y1-y5;
f2 BY y6-y10;

But is gives me an error "Variable is uncorrelated with all other variables: gender. All least one variable is uncorrelated with all other variables in the model. Check that this is what is intended." (The variable is gender, variance 0).

could you help me? Thank you in advance
 Linda K. Muthen posted on Tuesday, April 10, 2012 - 1:45 pm
You must have gender on the USEVARIABLES list. This is only for variables used in the analysis. Remove it.
 Kimberly Henderson posted on Friday, June 08, 2012 - 11:50 am
I am testing a measurement model at two waves (i.e., wave 3 and wave 4). The model is the same at each wave. Since, both measurement models will eventually be added to a two-time point longitudinal SEM, I was advised to test for measurement invariance between the measurement models of the two waves.

My estimator is WLSMV and I have categorical and continuous variables. Can I simply use the DiffTest option to do a multi-group comparison or is there another test that is more appropriate?

Thanks!!
 Linda K. Muthen posted on Friday, June 08, 2012 - 12:07 pm
No, the groups would not be independent. You would test the measurement invariance in a single-group analysis. The multiple indicator growth model example in the Topic 4 course handout shows how to do this for continuous variables. You would take the same approach for categorical variables but use the steps shown in the Topic 2 course handout under multiple group analysis.
 Kimberly Henderson posted on Friday, June 08, 2012 - 12:12 pm
Okay. Thanks for the reply. I will try this out.
 Kimberly Henderson posted on Monday, June 11, 2012 - 10:02 am
Do you know what heading or page that I would need to refer to in Topic 2 and Topic 4 courses?
 Linda K. Muthen posted on Monday, June 11, 2012 - 11:07 am
See the table of contents. For Topic 4, look for multiple indicator growth. For Topic 2, look for multiple group analysis.
 Kimberly Henderson posted on Monday, June 11, 2012 - 1:02 pm
I see. Thanks!
 Ebrahim Hamedi posted on Thursday, September 13, 2012 - 8:10 pm
Hi
I am doing a MGCFA, 3 groups. When testing each group separately, everything is fine and models are identified. when multi-group modeling however, I get this message:

THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE
COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL.
PROBLEM INVOLVING PARAMETER 61.

parameter 61 is for alpha (I guess something related to intercept) of item 14 in group 2. As I told you when doing the analysis separately in group 2, the model converges.

It is noteworthy that the MGCFA converges when using Amos on the same data. Sounds like this is something specific to Mplus way of calculating estimates.

In the MGCFA, the unstandardized coefficient for item 14 in group 2 is 1.292, while other itemsí coefficients are smaller. This is the only aspect of this item different from others. Interestingly, when I omit this item in the model, still it does not converge and says something is wrong with item 13. Can you please help me resolve this?

Many thanks,
ebi
 Linda K. Muthen posted on Friday, September 14, 2012 - 6:01 am
Please send the output and your license number to support@statmodel.com.
 Ebrahim Hamedi posted on Friday, September 14, 2012 - 4:20 pm
Hi
Thanks for your message. I think I found the problem. while in Amos, especifying factor means to be 0 leads to unidentification in a multigroup analyis, in mplus things are different. I had not specified the factor mean to be zero in my multigroup analysis in mplus, which resulted in an intercept issue in one of the groups. setting it to be zero resolved the issue.
 Ebrahim Hamedi posted on Thursday, November 01, 2012 - 8:17 pm
Hi
I have a question and I would appreciate if you could answer. in testing for scalar invariance, when we find that an intercept is not invariant, in order to follow up this finding it is attractive to compare the intercepts in various groups. In doing so, should we compare the unstandardization intercepts or the standardized (Est./S.E.) ones?

the results are sometimes contradictive. for example, based on the unstandardization estimates group 1 has the highest score but based on the standardized estimates, group 4 has the highest score. what do you suggest?

the same question can be asked about comparing the intercepts in a number of groups, estimating separate models for each (not multigroup CFA). which one should be used for comparing scale origins across groups: the unstandardized or starndardized estimates?

a marginal question is whether or not we could only examine observed item means in separate samples to follow up a noninvariant intercept.


Thank you very much in advance,
Ebi
 Bengt O. Muthen posted on Thursday, November 01, 2012 - 9:22 pm
I am not sure I understand the situation, but it seems that you should use the unstandardized intercepts since you are doing an unstandardized analysis when you compare groups.
 Ebrahim Hamedi posted on Thursday, November 01, 2012 - 10:03 pm
Thank you very much. Because this question is so critical for me at this point, I decided to give an example.

I have three groups. One item intercept is non-invariant. Below the unstandardized intercepts, standardized intercepts, and simple observed item means for each group are reported respectively:

g1: 4.354- 72.197- 4.35
g2: 4.738- 73.280- 4.74
g3: 4.492- 61.631- 4.01

the unstandardized and standardized intercepts are from the multigroup analysis for testing scalar invariance.

Now, I would like to report actual intercept differences among these three groups for that item to understand the results better.

As you can see it is so tricky. If I use the unstandardized intercepts, I should conclude that group 1 had the lowest scale origin (4.354), and the main difference is that group 2 scored remarkably differently from the other two groups.

Alternatively, if I use the standardized intercepts, I should conclude that group 3 had the lowest scale origin (61.631), and the noninvariance is because this group scored remarkably differently from the other two groups.

So the question is: in this multi group analysis, should I use standardized or unstandardized intercepts for comparison.


Thank you very much ,
Ebi
 Bengt O. Muthen posted on Sunday, November 04, 2012 - 11:09 am
You should not use standardized values such as intercepts when you compare groups. This is because standardization confounds the parameters of interest with group-varying variances.

Note also the the item mean is a function of the item intercept, the factor loading and the factor mean.
 Ebrahim Hamedi posted on Sunday, November 04, 2012 - 1:10 pm
Thank you very much professor Muthen. really helpful.
 H Steen posted on Tuesday, June 04, 2013 - 5:12 am
Hi,

I have a question about interpreting differences in model fit between groups.

Before doing a multigroup second order CFA, I compared the models in the groups separately. And the results are so different that I have no reason to investigate any type of invariance any further. However the results are a bit surprising and I would like your comments.
Comparing low, medium and high educated groups results in a mediocre model fit for the low educated (RMSEA 0.075) and reasonable fit in de high educated group (0.041). The groups are about the same size, N varies between 302 and 330.
The thing is, the MI's are very similar; in all groups 10 is the highest, and almost all are (much) lower. Furthermore, the factor loadings are much higher in de low educated group, which seems contradictory, as the model fit is worse. The low educated group have much smaller variance in the items in the CFA though, and I would like to whether this can explain the combination of worse fit and higher loadings. With small variance, there is less to model anyway.

Other types of Analysis (eg Mokken) which are not to be preferred though for my research, show best structure fit for the low educated.

Shortly stated, can it be the case that the fit is indeed best in de low educated group as the factor loadings are highest?

Thank you very much for any comments.
 Bengt O. Muthen posted on Tuesday, June 04, 2013 - 8:47 am
Fit assessments are affected by the size of correlations among the observed variables - higher correlations give higher power to reject, ceteris paribus. hose correlation sizes may vary across your groups.
 H Steen posted on Thursday, June 06, 2013 - 2:10 am
Thank you for your response. You are right, in the lower educated group (which shows worst fit) the correlations are much higher.

But what that this imply? That the fit indices are not meaningful?

Is there a way to correct fit indices for the level of interrelatedness of items?
 Bengt O. Muthen posted on Thursday, June 06, 2013 - 7:47 am
You should take a look at for instance Saris, Satorra, & Veld (2009) in the SEM journal about the weaknesses of fit indices.
 H Steen posted on Thursday, June 06, 2013 - 8:42 am
Thank you very much!
 Sabrina Thornton posted on Wednesday, October 09, 2013 - 9:17 am
Hi I managed to fit the model in each group, but it doesn't seem to fit in a multogroup CFA. I am performing a multi group CFA with a four factor solution across two groups. I followed the handout to the letter, but I get this message:
-----------------------------------
MAXIMUM LOG-LIKELIHOOD VALUE FOR THE UNRESTRICTED (H1) MODEL IS -7520.177


THE MODEL ESTIMATION TERMINATED NORMALLY

THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL.

PROBLEM INVOLVING PARAMETER 110.

THE CONDITION NUMBER IS -0.318D-06.
--------------------------------------

The syntax is as follows:


USEVARIABLES ARE A1 A2 A3 A5 B1 B2 B3 B4 C5 C6 C7 C9 D3 D4 D5 D7;
GROUPING IS Group (1=GROUP1 2=GROUP2);
MISSING are all (666);

MODEL:
IA BY A1-A5;
OE by B1-B4;
SRM by C5-C9;
WRM by D3-D7;
[IA@0 OE@0 SRM@0 WRM@0];

MODEL GROUP2:
IA BY A1-A5;
OE by B1-B4;
SRM by C5-C9;
WRM by D3-D7;
[A1-A5 B1-B4 C5-C9 D3-D7];

OUTPUT:STANDARDIZED MODINDICES SampStat Residual;
 Linda K. Muthen posted on Wednesday, October 09, 2013 - 10:06 am
You should not mention the first factor indicators in the group-specific MODEL command. When you do this, they are no longer fixed at zero and the model is therefore not identified,
 Sabrina Thornton posted on Wednesday, October 09, 2013 - 12:53 pm
That has just made my evening! Thanks, Linda.
 Liting Cai posted on Tuesday, January 14, 2014 - 1:56 am
Hi,

In an earlier post, as well as in the handout for Topic 2, it was mentioned that to conduct multiple group analysis on categorical variables, the steps are slightly different from that if the variables are continuous. The difference is that for categorical variables, we do not conduct the following steps:

(1) Fix the factor loadings across groups and free the thresholds, and

(2) Fix the thresholds across groups and free the factor loadings.

May I find out, what should I do if I have both continuous and categorical variables in my multiple group factor analysis? Do I go with the steps for categorical variables?
 Bengt O. Muthen posted on Wednesday, January 15, 2014 - 10:50 am
You can go with the steps for the categorical variables for all the variables, but you can also do all steps for the continuous variables.
 Liting Cai posted on Thursday, January 16, 2014 - 6:57 pm
Dear Dr Muthen,

Thank you for taking time to address my query!

May I understand the rationale for the different steps (for multiple group analysis) for continuous and categorical variables? Is there a paper that provides this rationale, that you could direct me to?
 Linda K. Muthen posted on Friday, January 17, 2014 - 8:53 am
See the Millsap 2011 book.
 Liting Cai posted on Friday, January 31, 2014 - 1:48 am
Thanks! Will check out the book.
 Kerry Zelazny posted on Tuesday, February 18, 2014 - 6:52 am
Hello,

I am trying to conduct multiple group invariance testing using categorical indicators for continuous latent factors. When I try to release the factor loadings and the thresholds for my second group, I get the following error:

THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL. PROBLEM INVOLVING PARAMETER 42.

The TECH output tells me Param 42 is the second diagonal entry in the THETA matrix for my second group. I'm not sure what I'm doing wrong:

VARIABLE: NAMES ARE DUMMY PTSD6-PTSD25;
USEVARIABLES PTSD6-PTSD25;
CATEGORICAL ARE PTSD6-PTSD25;
GROUPING is DUMMY (0 = NOTRAUMA 1 = TRAUMA);
ANALYSIS:
ESTIMATOR = WLSMV;
PARAMETERIZATION = THETA;
MODEL:
F1 BY PTSD6-PTSD13@1;
F2 BY PTSD14-PTSD25@1;

F1;
F2;

F1 with F2;

PTSD6-PTSD25;

[PTSD6$1-PTSD25$1@1];
MODEL TRAUMA:
F1 BY PTSD7-PTSD13;
F2 BY PTSD15-PTSD25;

[PTSD6$1-PTSD25$1];

Any ideas what I am doing wrong? Help is very much appreciated!
 Linda K. Muthen posted on Tuesday, February 18, 2014 - 8:20 am
See the Version 7.1 Mplus Language Addendum on the website. You will find described in this document a way to do invariance testing automatically and a full description of the models for testing for measurement invariance in various situations.

When factor loadings and thresholds are free across groups, residual variances should be one in all groups and factor means should be zero in all groups.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: