Measurement invariance PreviousNext
Mplus Discussion > Structural Equation Modeling >
Message/Author
 Marie Eisenkolb posted on Friday, February 03, 2012 - 1:07 am
I established a structural equation model for testing measurement invariance over two conditions in four groups and I tested by using the command grouping.
That leads me to bad model fits, but if I leave out this command, it fits better. Can you help me to explain these results?
Another question: I established these nested models by starting with configural invariance. To make mplus to test the configural model, I have to restrict the first factor loading to 1 and so I have to fix the first factor loadings in the weak, strong and strict invariance models, too,right? By keeping this restrictions, I achieve bad model fits.
Is there any chance to avoid the restiction of the first factor loadings? Here are my commands:
usevar = P_NEO_1 P_NEO_6 P_NEO_11 P_NEO_16 P_NEO_21 P_NEO_26
P_NEO_31 P_NEO_36 P_NEO_41 P_NEO_46 P_NEO_51 P_NEO_56 C_NEO_1 C_NEO_6 C_NEO_11 C_NEO_16 C_NEO_21 C_NEO_26
C_NEO_31 C_NEO_36 C_NEO_41 C_NEO_46 C_NEO_51 C_NEO_56 Reihe;
missing = all(99);
GROUPING IS Reihe (0=g1 1=g2 2=g3 3=g4);
MODEL: N_P BY P_NEO_1 P_NEO_6 P_NEO_11 P_NEO_16 P_NEO_21 P_NEO_26 P_NEO_31 P_NEO_36 P_NEO_41 P_NEO_46 P_NEO_51 P_NEO_56;
N_C BY C_NEO_1 C_NEO_6 C_NEO_11 C_NEO_16 C_NEO_21 C_NEO_26 C_NEO_31 C_NEO_36 C_NEO_41 C_NEO_46 C_NEO_51 C_NEO_56;
[N_P-N_C@0];N_P-N_C@1;
[P_NEO_1-P_NEO_56];[C_NEO_1-C_NEO_56];
N_P WITH N_C;
 Linda K. Muthen posted on Friday, February 03, 2012 - 8:57 am
When you use the GROUPING option, intercepts and factor loadings are held equal as the default. When you don't, the full sample is used and there are no equalities imposed.

You can set the metric by fixing the factor variance to one instead of the first factor loading to one:

f BY y1* y2 y3;
f@1;

See the Topic 1 course handout on the website under the topic Multiple Group Analysis. The inputs for measurement invariance are given there.
 Marie Eisenkolb posted on Friday, February 03, 2012 - 1:55 pm
Thank you very much for your answer. That helped me a lot.

So I have to test my groups against each other. Can you tell me, how to use only a part of the data within one variable? So that I can test within one variable the group of person 1 till 73 against the group of person 143 till 202?

I'd be happe for any advise.
 Linda K. Muthen posted on Friday, February 03, 2012 - 2:07 pm
Use the USEVARIABLES option to use part of the data.
 Marie Eisenkolb posted on Saturday, February 04, 2012 - 1:23 am
I did use the USEVARIABLES option, but all of my groups are in one variable and I need to test the model fit for example within only one group.
If I consider four variables (one for each group) instead of one, mplus says "FATAL ERROR", because the data matrix is too big (more variables than 350 variables).
 Linda K. Muthen posted on Saturday, February 04, 2012 - 6:59 am
Please send the outputs and your license number to support@statmodel.com.
 Marie Eisenkolb posted on Monday, February 13, 2012 - 11:12 am
For the FATAL ERROR I made a programming fault, but I found and corrected it. Thank you very much for your offering.

Now, to test the sequence effects, I need to override the default, that fixes the factor loadings and intercepts to be equal over the groups. How can I test a configural or weak Modell of measurement invariance?

The command * does only work for different conditions and having different variables loading on different factors, doesn't it?

Here are my commands:
GROUPING IS Reihe (1=t1 2=t2);
DEFINE:
IF (Reihe==0 OR Reihe==1) THEN Reihe=1;
IF (Reihe==2 OR Reihe==3) THEN Reihe=2;
MODEL:
N_P BY P_NEO_1* (a)
P_NEO_6 (b)
P_NEO_11 (c)
P_NEO_16 (d)
P_NEO_21 (e)
P_NEO_26 (f)
P_NEO_31 (g)
P_NEO_36 (h)
P_NEO_41 (i)
P_NEO_46 (j)
P_NEO_51 (k)
P_NEO_56;(l)
[P_NEO_1-P_NEO_56];
[N_P@0];
N_P@1;
 Linda K. Muthen posted on Tuesday, February 14, 2012 - 5:26 pm
See the Topic 1 course handout under multiple group analysis.
 Geneviève Taylor posted on Wednesday, March 28, 2012 - 1:04 pm
Dear Mplus team,

I am trying to test gender invariance in a path analysis model with continuous variables. I have looked at your Topics 1 handout but I am confused as to what I should specifiy exactly in my input file.
The only thing I changed to test whether the models are different for each gender in the GROUPING command. Here is the input I have so far:
...
VARIABLE:
MISSING ARE ALL (-999);
NAMES ARE.....
USEVAR ARE Sexe azagg azpop bzpop czpop dzpop aengcpt7 bengcpt7 cengcpt7 dengcpt7
eengcpt7 azaggami bzaggami czaggami dzaggami;

GROUPING IS Sexe (0 = filles 1 = garçons);
ANALYSIS:
ESTIMATOR = MLR;
MODEL:
dzaggami ON cengcpt7 czaggami czpop;
czaggami ON bengcpt7 bzaggami bzpop;
bzaggami ON aengcpt7 azaggami azpop;
dzpop ON czpop czaggami cengcpt7;
czpop ON bzpop bzaggami bengcpt7;
bzpop ON azpop azaggami aengcpt7;
eengcpt7 ON dengcpt7 dzaggami dzpop;
dengcpt7 ON cengcpt7 czaggami czpop;
cengcpt7 ON bengcpt7 bzaggami bzpop;
bengcpt7 ON aengcpt7 azaggami azpop;
azpop ON azagg;
azaggami ON azagg;
aengcpt7 ON azagg;

Is there anything else I should be adding to test this correctly?
Many thanks in advance for your help!
Genevieve Taylor
 Linda K. Muthen posted on Wednesday, March 28, 2012 - 1:32 pm
The GROUPING option should be used in all but the first step of testing for measurement invariance. The first step is to run the model separately for each group. The correct inputs are shown in the Topic 1 course handout under Multiple Group Analysis. Please refer to these inputs.
 Geneviève Taylor posted on Thursday, March 29, 2012 - 8:30 am
Hi Dr Muthen,

Thanks for your response. I understand the handout now. I will follow these steps for my analysis.
Many thanks,
Geneviève
 Yoonjeong Kang posted on Wednesday, August 28, 2013 - 2:31 pm
Dear Mplus team,

I am trying to understand a new approach to measurement invariance (approximate measurement invariance)implemented in Version of Mplus 7.11.

Q1. I ran a two-group CFA model for testing measurement invariance based on example 5.33. Under the DIFFERENCE OUT, I got average of estimate, standard deviation, deviations from the mean for each parameter and each group. I specified difference between two options like, N(0,0.01).

For example, I got
Average: 1.422
SD: 0.031
Deviation from the mean: -0.03 (Lamda1), 0.03(Lamda2)
--> How do I know whether the deviations from the mean in Lamda1 and 2 are significant or not?


Q2. Based on Muthen (2013) paper, it says that
" With only two groups/timepoints, the difference relative to the average can be augmented by the difference across the two groups/timepoints which can be expressed in MODEL CONSTRAINT.
If I want to test approximate measurement invariance between two group, what kinds of model constraint I need?

Thanks!!
 Bengt O. Muthen posted on Wednesday, August 28, 2013 - 6:24 pm
q1. There is an asterisk if the value is significant.

q2. Use parameter labels a and b in the MODEL command, where those parameters are the 2 parameters in question. Then use Model Constraint to do

New(diff);
diff = a-b;
 Yoonjeong Kang posted on Thursday, August 29, 2013 - 9:26 am
Dear Dr. Muthen,

Thanks a lot for your answer.

I have one more question. Can I test approximate measurement invariance in multilevel context? For example, can I conduct approximate measurement invariance test for between-level factor loadings? I have tried to do it by extending ex5.33 code but I couldn't. Please let me know.


Thanks a lot in advance.
 Bengt O. Muthen posted on Friday, August 30, 2013 - 2:41 pm
It is in principle possible but is quite complex given that the DO-DIFF options haven't been tailored to multilevel applications. I would not recommend trying.
 Elina Dale posted on Friday, February 07, 2014 - 12:15 pm
Dear Dr. Muthen,

I would like to test measurement invariance where my loadings are constant across groups, but thresholds are allowed to vary.

As per Ex. 5.16, since I am allowing thresholds to vary across groups, I fixed the scale factors to 1. I don't understand what is wrong with my input:

CATEGORICAL = i1-i9;
GROUPING IS g (1 = male 2 = female) ;
CLUSTER = clust;
MISSING = ALL (-9999) ;
Analysis: TYPE = COMPLEX ;
Model:
f1 BY i1 i2 i3 ;
f2 BY i4 i5 i6;
f3 BY i7 i8 i9 ;
Model female:
[i1$1 i2$1 i3$1 i4$1 i5$1 i6$1 i7$1 i8$1 i9$1 i1$2 i2$2 i3$2 i4$2 i5$2 i6$2 i7$2 i8$2 i9$2 i1$3 i2$3 i3$3 i4$3 i5$3 i6$3 i7$3 i8$3 i9$3];
{i1@1 i2@1 i3@1 i4@1 i5@1 i6@1 i7@1 i8@1 i9@1};

I keep getting an error message:
THE MODEL ESTIMATION TERMINATED NORMALLY
THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL. PROBLEM INVOLVING PARAMETER 112.THE CONDITION NUMBER IS -0.175D-16.

I have checked this parameter and it is Alpha for F1, which is an intercept I guess.

Thank you!!!
 Linda K. Muthen posted on Friday, February 07, 2014 - 12:25 pm
If you free the thresholds, you must fix the factor variances to zero.
 deana desa posted on Tuesday, March 18, 2014 - 6:56 am
I would like to know if factor scores computed from the alignment method and the convenient features (i.e., configural, metric or scalar) are (directly) comparable or related?

How much these scores are expected to be correlated?

Is there any literature out there that I can refer to for the scores computed from these different techniques?
 Linda K. Muthen posted on Tuesday, March 18, 2014 - 1:35 pm
No, the factor scores from alignment are not the same as those from configural, metric, or scalar. They start from a configural model and maximize measurement invariance. The correlation between the different factor scores would depend on the amount of measurement invariance.

I doubt there is any literature on this yet.
 Bilge Sanli posted on Friday, August 08, 2014 - 11:25 am
Drs. Muthen and Muthen,

Using the National Identity Module of the ISSP, I am adopting a two-level EFA approach in my exploratory research on different dimensions of nationhood, and their contextual and individual predictors. My cluster variable is countries, and my variables
are all at the ordinal level of measurement. In a subsequent two-level SEM analysis, (upon your suggestion in an earlier inquiry) I will use the factor scores I obtained from the initial two-level EFA analysis as dependent variables and regress them onto independent variables at both individual and contextual levels.
My question is the following: since I am engaging in a cross-national analysis, should I be establishing measurement invariance first? If I am to do this, is multiple group CFA the only option? In this scenario, how shall one take into account the multi-levelness of the data? Once I establish measurement invariance, shall I proceed with the two-level SEM?
Apologies for the deluge of questions. I'd greatly appreciate your help. Thank you very much in advance.
 Bengt O. Muthen posted on Friday, August 08, 2014 - 3:54 pm
These are good questions. I think you will be interested in reading the paper on our website (see Recent papers):

Muthén, B. & Asparouhov, T. (2013). New methods for the study of measurement invariance with many groups. Mplus scripts are available here.

This paper compares the fixed-effect multiple-group approach with the random-effect multilevel approach. It turns out that 2-level FA can be seen as a random intercept model, that is, measurement non-invariance that still makes factor comparisons possible.
 Kelly M Allred posted on Wednesday, December 17, 2014 - 10:43 am
I am struggling to conduct a analysis of measurement invariance in a 2-group CFA with categorical indicators each with three categories. I've included the code for the model in which factor loadings and thresholds are freed between the two groups:

GROUPING is SEX (1=male 0=female);
MODEL: FACTOR1 BY PFMS10 PFMS12 PFMS13 PFMS14 PFMS16 PFMS19 PFMS21 PFMS26;
FACTOR2 BY PFMS7 PFMS9 PFMS18 PFMS22 PFMS23 PFMS24 PFMS25 PFMS29
PFMS30 PFMS31 PFMS32 PFMS33;
[FACTOR1@0 FACTOR2@0];
MODEL female: FACTOR1 BY PFMS10 PFMS12 PFMS13 PFMS14 PFMS16 PFMS19 PFMS21 PFMS26;
FACTOR2 BY PFMS7 PFMS9 PFMS18 PFMS22 PFMS23 PFMS24 PFMS25 PFMS29
PFMS30 PFMS31 PFMS32 PFMS33;
[PFMS10$1 PFMS10$2 PFMS10$3
PFMS12$1 PFMS12$2 PFMS12$3
PFMS13$1 PFMS13$2 PFMS13$3
PFMS14$1...];
OUTPUT: STDYX MODINDICES;

When I conduct this model, I get the following message:

THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE
COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL.
PROBLEM INVOLVING THE FOLLOWING PARAMETER:
Parameter 82, Group FEMALE: FACTOR2 WITH FACTOR1

I'd appreciate any guidance on how to correctly identify the model!

Thank you!
 Linda K. Muthen posted on Wednesday, December 17, 2014 - 3:55 pm
In MODEL female do not mention the first factor indicator. When you do, the factor loading is not fixed at one and the model is not identified.
 Kelly M Allred posted on Thursday, December 18, 2014 - 7:27 am
Thanks, Dr. Muthen. I conducted the same model without mentioning the first factor indicators in the female model. See below:

GROUPING is SEX (1=male 0=female);
MODEL: FACTOR1 BY PFMS10 PFMS12 PFMS13 PFMS14 PFMS16 PFMS19 PFMS21 PFMS26;
FACTOR2 BY PFMS7 PFMS9 PFMS18 PFMS22 PFMS23 PFMS24 PFMS25 PFMS29
PFMS30 PFMS31 PFMS32 PFMS33;
[FACTOR1@0 FACTOR2@0];
MODEL female: FACTOR1 BY PFMS12 PFMS13 PFMS14 PFMS16 PFMS19 PFMS21 PFMS26;
FACTOR2 BY PFMS9 PFMS18 PFMS22 PFMS23 PFMS24 PFMS25 PFMS29
PFMS30 PFMS31 PFMS32 PFMS33;
[FACTOR1@0 FACTOR2@0];
[PFMS10$1 PFMS10$2 PFMS10$3
PFMS12$1 PFMS12$2 PFMS12$3
PFMS13$1 PFMS13$2 PFMS13$3
PFMS14$1 PFMS14$2 PFMS14$3
PFMS16$1 PFMS16$2 PFMS16$3
PFMS19$1 PFMS19$2 PFMS19$3
PFMS21$1 PFMS21$2 PFMS21$3];
OUTPUT: STDYX MODINDICES;

When I run this model, I get a different error message:

THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE
COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL.
PROBLEM INVOLVING THE FOLLOWING PARAMETER:
Parameter 162, Group MALE: { PFMS7 }

I'm struggling to figure out what is wrong with my output.

Thank you!
 Linda K. Muthen posted on Thursday, December 18, 2014 - 10:06 am
Scale factors must be fixed to one in all groups when the factor loadings are free across groups. See the Version 7.1 Language Addendum on the website with the user's guide under Multiple Group Analysis: Convenience Features where models for testing for measurement invariance are described.
 TA posted on Tuesday, May 12, 2015 - 8:52 am
Does Mplus have simple code to conduct measurement invariance like lavaan's R?

I used Millsap's measurement invariance Mplus code for categorical data here: http://www.myweb.ttu.edu/spornpra/catInvariance.html

What I noticed is the degrees of freedom are off between the R and Millsap's mplus code. This led me to wonder if there was a simple line in mplus to run a configural, weak, strong, strict models to avoid human error coding like in lavaan's R package.

Thanks!
 Linda K. Muthen posted on Tuesday, May 12, 2015 - 10:55 am
See the Version 7.1 Language Addendum on the website with the user's guide. The options for automatically testing for measurement invariance are shown there.
 Lucy Markson posted on Friday, August 07, 2015 - 4:17 am
I am testing the longitudinal measurement invariance of a 15 item measure from the child behavior checklist.
For the structural invariance model I have asked for the same factor items but factor loadings not constrained, the variances of scales fixed to 1, latent means fixed to 0 and no constraints on intercepts. For the weak model I have the same factor items and factor loadings, variances of scales fixed to 1, latent factor means fixed to 0 and no constraints on intercepts. For the strong model I have the same factor items and factor loadings, variances of scales fixed to 1, only the first latent factor mean fixed to 0 and the other means free to vary and intercepts set to be equal.
Have I put too many constraints on the structural and weak models? (ie with the variances and means?) Would it be possible to have some guidance on how the variances and means should be dealt with for the structural, weak and strong models?
Is it also possible to find out why you should allow residual correlations of corresponding items across time?
Many thanks
 Linda K. Muthen posted on Friday, August 07, 2015 - 6:27 am
We give detailed information about the models to test measurement invariance for various types of variables and estimators in the Version 7.1 Language Addendum on the website with the user's guide. They refer to multiple group models but the same constraints can be used across time for longitudinal measurement invariance.
 Jamie Vaske posted on Thursday, September 17, 2015 - 2:42 pm
Hi Linda & Bengt,
I conducted a measurement invariance test in my MPLUS 7.1 version and found configural & metric invariance when my items were coded as 1 = strongly agree, 2 = agree, 3 = disagree, and 4 = strongly disagree. I reverse coded the items so that 1=SD and 4=SA. Once I did this, I was not able to establish metric invariance. From Technical Appendix 11, I am guessing one reason why this might occur is because the thresholds move around and change in sign. From your experience, why might the results of measurement invariance change when items are reverse coded?
 Bengt O. Muthen posted on Thursday, September 17, 2015 - 6:02 pm
Sounds strange - like something isn't set up right. If you don't find it, please send the 2 outputs to support.
 Jamie Vaske posted on Friday, September 18, 2015 - 3:36 am
You are correct. Using the TECH1 outputs, I noticed that some of the thresholds were automatically constrained in the metric invariance model for one set of output but not the other (despite similarities in syntax). I'm following up with support. Thanks!
 Daniel Lee posted on Saturday, February 27, 2016 - 9:17 pm
Hi Dr. Muthen,

Is it possible for the model fit to improve from configural invariance to strong factorial invariance (tested for weak invariance as well, in between)? I generally find decrements in model fit as I impose more restrictions to the model, but, interestingly, the model fit has been incrementally improving from configural to strong factorial invariance for this scale. I'm wondering if you could tell me (1) if I'm doing something wrong (would be happy to send along data&input&output), or (2) in brief, what this improvement in model fit means conceptually.

Thank you, as always!
 Linda K. Muthen posted on Sunday, February 28, 2016 - 6:03 am
Which estimator are you using?
 Daniel Lee posted on Saturday, March 05, 2016 - 1:43 pm
Hi Dr. Muthen, I am using ML. Is that normal for ML?
 Daniel Lee posted on Saturday, March 05, 2016 - 1:51 pm
I'm sorry, I meant WLSMV! So again, in the multiple-group CFA (2 groups), I found it intriguing that the model fit improved from configural to strong invariance and was wondering what might be going on...normally, I would observe a decrement in model fit as I include more restrictions in the model.
 Linda K. Muthen posted on Saturday, March 05, 2016 - 3:06 pm
The chi-square values for WLSMV cannot be compared. Only the p-values can be comapred. This is why the DIFFTEST option must be used for difference testing of nested models.
 Daniel Lee posted on Sunday, March 06, 2016 - 1:28 pm
I understand! That makes perfect sense. I had one more question about testing invariance. I have been trying to establish configural invariance for a 2-factor model (grouping = gendeR) using WLSMV as an estimator, and the error message I get is:

The following MODEL statements are ignored:
* Statements in Group MALE:
[ D1 ]
[ D7 ]
[ D8 ]
[ D9 ]
[ D10 ]
[ D11 ]
[ D12 ]
[ D13 ]
[ D16 ]

So when I release equality restrictions on item intercepts for males, and when I estimate the model using WLSMV, I get the aforementioned error message. some insight about this error message would be greatly appreciated!!

Thank you, again!
 Linda K. Muthen posted on Sunday, March 06, 2016 - 2:47 pm
Please send the output and you license number to support@statmodel.com.
 Samantha Penney posted on Sunday, April 24, 2016 - 9:42 am
Hello,
I've trying to test the measurement invariance across gender however I get this error:
THE MODEL ESTIMATION TERMINATED NORMALLY
THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE
COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL.
PROBLEM INVOLVING PARAMETER 128.
THE CONDITION NUMBER IS -0.857D-07.
 Samantha Penney posted on Sunday, April 24, 2016 - 9:48 am
Syntax for measurement invariance

PhysFIW BY PG5@1 (L1) PG6* (L2) PG7* (L3) PG8* (L4);
EmotFIW BY EG5@1 (L5) EG6* (L6) EG7* (L7) EG8_b* (L8);
PhysWIF BY PG1@1 (L8) PG2* (L9) PG3* (L10) PG4* (L16);
EmotWIF BY EG1@1 (L11) (EG2* (L12) EG3* (L13) EG4* (L14);
[PG1*] (I1); [PG2*] (I2); [PG3*] (I3); [PG4*] (I4);[PG5*] (I5); [PG6*] (I6); [PG7*] (I7); [PG8*] (I8);
[EG1*] (I9); [EG2*] (I10); [EG3*] (I11); [EG5*] (I12);[EG5*] (I13); [EG6*] (I14); [EG7*] (I15); [EG8_b*] (I16);
Model Female:
PhysFIW BY PG5@1 PG5-PG8*;
EmotFIW BY EG5@1 EG6-EG8_b*;
PhysWIF BY PG1@1 PG2-PG4*;
EmotWIF BY EG1@1 EG2-EG4*;
Output: STANDARDIZED MODINDICES (ALL);
 Bengt O. Muthen posted on Sunday, April 24, 2016 - 10:48 am
You cannot have several labels on a line without separating them by semicolons.
 Ray Reichenberg posted on Monday, June 06, 2016 - 10:01 pm
Good evening. I am conducting measurement invariance testing for some IRT models (Samejima's Graded Response Model). I am using WLSMV estimation and, as such am using the DIFFTEST command to compute the Chi-square difference tests. The problem is that I have a very large sample (N = ~65,000) which yields a very liberal test (i.e., the null is almost always rejected). There are other approaches out there but I can't seem to implement them in Mplus. Todd Little (and others), for example recommend using a difference in CFI computed using the proper null model. I don't think the Chi-square value that results from using WLSMV can be used for these calculations. As evidence of this, the first test I conducted yielded a smaller Chi-square value for my metric invariance model (constrained loadings) than it did for my configural (free; less restricted) model. This, of course doesn't happen when using ML estimation. I tried to use ML estimation, even though it probably isn't appropriate for my ordinal (4 category) data, but you can't use numerical integration with multigroup models, evidently. Is there a correction I can apply to the WLSMV Chi-square that will allow for direct comparison between (nested) models? Is there another approach for conducting the invariance testing that might be more appropriate given my large sample? Your feedback is appreciated.
 Bengt O. Muthen posted on Tuesday, June 07, 2016 - 8:57 am
WLSMV produces CFI values so I don't see why you couldn't use CFI differences if that is what you like.

You can use ML with ordinal outcomes. Note that ML does not mean you have to have continuous-normal outcomes (that's a common misperception). You can use multiple-group analysis and numerical integration - you just have to do the multiple groups as Knownclass in a Type = Mixture run. Then you can use logL values to do get a chi-square using the loglikelihood ratio approach.
 Ray Reichenberg posted on Tuesday, June 07, 2016 - 9:36 am
Thank you for the response. I guess the part that is confusing to me is that, in my first set of models I am adding constraints and getting improved Chi-sq/CFI values when I should be seeing worse model fit. For example, I estimated a configural model (simple one factor with 10 items; 2 groups; N =~ 32,000 for each group) that yielded a Chi-sq ~19780 (70 df) and a CFI of .97. I then constrained the loadings to be equal across groups and got a Chi-sq ~12700 (79 df) and a CFI of .98. That suggested to me that that raw Chi-sq/CFI values resulting from WLSMV may not be comparable across models without some correction.
 Ray Reichenberg posted on Tuesday, June 07, 2016 - 10:28 am
Upon further review, this scenario where the Chi-sq, etc. improve when adding constraints to go from configural invariance to full metric invariance seems to be the case in every example I could find (in the realm of IRT with WLSMV, anyways). For the rest of the tests (e.g., going from full metric invariance to threshold invariance and so on) the fit indices behave in the usual way. Any idea why that would be?
 Linda K. Muthen posted on Tuesday, June 07, 2016 - 2:48 pm
With WLSMV, chi-square and related fit statistics like CFI cannot be compared. Difference testing can be carried out only using the DIFFTEST option.
 Ray Reichenberg posted on Tuesday, June 07, 2016 - 2:57 pm
Thank you. So, then. If I want to use the change in CFI approach as in Cheung & Rensvold (2002), Meade, et al. (2008), etc. I need to use ML estimation by utilizing the KNOWNCLASS command with TYPE=MIXTURE? Is it possible to work backwards from the Chi-sq statistic reported in the output when using WLSMV to get an uncorrected Chi-sq that I could then use to compute a CFI value that would be comparable across models? I apologize for all the questions but the LR test implemented using DIFFTEST seems rather unusable with large sample sizes such as those I am working with. The presence of any deviation in the parameters across groups yields a statistically significant difference test. Thank you again.
 Linda K. Muthen posted on Tuesday, June 07, 2016 - 4:12 pm
There is no way to work with the WLSMV chi-square other than the DIFFTEST option. You would need to use ML.
 Ray Reichenberg posted on Tuesday, June 07, 2016 - 5:47 pm
Thanks. Can you point me to any literature (a Tech Note, perhaps) that describes the mechanics underlying the DIFFTEST procedure?
 Ray Reichenberg posted on Tuesday, June 07, 2016 - 9:08 pm
Sorry for the second question but am I correct in saying that I cannot get CFI, TLI, or RMSEA when using the KNOWNCLASS/TYPE=MIXTURE approach with ML estimation? It appears that I can only get Chi-square. Can I compute CFI manually using the Chi-square/df values for the estimated and baseline models?
 Bengt O. Muthen posted on Wednesday, June 08, 2016 - 10:55 am
See Web Note 10 for Difftest documentation.

ML for mixtures requires raw data analysis which implies that a mean vector and a covariance matrix are not sufficient to summarized the modeling. Which in turn means that CFI etc are not relevant. Still, you can get chi-square to compare nested model by using a likelihood-ratio test.
 Ray Reichenberg posted on Wednesday, June 08, 2016 - 5:43 pm
Thanks for all of your help. One last question -- can you clarify a bit for me the new "convenience" commands for MG models in Mplus v7.3 and later? I'm currently using v7.11. With that version I cannot use ML estimation with MG specification (i.e., GROUPING=...) and categorical factor indicators due to Mplus not being able to implement numerical integration for these models. I've been told that I can use ML estimation with MG specifications in v7.3 an on by using the MODEL=... command with TYPE=complex under the ANALYSIS section of the syntax. Is that, in fact the case? If so, can I use any ML estimator (ML, MLM, MLR, MLMV)? Which fit indices are reported? Are these fit indices (CFI, TLI, and RMSEA in particular if available) comparable across models? I'm assuming they would be with ML, but perhaps not with an estimator that includes a Chi-square scaling correction. I'm very much interested in finding a robust method for conducting reasonably rigorous MI testing with categorical indicators and large samples without having to resort to using AIC/BIC, an "eyeball" test, or stratified sampling from my larger sample. Thanks in advance.
 Linda K. Muthen posted on Wednesday, June 08, 2016 - 5:50 pm
With maximum likelihood and categorical items, the KNONWCLASS option must be used instead of the GROUPING Option.

Chi-square and other fit statistics should only be compared using the same estimator.
 Ray Reichenberg posted on Wednesday, June 08, 2016 - 6:32 pm
So, let's say I had continuous indicators. In that case, a Chi-square-based GFI (e.g., CFI, TLI, RMSEA) from, say a configural MG model estimated using MLR could be compared to the same index resulting from a metric model (also using MLR) even if the S-B scaling factors are not equal?
 Linda K. Muthen posted on Wednesday, June 08, 2016 - 10:01 pm
You can use those two chi-square values along with the scaling correction factors to calculate a chi-square difference test.
 Ray Reichenberg posted on Wednesday, June 08, 2016 - 10:38 pm
Thank you. Can I make any comparison between GFIs that are computed using the Chi-square statistic? For example, if I am comparing two nested factor models using MLR and one yields a CFI of .97 (less constrained model) while the other yields a value of .95 (more constrained), is it appropriate to say that the former fits better than the latter at the global level (not necessarily in the statistically significant sense, of course)? If not, can I use the scaling factor to compute a fit index (CFI, for example) that is comparable across models? With ML the less constrained model will always fit better (per the Chi-square) than the more constrained model save for a few rare cases. I'm not sure if that's the case when using MLR, etc. due to the scaling correction.
 Linda K. Muthen posted on Thursday, June 09, 2016 - 6:13 am
Only a chi-square difference test can be used to say one model fits better than another in my opinion. You may want to ask this question on a general discussion forum like SEMNET to obtain the opinions of others.
 Ray Reichenberg posted on Friday, June 10, 2016 - 5:56 am
Thank you. Are the CFI, TLI, and/or RMSEA values on the same scale when using MLR estimation for hierarchically related models? I realize that I can't conduct any null hypothesis test of differences between these values from one model to the next. I know that they are on the same scale when using ML, but am not sure about whether this is true when using an estimator that requires a scale correction factor for model comparison. This question probably comes down to whether Mplus applies a scaling correction before calculating those indices. If not, then I can't imagine that they're comparable as reported in the output. Thank you.
 Linda K. Muthen posted on Friday, June 10, 2016 - 9:11 am
I do not believe you should compare CFI etc. when using MLR. These are based on the chi-square with uses the scaling correction factor.
 Hewa G posted on Friday, July 22, 2016 - 4:09 pm
Dear Dr. Muthen,
I have complex survey data with two-stage cluster sampling. I tried testing measurement invariance in a 2-group (TL=1 and CS=2) CFA with continuous latent factors. The sample size is unequal for the two groups. My Mplus syntax is as follows:
MISSING ARE y1-y20 (999);
grouping = OC (2 = TL 3 = CS);
Cluster Is MG;
ANALYSIS: Type=Complex;
Model = Configural metric scalar;
MODEL:
AD by y1-y4
AA by y6-y9………
OUTPUT: STANDARDIZED (STDYX);
How do I control for the two clusters? There is an error message in the output file “THIS IS MOST LIKELY DUE TO HAVING MORE PARAMETERS THAN THE NUMBER OF CLUSTERS MINUS THE NUMBER OF STRATA WITH MORE THAN ONE CLUSTER”. I tried Type= Twolevel complex command with both clusters but did not work. Any advice you can offer will be much appreciated.
 Bengt O. Muthen posted on Friday, July 22, 2016 - 4:35 pm
It sounds like you are saying you have 2 clusters. Type=Complex requires at least 20 clusters for good performance.
 Hewa G posted on Friday, July 22, 2016 - 5:08 pm
My sample consist of two types of clusters (Firms=30 and managers=104). Then data is at employee level with 624 employees.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: