Message/Author 


Hello everyone, I recently saw a presentation on what was, essentially, multilevel measurement invariance. The authors suggested that strict equality constraines could not be imposed across levels, as in regular invariance tests. Instead, the authors suggested a type of proportional invariance, where item 1's loading could be any given proportion of item 2's loading, but this proportion should hold across levels. OR, they suggested one could propose that the betweengroups loading for item 1 be an observed proportion of the withingroups loading of item 1, but that this proportion hold for all items. Clearly, this is a more complex way to set equality constraints than are usually allowed in Mplus. However, would constraints such as these be possible? Could one, for example, state that a factor loading or a structural relationship be proportionally equivalent across levels? If so, I think this could be a new way to get the word out about Mplus: It allows tests of multilevel measurement and structural invariance! Cheers, Mike 

bmuthen posted on Saturday, August 13, 2005  2:47 pm



I haven't seen this kind of argument, but maybe it arises when letting all loadings be free and held equal across levels  setting the metric of the factor by having the factor variance fixed at one. Because the within and between variances are different this would call for proportionality constraints rather than equality constraints. I would instead set the metric by a factor loading fixed at one and then holding the remaining loadings equal across levels, letting the within and between factor variances be free. But perhaps I am overlooking something. To answer your question about proportionality constraints, I think this can be done in Mplus using Model Constraint. For example, in abbreviated form, %Within% fw by y2 (p2w) y3 (p3w); %Between% fb by y2 (p2b) y3 (p3b); Model constraint: p2b = c*p2w; p3b = c*p3w; Here, c is a dummy parameter which is the proportion. c can be estimated by giving it a label connected with a dummy parameter. Such dummy parameter handling can be done in the current version, but will be easier in Version 4. I can tell you more about it if my response is on the right track. 


Hi Bengt, This is exactly the kind of modeling constraint I am imagining. Please do tell me more about how to estimate c by giving it a label connected with a dummy parameter. Also, regarding your initial sentence and example constraints, do you think this kind of model would only work when allowing loadings to be free but set to proportional invariance across levels? Could it not also be used by allowing the items to interrelate in a given way at one level and having the same interrelation at another level? For example: %Within% fw by y2 (p2w) y3 (p3w); %Between% fb by y2 (p2b) y3 (p3b); Model constraint: p2w = c*p3w; p2b = c*p3b; Here, c would specify a proportion but do so with a different meaning than in your example. Do you see any problems with doing one vs. the other? They seem to address different questions. Mplus is great, I can't wait for version 4! 

bmuthen posted on Monday, August 15, 2005  9:35 am



Proportionality for any kind of parameters would be possible in this approach. The dummy parameter specification in the current Mplus is done by labeling a parameter that would not in itself be identified, for example, fdum by y2 (c); fdum@0; [fdum@0]; such that this loading has no impact on the model or its likelihood. Note that fdum should also have zero correlation with other factors. Then, "c" in the Model Constraint paragraph has a place in the parameter array  and is identified since the Model Constraint statements place constraints on the parameters that c has to try to match. 


Any thoughts on this? In line with your recomendations, I entered the following model and it says, "*** ERROR Unknown parameter label in MODEL CONSTRAINT: C" TITLE: Multilevel Structural Invariance DATA: FILE IS Data.dat; VARIABLE: NAMES ARE BRANCH TW SC CS YEAR GROUP; USEVARIABLES ARE TW SC CS BRANCH; MISSING ARE ALL (999); CLUSTER IS BRANCH; CENTERING IS GRANDMEAN (TW SC CS); ANALYSIS: TYPE IS TWOLEVEL MISSING; ESTIMATOR = ML; MODEL: %BETWEEN% CS ON TW (B1); CS ON SC (B2); cs@1.0; tw@1.0; sc@1.0; %WITHIN% CS ON TW (W1); CS ON SC (W2); cs@1.0; tw@1.0; sc@1.0; DUMMY by CS (C); DUMMY@0; [DUMMY@0]; DUMMY WITH CS@0; DUMMY WITH TW@0; DUMMY WITH SC@0; MODEL CONSTRAINT: B1 = C*W1; B2 = C*W2; OUTPUT: SAMPSTAT RESIDUAL MODINDICES CINTERVAL STANDARDIZED; 


Nevermind, sorry, all that was required (of course) was to free the parameter by saying: DUMMY by CS* (C); 


Again, sorry to bother, but one real question is: If a parameter "c" is freely estimated, do you think that the reduction in the degrees of freedom for the model (1) because of estimating the parameter is justified? In other words, because the parameter doesn't influence the model, for difference testing could df be upwardly adjusted by +1 to pretend that no parameter was estimated? 

bmuthen posted on Sunday, August 21, 2005  9:15 am



The model with C appearing in Model Constraint expresses the 2 B1, B2 slopes in terms of just 1 parameter, C (and other already existing parameters). Therefore this model has 1 fewer (unrestricted) parameters and should have 1 more degree of freedom. Difference testing of this C constraint should have 1 df. 

MarcReis posted on Tuesday, January 03, 2006  7:48 am



Hello Dr. Muthén, regarding the test of measurement invariance across levels as you described above (setting the metric by fixing the loading of item 1 to 1.0 and constraining the others to be equal across levels): Is this kind of test still appropriate, if (a) it´s necessary to fix some nonsignificant betweenresiduals to zero and/or (b) allowing correlated residuals on the within but not on the betweenlevel? In this case, the models wouldn´t be exactly the same on both levels. I would appreciate a comment on this, thanks! 


It is usually the case that a different number of factors are found at the within and between levels of a multilevel model. And even when there are, the factors most likely have a different meaning for the two levels. The suggestion that I see from the previous postings is that holding the factor loadings equal across levels is most likely not appropriate. 

MarcReis posted on Wednesday, January 04, 2006  2:19 am



Many thanks for the answer, but I'am unsure whether my question was not clear enough. It`s not that I want to impose an equality constraint for the factor loadings across levels a priori. As a result of a MFA, I found that a model with the same factor structure between and within fits the data very well (however, I had to allow some correlated residuals within and fix some nonsignificant residuals between to zero). Additionally, judging by the MLRchi²difference test as described on the homepage, there is no significant worsening in the model fit, when I impose the equality constraint on the factor loadings across levels. As far as I understand, this is mentioned in the Metha & Neale  article (2005). So, does your previous answer fits to this situation? 


I think correlated residuals on within probably represent a hidden factor. I suspect that you have not actually done an EFA on the pooledwithin and Sigma between matrices as a first step to see how many factors there are on each level. If you have not, I would start there. I think it would be hard to support that the factors are the same when you have to add residual covariances on the within level. 

MarcReis posted on Wednesday, January 04, 2006  12:15 pm



Actually I did initial EFA and the accordingly specified CFA/MFA fits well in terms of CFI, RMSEA, etc., but there are some high MI. I will see whether the residuals are really necessary, thanks a lot for your time! 


Hi, I’d like to investigate measurement invariance in a multilevel context using multiple group models. I have factors on the within and between level; some of them have continuous and some have categorical factor indicators. For continuous indicators, I adopted a suggestion by Thompson and Green (2006) to test for both invariance of factor loadings and intercepts. Invariance of factor loadings is tested first by analyzing the covariance structure only, not the mean structure. Having explored invariance of loadings, invariance of intercepts is tested next. For step 1 (covariance structure only) I decided to extract the estimated sigma within and sigma between matrices for the groups to be compared and to analyze within and between separately. 1a) Is it correct that intercepts of the outcomes are only of interest on the between level and analyzing a mean structure to test invariance of intercepts on the within level does not make sense? 1b) If yes, is it sufficient to check for invariance of the loadings on the within level? 2) Since means of factor indicators are given on the between level, would it be correct to first analyze the between covariance matrix only (using the estimated sigma between), then "add" the means (given in the output for a type=twolevel analysis) to the covariance matrix used as input and then test for intercept invariance? 


Invariance for categorical outcomes causes even more trouble to me. As far as I know, threshold invariance has to be analyzed along with invariance of loadings. Within level: 3) Is it possible to use an extracted correlation matrix to check for measurement invariance  maybe in the same way this is done for continuous outcomes? Between level: As far as I know, a between matrix cannot be extracted for categorical outcomes. So I thought about using type=twolevel. 4a) Is this starting setup correct:  defining the factor on the between level while having no variables on the within level if the factor indicators are measured on the between level  defining the factor on the between level while allowing just for indicator correlations on the within level if the indicators are measured on the within level Next step would be to do a comparison of models a) with thresholds and loadings fixed between groups and factor means freely estimated with the exception of the mean in one group fixed to zero and b) with thresholds and loadings freely estimated and the means in all groups fixed to zero. 4b) Is this specification correct? 4c) If yes, how would I compare these models? I'm not sure if they are nested. Besides a direct comparison: What differences in values for thresholds and (standardized) loadings would be troubling? Thank you very much for your advice! 


The answer to all of your questions is yes with the following additional information. For categorical outcomes, within and between matrices are not available at this time. You can create a data set where each observation is a cluster and analyze that. The models you mention are nested. 


1) If each observation is a cluster, this is (in case of a binary outcome) a multilevel logistic regression where probability of endorsement is a function of loading and threshold on lvl1 and of factor score on lvl2? 2a) If yes, invariance for within structure could be tested by a crosslvl interaction of group membership (dummy vars on lvl2) on the lvl1 regression coefficients with lvl 2 being persons? 2b) For invariance of between structure, it would be analogous with the units of between structure (in my case school classes) on lvl2? Thank you. 


I think Linda's answer referred to your 4a) statement:  defining the factor on the between level while having no variables on the within level if the factor indicators are measured on the between level. That is, you will then have a singlelevel factor model for your between variables, where the clusters is your sample. 

Nikolai Eton posted on Thursday, September 04, 2008  5:35 am



Greetings, I am relatively new to multilevel modeling and would appreciate it a lot if you can give me a reference on testing measuring and factor invariance across levels. Furthmore, how can this be applied in Mplus? By using the "model constraints" command? Many thanks in advance. cheers, Nikolai 


This can be done using equality constraints. These are demonstrated for a multilevel multiple indicator growth model in Example 9.15. The principles are the same for a factor analysis model. 


Drs. Muthen, I believe I am getting the same error message as user Michael J. Zyphur above. I used proportionality constraints (p2d, p3d, and p4d; and p2d, p3d, and p4d) which load onto two different latent slopes. I employed the "Model constraint" section of code to define the proportionality constraints. But I receive the error message: *** ERROR in MODEL command Unknown variable(s) in a BY statement: (P2D) Do I need to define the constraint variables in the USEVARIABLES statement Portions of my code are below. Thank you, Lisa !level and slope for depression d0 by ldep1@1; d1 by ddep2 (p2d) ddep3 (p3d) ddep (p4d); !< proportionality constraints d0; d1; [d0 d1]; d0 with d1*.25; !level and slope for bmi b0 by lbmi1@1; b1 by dbmi2 (p2b) dbmi3 (p3b) dbmi4 (p4b);! < proportionality constraints b0; b1; [b0 b1]; b0 with b1*.25; Model constraint: p3d = 2p2d; p4d = 4p2d; p3b = 2p2b; p4b = 4p2b; OUTPUT: SAMP STAND RES MODINDICES(0) tech1; *** ERROR in MODEL command Unknown variable(s) in a BY statement: (P2D) 


Please send the full output and your license number to support@statmodel.com. 

Juliette posted on Wednesday, March 28, 2012  6:00 am



Hello, I am trying to understand if it is possible/acceptable to test measurement equivalence across two time points in a multilevel CFA model. I have come across some responses regarding measurement equivalence across groups in a multilevel model, but I am not clear how this applies to equivalence across time. If it is possible to do, do you go through the same steps as in single level measurement invariance testing (allowing all to vary but fixing the factor variances to one and means to zero respectively, constraining factor loadings, constraining loadings and intercepts across time)but do this within each level? Thank you. 


See the Topic 4 course handout on the website under multiple indicator growth. Testing for measurement invariance across time is shown here. This can be done only if the data are in wide multivariate format. 

Juliette posted on Friday, March 30, 2012  3:55 am



Thank you for pointing me in the right direction. So, I follow the steps outlined in the handout, testing for invariance in the within part and the between part of the model simultaneously? Or do you have to do this for the within part and the between part separately so that you are testing for invariance at one level at a time? Thank you. 


I would test the within and between parts separately. 


Dear Dr. Muthen, To confirm a hypothesis in multiple group analysis I have to free each structural path and then compare them with the unconstrained model. In Mplus I constrained it using the following syntaxis: f3 on f2 (1); f3 on f5 (2); f3 on f7 (3); Then I compare it with: f3 on f2 f5 f7; !(unconstrained) I would like to ask you: 1) Do you if I can constrain all the paths togheter and compare it with the unconstrained model? 2) Is it correct my syntax to make it? Thank you very much, Mario 


1. Yes. 2. Yes. 


Thank you, I just have one more question. When I tried it togheter (like I described before) the Xi2 difference is significative, supporting the hypothesis. However, when I tried each path independently the differences are not significative. Can I constrian all the paths togheter and compare it with the unconstrained model to support the hypothesis? 


The overall test likely has more power than the individual tests. The way you approach this should take into account your hypotheses. If you have specific hypotheses, you should test those. Otherwise, you can use an overall test. 


Dear Drs Muthen, I have created a structural model using the wlsmv estimator. It is a MIMIC model and includes 7 latent factors and 8 cause indicators: f1 on f2; f2 on f3; f3 on f4; f1 on f5; f5 on f6 f7; f2 on v1; f7 on v2 v3 v4 v5 v6 v7; All factor indicator variables are ordered categoric with the exception of one continuous indicator variable. I would now like to test for measurement invariance across six groups and also test for latent means differences. 1. Can I perform these procedures from within the above SEM framework, or is this form of testing only feasible from within a CFA framework? 2. If only feasible from within the CFA framework, should I remove the 8 cause indicators and carry out the invariance tests on my sevenfactor measurement model? Thanks, 


Some people advocate 1 and others 2. It is your choice. 


Thanks. So, 1. If I choose to do the tests from within SEM, do I remove the cause indicators from the model? 2. If I choose to do the tests from within CFA, do I remove those same sets of observed variables from the model? 


Also, kindly, another question concerning code. In terms of identifying item thresholds , I have seen two different expressions of code: a. [v1$1] b. [v1$1*] http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1083&context=cehsdiss (p. 119) and http://psych.unl.edu/psycrs/948_2011/13a_MG_IFA_Invariance_Example.pdf 3. So, my third question is, how does freeing the parameter of the threshold change anything in the context of testing measurement invariance? 


1. You don't need to. 2. You should remove them. 


4. Another question, please. When carrying out config, metric, scalar, and structural invariance tests, in each case, I constrain the factor means to equal "0" under the MODEL command. Lastly, I use the savedata command to save the output of the structural test (DIFFTEST=structural.dat). Next, to test for the equivalence of latent factor means, as I understand it, I am supposed to constrain the latent means of one group to zero. To do this, do I now omit the "[f1@0]" command from under the MODEL command and place this restriction under the MODEL GROUP2 command? Thanks so much for your time. 


Thanks so much for your answers to questions 1 and 2. Looks like you just missed my posting of question 3. I kindly ask you to consider questions 3 and 4 above. 


Please see the Topic 2 course handout on the website under multiple group analysis where the inputs for testing measurement invariance for binary items are shown. 


Thanks for your reply. It seems to answer question 3 that I posted. However, as for question 4, the chapter you suggested in the handout, http://www.statmodel.com/download/Topic%202v14.pdf (I'm assuming slides 167171), doesn't seem to provide anything about assessing means and covariance structures (i.e., equivalence of latent factor means) with categorical variables. I kindly ask you for a little more direction with regards to my question 4, above. 


Basically, I get an interpretable output, but I'm wondering if the following code is correct: Analysis: type = general; estimator = wlsmv; parameterization = theta; DIFFTEST = structural.dat; Model: f1 by v1* (A1); f1 by v2v3* (A2A3); [v1$1] (B1); [v1$2] (B2); [v2$1] (B3); [v2$2] (B4); [v3$1] (B5); [v3$2] (B6); v1v3@1; f1@1; f2 by v4* (C1); f2 by v5v6* (C2C3); [v4$1] (D1); [v4$2] (D2); [v5$1] (D3); [v5$2] (D4); [v6$1] (D5); [v6$2] (D6); v4v6@1; f2@1; f1 with f2 (D7); Model group2: f1 by v1v3* (A1A3); [v1$1] (B1); [v1$2] (B2); [v2$1] (B3); [v2$2] (B4); [v3$1] (B5); [v3$2] (B6); v1v3@1; f1*; [f2@0]; f2 by v4v6* (C1C3); [v4$1] (D1); [v4$2] (D2); [v5$1] (D3); [v5$2] (D4); [v6$1] (D5); [v6$2] (D6); v4v6@1; f2*; [f2@0]; f1 with f2 (D7); 


Posts on Mplus Discussion should be limited to one window. See the Topic 1 course handout under multiple group regression where comparing the means, variances, and covariances of factors is shown. 


Dear Drs Muthen, We are trying to run a multilevel (2 levels, pupils within schools)model with a pupil variable measured at 2 time points using the same measure with 10 categorical (0,1,2) items. To achieve measurement invariance over time we have constrained factor loadings and thresholds to be equal over time as illustrated below !time 1 EMO1 BY X1_T1 (1) X2_T1 (2) !time 2 EMO2 BY X1_T2 (1) X2_T2 (2) !threshold constraining [X1_T1$1 X1_T2$1] (1); [X1_T1$2 X1_T2$2] (2); Excluding the multilevel component the model runs. When we include the multilevel bit the following error message appears. *** ERROR in MODEL command The thresholds of variables on the between and within levels exist only on the between level. We are not sure how to proceed. The multilevel model runs if the threshold constraint is removed, so can we run it without constraining thresholds over time? I should also note here that we do not have any level2 predictors and are using a multilevel approach to account for schoollevel variation. 


Move your threshold statements to the Between level. 


Thank you! The model runs when I follow your suggestion and move just the thresholds to the between level. Just for my own understanding can you please explain to me why the threshold constraints are between and the rest of the measurement model within? (or direct me to somewhere that does). Thanks! 


The means are overall means. By convention is multilevel modeling, the means are reported at the highest level. 

jml posted on Sunday, January 05, 2014  8:34 am



Dear Drs. Muthen, I will be fitting a multilevel SEM to examine the influence of how a change in schoollevel variables from elementary school to middle school affects children's academic and behavioral outcomes in middle school. But first I'd like to just examine the measurement models for the schoollevel variables (one for elementary schools and one for middle schools). Unfortunately, there are only about 20 elementary schools and 90 middle schools, which makes things tough in terms of having large enough Ns in the measurement models, particularly for elementary schools. However, since the schoollevel variables will eventually have children "below" them in the SEM, can I artificially increase my Ns when I test the schoollevel measurement models by duplicating the set of schoollevel variables as many times as there are children for that school and then fitting a CFA model to those augmented data sets? So instead of having one set of observations for each school on the schoollevel variables, I have N_j, where N_j is the number of children that were in school j when those schoollevel variables were collected? If not, is there a better way to handle this smallN measurementmodel problem? Thanks very much! 


I'm afraid that wouldn't be correct if the schoollevel variables are truly measurements of schools, not students. With only 20 observations you can't investigate a measurement instrument, so you have to rely on past research to form a scale. 

jml posted on Sunday, January 05, 2014  10:02 am



That's what I was afraid of... Well, thanks for setting me straight on that! 


Hello, I have a sample of individuals nested in teams across three conditions. I ran a multigroup MCFA to test for measurement/structural invariance across the conditions. When I change the referent group/condition in the models, I get almost identical results (fit indices, etc.) across both the configural model and the metric model. However, I get very different fit indices and parameter estimates across referent groups in the scalar model. Any idea why this might be happening? Thanks! 


I would need to see the outputs and your license number at support@statmodel.com to be able to comment on this. 


Greetings, As part of a scale validation project, I recently performed CFA and IRT analyses of a multilevel data set of 63 3category ordinal survey items originating from 1181 participants from 22 villages. For the CFAs I used TYPE=COMPLEX due to the modest number of villages and found good support for a 7dimension correlated factors CFA. However, a reviewer of our manuscript feels we haven't done enough to validate the construct at the village level. ICCs for the subscales were modest, ranging from .02 to .15. The reviewer believes the low ICCs in conjunction with the analysis approach taken indicate insufficient validation of the scale at the group level. Given the moderate number of villages, my coauthors and I are curious if there are additional validation techniques available in Mplus that we could consider or whether we're stuck concluding that while there is solid support for the 7dimension construct at the person level, support at the community level is limited? For instance, could the newer alignment methods be tried to help address these issues? Any suggestions appreciated. Tor Neilands 


Alignment is available with maximum likelihood estimation for ordinal outcomes so that could be a possibility. 

Back to top 