Message/Author 


My question concerns tests of longitudinal invariance when an instrument has categorical indicators. Assume that I have onefactor construct which is measured with 3 dichotomous items at 2 points in time. 1. In a *multiple group* approach, typically a basline model is estimated in which all parameters are freely estimated. Subseq. models then impose constraints on thresholds/loadings, latent (co)variances, and eventually latent means. HOWEVER, in the case of longitudinal data this is not possible, since freely estimating *all* thesholds will necessarily result in underindentified latent means. That is, even after I fix the latent mean at time 1 to 0, the latent mean at time 2 is not identified (without constraints). However if I constrain thresholds (and factor loadings by association) to be equal over time, this baseline model seems much more stringent than is the multiple group case. Am I missing something or is that all I can do? 2. If I assume that the baseline model necessarily imposes constraints on thresholds/factor loadings across time, I would still like to compare this model against models with constraints on latent (co)variances and latent means. Imposing constraints on latent (co)variances is not a problem. However constraining latent means is. Specifically, initially the time 1 latent mean is fixed to 0 and the time 2 mean is interpreted as the change in mean level relative to time 1. In the model that I try to equate latent means in the customary way (e.g., [latentxt1 latentxt2] (##);) I get an error implying underidentification. If I continue to fix the latent mean at time 1 to 0, obviously there is no constraint being imposed and the latent mean at time 2 continues to be interpretted as relative change. Again, if I were doing a multiple group analysis, I could simply equate latent means separately using "model" lines (as Linda suggested Millsap in her 11/27/00 post to this board). However in the case of long. invariance, I can't seem to impose constraints on latent means over time and still have them be identified. Again, am I missing something? I appreciate your thoughts  mikew@unc.edu 

bmuthen posted on Tuesday, December 19, 2000  2:02 pm



A useful first model is to analyze both time points together with no acrosstime invariance. In this model you have the factor means fixed at zero at both time points since you allow the thresholds to be different. Then you turn to the model with invariant thresholds and loadings, letting the delta scale factors be different across time (see Users Guide, Appendix). In this model the factor means are zero at time 1 and free at time 2. The factor covariance matrices are different across time. Following that you can use WLS chisquare difference testing and restrict factor means to zero at time 2 to test that they are different from time 1, and you can restrict the factor covariance matrices to test that. 


I have data (n=1300) from a 2wave panel on a single construct measured by 10 dichotomous items. What is the most appropriate model to assess longitudinal measurement invariance? Thanks. 


You would compare the same models across time as you would normally compare across groups. The Topic 4 course handout shows this for continuous outcomes under Multiple Indicator Growth. The measurement invariance models we suggest for categorical outcomes are described in the new Chapter 14 at the end of the discussion of multiple group analysis. 


Linda and Bengt, I am analyzing longitudinal categorical data with 9 indicators of a single factor measured 3 times. The data are quite similar in structure to Example 6.15, the multiple indicator growth model with categorical indicators. As a preliminary step to the growth model, I am doing a multiwave CFA to test for measurement invariance (modeling the longitudinal data simultaneously, not as separate groups, to preserve the longitudinal structure). In your recommendations concerning testing for measurement invariance (Ch. 14, p. 433 of the User's Guide), you recommend that the more restrictive model for categorical data should hold factor loadings and thresholds to equality over time, to fix scale factors at 1 at one wave and free at other waves, and to fix the factor means at 0 in wave waves and free in the other waves. With such data, Mplus sets all factor means to 0 by default (this is true in ex6.15, too). When I fix wave 1 factor mean at 0 and free factor means in the other groups (per the User's Guide suggestion), I get small and nonsignificant factor means at waves 2 and 3 and model fit improves to a nonsignificant degree (relative to model with all factor means at 0). 


(continued from above because post size is limited) So, my inclination is to keep all factor means fixed at 0 for parsimony. But I am not clear on the effect that this has on the interpretation and validity of factor loadings and factor covariances, which I'm interested in. My sense is one fixes the factor mean at 0 in one group to provide a scaling of the factor means relative to the other waves (roughly mean differences in theta, in IRT terms). So, the fact that the means are not really different when freed in waves 2 and 3 suggests that there is little mean level change in the construct (i.e., symptom severity is roughly equal across time in my case of a onefactor severity construct). So, I think the specific questions are: 1) Is it valid to fix the means at zero at all waves or does this muddy the interpretation? 2) Is my interpretation of the nonsignificant factor mean differences valid (i.e., little mean level change in the construct)? 3) Are there any concerns of model nonidentification? (I know estimating all factor means freely will cause nonidentification...) Thanks so much for your input, Michael 


1) Yes, but I would not recommend it because it is like model trimming  report the nonsignificance instead. 2) Yes. 3) Note that in the factor analysis case, the means are free for all but the reference time point. But for growth modeling the factor intercepts are fixed at zero for all time points because the factor mean changes are handled by the growth factors. Yes, post size is limited  it is intentional: We only have time for short questions 


Hi I have read a lot of posts about this but i still don't get it straight. I have a CFA model with four factors and categorical indicators that fit well in the two time points available separately. I would like to assess if the loadings are similar in the two time points I have specified two models as stated in the manual with both time points together: "1. Thresholds and factor loadings free across groups (time); scale factors fixed at one in all groups (time); factor means fixed at zero in all groups (time) 2. Thresholds and factor loadings constrained to be equal across groups (time); scale factors fixed at one in one group (time) and free in the others; factor means fixed at zero in one group (time) and free in the others" My question is; Can I compare this two models with difftest to assess the equality of loadings and thresholds over time? I don't see clearly that this are nested models because of the second one having some free parameters that the first one doesn't have. Thank you J. 


Yes, these two models are nested and can be tested with DIFFTEST. 


I have an autoregressive path model with parameterization=theta (3 waves) categorical are u1 u2 u3; model: !autoregressive paths f3 on f2; f2 on f1; u3 on u2; u2 on u1; !crosslagged paths f3 on u2; f2 on u1; u3 on f2; u2 on f1; !withintime covariances u1 with f1; u2 with f2; u3 with f3; !residual correlations u1 with u2 u3; u2 with u3; f1 with f2 f3; f2 with f3; !covariates u1 f1 on male age; How would I set up an autoregressive path model model up to capture similarities in measurements across time in the u variables? I would like to assume the u variable has measurement invariance across time, and also, the measurement error of u is correlated across time. Is that possible? As for the residual correlations, The "u1 with u2 u3" presently statements show up in the psi matrix and not the theta matrix in TECH1, so it looks like they are simply correlations of the variables. Thank you! 


Starting from your last statement, the fact that u1 with u2 ends up in Psi instead of Theta should be ignored  it still refers to the residual covariance between the variables (the reason it ends up there is that a latent variable gets put behind the u's that have regressions; it's just an Mplus trick). With a continuous outcome that is repeatedly measured over time, it is possible to identify and estimate a measurement error. But I don't think you can identify correlated errors over time of say y1 with y2 if you also have y2 on y1. Same for a categorical outcome. 


Thank you for your quick response! The model appears to be identified with these crosstime correlations and regressions specified, in that it has no problems with convergence and has good fit. Does that sound wrong? I am suspicious though as to why the crosstime residual correlations are negative for the observed categorical and continuous variables. Does this sound like an incorrectly specified model? 


Then we need to see the output  please send to Support. 


Looks like the model is identified by restrictions across the different processes. Don't know what to say about the negative crosstime residual correlations. 


Are the syntax associated with the "Item Response Modeling in Mplus: A MultiDimensional, MultiLevel, and MultiTimepoint Example" paper going to be posted to statmodel? I had not seen anything uploaded and was uncertain. Thanks 


Haven't done this yet, but should. I can send you the ones you are interested in. 


That would be great. I'm trying to establish longitudinal measurement invariance of a bifactor model over 9 waves with an accelarated cohort design and have been running into a few problems. I believe the syntax related to your Table 6 dealing with longitudinal twolevel analysis using bifactor CFA and the Bayes estimator would be helpful, or any others you suggest. Thanks again. dap38@pitt.edu 


Will send the Table 6 output. 


Dear Professor Muthen, I am also interested in Mplus syntax/output associated with the Table 6 (and Table 3) from "Item Response Modeling in Mplus: A MultiDimensional, MultiLevel, and MultiTimepoint Example" paper. I would be very grateful if you could send it to me also (grygiel@uniwersytet.rzeszow.pl). 


Will have it posted today. 


The Table 6 run has been posted. 

RuoShui posted on Tuesday, April 01, 2014  4:20 am



Dear Drs. Muthen, I am pretty new to longitudinal measurement invariance with categorical indicators. After reading the users' guide, I am not sure if I had this correct. Could you please kindly have a look at the syntax? 1. Testing configural invariance USEVARIABLES ARE ..... CATEGORICAL= a1a3 b1b3 c1c3; Model: f1 BY a1a3; f2 BY b1b3; f3 BY c1c3; [f1f3@0]; {a1a3@1 b1b3@1 c1c3@1}; a1 WITH b1; a1 WITh c1; b1 WITH c1; ... savedata: difftest is configural.dat 2. Testing metric and scalar invariance in tandem Analysis; difftest is configural.dat Model: f1 BY a1 a2 (1) a3 (2) f2 BY b1 b2 (1) b3 (2) f3 BY c1 c2 (1) c3 (2) [a1$1 b1$1 c1$1](3); [a1$2 b1$2 c1$2](4); [a2$1 b2$1 c2$1](5); [a2$2 b2$2 c2$2](6); [a3$1 b3$1 c3$1](7); [a3$2 b3$2 c3$2](8); [a1@0]; {a1a3@1}; a1 WITH b1; a1 WITh c1; b1 WITH c1; ... Thank you so much for your help! 


See the Version 7.1 Language Addendum on the website with the user's guide. All models for testing for measurement invariance along with new features for testing for measurement invariance are described starting on page 4. 

RuoShui posted on Tuesday, April 01, 2014  4:22 pm



Thank you so much Dr. Muthen. That was really helpful. I have a related question. From the user's guide, it seems that the factor loading and threshold should be constrained in tandem when testing measurement invariance for ordinal indicators. But the language addendum carried out metric and scalar invariance in separate steps. When I tested scalar invariance, the modification indices suggested relaxing the two thresholds for 1 item. My question is, do I also need to relax its factor loading at the same time I relax its two thresholds? Another question is when I fit a linear growth curve modeling, there is this warning message. Could you please let me know what this means? Thank you very much! THE MODEL CONTAINS A NONZERO CORRELATION BETWEEN DEPENDENT VARIABLES. SUCH CORRELATIONS ARE IGNORED IN THE COMPUTATION OF THE FACTOR SCORES. 


The Version 7.1 is a more thorough description of measurement invariance models for more specific situations. As stated there, thresholds can be tested separate from factor loadings for ordinal items. You must have a WITH statement between two dependent variables which means that factor scores cannot be computed. 

RuoShui posted on Wednesday, April 02, 2014  12:52 pm



Dear Dr. Muthen, Thank you very much! Yes, I did correlate the error variance between the same ordinal indicators at different time points using WITH statement. But i did not get the error message when I fitted similar models with continuous indicators. Does this only happen for ordinal indicators? And is it a concern that factor scores cannot be computed? In other words, does using WITH statement between two ordinal indicators invalidate the model? Thank you again for your time. 


This situation happens only with categorical items and weighted least squares estimation. No, the WITH statement does not invalidate the model. This is only a problem if you want factor scores based on a model with a WITH statement. 

RuoShui posted on Wednesday, April 02, 2014  3:43 pm



Thank you very much Dr. Muthen! 

Tom Booth posted on Thursday, October 01, 2015  6:00 am



Dear Linda/Bengt, I have a situation where I am fitting a longitudinal invariance model, and including a crosslagged set up in my structural model. The structural model also includes a latent interaction. This combination of things doesn't seem doable in MPlus, as the latent interaction estimation does not run when correlated residuals across time are present in the longitudinal invariance measurement model. Is there a way round this? Best Tom 


Please send the output that shows the problem and your license number to support@statmodel.com. 

IW posted on Friday, October 16, 2015  10:10 pm



Dear Drs. Muthen, I have a similar model as Tom Booth where I'm modeling longitudinal and multigroup invariance with categorical latent factors and I cannot seem to model the latent factor interactions. Specifying KNOWNCLASS with TYPE = mixture random, and ALGORITHM = integration causes two issues: 1) Mplus will ignore correlations between indicator residuals across time points 2) Problems with threshold constraints Thank you for your help. 


To give guidance here we need to see the failed outputs and data. Include license number and send to Support. 

Margarita posted on Thursday, October 13, 2016  3:59 am



Dear Dr. Muthen, As described in the manual, for scalar measurement invariance, categorical indicators and THETA, the residual variances are set to 1 in the first group and freely estimated in the others. I just wanted to double check that when testing for longitudinal invariance, the residual variances should be fixed at 1 in Time 1, and freely estimated in Times 2 and 3. Would that be the way to go? Thank you 


Yes. 


Dear Professors Muthén, I am hoping to confirm that I used the correct technique for assessing factor loading and threshold invariance in testing longitudinal MI with categorical data that includes correlating residuals between study waves. I based my syntax on Topic 4 slide 94 along with slides from Topic 2, as well as syntax from Newsom (2016) and Brown (2016). If this isn't correct, what needs to be changed? ESTIMATOR IS WLSMV; Parameterization is Delta; !Factor loading invariance MODEL: factor1 BY a1 b1j1 (19); factor2 BY a2 b2j2 (19); !Correlating Residuals between waves a1 WITH a2; b1 WITH b2; c1 WITH c2; d1 WITH d2; e1 WITH e2; f1 WITH f2; g1 WITH g2; h1 WITH h2; i1 WITH i2; j1 WITH j2; !Within wave corr resid i2 WITH j2; !Threshold invariance for categorical vars [a1$1a2$1] (10); [a1$2a2$2] (11); [a1$3a2$3] (12); ... [j1$1j2$1](37); [j1$2j2$2](38); [j1$3j2$3] (39); [factor1@0 factor2]; 


Also, I tried turning the data into long format and using the simplified MODEL = CONFIGURAL METRIC SCALAR command with grouping variable as Wave 1 or 2 and items listed as "a b c d." However, I'm not sure how to correlate residuals between waves with data in long format. Would something like "a WITH a" suffice? Thank you. 


It looks like you have 2 time points in which case you want to free the Delta parameters at time 2. You can't get the acrosswave residual correlations in long format. 


Thank you for your response, Dr. Muthén. I should have mentioned this is for testing strong/scalar MI. When you speak of freeing Delta parameters, do you mean freeing the scale factor for time 2 like the following? !Fixing scale factor at time 1 to 1 {a1j1@1}; !Freeing scale factor at time 2 {a2j2*}; If the above is correct, am I missing anything else? The various examples online and in textbooks show slightly different ways to run longitudinal MI with categorical data, so I just want to make sure I am doing what is recommended. Good to know that I was interested in doing the currently impossible re: long formatted data! 


Based on the Mplus hand book version 7, page 137, it looks like I could use this syntax to fix and free the respective scale factors: {a1j1@1 a2j2}; However, when I run this syntax, I receive the following error: THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL. PROBLEM INVOLVING THE FOLLOWING PARAMETER: Parameter 93, { a2 } THE CONDITION NUMBER IS 0.510D16. I tried both syntax examples I included in this and the above post for scale factors and received similar errors. If I want to switch to using Theta parameterization, would I be okay to use my original model? 


Never mind, I got the syntax to work. Thank you! 


Hello, I am helping with longitudinal invariance testing with categorical data AND hierarchical modeling. I am following Chen, Sousa, and West's (2005) suggestions in Testing Measurement Invariance of SecondOrder Models (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.457.5719&rep=rep1&type=pdf) The following is recommended in the MPLUS user guide: "The configural model has factor loadings and thresholds free across groups, residual variances fixed at one in all groups, and factor means fixed at zero in all groups...The scalar model has factor loadings and thresholds constrained to be equal across groups, residual variances fixed at one in one group and free in the other groups, and factor means fixed at zero in one group and free in the other groups." Thus, I would be unable to run Chen et al.'s (2005) model 6 and 7 because factor means and residual variances would be constrained to 1 or 0. What are your recommendations for completing longitudinal invariance testing with categorical data and hierarchical models? Thanks! Hillary 


I see no reason that you can't do models 6 and 7. Using the scalar model that we mention plus hold the residual variances fixed at 1 at all time points (this is the same as holding them equal given that with categorical outcomes they cannot be identified for all time points but must have one reference time point where they are 1). 


Ok, thank you for this information! We will try this approach. Hillary 


Hi, With a threefactor hierarchical model with categorical data AND a twoitem component/ factor, is invariance testing possible? Would the twoitem component/ factor prevent convergence in MPLUS, even if using WLSMV estimation and Theta parameterization? If it is not possible, would the best invariance testing approach be to test the components? Thanks! Hillary 


*Correction: This is for longitudinal invariance testing. 


If you have a general factor and a specific factor with only 2 indicators, the 2 loadings have to be fixed at 1 to make the model identified. 


Thank you for your response! Is this the correct syntax for doing so? FACTOR BY INDICATOR1@1 INDICATOR2@1; Thanks! Hillary 


Right. 


When testing the configural model using Theta parameterization, the Version 7.1 Language Addendum states that residual variances should be constrained to 1 when doing multiple group comparisons. Should I skip this constraint when conducting longitudinal analyses where indicatorspecific residuals are correlated across time. The following syntax runs fine, but I worry that I am missing something. Analysis: ESTIMATOR= WLSMV; PARAMETERIZATION = THETA; MODEL: [LE4@0 LE5@0 LE6@0]; LE4@1 LE5@1 LE6@1; LE4 BY tngprs4* tngONT4 tngMPS4 tngATI4 tngNEG4; LE5 BY tngprs5* tngONT5 tngMPS5 tngATI5 tngNEG5; LE6 BY tngprs6* tngONT6 tngMPS6 tngATI6 tngNEG6; tngONT4 tngONT5 tngONT6 with tngONT4 tngONT5 tngONT6; tngPRS4 tngPRS5 tngPRS6 with tngPRS4 tngPRS5 tngPRS6; tngMPS4 tngMPS5 tngMPS6 with tngMPS4 tngMPS5 tngMPS6; tngATI4 tngATI5 tngATI6 with tngATI4 tngATI5 tngATI6; tngNEG4 tngNEG5 tngNEG6 with tngNEG4 tngNEG5 tngNEG6; Thank you 


See page 752 in the V8 UG. 


Thank you for your quick response. I just want to clarify that I am testing measurement invariance across time, not running a growth model. My variables are ordinal (likert, 15). I am still unsure how to modify the ordinal measurement invariance analyses for timevarying (rather than multigroup) analyses. Based on page 752, it seems as though residual variances should be free (rather than all fixed to one in the configural model or fixed to one in only the first group for the metric and scalar models). This would allow me to specify residual correlations within items over time to account for residual dependence. Am I correct in this conclusion? My configural model runs fine when I do not constrain the residual varainces, but the psi matrix is not positive definite in my metric and scalar models. The problem involves the first indicator at the third time point. The problem persists regardless of how I scale/standardize my factors, but I am having trouble identifying the exact issue. Would I be justified in using delta parameterization instead of theta parameterization for this analysis? When I use Delta parameterization, my models run without issue. However, I read that Theta parameterization is preferable for longitudinal analyses. Is this true for longitudinal measurement invariance, or does it only truly matter when measuring growth? Thank you again for your time. 


Longitudinal invariance is handled the same way as multigroup invariance. Residual variances can be free in all but the first time point when using Theta param. But you can include correlated item residuals also with Delta. The configural model should have Theta residual variances fixed at 1 for all time points. Usually, Delta provides easier computations (Theta is more sensitive to which loading is fixed at 1 for instance); this is why it is the default. See our web note 4. 


Hello, I am examining measurement invariance in a bifactor model of ordinal data across time and groups using the invariance testing convenience features and the WLSMV estimator. I have two questions: 1. From reading previous posts and the Version 7 Language Addendum, it seems that the METRIC step is not identified when items load on more than one factor. This would mean the Scalar model is compared with the Configural model. Could you confirm that this is correct? 2. Is it possible to use the invariance testing convenience features to test longitudinal invariance when the data are in wide format? If so, I would be grateful if you could point me to an example of a similar input. Thank you for your time. 


1. I recommend skipping the Metric step with ordinal outcomes and instead compare the scalar model to the configural. 2. Yes. See the UG page 751 script and also UG ex 6.15. Plus, the discussion in the IRT chapter http://www.statmodel.com/download/V1C31.pdf 


Although page 751 is for a growth model, you can translate that into a longitudinal factor model  the scalar measurement invariance constraints are the same. 


Hello Dr. Muthen, Thank you for this quick and very helpful response. In modifying the script in UG ex 6.15 and UG page 751, would the scale factors portion still apply? Also, I have seen in invariance scripts that the factor means must be fixed to zero for model identification. Is this done automatically when using the MODEL=CONFIGURAL analysis command, or is it not necessary for ordinal data? Thank you again for your help. 


Yes, scale factors apply. Longitudinal invariance is analogous to multiplegroup invariance. You should not use the MODEL=Configural statement for longitudinal data unless you have multiple groups. You have to set up this model yourself. Factor means should be fixed at zero for all time points  also for ordinal data. 


Hi Dr. Muthen, I'm testing longitudinal invariance of a hierarchical model (two waves, three firstorder factors) based on categorical data. I'm using WLSMV and delta parameterization, using the descriptions of configural and scalar invariance models in the v7.1 language addendum. This setup worked for a correlated factors version of the model, and also for the configural invariance version of the hierarchical model, but is not identified in the scalar setting for the hierarchical model. I think this is because all factor means are free at wave 2, but I'm not sure what the right constraints would be to test scalar invariance in this context. Could you advise on what you would recommend in this context? (I saw the earlier post on a similar idea, but based on theta parameterization, where you recommended fixing residual variances to 1 at all waves. Would this generalise to fixing scale factors and/or firstorder factor variances to 1 across waves?) Thanks very much for your help! Miri 


You say 3 firstorder factors  does that mean that there is a secondorder factor behind them? 


Yes that's right three firstorders with a higherorder factor accounting for the factor correlations. 


Perhaps you overlook fixing to zero the intercepts of the firstorder factors in their regression on the secondorder factor. At both time points. 


That does the trick. Thanks very much for your help! 


I am testing the longitudinal measurement invariance of a scale with 1 factor and 7 ordered categorical indicators across 2 time points (N=281, Estimator WLSMV, Theta parameterization). My baseline model fails the ChiSquare test (X2= 129.379 (69), p<0.001) and I am struggling to understand some of the modification indices I got. For instance: ON/BY Statements Ft1 BY u1t2 18.038 WITH Statements u1t2 WITH Ft1 18.044 u1t2 WITH Ft2 18.049 I also got some ON statements involving the same u1t2 indicator and other indicators measured at t1. Could these modification Indices (particularly the BY Statement) be an indication of noninvariance of the factor loading for the u1t2 indicator? or is there another plausible interpretation for them? When I constrain the factor loadings to be equal across time, the model fails again the Chisquare test, but the DIFFTEST indicates that its fit isn't worse than the fit of the baseline model. In this case, I get again the Ft1 BY u1t2 statement but smaller (9.002). Other fit indices such as CFI, TLI and RMSEA are good for both models according to the conventional cutoffs, but I recently read that they may fail to discover data misfit when using DWLS and ULS estimators and therefore I am more interested in the interpretation of the modification indices. I appreciate your thoughts on these issues. 


I would first make sure the model fits at each measurement occasion. If it does, then misfit must come from the model failing to capture acrosstime correlations. This is most typically handled by allowing all residual correlations for the same item across time. 

Back to top 