Message/Author 

daniel posted on Tuesday, February 18, 2003  8:59 am



If I want to know the covariance among endogenous factors f2 f3 f4, I use program like this: model: f1 by sbv26t1 sbv27t1 sbv29t1 ; f2 by magsabt2 magvvbt2 magsvbt2 ; f3 by mpmpcot2 mpmptrt2 mpmpint2 ; f4 by mpbdevt2 height2 ; f5 by sbv26t3 sbv27t3 sbv29t3 ; f4 f3 f2 on f1 ; f5 on f2 f3 f4 f1; f3 with f4; f2 with f3; f2 with f4; what problems will occur at interpretation of the covariance? 


The covariances among endogenous factors are residual covariances. If you ask for TECH4 in the OUTPUT command, you will obtain covariances and correlations for the latent variables in the model. 

Anonymous posted on Friday, January 14, 2005  8:13 am



Hello Linda, I am running a path analysis with categorical and continuous mediating variables and a continuous final outcome. My question is about the use of with statement regarding one independent and one dependent variable. For example, if I have x1(dependent predicted by other variable i.e not x2 ) and x2 (independent) X1 with x2  can this be estimated in mplus and does it make sense to estimate this correlation? It was suggested by modification indices in mplus Thanks a lot. 


I cannot understand your model from what you say. If x1 is regressed on x2 then x1 WITH x2 is not identified if this is what you are asking. 

Anonymous posted on Sunday, January 16, 2005  2:12 pm



Hi Linda, Thanks for the reply. Let me put my question as mplus program. X1 on x2 x3 x4; X5 on x3 x4; Y on x5 x6; X6 with x5; Here is an independent variable x6 correlating with a dependent variable x5(its error). Once I correlate x6 with x5 the mdoel improves much and the path from x6 to y is no more significant. How do we interpret this? is it common in SEM? Thanks 

BMuthen posted on Tuesday, January 18, 2005  3:44 pm



Without your WITH statement, x5 and x6 are uncorrelated in your model. It may be more natural to regress x5 on x6 than to correlate them. In path analysis as you are doing, if the model improves when you add a path, then this path is needed to reproduce the correlation matrix. I think this is a common phenomena. 

Annonymous posted on Tuesday, January 17, 2006  1:07 pm



With regard to the previous post: is it possible to correlate just the error and not the observed variable? i know that in AMOS, there is a difference between covarying the error terms of Y1 and Y2 with each other, compared to covarying Y1 and Y2. 

Annonymous posted on Tuesday, January 17, 2006  1:08 pm



PS Y1 and Y2 are continuous variables in a CFA. 


If y1 and y1 are factor indicators, then you estimate their residual covariance as part of the model because they are endogenous variables. If they were exoogenous variables, their variance would be estimated. 

Annonymous posted on Wednesday, January 18, 2006  7:19 am



Ok. How would that be programmed? if a latent var (L) is measured by Y1 and Y2 and is endogenous to some other exogeneous factor (EX),it seems that if we wrote EX on L; L by Y1 Y2; Y1 with Y2; then we would covary the values of Y1 and Y2 and not just their residuals. 


The meaning of y1 WITH y2 depends on the context in which it is used in a model. In the model you show above, y1 and y2 are endogenous variables, therefore y1 WITH y2 represents a residual covariance. In the following model, y1 WITH y2 represents a covariance: MODEL: y1 WITH y2; 


hi, i am regressing an observed v. on one latent variable and several manifest variables. as i understand the default is that the correlation among the DV is estimated. but I don't get it in the model output. Only in tech 4. if i specify the correlations via with statements i get an error message about non positive prod. matr. I am wondering why. thanks 


What are your DVs? You mention an observed variable. Is the latent variable that you mention a second DV? It is not clear what your model is. Note that you cannot identify both a residual covariance between two DVs and the regression of one on the other. 


hi, my dv are some observed and one latent variable latent_var by x1 x2; observ_var on observ_var1 observ_var2 latent_var; thanks 


The covariances between x1, x2, and observ_var are not estimated as the default. 


Hello, I have a question concerning a path model with two dependent variables. I understand that the residuals are allowed to covary (by default), but what does it actually mean? Does it mean that two variables share some common `cause` not explained by the specified predictors. And, if I want to examine unique associations between the predictors and y1 (controlling for the y2 effect on y1), should I regress y1 on y2 (rather than estimating the covariance among the residuals)? Thank you! 


Yes to your first paragraph. You can include y2 as a predictor of y1 instead of having the residual covariance between y1 and y2. 


Dear Mr. and Mrs. Muthén, I am conducting a SEMmodel with 2 independent latent variables (measured by two parcels each) and 4 dependent latent variables (measured by two parcels each). Since 3 of the dependent variables were measured with the same method (questionnaire), I allow for residual covariance by using the default option and restricting the residual covariance with the fourth dependent variable (lat_DV1 WITH lat_DV4@0; lat_DV2 WITH lat_DV4@0; lat_DV3 WITH lat_DV4@0;). Now, I'm not sure how to visualize the relations in a SEMmodel correctly. Do I have to include correlated residuals between the 6 manifest variables (parcels) or is it sufficient to include solely the correlation between the 3 latent dependent variables (which I get from TECH4)? Kind regards Tony 


I think there are more reasons that the 4 dependent factors have residual covariances than merely that 3 of them share the same method. Any leftout covariates predicting the 4, but left out in the model, would cause residual covariances. You are right that indicators obtained by the same method may call for residual covariances among them. But this has to be modeled in a careful fashion. See for example the MTMM literature, for example in the CFA book by Tim Brown. 


Dear Mr. Muthén, thanks for your quick reply. If I include all residual covariances between the 4 latent dependent variables, the model fit worsen (but fit indices indicate still acceptable model fit) and the output shows, that the fourth dependent variable (which is measured with a different method) has no significant residual correlations with the other three latent dependent variables. Thus, in terms of parsimony I might left out the residual covariances with this fourth variable. Is that right? I already studied the literature on method effects, but for my data none of the modeling techniques seem appropriate (besides, modeling method effects explicitly is not the aim of my article). However, I need an argument why the three latent dependent variables may correlate, since in some articles it is stated, that in "good models there are no correlated errors"). Do you have any advice? Kind regards Tony 


People may have different opinions about these matters; here are mine. I don't agree with the quote. I think a natural baseline model is one where residuals for the dependent variable factors do correlate. Why would a priori the independent variables be the only ones causing correlations between the 4 dependent variable factors? That's a very strong statement. Also, I am not a fan of "model trimming" where one deletes insignificant parameters  reporting that they are insignificant seems better to me. 


Dear Mr. and Mrs. Muthén, I am conducting a path analysis (longitudinal data) with only observed variables. I am wondering about the meaning of the estimate "Y1 WITH Y2" in this case. Does it refer to Y1 and Y2 covariance or to their residual covariance? Is there a way to let the residuals covariate without freeing the covariance of the 2 variables? Thank you so much. Best Regards, michela 


With endogenous variables WITH is a residual covariance. With exogenous variables, it is a covariance. 


Thank you very much for your quick and helpful reply. Best, michela 


Dear Mr. Muthén, From the preceding discussion i understand that, in the following model p represents a residual covariance between f1 and f2 f1 by y1 y2 y3; f2 by y4 y5 y6; f1 with f2 (p); It however remains unclear to me how one labels a residual variance? as f1 with f1 (q) generates an error message and f1(q) would represent the factor variance? Best, Steven 


p is a covariance. q is a variance. 


As f1 and f2 are two endogenous variables, q is not a covariance but a residual covariance, no? If so, how do i label the residual variance? Steven 


In a factor model, the factors are covariates. The factor indicators are regressed on the factors. Unless, there are other covariates in the model, q is a covariance. 


Ok, Actually, i want to specify some error covariance restrictions on the structural part of a sem model. In order to do so i need to label the residual variances and covariances. So in the following model p does represent a residual covariance between f1 and f2 f1 by y1 y2 y3; f2 by y4 y5 y6; f2 on f1; f1 with f2 (p); If in the above model p represents a residual covariance, how do i label the residual variance? i tried f1 with f1 (q) but this generates an error message. Best, Steven 


F1 WITH f2 is not a residual covariance. It would not be identified as stated. Following is an example of a model with two residual variances and one residual covariance: f1 by y1 y2 y3; f2 by y4 y5 y6; f1 f2 ON x; f1 WITH f2 (a); f1 (b); f2 (c); 


Dear Dr. Linda Muthen. About your example above: 1. How can I interpret the magnitude of the residual covariance (parameter "a") in both estimated and standardized units? 2. How can I relate "a" with the total covariance between the 2 variables? 3. Is it easier to explain "a" in terms of correlation between f1 and f2? 4. Can I call the latter, the residual correlation between f1 and f2 (which is not explained by "x")? Thank you. 


1 and 3. You can look at the significance of the raw coefficient and at the standardized coefficient which is a correlation. 2. Look at the ratio of the residual covariance to the covariance. 4. Yes. 

Mark Boons posted on Friday, April 25, 2014  10:09 am



Dear Drs Muthen, I would like to hear your opinion on the following: Me and my coauthors are currently working on a paper in which we test a SEM model in MPlus (see below). As you can see, we allow the residual covariances between factors 1 and 2 to be freely estimated in our model as these two factors are theoretically argued to be “related, but distinct” and a review of previous research including these two constructs has (in all cases) found moderate correlations between these constructs ranging from .35 to .60. However, while one reviewer is satisfied with our justification of our decision to allow these residual covariances to be freely estimated, another reviewer argues that one should NEVER allow for residual covariances to be freely estimated among endogenous latent constructs as this will inflate model fit. While the overall fit of the model does improve by allowing the residual covariances to be freely estimated, none of the results of the hypothesized relationships changes in a way that ‘helps’ (they actually become less significant). I would really like to hear your expert opinions on this matter. MODEL: F1 BY x1x4; F2 BY x5x7; F3 BY x8x12; F4 BY x13x16; F1 ON x17; F2 ON x18; F3 ON f1 f2; F4 ON f1 f2; x19 ON f1 f2; f1 WITH f2; Thanks in advance for your response! Mark 


I would include f1 WITH f2. There is very likely a correlation due to leftout predictors. Do you mean x19 ON f1 f2 or f1 f2 ON x19? 

Mark Boons posted on Friday, April 25, 2014  11:55 am



Dear Linda, Thank you for your quick reply. With regard to x19: In our model we have an outcome measure that captures 'activity'. Since this is a count variable with about 50% zeros, we include this as a variable instead of a singleitem factor and estimate a negative binomial model by using 'count = x19(nb)' in the variables section. Please let me know if you would do this differently. Again thank you for your feedback! This is greatly appreciated! Mark 


Dear, I would like to hear your opinion on following situation: I want to do a path analysis with 6 latent variables. Before measuring different structural models, I did a CFA: INPUT: Org_inf BY SSI_1a  SSI_1g; DS_inf BY SSI_2a  SSI_2g; Coll_inf BY SSI_3a  SSI_3g; ISB BY PB21 PB22 PB23 PB24; NSA BY NSA1 NSA2 NSA3 NSA4; RC BY RC1 RC2 RC3; OUTPUT: Number of free parameters: 108; Chisquare, degrees of freedom: 716.151, 419; RMSEA: 0.053; CFI: 0.923; TLI: 0.914; SRMR: 0.058. Subsequently, I moved on to my first structural model where I use Org_inf, DS_inf, Coll_inf and ISB as exogenous variables and NSA and RC as endogenous variables. The MODELcommand is as follows: Org_inf BY SSI_1a  SSI_1g; DS_inf BY SSI_2a  SSI_2g; Coll_inf BY SSI_3a  SSI_3g; ISB BY PB21 PB22 PB23 PB24; NSA BY NSA1 NSA2 NSA3 NSA4; RC BY RC1 RC2 RC3; NSA ON Org_inf DS_inf Coll_inf ISB; RC ON Org_inf DS_inf Coll_inf ISB; When I look at my fit indices and # of free parameters, it is remarkable that the results are identical with those of my CFA. According to Hair et al. (2010), this is because my structural model is a saturated one. How do I interpret this, and which actions can I take to resolve this issue? Can I somehow delete the residual covariance between my endogenous variables (NSA and RC)? Thanks in advance for your response. Annelies 


Model fit cannot be assessed for the saturated part of your model. You would need to put some restrictions on the structural model, for example, make some paths zero if this fits your research hypothesis. 

Anonymous posted on Thursday, November 12, 2015  8:50 am



I have an SEM with an exogenous predictor, 3 latent endogenous mediators and a final outcome variable. What are the advantages or disadvantages to correlated the endogenous residuals of the latent variables in the structural model? 


I think you refer to the residuals of the 3 latent mediators. If you add those you make the model fit better and may therefore avoid distorted parameter estimates due to misfit. You also learn about their sizes. 


Hello Dr. Muthen, I am working on a longitudinal SEM model. However, many of my endogenous variables should not have a correlation among them. Therefore, I fixed the covariance between these variables to Zero (f1 with f2 @0). Is this a good solution? And does this change the other regression coefficient values? Thank you, Owis 


Q1. Yes. Q2. Typically not  try it. 


Thank you, I checked, the values of regression coefficient did not change. Thank you, Owis 


I have a quick question in regards to the post from November 12, 2015. The response to that post indicates that modeling correlated errors among latent mediators will improve model fit and reduce the likelihood of distorted parameter estimates due to misfit. I'm wondering whether one also needs a theoretical rationale for modeling correlated residuals among latent mediators? In a model I'm currently running, modification indices suggest adding a with statement between two latent mediators in order to improve fit. The literature indicates the two mediators should be highly related (though they are distinct constructs), but it doesn't suggest much of a reason for why their disturbances would be correlated. Is it still appropriate to add the with statement in this case? 


I think it is a good default approach to correlate the residuals of the mediators. There are most likely many predictors not included in the model influencing both mediators  that would then result in residual correlation. 

Lijin Zhang posted on Saturday, September 05, 2020  5:57 am



Hi, I am wondering what is the default prior for the covariance among endonenous factors in Bayesian analysis? Thanks! 


These can be found in output:tech1  it depends on how many factors. Also see page 775 in the Mplus User's Guide. 

Back to top 