Covariance among endonenous factors
Message/Author
 daniel posted on Tuesday, February 18, 2003 - 8:59 am
If I want to know the covariance among endogenous factors f2 f3 f4, I use program like this:

model: f1 by sbv26t1 sbv27t1 sbv29t1 ;
f2 by magsabt2 magvvbt2 magsvbt2 ;
f3 by mpmpcot2 mpmptrt2 mpmpint2 ;
f4 by mpbdevt2 height2 ;
f5 by sbv26t3 sbv27t3 sbv29t3 ;

f4 f3 f2 on f1 ;
f5 on f2 f3 f4 f1;

f3 with f4;
f2 with f3;
f2 with f4;

what problems will occur at interpretation of the covariance?
 Linda K. Muthen posted on Tuesday, February 18, 2003 - 9:40 am
The covariances among endogenous factors are residual covariances. If you ask for TECH4 in the OUTPUT command, you will obtain covariances and correlations for the latent variables in the model.
 Anonymous posted on Friday, January 14, 2005 - 8:13 am
Hello Linda,

I am running a path analysis with categorical and continuous mediating variables and a continuous final outcome. My question is about the use of with statement regarding one independent and one dependent variable. For example, if I have x1(dependent- predicted by other variable i.e not x2 ) and x2 (independent)

X1 with x2 - can this be estimated in mplus and does it make sense to estimate this correlation? It was suggested by modification indices in mplus

Thanks a lot.
 Linda K. Muthen posted on Friday, January 14, 2005 - 8:24 pm
I cannot understand your model from what you say. If x1 is regressed on x2 then x1 WITH x2 is not identified if this is what you are asking.
 Anonymous posted on Sunday, January 16, 2005 - 2:12 pm
Hi Linda,

Let me put my question as mplus program.

X1 on x2 x3 x4;
X5 on x3 x4;
Y on x5 x6;

X6 with x5;

Here is an independent variable x6 correlating with a dependent variable x5(its error). Once I correlate x6 with x5 the mdoel improves much and the path from x6 to y is no more significant. How do we interpret this? is it common in SEM?

Thanks
 BMuthen posted on Tuesday, January 18, 2005 - 3:44 pm
Without your WITH statement, x5 and x6 are uncorrelated in your model. It may be more natural to regress x5 on x6 than to correlate them. In path analysis as you are doing, if the model improves when you add a path, then this path is needed to reproduce the correlation matrix. I think this is a common phenomena.
 Annonymous posted on Tuesday, January 17, 2006 - 1:07 pm
With regard to the previous post: is it possible to correlate just the error and not the observed variable? i know that in AMOS, there is a difference between covarying the error terms of Y1 and Y2 with each other, compared to covarying Y1 and Y2.
 Annonymous posted on Tuesday, January 17, 2006 - 1:08 pm
PS Y1 and Y2 are continuous variables in a CFA.
 Linda K. Muthen posted on Tuesday, January 17, 2006 - 1:47 pm
If y1 and y1 are factor indicators, then you estimate their residual covariance as part of the model because they are endogenous variables. If they were exoogenous variables, their variance would be estimated.
 Annonymous posted on Wednesday, January 18, 2006 - 7:19 am
Ok. How would that be programmed? if a latent var (L) is measured by Y1 and Y2 and is endogenous to some other exogeneous factor (EX),it seems that if we wrote

EX on L;
L by Y1 Y2;
Y1 with Y2;

then we would covary the values of Y1 and Y2 and not just their residuals.
 Linda K. Muthen posted on Wednesday, January 18, 2006 - 7:40 am
The meaning of y1 WITH y2 depends on the context in which it is used in a model. In the model you show above, y1 and y2 are endogenous variables, therefore y1 WITH y2 represents a residual covariance. In the following model, y1 WITH y2 represents a covariance:

MODEL: y1 WITH y2;
 Alexander Kapeller posted on Friday, August 28, 2009 - 9:14 am
hi,
i am regressing an observed v. on one latent variable and several manifest variables. as i understand the default is that the correlation among the DV is estimated. but I don't get it in the model output. Only in tech 4. if i specify the correlations via with statements i get an error message about non -positive prod. matr. I am wondering why. thanks
 Bengt O. Muthen posted on Friday, August 28, 2009 - 6:08 pm
What are your DVs? You mention an observed variable. Is the latent variable that you mention a second DV? It is not clear what your model is. Note that you cannot identify both a residual covariance between two DVs and the regression of one on the other.
 Alexander Kapeller posted on Wednesday, September 02, 2009 - 1:25 pm
hi,

my dv are some observed and one latent variable

latent_var by x1 x2;
observ_var on observ_var1 observ_var2 latent_var;

thanks
 Linda K. Muthen posted on Wednesday, September 02, 2009 - 1:40 pm
The covariances between x1, x2, and observ_var are not estimated as the default.
 Kätlin Peets posted on Monday, September 07, 2009 - 6:57 am
Hello,

I have a question concerning a path model with two dependent variables. I understand that the residuals are allowed to covary (by default), but what does it actually mean? Does it mean that two variables share some common `cause` not explained by the specified predictors.

And, if I want to examine unique associations between the predictors and y1 (controlling for the y2 effect on y1), should I regress y1 on y2 (rather than estimating the covariance among the residuals)?

Thank you!
 Linda K. Muthen posted on Monday, September 07, 2009 - 9:40 am

You can include y2 as a predictor of y1 instead of having the residual covariance between y1 and y2.
 Tony Stoneriver posted on Wednesday, June 23, 2010 - 12:21 am
Dear Mr. and Mrs. Muthén,

I am conducting a SEM-model with 2 independent latent variables (measured by two parcels each) and 4 dependent latent variables (measured by two parcels each).

Since 3 of the dependent variables were measured with the same method (questionnaire), I allow for residual covariance by using the default option and restricting the residual covariance with the fourth dependent variable (lat_DV1 WITH lat_DV4@0; lat_DV2 WITH lat_DV4@0; lat_DV3 WITH lat_DV4@0;).

Now, I'm not sure how to visualize the relations in a SEM-model correctly. Do I have to include correlated residuals between the 6 manifest variables (parcels) or is it sufficient to include solely the correlation between the 3 latent dependent variables (which I get from TECH4)?

Kind regards
Tony
 Bengt O. Muthen posted on Wednesday, June 23, 2010 - 12:04 pm
I think there are more reasons that the 4 dependent factors have residual covariances than merely that 3 of them share the same method. Any left-out covariates predicting the 4, but left out in the model, would cause residual covariances.

You are right that indicators obtained by the same method may call for residual covariances among them. But this has to be modeled in a careful fashion. See for example the MTMM literature, for example in the CFA book by Tim Brown.
 Tony Stoneriver posted on Wednesday, June 23, 2010 - 11:19 pm
Dear Mr. Muthén,

If I include all residual covariances between the 4 latent dependent variables, the model fit worsen (but fit indices indicate still acceptable model fit) and the output shows, that the fourth dependent variable (which is measured with a different method) has no significant residual correlations with the other three latent dependent variables. Thus, in terms of parsimony I might left out the residual covariances with this fourth variable. Is that right?

I already studied the literature on method effects, but for my data none of the modeling techniques seem appropriate (besides, modeling method effects explicitly is not the aim of my article). However, I need an argument why the three latent dependent variables may correlate, since in some articles it is stated, that in "good models there are no correlated errors"). Do you have any advice?

Kind regards
Tony
 Bengt O. Muthen posted on Thursday, June 24, 2010 - 9:20 am
People may have different opinions about these matters; here are mine. I don't agree with the quote. I think a natural baseline model is one where residuals for the dependent variable factors do correlate. Why would a priori the independent variables be the only ones causing correlations between the 4 dependent variable factors? That's a very strong statement. Also, I am not a fan of "model trimming" where one deletes insignificant parameters - reporting that they are insignificant seems better to me.
 michela addis posted on Sunday, March 20, 2011 - 6:50 am
Dear Mr. and Mrs. Muthén,

I am conducting a path analysis (longitudinal data) with only observed variables.
I am wondering about the meaning of the estimate "Y1 WITH Y2" in this case. Does it refer to Y1 and Y2 covariance or to their residual covariance? Is there a way to let the residuals covariate without freeing the covariance of the 2 variables?
Thank you so much.
Best Regards,
michela
 Linda K. Muthen posted on Sunday, March 20, 2011 - 9:42 am
With endogenous variables WITH is a residual covariance. With exogenous variables, it is a covariance.
 michela addis posted on Monday, March 21, 2011 - 3:49 am

Best,
michela
 stevengroenez posted on Monday, June 13, 2011 - 11:38 pm
Dear Mr. Muthén,

From the preceding discussion i understand that, in the following model p represents a residual covariance between f1 and f2

f1 by y1 y2 y3;
f2 by y4 y5 y6;
f1 with f2 (p);

It however remains unclear to me how one labels a residual variance?

as f1 with f1 (q) generates an error message
and f1(q) would represent the factor variance?

Best,
Steven
 Linda K. Muthen posted on Tuesday, June 14, 2011 - 6:02 am
p is a covariance. q is a variance.
 stevengroenez posted on Thursday, June 16, 2011 - 7:44 am
As f1 and f2 are two endogenous variables, q is not a covariance but a residual covariance, no?

If so, how do i label the residual variance?

Steven
 Linda K. Muthen posted on Thursday, June 16, 2011 - 10:47 am
In a factor model, the factors are covariates. The factor indicators are regressed on the factors. Unless, there are other covariates in the model, q is a covariance.
 stevengroenez posted on Friday, June 17, 2011 - 12:05 am
Ok,

Actually, i want to specify some
error covariance restrictions on the structural part of a sem model. In order to do so i need to label the residual variances and covariances.

So in the following model p does represent a residual covariance between f1 and f2

f1 by y1 y2 y3;
f2 by y4 y5 y6;

f2 on f1;

f1 with f2 (p);

If in the above model p represents a residual covariance, how do i label the residual variance?

i tried f1 with f1 (q) but this generates an error message.

Best,
Steven
 Linda K. Muthen posted on Friday, June 17, 2011 - 8:10 am
F1 WITH f2 is not a residual covariance. It would not be identified as stated. Following is an example of a model with two residual variances and one residual covariance:

f1 by y1 y2 y3;
f2 by y4 y5 y6;
f1 f2 ON x;
f1 WITH f2 (a);
f1 (b);
f2 (c);
 Andres Fandino-Losada posted on Friday, August 24, 2012 - 2:03 am
Dear Dr. Linda Muthen.
1. How can I interpret the magnitude of the residual covariance (parameter "a") in both estimated and standardized units?
2. How can I relate "a" with the total covariance between the 2 variables?
3. Is it easier to explain "a" in terms of correlation between f1 and f2?
4. Can I call the latter, the residual correlation between f1 and f2 (which is not explained by "x")?
Thank you.
 Linda K. Muthen posted on Friday, August 24, 2012 - 8:15 am
1 and 3. You can look at the significance of the raw coefficient and at the standardized coefficient which is a correlation.

2. Look at the ratio of the residual covariance to the covariance.

4. Yes.
 Mark Boons posted on Friday, April 25, 2014 - 10:09 am
Dear Drs Muthen,

I would like to hear your opinion on the following:
Me and my co-authors are currently working on a paper in which we test a SEM model in MPlus (see below). As you can see, we allow the residual covariances between factors 1 and 2 to be freely estimated in our model as these two factors are theoretically argued to be “related, but distinct” and a review of previous research including these two constructs has (in all cases) found moderate correlations between these constructs ranging from .35 to .60. However, while one reviewer is satisfied with our justification of our decision to allow these residual covariances to be freely estimated, another reviewer argues that one should NEVER allow for residual covariances to be freely estimated among endogenous latent constructs as this will inflate model fit. While the overall fit of the model does improve by allowing the residual covariances to be freely estimated, none of the results of the hypothesized relationships changes in a way that ‘helps’ (they actually become less significant). I would really like to hear your expert opinions on this matter.

MODEL:
F1 BY x1-x4;
F2 BY x5-x7;
F3 BY x8-x12;
F4 BY x13-x16;
F1 ON x17;
F2 ON x18;
F3 ON f1 f2;
F4 ON f1 f2;
x19 ON f1 f2;
f1 WITH f2;

Mark
 Linda K. Muthen posted on Friday, April 25, 2014 - 10:54 am
I would include f1 WITH f2. There is very likely a correlation due to left-out predictors.

Do you mean x19 ON f1 f2 or f1 f2 ON x19?
 Mark Boons posted on Friday, April 25, 2014 - 11:55 am
Dear Linda,

Thank you for your quick reply. With regard to x19: In our model we have an outcome measure that captures 'activity'. Since this is a count variable with about 50% zeros, we include this as a variable instead of a single-item factor and estimate a negative binomial model by using 'count = x19(nb)' in the variables section. Please let me know if you would do this differently.

Again thank you for your feedback! This is greatly appreciated!

Mark
 Anneies De Vuyst posted on Wednesday, August 06, 2014 - 7:55 am
Dear,

I would like to hear your opinion on following situation: I want to do a path analysis with 6 latent variables. Before measuring different structural models, I did a CFA:

INPUT:
Org_inf BY SSI_1a - SSI_1g;
DS_inf BY SSI_2a - SSI_2g;
Coll_inf BY SSI_3a - SSI_3g;
ISB BY PB21 PB22 PB23 PB24;
NSA BY NSA1 NSA2 NSA3 NSA4;
RC BY RC1 RC2 RC3;

OUTPUT:
Number of free parameters: 108; Chi-square, degrees of freedom: 716.151, 419; RMSEA: 0.053; CFI: 0.923; TLI: 0.914; SRMR: 0.058.

Subsequently, I moved on to my first structural model where I use Org_inf, DS_inf, Coll_inf and ISB as exogenous variables and NSA and RC as endogenous variables. The MODEL-command is as follows:

Org_inf BY SSI_1a - SSI_1g;
DS_inf BY SSI_2a - SSI_2g;
Coll_inf BY SSI_3a - SSI_3g;
ISB BY PB21 PB22 PB23 PB24;
NSA BY NSA1 NSA2 NSA3 NSA4;
RC BY RC1 RC2 RC3;

NSA ON Org_inf DS_inf Coll_inf ISB;
RC ON Org_inf DS_inf Coll_inf ISB;

When I look at my fit indices and # of free parameters, it is remarkable that the results are identical with those of my CFA. According to Hair et al. (2010), this is because my structural model is a saturated one. How do I interpret this, and which actions can I take to resolve this issue? Can I somehow delete the residual covariance between my endogenous variables (NSA and RC)?

Annelies
 Linda K. Muthen posted on Wednesday, August 06, 2014 - 7:58 am
Model fit cannot be assessed for the saturated part of your model. You would need to put some restrictions on the structural model, for example, make some paths zero if this fits your research hypothesis.
 Anonymous  posted on Thursday, November 12, 2015 - 8:50 am
I have an SEM with an exogenous predictor, 3 latent endogenous mediators and a final outcome variable. What are the advantages or disadvantages to correlated the endogenous residuals of the latent variables in the structural model?
 Bengt O. Muthen posted on Thursday, November 12, 2015 - 5:52 pm
I think you refer to the residuals of the 3 latent mediators. If you add those you make the model fit better and may therefore avoid distorted parameter estimates due to misfit. You also learn about their sizes.
 Owis Eilayyan posted on Wednesday, November 23, 2016 - 7:34 am
Hello Dr. Muthen,

I am working on a longitudinal SEM model. However, many of my endogenous variables should not have a correlation among them. Therefore, I fixed the covariance between these variables to Zero (f1 with f2 @0). Is this a good solution? And does this change the other regression coefficient values?

Thank you,
Owis
 Bengt O. Muthen posted on Wednesday, November 23, 2016 - 12:03 pm
Q1. Yes.

Q2. Typically not - try it.
 Owis Eilayyan posted on Wednesday, November 23, 2016 - 5:05 pm
Thank you,

I checked, the values of regression coefficient did not change.

Thank you,
Owis
 Timothy Allen posted on Thursday, July 06, 2017 - 12:17 pm
I have a quick question in regards to the post from November 12, 2015. The response to that post indicates that modeling correlated errors among latent mediators will improve model fit and reduce the likelihood of distorted parameter estimates due to misfit. I'm wondering whether one also needs a theoretical rationale for modeling correlated residuals among latent mediators?

In a model I'm currently running, modification indices suggest adding a with statement between two latent mediators in order to improve fit. The literature indicates the two mediators should be highly related (though they are distinct constructs), but it doesn't suggest much of a reason for why their disturbances would be correlated. Is it still appropriate to add the with statement in this case?
 Bengt O. Muthen posted on Thursday, July 06, 2017 - 6:55 pm
I think it is a good default approach to correlate the residuals of the mediators. There are most likely many predictors not included in the model influencing both mediators - that would then result in residual correlation.