I'm doing an ESEM model using WLSMV and was wondering if the residual correlations from the "Residuals for Covariances/Correlations/Residual Correlations" matrix in Mplus can be interpreted in a meaningful way. Can these be interpreted as correlations and used to diagnose areas of misfit? Also, is there any way to get standardized residuals (z-scores) with WLSMV? Thank you,
They are not correlations. In principle one could get z scores for them, but it may not be easy to do by hand.
Perhaps a better approach is to use Modification indices which point more directly to the part of the model that needs improvement. MIs are available with ESEM.
Doug posted on Monday, November 14, 2011 - 10:52 am
I found invariance of the loadings/thresholds but non-invariance of the residual variances. I identified the residuals of two items that were non-invariant (partial invariance). In researching on how to interpret these findings I found several articles, for continuous data, which have found non-invariance of the uniqueness component (not residuals variances). In these articles they suggested non-invariance of uniqueness would mean that even though the instrument measured on the same scale units across groups i.e. equality of factor loadings, they did so with a different degree of reliability. They define reliability as:
How would this interpretation relate to non-invariance of residual variances in Mplus? I know from other postings that residual variance is viewed slightly different than uniqueness in Mplus, but would the same interpretation as above hold for residual variance and categorical data, i.e. that the items are measured with a different degree of reliability? Would the same formula hold, but just substitute residual variance for uniqueness?
Doug posted on Tuesday, November 15, 2011 - 4:12 pm
Thanks Linda. Do you think that the definition of reliability is correct based on how Mplus handles MGCFA with categorical data? In Mplus loadings and thresholds are constrained equal at the same time, and I am wondering how you would incorporate thresholds in the above formula.
Heike B. posted on Wednesday, November 16, 2011 - 4:01 am
I am struggling with the interpretation of the matrices from the RESIDUAL Output and the correlation matrix from the SAMPSTAT output.
My manifest path model contains two continous and two categorical mediator variables and two categorical final outcome variables, i.e. I have a model with categorical variables containing covariates. And I am using WLSMV.
From the user guide and previous postings here I supposed that
1. the non-diagonals of the SAMPSTAT correlation matrix contain sample probit residual correlations.
2. the diagonal of the SAMPSTAT correlation matrix contains variances for the continous variables.
3. the Model Estimated Covariances/Correlations/Residuals Correlation Matrix from the RESIDUAL output follows the same logic just that this matrix contains estimated values not sample values.
4. the Residuals for Covariances/Correlations/Residuals Correlation Matrix from the RESIDUAL output should contain the difference between the correlation matrix from the SAMPSTAT output and the Model Estimated Covariances/Correlations/Residuals Correlation Matrix from the RESIDUAL output. However in my output the differences do not match completely. They match if the element refers to categorical variables only or if they are the diagonal elements otherwise they don't.
What would be the right interpretation of all of these matrice's contents in my context?
Thresholds are not involved in reliability. With categorical outcomes, this reliability is for the latent response variable underlying the categorical outcome.
Doug posted on Friday, November 18, 2011 - 8:34 am
Thanks again. One last question. I found non-invariance of the loadings/thresholds and I identified the items that were contributing to the non-invariance. For some items, the thresholds are larger in group A than group B, and the corresponding factor loadings are smaller in group A than in group B. For two items, the opposite is true, the thresholds are larger in group B than group A, and the corresponding factor loadings are smaller in group B than in group A. Regardless of the group findings, the pattern seems to be that larger thresholds are associated with smaller factor loadings. Going through the Mplus articles I am trying to figure out what is the relationship between the loadings and thresholds in Mplus, in order to explain my findings. Can you provide some insight into the relationship between the loadings and thresholds in MPlus?
I don't think there necessarily is a relationship in general. A high threshold means that the item is difficult or, with attitude or behavioral items, extreme. Sometimes such items have large loadings, meaning that when your ability is high enough, the probability of getting it right goes up very quickly. On the other hand a difficult item may have a weak relationship with ability because the students weren't exposed to the content of the item, so high threshold and low loading.
Doug posted on Wednesday, November 23, 2011 - 11:31 am
Thanks Bengt. What did you mean by "students weren't exposed to the content of the item" in the above message? I am having a little difficulty following the example you have used in your answer. In my situation, I am using a social desirability responding measure. For 5 of the 40 items, the thresholds were higher and loadings were smaller in group A than in group B.
My example of a reason for a low loading concerned an item that is not strongly related to its factor. If the factor is ability and you have a low-ability student and a high-ability student, you won't see much of a difference in their performance if they haven't been exposed to the topic - this gives a low slope. That prototype can be translated to other situations.
H. R. posted on Tuesday, March 06, 2012 - 11:55 pm
I have a question regarding the Residual Correlation Matrices in the SAMPSTAT and RESIDUAL Outputs.
To my understanding the residual is the difference between the observed value and the estimated value. Hence there should be one residual per variable that is derrived from sample data and estimated data.
But why are there two different residual correlations? And how is the residual correlation calculated from sample data only in the SAMPSTAT section?
The RESIDUAL output gives the difference between the observed and model estimated correlations. The residual correlations given with SAMPSTAT are part of the sample statistics used for model estimation of a conditional model using WLSMV. They are not the same thing.
H. R. posted on Wednesday, March 07, 2012 - 10:11 am
Thank you, Linda.
Just for my understanding: how is the residual correlation in the SAMSPSTAT section defined?
In a conditional model, each variable is regressed on the covariates to obtain the probit regression coefficients. Each pair of variables is regressed on the covariates to obtain the residual correlations.
H. R. posted on Wednesday, March 07, 2012 - 8:40 pm
My question is about the Residual output for a CFA I am conducting using categorical data (WLSMV estimator). In the Mplus 7 user's guide the example given on pg 724-725 shows a heading for Standardized Residuals for Covariances/Correlations/Residual Correlations but I do not get this in my output. My questions are as follows:
1)I'm assuming the output under the heading Residuals for Covariance/Correlations/Residual Correlations is unstandardized. Is this correct? 2) Am I not getting this output because I am using WLSMV as an estimator and is it possible to get the standardized residuals?
3) What are the suggested unstandardized residual values (e.g., values >.10) to look for in determining areas of strain or to indicate that the model does not explain the corresponding sample correlation very well?
I am having a problem similar to Heike’s (from above). I am using ESEM with WLSMV in Mplus version 7.4 to evaluate a measurement model.
The analyses are being conducted on a set of imputed files. The variables are in a mixture of ordinal and continuous formats.
Because TECH10 and MODINDICES are not available, I am trying to obtain correlation residuals to identify possible item-level misfit
Like Heike's, my correlation residuals in the Residuals for Covariances/Correlations/Residuals Correlation Matrix portion of the RESIDUAL output exactly match hand-calculated differences between the observed and model expected correlation values, but only when both items in a pair are categorical. When one item is categorical and the other is continuous, the residual values in the output do not match hand-calculated differences.
Do you know how I can interpret the residuals for the mixed categorical/continuous item pairs? Or how I might find the correlation residuals for these item pairs?