Hello Dr. Muthen: I have estimated a growth model in which I know that my covariance matrix is singular because I am including a person mean in the presence of a time-varying covariate. Other SEM programs will not estimate the model because of this singularity; I must instead invoke some form of ridge estimator. However, in Mplus I get the message:
WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES IS SINGULAR. PROBLEM INVOLVING VARIABLE XBAR.
THE MODEL ESTIMATION TERMINATED NORMALLY
Could you please briefly clarify what Mplus is doing to get a solution here? Does this involve a ridge estimator? Or some other sandwich-type approach?
Dr. Muthen: Thank you for your kind reply. Very briefly, can results of ML with gentle ridging be treated as true ML estimates, or are there some limitations due to the adding of an artificial constant to the diagonal of sigma?
Thank you yet again. You are most kind. One final question, if I may.
Even if the model is estimated conditioned on the independent variables, would the addition of an artificial value to the variance of the exogenous variables potentially change the correlation structure of the entire set of measures? And might this in turn bias other model estimates relative to their population counterparts?
No, this would not happen. Only the diagonal of the xx covariance matrix is affected, not the yx or yy covariance matrices.
csulliva posted on Thursday, October 08, 2009 - 9:25 am
I posted a similar question in another thread last week:
"Is it possible to include both a time stable [mean] and time varying [deviation score] component of the same covariate in a GMM or LCGA model?"
Linda K. Muthen posted on Saturday, October 03, 2009 - 8:56 am
I don't think this will work because of singularity among the covariates. You can try it out. You may have to exclude some deviations.
I did in fact get a warning that the covariance of the independent variables was singular. My question--given the discussion in this thread--is whether the estimates are useable? It sounds like the same problem and the previous responses seem to suggest that the ridge process helps to deal with this issue
csulliva posted on Friday, October 09, 2009 - 10:23 am
Thank you very much. If you might permit one more brief question...is the situation I'm posing qualitatively different from the one posted at the top of this thread? The premise and the warning message seem the same, so I am slightly confused about where the ridging process fits into the estimation and the viability of the estimates. I couldn't find anything in the technical appendices on this.
It is the same situation. Ridging may lead to poor estimates in some cases. We allow ridging for users who are sure that the estimates are good for their situation. The safest approach is to avoid singularity in the first place.
You should not go on without having resolved this problem. It sounds like some of your variables have a linear dependency among them. This would for instance happen if you use both the sum and each variable in the sum, or if you use 3 dummy variables for a 3-category variable.
Stine Hoj posted on Sunday, May 04, 2014 - 11:44 pm
Dear Bengt and Linda,
I have constructed a four-class linear GMM from continuous indicators measured at three time points, with individually-varying times of observation.
I now wish to add covariates to the model to predict class membership and within-class variance in intercepts and slopes. I have tried to run the model with (1) a single binary covariate, gender; (2) a single continuous covariate, log income; and (3) both of these.
In each case I get the message, WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES IS SINGULAR.
I am hoping you might be able to help me understand what is causing this problem. My MODEL code is as follows:
i s | QOLA1 QOLA2 QOLA3 AT TimeT1 TimeT2 TimeT3; i s ON Male LogInc1; c#1 ON Male LogInc1; c#2 ON Male LogInc1; c#3 ON Male LogInc1;
%C#1% i s; [i*65]
%C#2% i s; [i*54]
%C#3% i s; [i*89]
%C#4% i s; [i*118]
Classes 2 & 3 have only a small membership (1.2% and 2.7% from a sample size of 1050). Is this likely to be creating these issues? Thank you in advance.
This is most likely caused by no variability on the covariates within class. Ask for TECH12 in the OUTPUT command to get descriptives for the classes.
Stine Hoj posted on Saturday, March 07, 2015 - 6:43 pm
I am still experiencing difficulties with the following error message:
WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES IS SINGULAR
I have a 3 class lienar GMM. The smallest class contains 9% of respondents (n=81).
I add a single binary (0,1) covariate to the model to predict class membership and within-class intercepts and slopes.
%OVERALL% i s | Y1 Y2 Y3 AT T1 T2 T3; c ON Educ1; i s ON Educ1;
From TECH7, the mean of Educ1 in the three classes ranges from 0.44 to 0.61 and its variance ranges from 0.24 to 0.25.
It doesn't appear to me that the singularity is being caused by a lack of variability on the covariate within class, nor by linear dependencies between covariates (because I include only one). Can you provide any further guidance on this issue?