Singularity in covariance matrix PreviousNext
Mplus Discussion > Growth Modeling of Longitudinal Data >
Message/Author
 Amanda Hugan-Kiss posted on Friday, December 01, 2006 - 3:03 pm
Hello Dr. Muthen: I have estimated a growth model in which I know that my covariance matrix is singular because I am including a person mean in the presence of a time-varying covariate. Other SEM programs will not estimate the model because of this singularity; I must instead invoke some form of ridge estimator. However, in Mplus I get the message:

WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES IS SINGULAR. PROBLEM INVOLVING VARIABLE XBAR.

THE MODEL ESTIMATION TERMINATED NORMALLY

Could you please briefly clarify what Mplus is doing to get a solution here? Does this involve a ridge estimator? Or some other sandwich-type approach?

Thank you very much for your time.

Amanda Hugan
 Bengt O. Muthen posted on Saturday, December 02, 2006 - 8:46 am
With a singular sample covariance matrix, Mplus automatically does a gentle ridging (adding epsilon to the diagonal).
 Amanda Hugan-Kiss posted on Sunday, December 03, 2006 - 8:26 pm
Dr. Muthen: Thank you for your kind reply. Very briefly, can results of ML with gentle ridging be treated as true ML estimates, or are there some limitations due to the adding of an artificial constant to the diagonal of sigma?

Thank you Sir.

Amanada Hugan
 Linda K. Muthen posted on Monday, December 04, 2006 - 8:34 am
Ridging is done on the indepedent variables in the model, and the model is estimated conditioned on the independent variables so the results are true ML.
 Amanda Hugan-Kiss posted on Saturday, December 09, 2006 - 3:08 pm
Thank you yet again. You are most kind. One final question, if I may.

Even if the model is estimated conditioned on the independent variables, would the addition of an artificial value to the variance of the exogenous variables potentially change the correlation structure of the entire set of measures? And might this in turn bias other model estimates relative to their population counterparts?

Thank you for your patience with my inquiries.

Amanda
 Linda K. Muthen posted on Sunday, December 10, 2006 - 12:07 pm
No, this would not happen. Only the diagonal of the xx covariance matrix is affected, not the yx or yy covariance matrices.
 csulliva posted on Thursday, October 08, 2009 - 9:25 am
I posted a similar question in another thread last week:

"Is it possible to include both a time stable [mean] and time varying [deviation score] component of the same covariate in a GMM or LCGA model?"

Linda K. Muthen posted on Saturday, October 03, 2009 - 8:56 am

I don't think this will work because of singularity among the covariates. You can try it out. You may have to exclude some deviations.

I did in fact get a warning that the covariance of the independent variables was singular. My question--given the discussion in this thread--is whether the estimates are useable? It sounds like the same problem and the previous responses seem to suggest that the ridge process helps to deal with this issue
 Linda K. Muthen posted on Friday, October 09, 2009 - 9:02 am
I would not use results when this message appears. I would attempt to get rid of it by, for example, getting rid of the first deviation score.
 csulliva posted on Friday, October 09, 2009 - 9:17 am
Would that create a situation where I was conditioning on time varying covariates at t2-t4, but not t1?
 Linda K. Muthen posted on Friday, October 09, 2009 - 9:49 am
It sounds that way.
 csulliva posted on Friday, October 09, 2009 - 10:23 am
Linda:

Thank you very much. If you might permit one more brief question...is the situation I'm posing qualitatively different from the one posted at the top of this thread? The premise and the warning message seem the same, so I am slightly confused about where the ridging process fits into the estimation and the viability of the estimates. I couldn't find anything in the technical appendices on this.

Thanks again.
 Bengt O. Muthen posted on Friday, October 09, 2009 - 5:12 pm
It is the same situation. Ridging may lead to poor estimates in some cases. We allow ridging for users who are sure that the estimates are good for their situation. The safest approach is to avoid singularity in the first place.
 Annie Desrosiers posted on Thursday, October 20, 2011 - 5:47 am
Hi Dr Muthen,
I'm running a piecewise LGM on 5 time points with covariates:

model:
i s1 | vioT1@0 vioT2@1 vioT3@1 vioT4@1 vioT5@1;
i s2 | vioT1@0 vioT2@0 vioT3@1 vioT4@2 vioT5@3;
s1@0;
i on ageT1;
s1 on lowT2 highT2;
s2 on lowT5r highT5r;

I have this message error :

WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES
IS SINGULAR. PROBLEM INVOLVING VARIABLE HIGHT5R.

THE MODEL ESTIMATION TERMINATED NORMALLY

THE STANDARD ERRORS FOR THE STANDARDIZED COEFFICIENT
COULD NOT BE COMPUTED DUE TO FAILURE OF THE
STANDARD ERROR COMPUTATION FOR THE H1 MODEL.

The results seam correct and coherent with precedent analysis on the same sample.

1-Can I trust the model even if I have a singular matrix ?
2-If not, how can I avoid the singularity ?

Thank you very much
 Linda K. Muthen posted on Thursday, October 20, 2011 - 1:51 pm
How did you create lowT5r and highT5r?
 Annie Desrosiers posted on Monday, October 24, 2011 - 5:45 am
LowT5r: A score of 1 was attributed to participants from our experimental group who had a low level of exposure to the programme from T3 to T5.

HighT5r: A score of 1 was attributed to participants from our experimental group who had a high level of exposure to the programme from T3 to T5.

Participants from the control group figure as the reference group.

Regards
 Bengt O. Muthen posted on Monday, October 24, 2011 - 8:32 pm
Do a Type=Basic run using only your covariates ageT1 lowT2 highT2. Look for unit correlations.
 Annie Desrosiers posted on Tuesday, October 25, 2011 - 8:44 am
I don't have unit correlations.

The warning says :
WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES
IS SINGULAR. PROBLEM INVOLVING VARIABLE HIGHT5R.

THE MODEL ESTIMATION TERMINATED NORMALLY

1-Can I trust the model even if I have a singular matrix ?

Thank you very much
 Bengt O. Muthen posted on Tuesday, October 25, 2011 - 2:53 pm
You should not go on without having resolved this problem. It sounds like some of your variables have a linear dependency among them. This would for instance happen if you use both the sum and each variable in the sum, or if you use 3 dummy variables for a 3-category variable.
 Stine Hoj posted on Sunday, May 04, 2014 - 11:44 pm
Dear Bengt and Linda,

I have constructed a four-class linear GMM from continuous indicators measured at three time points, with individually-varying times of observation.

I now wish to add covariates to the model to predict class membership and within-class variance in intercepts and slopes. I have tried to run the model with (1) a single binary covariate, gender; (2) a single continuous covariate, log income; and (3) both of these.

In each case I get the message, WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES IS SINGULAR.

I am hoping you might be able to help me understand what is causing this problem. My MODEL code is as follows:

%OVERALL%

i s | QOLA1 QOLA2 QOLA3 AT TimeT1 TimeT2 TimeT3;
i s ON Male LogInc1;
c#1 ON Male LogInc1;
c#2 ON Male LogInc1;
c#3 ON Male LogInc1;

%C#1%
i s;
[i*65]

%C#2%
i s;
[i*54]

%C#3%
i s;
[i*89]

%C#4%
i s;
[i*118]

Classes 2 & 3 have only a small membership (1.2% and 2.7% from a sample size of 1050). Is this likely to be creating these issues? Thank you in advance.
 Linda K. Muthen posted on Monday, May 05, 2014 - 9:37 am
This is most likely caused by no variability on the covariates within class. Ask for TECH12 in the OUTPUT command to get descriptives for the classes.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: