Singularity in covariance matrix PreviousNext
Mplus Discussion > Growth Modeling of Longitudinal Data >
Message/Author
 Amanda Hugan-Kiss posted on Friday, December 01, 2006 - 3:03 pm
Hello Dr. Muthen: I have estimated a growth model in which I know that my covariance matrix is singular because I am including a person mean in the presence of a time-varying covariate. Other SEM programs will not estimate the model because of this singularity; I must instead invoke some form of ridge estimator. However, in Mplus I get the message:

WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES IS SINGULAR. PROBLEM INVOLVING VARIABLE XBAR.

THE MODEL ESTIMATION TERMINATED NORMALLY

Could you please briefly clarify what Mplus is doing to get a solution here? Does this involve a ridge estimator? Or some other sandwich-type approach?

Thank you very much for your time.

Amanda Hugan
 Bengt O. Muthen posted on Saturday, December 02, 2006 - 8:46 am
With a singular sample covariance matrix, Mplus automatically does a gentle ridging (adding epsilon to the diagonal).
 Amanda Hugan-Kiss posted on Sunday, December 03, 2006 - 8:26 pm
Dr. Muthen: Thank you for your kind reply. Very briefly, can results of ML with gentle ridging be treated as true ML estimates, or are there some limitations due to the adding of an artificial constant to the diagonal of sigma?

Thank you Sir.

Amanada Hugan
 Linda K. Muthen posted on Monday, December 04, 2006 - 8:34 am
Ridging is done on the indepedent variables in the model, and the model is estimated conditioned on the independent variables so the results are true ML.
 Amanda Hugan-Kiss posted on Saturday, December 09, 2006 - 3:08 pm
Thank you yet again. You are most kind. One final question, if I may.

Even if the model is estimated conditioned on the independent variables, would the addition of an artificial value to the variance of the exogenous variables potentially change the correlation structure of the entire set of measures? And might this in turn bias other model estimates relative to their population counterparts?

Thank you for your patience with my inquiries.

Amanda
 Linda K. Muthen posted on Sunday, December 10, 2006 - 12:07 pm
No, this would not happen. Only the diagonal of the xx covariance matrix is affected, not the yx or yy covariance matrices.
 csulliva posted on Thursday, October 08, 2009 - 9:25 am
I posted a similar question in another thread last week:

"Is it possible to include both a time stable [mean] and time varying [deviation score] component of the same covariate in a GMM or LCGA model?"

Linda K. Muthen posted on Saturday, October 03, 2009 - 8:56 am

I don't think this will work because of singularity among the covariates. You can try it out. You may have to exclude some deviations.

I did in fact get a warning that the covariance of the independent variables was singular. My question--given the discussion in this thread--is whether the estimates are useable? It sounds like the same problem and the previous responses seem to suggest that the ridge process helps to deal with this issue
 Linda K. Muthen posted on Friday, October 09, 2009 - 9:02 am
I would not use results when this message appears. I would attempt to get rid of it by, for example, getting rid of the first deviation score.
 csulliva posted on Friday, October 09, 2009 - 9:17 am
Would that create a situation where I was conditioning on time varying covariates at t2-t4, but not t1?
 Linda K. Muthen posted on Friday, October 09, 2009 - 9:49 am
It sounds that way.
 csulliva posted on Friday, October 09, 2009 - 10:23 am
Linda:

Thank you very much. If you might permit one more brief question...is the situation I'm posing qualitatively different from the one posted at the top of this thread? The premise and the warning message seem the same, so I am slightly confused about where the ridging process fits into the estimation and the viability of the estimates. I couldn't find anything in the technical appendices on this.

Thanks again.
 Bengt O. Muthen posted on Friday, October 09, 2009 - 5:12 pm
It is the same situation. Ridging may lead to poor estimates in some cases. We allow ridging for users who are sure that the estimates are good for their situation. The safest approach is to avoid singularity in the first place.
 Annie Desrosiers posted on Thursday, October 20, 2011 - 5:47 am
Hi Dr Muthen,
I'm running a piecewise LGM on 5 time points with covariates:

model:
i s1 | vioT1@0 vioT2@1 vioT3@1 vioT4@1 vioT5@1;
i s2 | vioT1@0 vioT2@0 vioT3@1 vioT4@2 vioT5@3;
s1@0;
i on ageT1;
s1 on lowT2 highT2;
s2 on lowT5r highT5r;

I have this message error :

WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES
IS SINGULAR. PROBLEM INVOLVING VARIABLE HIGHT5R.

THE MODEL ESTIMATION TERMINATED NORMALLY

THE STANDARD ERRORS FOR THE STANDARDIZED COEFFICIENT
COULD NOT BE COMPUTED DUE TO FAILURE OF THE
STANDARD ERROR COMPUTATION FOR THE H1 MODEL.

The results seam correct and coherent with precedent analysis on the same sample.

1-Can I trust the model even if I have a singular matrix ?
2-If not, how can I avoid the singularity ?

Thank you very much
 Linda K. Muthen posted on Thursday, October 20, 2011 - 1:51 pm
How did you create lowT5r and highT5r?
 Annie Desrosiers posted on Monday, October 24, 2011 - 5:45 am
LowT5r: A score of 1 was attributed to participants from our experimental group who had a low level of exposure to the programme from T3 to T5.

HighT5r: A score of 1 was attributed to participants from our experimental group who had a high level of exposure to the programme from T3 to T5.

Participants from the control group figure as the reference group.

Regards
 Bengt O. Muthen posted on Monday, October 24, 2011 - 8:32 pm
Do a Type=Basic run using only your covariates ageT1 lowT2 highT2. Look for unit correlations.
 Annie Desrosiers posted on Tuesday, October 25, 2011 - 8:44 am
I don't have unit correlations.

The warning says :
WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES
IS SINGULAR. PROBLEM INVOLVING VARIABLE HIGHT5R.

THE MODEL ESTIMATION TERMINATED NORMALLY

1-Can I trust the model even if I have a singular matrix ?

Thank you very much
 Bengt O. Muthen posted on Tuesday, October 25, 2011 - 2:53 pm
You should not go on without having resolved this problem. It sounds like some of your variables have a linear dependency among them. This would for instance happen if you use both the sum and each variable in the sum, or if you use 3 dummy variables for a 3-category variable.
 Stine Hoj posted on Sunday, May 04, 2014 - 11:44 pm
Dear Bengt and Linda,

I have constructed a four-class linear GMM from continuous indicators measured at three time points, with individually-varying times of observation.

I now wish to add covariates to the model to predict class membership and within-class variance in intercepts and slopes. I have tried to run the model with (1) a single binary covariate, gender; (2) a single continuous covariate, log income; and (3) both of these.

In each case I get the message, WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES IS SINGULAR.

I am hoping you might be able to help me understand what is causing this problem. My MODEL code is as follows:

%OVERALL%

i s | QOLA1 QOLA2 QOLA3 AT TimeT1 TimeT2 TimeT3;
i s ON Male LogInc1;
c#1 ON Male LogInc1;
c#2 ON Male LogInc1;
c#3 ON Male LogInc1;

%C#1%
i s;
[i*65]

%C#2%
i s;
[i*54]

%C#3%
i s;
[i*89]

%C#4%
i s;
[i*118]

Classes 2 & 3 have only a small membership (1.2% and 2.7% from a sample size of 1050). Is this likely to be creating these issues? Thank you in advance.
 Linda K. Muthen posted on Monday, May 05, 2014 - 9:37 am
This is most likely caused by no variability on the covariates within class. Ask for TECH12 in the OUTPUT command to get descriptives for the classes.
 Stine Hoj posted on Saturday, March 07, 2015 - 6:43 pm
Dear Linda,

I am still experiencing difficulties with the following error message:

WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES IS SINGULAR

I have a 3 class lienar GMM. The smallest class contains 9% of respondents (n=81).

I add a single binary (0,1) covariate to the model to predict class membership and within-class intercepts and slopes.

%OVERALL%
i s | Y1 Y2 Y3 AT T1 T2 T3;
c ON Educ1;
i s ON Educ1;

From TECH7, the mean of Educ1 in the three classes ranges from 0.44 to 0.61 and its variance ranges from 0.24 to 0.25.

It doesn't appear to me that the singularity is being caused by a lack of variability on the covariate within class, nor by linear dependencies between covariates (because I include only one). Can you provide any further guidance on this issue?
 Bengt O. Muthen posted on Sunday, March 08, 2015 - 9:27 am
Note that the independent variables also include T1, T2, and T3. Check that these are scored correctly.

If this doesn't help, please send input, output, data and license number to support@statmodel.com.
 Ebrahim Hamedi posted on Friday, July 20, 2018 - 2:26 am
hello
I wonder if mplus considers the relationship between a centered variable and its squared term as singular? My variables are not singular, but mplus says they are. My only guess is the squared term!?

many thanks
 Bengt O. Muthen posted on Friday, July 20, 2018 - 10:40 am
If you create

y = (x-xmean)*(x-xmean)

y and x shouldn't correlate 1. So I don't think that's the reason for the message. Send your output to Support along with your license number. Perhaps we can see what's going on - sending the data as well helps.
 Jordan posted on Tuesday, September 04, 2018 - 4:08 pm
Hello,

I tried to run a type=complex analysis, but was told that I was estimating two many parameters (for the number of groups/units my data has, which is 28).

To work around this, I created dummy variables for the 28 units. But when I run the analysis I get the following error:

WARNING: THE SAMPLE COVARIANCE OF THE INDEPENDENT VARIABLES
IS SINGULAR. PROBLEM INVOLVING VARIABLE UNIT27.


NO CONVERGENCE. SERIOUS PROBLEMS IN ITERATIONS.
CHECK YOUR DATA, STARTING VALUES AND MODEL.

I am not sure what to do from here.
 Bengt O. Muthen posted on Tuesday, September 04, 2018 - 5:51 pm
If you use only 27 dummies instead of 28, you should not have a problem - unless some of them are directly/mathematically related.
 Jordan posted on Wednesday, September 05, 2018 - 11:42 am
After removing one of the dummies, I received this error message:
THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES MAY NOT BE
TRUSTWORTHY FOR SOME PARAMETERS DUE TO A NON-POSITIVE DEFINITE
FIRST-ORDER DERIVATIVE PRODUCT MATRIX. THIS MAY BE DUE TO THE STARTING
VALUES BUT MAY ALSO BE AN INDICATION OF MODEL NONIDENTIFICATION. THE
CONDITION NUMBER IS 0.365D-17. PROBLEM INVOLVING THE FOLLOWING PARAMETER:
Parameter 101, PERF ON UNIT11

But it was followed by normal convergence message:

THE MODEL ESTIMATION TERMINATED NORMALLY
 Linda K. Muthen posted on Thursday, September 06, 2018 - 1:50 pm
Please send the output and your license number to support@statmodel.com.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: