How to interpret warning saddle point
Message/Author
 Charlotte Vrijen posted on Friday, August 12, 2016 - 7:33 am
I received this warning (see below) when trying to run a multilevel model with an MLR estimator. On this forum I read other comments on this type of warning and it was suggested that the results can still be interpreted (since in the end the model estimation ended normally) and it was not necessary to use an MLF estimator. But how can I interpret this warning? How do I know what type of adjustment has been made to the information matrix? What does the condition number mean? Thanks!

WARNING: THE MODEL ESTIMATION HAS REACHED A SADDLE POINT OR A POINT WHERE THE OBSERVED AND THE EXPECTED INFORMATION MATRICES DO NOT MATCH. AN ADJUSTMENT TO THE ESTIMATION OF THE INFORMATION MATRIX HAS BEEN MADE. THE CONDITION NUMBER IS -0.214D-02. THE PROBLEM MAY ALSO BE RESOLVED BY DECREASING THE VALUE OF THE MCONVERGENCE OR LOGCRITERION OPTIONS OR BY CHANGING THE STARTING VALUES OR BY USING THE MLF ESTIMATOR.

THE MODEL ESTIMATION TERMINATED NORMALLY
 Linda K. Muthen posted on Friday, August 12, 2016 - 10:50 am
If your results include standard errors, you can interpret them. Or you can try the suggestions in the error message.
 Charlotte Vrijen posted on Saturday, August 13, 2016 - 5:29 am
Thank you for your reply. But could you advice what starting values I would need to choose and how do I change them? In the manual I read something about adding an * followed by a starting value in the model specifications. Or would you recommend using the STARTS option in this case? Below I print my input. Would it be possible to indicate what changes should be made to the syntax to try different starting values and what ranges would you recommend me to try?

Thanks again!

Input:
INPUT INSTRUCTIONS

[...]

Variable:
Names are short_ID Time Day day_part Group ID INT
JOY SAD IRR WOR POS NEG INT_1 JOY_1 SAD_1 IRR_1 WOR_1 POS_1
NEG_1 afternoo evening morning;

Usevariables = INT INT_1 JOY_1 SAD_1 IRR_1 WOR_1 POS_1 NEG_1
afternoo evening;
Missing are ALL (-999);
Within = INT_1 JOY_1 SAD_1 IRR_1 WOR_1 POS_1 NEG_1 afternoo
evening;

Cluster = short_ID;

DEFINE:
CENTER INT_1 JOY_1 SAD_1 IRR_1 WOR_1 POS_1 NEG_1 (GROUPMEAN);

Analysis:

Type = twolevel random;
Estimator = MLR;

Model:
%within%
sINT| INT on INT_1;
sJOY| INT on JOY_1;
sIRR| INT on IRR_1;
sWOR| INT on WOR_1;
sPOS| INT on POS_1;
sNEG| INT on NEG_1;
INT on afternoo evening;

%between%
 Linda K. Muthen posted on Saturday, August 13, 2016 - 6:44 am
I would not use starting values. For further help, please send the output and your license number to support@statmodel.com.
 Charlotte Vrijen posted on Saturday, August 13, 2016 - 11:48 am
I just sent the output and licence number. Many thanks in advance!
 Charlotte Vrijen posted on Wednesday, October 12, 2016 - 2:28 am

WARNING: THE MODEL ESTIMATION HAS REACHED A SADDLE POINT OR A POINT WHERE THE OBSERVED AND THE EXPECTED INFORMATION MATRICES DO NOT MATCH. AN ADJUSTMENT TO THE ESTIMATION OF THE INFORMATION MATRIX HAS BEEN MADE. THE CONDITION NUMBER IS -0.214D-02. THE PROBLEM MAY ALSO BE RESOLVED BY DECREASING THE VALUE OF THE MCONVERGENCE OR LOGCRITERION OPTIONS OR BY CHANGING THE STARTING VALUES OR BY USING THE MLF ESTIMATOR.

THE MODEL ESTIMATION TERMINATED NORMALLY

And after sending my output you recommended that I could use the results because the output contained standard errors. I followed your advice to ignore the warning. Now me and my supervisors are wondering whether it would not be better to use the same information matrix estimation for all analyses, so is there a way we could make the same adjustment to the information matrix for our other multilevel analyses? We feel they would be better comparable in this way. We have several analyses that have the same input as described above (only different variables) and while several analyses do result in the warning and thus the information matrix is adjusted, others do not, with the result that we now use two different information matrix estimation methods whereas we would like to use the same method for all analyses.

Is this possible in Mplus? Many thanks!
 Tihomir Asparouhov posted on Wednesday, October 12, 2016 - 4:14 pm
There are 4 suggestions in the last sentence

THE PROBLEM MAY ALSO BE RESOLVED BY DECREASING THE VALUE OF THE MCONVERGENCE OR LOGCRITERION OPTIONS OR BY CHANGING THE STARTING VALUES OR BY USING THE MLF ESTIMATOR.

You should try to use those to resolve the problem before proceeding.

But if the message persists the last suggestion estimator=mlf can be used to avoid the problem and have the same method be used for all SE computations.
 Charlotte Vrijen posted on Wednesday, October 12, 2016 - 11:56 pm
Thank you for your reply. Of course I noticed the 4 suggestions, but when I asked on this forum what the best strategy would be (since I am not a very experiences Mplus user) and sent my output files to the Mplus support desk I was adviced: "You receive standard errors. They are in the column S.E. Therefore, you can ignore the message." I also read that the MLF estimator overestimated the standard errors and on the forum I noticed that people were adviced against using it in cases comparable to mine. Because (in fact I think it was your article) I also read about the possibility of using a simplified version of the actual information matrix, I figured that this may be another way to solve the problem. For example to assume orthogonality (since there are simulation studies showing that in the type of analyses I am doing this does not have a large effect on the results).

I hope this is a little bit clear! So my question is whether/how I can use a simplified version of the information matrix in my multilevel analyses (for an example of one of my inputfiles see above) and also whether you would recommend this. Or would you rather recomend something else? (If it is changing the starting values after all could you please advice me on which values to choose and how?)

 Tihomir Asparouhov posted on Thursday, October 13, 2016 - 2:41 pm
There are three things to keep in mind.

1) Figure out the reason for the problem

2) Decide if you should stick with that model or you should change the model

3) If you decide to keep the model what to do to improve the estimation.

For 1) looking at your input this is almost certainly caused by an effect that is not random (near zero variance .. remove it ... use fixed effect) or by two random effects that have a correlation of +-1 (I would use estimator=bayes for this problem)

2) Given that this is a part of a bigger set of model runs I assume you don't want to change the model but that will be most desirable, however the ML estimator tends to get into singular random effect matrices simply because of small number of clusters and the Bayes estimator switch is best then.

3) The saddle point message is 75% of the time incomplete convergence most likely you can resolve using miter=50000; mconv=0.000001; The slow convergence is due to between level singularities. About 25% it is caused by unidentified model.

You best bet apart from that is the MLF estimator. The adjustment amounts to removing negative eigenvalues - not something you can do when there is no problem.

You can add starting values using simpler runs that converge such as models with one or two fewer random effects (I don't think this is likely to solve the problem though) or use starts.

To summarize estimator = bayes or mlf is the best bet or modify the model.
 Tihomir Asparouhov posted on Thursday, October 13, 2016 - 2:42 pm
One last note - if none of the above improves the situation - it is safe to use the standard errors that Mplus gives you.
 Charlotte Vrijen posted on Monday, October 17, 2016 - 7:51 am
I will try the options you suggested. Thank you very much for this detailed advice!
 rgm smeets posted on Wednesday, December 19, 2018 - 10:20 pm
Dear,

I received the same warning as described earlier in this topic (about the saddle point) when using estimator MLR. Nevertheless, in my output there are standard errors given and the model estimation terminated normally. I assume that this tells me I can just interpret the model (results) and ignore the warning (it seems like you advised that before).

Thank you!!
 Bengt O. Muthen posted on Thursday, December 20, 2018 - 4:10 pm
You may first want to sharpen the convergence criterion Mconvergence to see if you get a better logL that avoids the message.
 rgm smeets posted on Thursday, December 20, 2018 - 10:47 pm
Ok thank you once more mister Muthen. I will try to sharpen the mconvergence. What if I do not get a better LogL? Is it then safe to stick with the MLR and ignore the warning (given that the model estimation terminates normally and produces standard errors)
 Bengt O. Muthen posted on Friday, December 21, 2018 - 3:46 pm
Most likely.
 rgm smeets posted on Saturday, December 22, 2018 - 4:53 am
Ok. Then I think I will keep my model as it is an interesting one (content-wise), despite the warning.

It is still a bit frustrating that I can't explain where the warning about the saddle point comes from. I tried every action you mentioned in your webnote on saddle point:

1. decreasing value of MCONVERGENCE
2. using more INTEGRATION points
3. using different starting values and randomized STARTS options
4. using INFORMATION in ANALYSIS

But none of these actions removes the warning on the saddle point (and each time the same best LogL and same model results are given).The only action that avoids the warning is using estimator MLF, but you advised me before not to use it due to high standard errors.

Thanks a lot!!
 Bengt O. Muthen posted on Sunday, December 23, 2018 - 5:23 pm
Accept it.
 rgm smeets posted on Wednesday, February 06, 2019 - 3:08 am
Like Charlotte Vrijen, the warning about the saddle point includes a negative condition number (-0.315D-05). Also the condition number for the information matrix (ratio of smallest to largest eigenvalue) is negative (-0.315E-05). Should I worry about this? I notice that many articles do not even report the condition number. Thank you!
 Bengt O. Muthen posted on Wednesday, February 06, 2019 - 5:20 pm
 Aurelie Lange posted on Saturday, May 09, 2020 - 9:43 am
Dear Prof Muthen,

I am analysing multiply imputed data. I get the warning about the saddle point:
"THE MODEL ESTIMATION HAS REACHED A SADDLE POINT OR A POINT WHERE THE OBSERVED AND THE EXPECTED INFORMATION MATRICES DO NOT MATCH."
This warning is present for most of the datasets, but not all.

In the messages above, you suggest that this warning can be ignored if s.e. is printed. Is this also true in my case?
I was thinking that, perhaps, the s.e. would be calculated only based on the datasets without the warning. In that case, you wouldn't know whether the datasets issuing the warning also printed s.e. or not and you cannot know whether it is ok to ignore the warning.

Can I ignore the warning if I have s.e., but the warning is only issued by some of my datasets?

Thank you!
 Aurelie Lange posted on Monday, May 18, 2020 - 11:01 pm
Dear Dr Muthen,

I hope you and your family are well in these times.
I don't want to be rude, but I was wondering whether you could advise me on the above problem, which I posted 10 days ago. It would be hugely helpful to know how to proceed.

Thank you so much!

kind regards,
Aurelie
 Bengt O. Muthen posted on Tuesday, May 19, 2020 - 2:56 pm
Often, this is avoided by sharpening (lowering) the convergence criterion e.g. by:

mconvergence = 0.000005;