Replicating the best loglikelihood
Message/Author
 Gareth posted on Saturday, November 13, 2010 - 6:41 am
Should the best loglikelihood be replicated to two decimal places, or is the nearest whole number sufficient?

For example:

Final stage loglikelihood values at local maxima, seeds, and initial stage start numbers:
-46462.753 514326 325
-46462.754 635245 121
-46462.754 991329 182
-46462.756 227563 63
-46462.757 930323 267
-46462.757 146900 391
...
 Bengt O. Muthen posted on Saturday, November 13, 2010 - 4:01 pm
If this is obtained by numerical integration, you don't need all 3 decimals to agree. When uncertain, use OPTSEED to run contending solutions and see how different the parameter estimates are - often they are essentially the same in these situations.
 John Woo posted on Wednesday, September 16, 2015 - 11:49 am
Hi Dr. Muthen, I am seeing a rather strange case of log likelihood replication. No matter whether start=100 or start=1000, I see this:
-706.710 846194 93
-708.841 311214 64
-708.841 915107 54
-708.841 354559 73
-708.841 650371 14
-708.841 392418 28
-708.841 804561 59
-708.841 902278 21
-708.841 565819 65
-708.842 207896 25

I am running GMM, and the results associated with the second best LL (-708.841, which is replicated) is very reasonable. But the result associated with the best but non replicated LL (-706.841) has negative residual variances for my growth factors and is problematic. What should I make of this? Why is there always that gap between the best and the second-best no matter the number of random starts? Is there a case where using the second best but replicated LL is acceptable? Thank you in advance.
 Bengt O. Muthen posted on Wednesday, September 16, 2015 - 5:24 pm
An inadmissible estimated model like your first one should be ignored. If a variance is zero the loglikelihood can shoot up.
 chioma nwaru posted on Thursday, January 12, 2017 - 1:24 am
Dear Bengt & Linda,

I have been running a LCGM on a dataset of N = 1612. So far, the best loglikehood replicated for classes 2 to 4. But for the 5th cIass solution, the best loglikelihood did not replicate even after several increase of the starting values. The latest was 1500 380, still I get same warning that the solution may not be trustworthy. Does it mean that the 4-class solution is sufficient and that the data do not support an extra class?

Best regards,
Chioma
 Linda K. Muthen posted on Thursday, January 12, 2017 - 6:11 am
You can try more starts, for example, 4000 1000 or even more. But it may be that you are trying to extract too many classes. You should also include the interpretability of the classes.
 Line Auneau posted on Thursday, July 26, 2018 - 2:52 am
Good morning Dr Muthen, I am running GMM and I do not understand how many time the best loglikelihood should be replicated for have this message : "THE BEST LOGLIKELIHOOD VALUE HAS BEEN REPLICATED."

In this example, the best likelihood has been replicated once, but the message says that she has been replicated.
STARTS = 80 10;
STITERATIONS = 20;

-3769.994 650371 14
-3770.005 462953 7
-3804.222 68985 17
-3804.534 533738 11

6 perturbed starting value run(s) did not converge.

THE BEST LOGLIKELIHOOD VALUE HAS BEEN REPLICATED. RERUN WITH AT LEAST TWICE THE
RANDOM STARTS TO CHECK THAT THE BEST LOGLIKELIHOOD IS STILL OBTAINED AND REPLICATED.

In another exemple, the 2 first best loglikelihood are also close but in this case, the best loglikelihood is not replicated.
STARTS = 80 10;
STITERATIONS = 20;

-3966.466 784664 75
-3967.537 966014 37
-3968.095 127215 9
-3974.316 967237 48
-3974.316 963053 43

5 perturbed starting value run(s) did not converge.

WARNING: THE BEST LOGLIKELIHOOD VALUE WAS NOT REPLICATED. THE
SOLUTION MAY NOT BE TRUSTWORTHY DUE TO LOCAL MAXIMA. INCREASE THE
NUMBER OF RANDOM STARTS.

 Bengt O. Muthen posted on Thursday, July 26, 2018 - 6:09 pm
Two logL that are the same is sufficient. But there are different tolerances used when numerical integration is involved - that's probably what's happening in the first instance.

Personally, I think you want to have say 5 of the same logL (allowing for tolerance).
 Line Auneau posted on Tuesday, July 31, 2018 - 6:53 am
You talk about tolerance in your post. How is defined tolerance in Mplus ? And it is possible to change this tolerance to improve replication of logLL ?