We also noted the huge text files with TECH8, so now you always get TECH8 to the screen but you only get TECH8 to the output file if you request TECH8 in the OUTPUT command. We felt users would always want to monitor their progress on screen even when they did not want the technical details in the output file. How do you feel about this?
Jon Heron posted on Monday, November 20, 2006 - 7:25 am
I think that's a perfect solution!
Jon Heron posted on Monday, November 27, 2006 - 3:49 am
... unless you're running a BLRT
I use the initial bit of Tech14 output to establish whether I've located the optimal n/n-1 class models + then I stop and try a new OPTSEED if this is not the case.
If I can't turn off tech8 then the output whizzes past so quickly that I can't make this judgement.
Our most recent suggestion regarding TECH14 which is under TECH14 in the user's guide on the website is to first find a replicated solution without using TECH14. Then use OPTSEED and TECH14 in conjunction with the LRTSTARTS option.
Jon Heron posted on Tuesday, November 28, 2006 - 12:44 am
I find optimal/replicated solutions for n and n-1 class models without using Tech14 and then bring in tech14 to carry out the BLRT.
Unfortunately, not all optseeds that replicate the n-class model will recreate the optimal model for *both* n and n-1 classes.
One way to ensure that it does, appears to be to make sure the classes for the n-class model are ordered in increasing size, however this is not always possible (in my example at least).
A quicker alternative I have found is to ensure that the H0LL and H1LL values that are quoted at the start of a tech14 run correspond to the likelihoods for the replicated n and n-1 class models. I then know that the BLRT is going to be comparing the two models I want it to.
I am now experimenting with different LRTSTARTS/LRTBOOTSTRAP options to see if I can get it to stop after quoting these initial H0LL/H1LL.
The OPTSEED option is only for the k class model. I think you should modify the LRTSTARTS option. The first two numbers are for the k-1 class analysis. The last two are for the k class analysis. I would try
LRTSTARTS = 2 1 150 15;
If that does not work, I would increase the last two numbers. You may just have a difficult model. Some are tougher than others.
Jon Heron posted on Thursday, November 30, 2006 - 5:29 am
we may be going round in circles due to my lack of understanding, so thankyou for your patience.
I have found that the k-1 class model referred to with 'H0 Loglikelihood Value ' in the BLRT output is strongly dependent on the ordering of classes for the k class model (and hence on the OPTSEED which generates the k-class model).
The restriction that the largest class is last (as described in the manual) does not seem sufficient for my model - I have found that the only way to be certain of obtaining the correct k-1 class model is to ensure monotonically increasing class sizes within the k class model.
Hence I have come up with a way of running a quick BLRT to ensure that the correct models are being referred to, and then running a longer BLRT to estimate the p-value. This is quicker than attempting (and often failing) to have the k-classes in increasing order of size.
In our experience, ordering the classes is not necessary. If you would like, you can send your input, data, output, and license number to email@example.com and we can see why you need to do this.
Matt Moehr posted on Tuesday, March 06, 2007 - 8:43 am
I was wondering if/how this thread was resolved because I have a related question.
In my case, I used the strategy of choosing a model based on BIC and then confirming with BLRT (tech14). I think the correct solution is somewhere between 3 and 5 classes. The BIC for the four models M5, M4, M3, M2 in the same order: 4831, 4784, 4784, 4809. All of the models seemed to have stable class counts and were well replicated with starts=100 25.
When I started using BLRT, the results seem much less clear cut. I think the root of the problem is that the log likeliehood reported in the tech14 section of the output is not the same as the LL I got when I was using the BIC criteria. For example when running a model with 5 classes, the H0 model is a 4-class solution and tech14 shows:
H0 Loglikelihood Value -2139.25
But when I separately estimated the 4-class model, the replicated LL was -2128.5. So two questions: 1) Is it a problem that H0LL != LL(n-1) ?
2) Can I compute the LR test statistic based on the replicated LL values (kind of like a naive chi-square), and then compare that LR to the distribution of bootstrap draws?
linda beck posted on Thursday, August 07, 2008 - 9:26 am
Using the OPTSEED from am model without Tech14 is intended (sometimes) for not doing the k-class analysis again, when using tech14. But this k-class solution derived from the optseed should be the same as doing the k-class analysis again, am I right!? I ask this, because my k-class model is already very complex and time consuming. I directly want to compute BLRT without computing the stable k-class solution again, so optseed is the choice?
Is ordering of classes necessary when using tech14 and tech11? From your experience, any news since 2006? Unfortunately, I have solutions where the last class is extracted first, but I get the impression, that BLRT and LRT are not influenced by that.
Oddity I am wondering if any others have noticed: If I request Tech8 in the output and then remove it and rerun the model, it still prints to the output. Do I need to manually delete the original output file or should the new (much smaller output file) overwrite the old one?