Strictly positive chi-square dif test... PreviousNext
Mplus Discussion > Latent Variable Mixture Modeling >
Message/Author
 Mike Stoolmiller posted on Wednesday, June 01, 2011 - 8:32 am
I'm trying to implement the Satorra-Bentler strictly positive chi-square difference test as described in Mplus Web note 12 for a factor mixture model with 4 classes. I have followed the procedure outlined in the web note except that to prevent Mplus from updating the start values, I have set convergence, mconvergence, logcriteria and rlogcriteria parameters to large values. Despite all this, Mplus takes a 2nd iteration in the EM algorithm regardless of how high I set the convergence parameters. If I put in start values with more then 3 significant digits, like 6 significant digits, which is considerably more work then using the svalues feature, Mplus still takes a 2nd iteration even though almost nothing changes. Is there something else that has to be set?
 Tihomir Asparouhov posted on Wednesday, June 01, 2011 - 9:08 am
Mike

Try miter=1 or send your example to support@statmodel.com

Tihomir
 Mike Stoolmiller posted on Wednesday, June 01, 2011 - 5:44 pm
I tried setting miter=1 and then Mplus tells me that an insufficient number of E steps have been taken and I don't get the MLR scaling factor that I need to compute the strictly positive test. I will send you the example and data.
 Alithe van den Akker posted on Wednesday, January 25, 2012 - 7:08 am
I am running into the same problem as presented above with estimating an m10 model. I have tried increasing the convergence, but there are iterations in the 'gradient' and 'quasi-newton' sections. Is there any more news on this issue? Thank you.
 Linda K. Muthen posted on Wednesday, January 25, 2012 - 7:42 am
Please send the relevant files and your license number to support@statmodel.com.
 Naomi Friedman posted on Wednesday, August 08, 2012 - 4:28 pm
I am also running into this same problem with type=random and algorithm=integration. When I set miter=1, it tells me "THE MODEL ESTIMATION DID NOT TERMINATE NORMALLY DUE TO AN INSUFFICIENT NUMBER OF E STEPS. INCREASE THE NUMBER OF MITERATIONS. ESTIMATES CANNOT BE TRUSTED" and it does not give me any loglikelihood or scaling factors.

Was there a resolution to this problem that would allow me to get the scaling factor for the M10 model?
 Linda K. Muthen posted on Thursday, August 09, 2012 - 10:37 am
Please send the output and your license number to support@statmodel.com.
 Nicholas Bishop posted on Wednesday, November 14, 2012 - 3:20 pm
Can anyone report back on the solution to this issue?

Here is the analysis statement I am using to produce the m10 model:

ANALYSIS:
type=mixture random;
ESTIMATOR=mlr;
PROCESSORS = 8;
algorithm=integration;
!convergence=100000000;
miter=1;

Thanks!
 Linda K. Muthen posted on Thursday, November 15, 2012 - 12:03 pm
You should not comment out the CONVERGENCE option.
 Nicholas Bishop posted on Thursday, November 15, 2012 - 2:01 pm
Hi Linda,
When I include the MITER statement, I received the following warning whether or not I have the convergence=100000000 statement included:

THE MODEL ESTIMATION DID NOT TERMINATE NORMALLY DUE TO AN INSUFFICIENT NUMBER OF E STEPS. INCREASE THE NUMBER OF MITERATIONS. ESTIMATES CANNOT BE TRUSTED.
 Linda K. Muthen posted on Friday, November 16, 2012 - 12:01 pm
Please send the output and your license number to support@statmodel.com.
 Sarah Herpertz posted on Friday, June 26, 2015 - 1:54 pm
Dear Muthéns,
I performed a LMS model with 2 groups (type=mixture random; algorithm=integration).
Subsequently, I computed a SB scaled Chi square difference test to compare a model without the interaction term (Model 0) to a model with the interaction term (Model 1). Thus the result of the SB scaled Chi square difference test was negative, I tried to estimate a third model (Model M10).
I followed web note 12, example 1.
My models are:
M0 – Model without interaction term, output: svalues;
M1 – Model with interaction term in each group;
M10 – 1) Model produced by svalues; 2) adding the interaction term in each group (f3 on f1xf2)
Question: Is this the right procedure for LMS models? I am asking because the scaling correction factor of M10 is smaller than the scaling correction factor of the M1 model – thus the chi-square is still negative. Mplus is fixing the interaction term automatically to zero.
Thank you very much.
 Bengt O. Muthen posted on Saturday, June 27, 2015 - 4:38 pm
If you have only a single XWITH interaction you only need the z-test for its slope.
 Sarah Herpertz posted on Monday, June 29, 2015 - 8:54 am
Thanks!
 Michelle Colder Carras posted on Friday, September 11, 2015 - 6:08 am
Good morning,

I'm also having a hard time figuring out where my strictly-positive chi-square difference test is going wrong. I'm doing an LCA with KNOWNCLASS , so my restricted M0 model is constraining the parameters to be equal between the two known classes (sex), while the M1 model is allowing the parameters of the 2 latent classes to differ by the KNOWNCLASS. I have tried using the starting values from the M0 output and setting CONVERGENCE=1000000000; MITER=1; or both. The resulting models either take iterations in the gradient or quasi-newton sections or fail to converge (INSUFFICIENT NUMBER OF E STEPS). Any help would be appreciated.

Thank you,

Michelle Carras
 Tihomir Asparouhov posted on Friday, September 11, 2015 - 8:54 am
Try using

miter=1; STARTS = 0; algo=ODLL; mconv=100000000;
 Michelle Colder Carras posted on Sunday, September 13, 2015 - 7:31 am
Dr. Asparahouv,

Thank you very much for that suggestion. I found that making that change fixed the problem with the analysis taking iterations, but some other problems emerged. I was unable to get Mplus to estimate more parameters when I cut and pasted the starting values from the M0 output where parameters were constrained between the two knownclasses (sex). I ended up removing the starting values from the 2nd knownclass (i.e., the G#2.C#1 and G#2.C#2 specific statements) to fix this problem. This produced a saddle point error message in estimation (WARNING: THE MODEL ESTIMATION HAS REACHED A SADDLE POINT OR A POINT WHERE THE OBSERVED AND THE EXPECTED INFORMATION MATRICES DO NOT MATCH.
AN ADJUSTMENT TO THE ESTIMATION OF THE INFORMATION MATRIX HAS BEEN MADE.) It did estimate the correct number of parameters and examination of the latent class patterns showed that thresholds differed. The computation of chidiff using this LL1 and cd value seemed to be feasible, but I wonder if taking the additional step of removing the starting values for the G2 knownclass invalidates or defeats the purpose of using STARTS=0 and taking other steps to ensure that the M10 model is correctly specified?

Thanks for your input,

Michelle
 Tihomir Asparouhov posted on Monday, September 14, 2015 - 9:17 am
If I am not mistaken, in the M10 model you need to remove the parameters constrains between the two knownclasses. If the log-likelihood value for the M10 model is not the same as the log-likelihood value for the M0 model the procedure is not valid.
 Marc Goodrich posted on Tuesday, February 23, 2016 - 12:58 pm
Dear Drs. Asparouhov and Muthen,

I have a question regarding computing the correct scaling correction factor for the strictly positive Satorra-Bentler chi-square for comparing a one-factor CFA to a two-factor CFA.

However, I am having the same problems as the previous poster on this thread.

Here is the relevant syntax I am using for the models:

M0 Model:

ANALYSIS:
TYPE=COMPLEX;
ESTIMATOR=MLR;
MODEL:
DV BY SDV1-SDV70 EDV1-EDV70;
OUTPUT:
STANDARDIZED SVALUES;

M1 MODEL:

ANALYSIS:
TYPE=COMPLEX;
ESTIMATOR=MLR;
MODEL:
SDV BY SDV1-SDV70;
EDV BY EDV1-EDV70;
OUTPUT:
STANDARDIZED;

M10 Model:

ANALYSIS:
CONVERGENCE=100000000;
TYPE=COMPLEX;
ESTIMATOR=MLR;
MODEL:
SDV BY SDV1-SDV70;
EDV BY EDV1-EDV70;
OUTPUT:
STANDARDIZED TECH5;

Also in the syntax for the M10 model I included the starting values for the parameters obtained in the output for the M0 model. I did not list them here in the interest of conserving space, as there are many indicators of each factor

Is this the correct way to specify the M10 model for the one-factor vs. two-factor comparison that I am interested in?

Thanks!
 Tihomir Asparouhov posted on Wednesday, February 24, 2016 - 5:33 pm
Maybe there is some confusion regarding the scale of the latent variables. I think it is easier to do this using the metric where the latent variables have a variance fixed to 1. Try these

MODEL M0:
DV BY SDV1-SDV70*1 EDV1-EDV70*1; DV@1;

MODEL M1:
SDV BY SDV1-SDV70*1; SDV@1;
EDV BY EDV1-EDV70*1; EDV@1;
SDV with EDV;

MODEL M10:
SDV BY SDV1-SDV70*(M0 sv); SDV@1;
EDV BY EDV1-EDV70*(M0 sv); EDV@1;
SDV with EDV*1;
 Marc Goodrich posted on Thursday, February 25, 2016 - 10:28 am
Dr. Asparouhov,

Thank you for the quick response. This seems like it should work, but I think I may be misunderstanding the M10 syntax.

When I specify each starting value from the M0 model, I get the following error messages:

THE ESTIMATED COVARIANCE MATRIX COULD NOT BE INVERTED. COMPUTATION COULD NOT BE COMPLETED IN ITERATION 1. CHANGE YOUR MODEL AND/OR STARTING VALUES.

THE MODEL ESTIMATION DID NOT TERMINATE NORMALLY DUE TO AN ERROR IN THE COMPUTATION. CHANGE YOUR MODEL AND/OR STARTING VALUES.

Here is the syntax I am using for the M10 model (x.xxx represents the starting values from the M0 model):

SDV BY SDV1*x.xxx
SDV2*x.xxx
...
SDV70*x.xxx; SDV@1;
EDV BY EDV1*x.xxx
EDV2*x.xxx
...
EDV70*x.xx; EDV@1;
SDV WITH EDV*1;

Because of this error, there is no log likelihood value to compare to the value from the M0 value.

I am guessing that I am just specifying the syntax incorrectly, but am not sure how it should be re-specified. Do you have any suggestions?

Thanks again!

Marc Goodrich
 Tihomir Asparouhov posted on Thursday, February 25, 2016 - 10:46 am
These indicators must be categorical then. Try
SDV WITH EDV*0.9999;
If this doesn't work send it to support@statmodel.com
 Marc Goodrich posted on Thursday, February 25, 2016 - 12:21 pm
Thank you. This suggestion worked, when used in conjunction with:
miter=1; STARTS = 0; algo=ODLL; mconv=100000000;

My apologies for not mentioning that the indicators were categorical previously.
 Ingrid Weigold posted on Tuesday, January 03, 2017 - 10:56 pm
Hello,

I am new to MPlus (although not SEM). I am using CFA to try to compare a single-factor model with 16 items and a first-order model with four correlating factors. I followed the helpful instructions in previous posts and on Web Notes 12 to calculate the strictly positive Satorra-Bentler chi-square (as I am using MLR estimation and have a negative scaling correction). However, I am having difficulty with the M10 model, as I am not certain what to do with parameters that are not in both models. Here is the input for Model M0 and Model M1:

MODEL M0

MODEL:
F1 by T1PGI01-T1PGI16;

MODEL M1

MODEL:
RC by T1PGI02 T1PGI08 T1PGI11 T1PGI16;
PL by T1PGI01 T1PGI03 T1PGI05 T1PGI10
T1PGI13;
UR by T1PGI06 T1PGI12 T1PGI14;
IB by T1PGI04 T1PGI07 T1PGI09 T1PGI15;

When creating M10, I know to copy the starting values for the parameters, but since I am using different factors, I am not certain how to do this correctly.

Thank you! Any help would be appreciated.
 Tihomir Asparouhov posted on Wednesday, January 04, 2017 - 9:35 pm
You should read this paper first as there are some limitations in the likelihood ratio test for that purpose

Hayashi, K., Bentler, P. M., & Yuan, K. (2007). On the likelihood ratio test for the number of factors in exploratory
factor analysis.Structural Equation Modeling, 14, 505–526.


Here is the proper setup for the strictly positive test

MODEL M0: F1 by T1PGI01-T1PGI16*1; F1@1;

MODEL M10: (put in the starting values from M0 results)
RC by T1PGI02* T1PGI08* T1PGI11* T1PGI16*;
PL by T1PGI01* T1PGI03* T1PGI05* T1PGI10*
T1PGI13*;
UR by T1PGI06* T1PGI12* T1PGI14*;
IB by T1PGI04* T1PGI07* T1PGI09* T1PGI15*;
RC-IB with RC-IB@1;
RC-IB@1;

MODEL M1:
RC by T1PGI02*1 T1PGI08 T1PGI11 T1PGI16;
PL by T1PGI01*1 T1PGI03 T1PGI05 T1PGI10
T1PGI13;
UR by T1PGI06*1 T1PGI12 T1PGI14;
IB by T1PGI04*1 T1PGI07 T1PGI09 T1PGI15;
RC-IB@1;
 Ingrid Weigold posted on Thursday, January 05, 2017 - 12:06 pm
Dear Dr. Asparouhov,

Thank you for your quick reply! I will look into the paper and make the corrections to the models. I appreciate your help as I learn this process!
 Ingrid Weigold posted on Thursday, January 05, 2017 - 9:16 pm
Hello again Dr. Asparouhov,

I've run the models using your suggestions. M0 and M1 look good. Unfortunately, when I run M10, I receive the following error message:

THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE
COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL.
PROBLEM INVOLVING THE FOLLOWING PARAMETER:
Parameter 19, RC BY T1PGI02
THE CONDITION NUMBER IS -0.249D+00.
THE ROBUST CHI-SQUARE COULD NOT BE COMPUTED.

I must not be specifying something correctly in the M10 syntax. (I did make sure to set the convergence to equal 100000000.) Do you know what might be causing this error message?

Thank you again for your help!
 Bengt O. Muthen posted on Friday, January 06, 2017 - 3:03 pm
Please send your output to Support along with your license number.
 Tihomir Asparouhov posted on Monday, January 09, 2017 - 12:23 pm
Ingrid you have done the inputs correctly and the outcome is not unexpected.

1. I calculated the regular SB chi-square using your output files and it is not negative. See
http://statmodel.com/chidiff.shtml
There is no point in computing the more complicated version given that result, particularly because of the fact the result is so decisive, chi-2 = 144 with 6 df, the more restricted model is rejected, and particularly because this result agrees with BIC and regular LRT.

2. The strictly positive SB chi-square assumes that the models are "close" to work properly. When the models are not close you can get a result as yours. Thus we would not recommend using the strictly positive SB test when the regular SB is not negative or in disagreement with other criteria.
 Ingrid Weigold posted on Monday, January 09, 2017 - 1:09 pm
Thank you Dr. Asparouhov! I must have calculated the original SB chi-square incorrectly. I apologize, as I double-checked the information I'd entered. I will look back at the link you've provided and work through it.

I appreciate the time you've taken to help me with this.
 Ingrid Weigold posted on Wednesday, January 11, 2017 - 3:01 pm
Hello again Dr. Asparouhov,

Somehow, I am still calculating a negative chi-square. I'm using the formula for calculating the difference based on MLR chi-square, and I wonder if I'm using the wrong correction factor when calculating cd. I'm using the "H0 Scaling Correction Factor for MLR" provided under Loglikelihood for both models. Should I be using the "Scaling Correction Factor for MLR" provided under Chi-Square Test of Model Fit instead?

Thank you again for all of your help!
 Tihomir Asparouhov posted on Wednesday, January 11, 2017 - 4:27 pm
Both methods yield the same result (always).

Here are the details of the computation:

H0
Number of Free Parameters 48
H0 Value -6634.193
H0 Scaling Correction Factor 1.1694

H1
Number of Free Parameters 54
H0 Value -6443.525
H0 Scaling Correction Factor 1.3330

Using the notation from
http://statmodel.com/chidiff.shtml

cd=(54*1.3330-48*1.1694)/(54-48)=2.6418
TRd=2*(-6443.525-(-6634.193))/2.6418=144.347
 Ingrid Weigold posted on Wednesday, January 11, 2017 - 7:04 pm
Thank you Dr. Asparouhov! I have one question. According to the notation on http://statmodel.com/chidiff.shtml, the first number for TRd is -2, not 2. This would then make the result negative. Am I looking at this correctly?
 Tihomir Asparouhov posted on Thursday, January 12, 2017 - 9:09 am
Ingrid a very basic check one your computations is the following: if the correction factors for H0 and H1 are identical - so will be cd. In addition if this correction factor is 1 then the result should be identical to the regular LRT.

You can use either of this versions
TRd = -2*(L0 - L1)/cd
or
TRd = 2*(L1 - L0)/cd

I used the second above.
 Ingrid Weigold posted on Thursday, January 12, 2017 - 11:15 am
OK, this makes sense. Thank you again for all of your time and help!
 Sebastian Siegler posted on Wednesday, May 31, 2017 - 5:00 am
I am running into the same problem as some other users. I am trying to compare a two-level random-intercept with a random-slope model using the Satorra-Bentler strictly positive chi-square difference test. To estimate the M10 I used the following ANALYSIS-command:
ANALYSIS: TYPE IS TWOLEVEL RANDOM;
ALGORITHM=INTEGRATION;
INTEGRATION=MONTECARLO (100);
CONVERGENCE = 100000000;
MITER = 1;
In the output, I get the error message "THE MODEL ESTIMATION DID NOT TERMINATE NORMALLY DUE TO AN INSUFFICIENT NUMBER OF E STEPS. INCREASE THE NUMBER OF MITERATIONS. ESTIMATES CANNOT BE TRUSTED." No correction factor b10 is reported.

May I ask you to help me with this? What do I have to change to get the required coefficients for M10? Thank you!
 Linda K. Muthen posted on Wednesday, May 31, 2017 - 6:51 am
It does not sound like the model converged. Please send the output and your license number to support@statmodel.com.
 Tao Yang posted on Friday, July 28, 2017 - 9:40 pm
My base model M0 is a 2-level model with random slopes of level 1 predictors. The alternative model M1 is to add level 2 predictors of the random slopes. Because the regular S-B test has negative value, I used the strictly positive S-B test and constructed the M10 model following web note 12, but the strictly positive S-B value is also negative.

The M10 model terminated normally, with a saddle point message (THE MODEL ESTIMATION HAS REACHED A SADDLE POINT OR A POINT WHERE THE OBSERVED AND THE EXPECTED INFORMATION MATRICES DO NOT MATCH. AN ADJUSTMENT TO THE ESTIMATION OF THE INFORMATION MATRIX HAS BEEN MADE).

Anything I can try to get a positive S-B value? Thanks!
 Bengt O. Muthen posted on Sunday, July 30, 2017 - 5:07 pm
Send your outputs to Support along with your license number.
 Lior Abramson posted on Wednesday, November 21, 2018 - 3:58 am
Hello,
I would like to make sure I understand how to use the strictly positive sattora-bentler difference test according to mplus note 12:

Specifically, this is what I did:

1)I ran model M0 with "svalues" output
2) then I ran model M1 as usual
3) finally, to create model M10, I copied the "svalues" output from model M0 and added it to model M1 (I did not erase model M1 specifications). I added the suffix *0 to the parameters that differ between M0 and M1, to make their starting values equal zero.

If I understand correctly, the only thing I need to take from the output of model M10 is the value of the Scaling Correction Factor, and then to compute the test written in note 12 manually, using the other values from the outputs of model M0 and M1. That is, I do not need to look at anything else in model M10. Is this correct?
(I ask because many things turn out different between the output of M10 model and the outputs of the other two models).

Also, I understand that the log likelihood of M10 and M0 model should be the same. The log liklihood itself came out differently in my output, but the MAXIMUM LOG-LIKELIHOOD VALUE FOR THE UNRESTRICTED (H1) MODEL was the same. What does it mean?



Thank you very much, I appreciate your help.
 Tihomir Asparouhov posted on Wednesday, November 21, 2018 - 3:05 pm
Did you see footnote 1 on page 5?
http://statmodel.com/examples/webnotes/SB5.pdf

The log-likelihood of M10 and M0 should be the same if it is setup correctly. If they are not the same the procedure should not be applied as it is not setup correctly. The additional parameters may not be set as *0. It depends on the model - see example 4 in the web note.
 Lior Abramson posted on Thursday, November 22, 2018 - 8:36 am
Thank you for your quick response!
I looked now at footnote 1 and accordingly tried the MCONVERGENCE=100000000; ALGO=ODLL; or the MITER=1; command. I actually got the exact fit indices and log-likelihood as in the model M1. Also, looking at the tech5 output, it seems that a lot of iterations occurred. Thus, I assume it is not the right direction. I don't think that example 4 fits my model, as I am simply trying to compare between two (bifactor) CFA models (with no groups).

Is there anything else that I could try?


I attach the beginning of the input of model M10 (shortened due to length limitation)

Thank you very much

!new parameters from M1
curis9r with explr9r*0;
poant9r with enths9r*0;
negaf9r with sad9r*0;

!output from svalues in M0
na BY negaf9r*1.054;
na BY frust9r*0.438;
na BY anger9r*0.646;
na BY sad9r*0.611;
........
 Tihomir Asparouhov posted on Thursday, November 22, 2018 - 12:38 pm
There should be no iterations in tech5 and tech8. You can try the rest of the options that control that LOGCRITERION, RLOGCRITERION, CONVERGENCE, MCCONVERGENCE, MUCONVERGENCE, MCITERATIONS, MUITERATIONS, ITERATIONS. If nothing works for you send it to support@statmodel.com
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: