I'm trying to implement the Satorra-Bentler strictly positive chi-square difference test as described in Mplus Web note 12 for a factor mixture model with 4 classes. I have followed the procedure outlined in the web note except that to prevent Mplus from updating the start values, I have set convergence, mconvergence, logcriteria and rlogcriteria parameters to large values. Despite all this, Mplus takes a 2nd iteration in the EM algorithm regardless of how high I set the convergence parameters. If I put in start values with more then 3 significant digits, like 6 significant digits, which is considerably more work then using the svalues feature, Mplus still takes a 2nd iteration even though almost nothing changes. Is there something else that has to be set?
I tried setting miter=1 and then Mplus tells me that an insufficient number of E steps have been taken and I don't get the MLR scaling factor that I need to compute the strictly positive test. I will send you the example and data.
I am running into the same problem as presented above with estimating an m10 model. I have tried increasing the convergence, but there are iterations in the 'gradient' and 'quasi-newton' sections. Is there any more news on this issue? Thank you.
I am also running into this same problem with type=random and algorithm=integration. When I set miter=1, it tells me "THE MODEL ESTIMATION DID NOT TERMINATE NORMALLY DUE TO AN INSUFFICIENT NUMBER OF E STEPS. INCREASE THE NUMBER OF MITERATIONS. ESTIMATES CANNOT BE TRUSTED" and it does not give me any loglikelihood or scaling factors.
Was there a resolution to this problem that would allow me to get the scaling factor for the M10 model?
Dear Muthéns, I performed a LMS model with 2 groups (type=mixture random; algorithm=integration). Subsequently, I computed a SB scaled Chi square difference test to compare a model without the interaction term (Model 0) to a model with the interaction term (Model 1). Thus the result of the SB scaled Chi square difference test was negative, I tried to estimate a third model (Model M10). I followed web note 12, example 1. My models are: M0 – Model without interaction term, output: svalues; M1 – Model with interaction term in each group; M10 – 1) Model produced by svalues; 2) adding the interaction term in each group (f3 on f1xf2) Question: Is this the right procedure for LMS models? I am asking because the scaling correction factor of M10 is smaller than the scaling correction factor of the M1 model – thus the chi-square is still negative. Mplus is fixing the interaction term automatically to zero. Thank you very much.
I'm also having a hard time figuring out where my strictly-positive chi-square difference test is going wrong. I'm doing an LCA with KNOWNCLASS , so my restricted M0 model is constraining the parameters to be equal between the two known classes (sex), while the M1 model is allowing the parameters of the 2 latent classes to differ by the KNOWNCLASS. I have tried using the starting values from the M0 output and setting CONVERGENCE=1000000000; MITER=1; or both. The resulting models either take iterations in the gradient or quasi-newton sections or fail to converge (INSUFFICIENT NUMBER OF E STEPS). Any help would be appreciated.
Thank you very much for that suggestion. I found that making that change fixed the problem with the analysis taking iterations, but some other problems emerged. I was unable to get Mplus to estimate more parameters when I cut and pasted the starting values from the M0 output where parameters were constrained between the two knownclasses (sex). I ended up removing the starting values from the 2nd knownclass (i.e., the G#2.C#1 and G#2.C#2 specific statements) to fix this problem. This produced a saddle point error message in estimation (WARNING: THE MODEL ESTIMATION HAS REACHED A SADDLE POINT OR A POINT WHERE THE OBSERVED AND THE EXPECTED INFORMATION MATRICES DO NOT MATCH. AN ADJUSTMENT TO THE ESTIMATION OF THE INFORMATION MATRIX HAS BEEN MADE.) It did estimate the correct number of parameters and examination of the latent class patterns showed that thresholds differed. The computation of chidiff using this LL1 and cd value seemed to be feasible, but I wonder if taking the additional step of removing the starting values for the G2 knownclass invalidates or defeats the purpose of using STARTS=0 and taking other steps to ensure that the M10 model is correctly specified?
If I am not mistaken, in the M10 model you need to remove the parameters constrains between the two knownclasses. If the log-likelihood value for the M10 model is not the same as the log-likelihood value for the M0 model the procedure is not valid.
I have a question regarding computing the correct scaling correction factor for the strictly positive Satorra-Bentler chi-square for comparing a one-factor CFA to a two-factor CFA.
However, I am having the same problems as the previous poster on this thread.
Here is the relevant syntax I am using for the models:
ANALYSIS: TYPE=COMPLEX; ESTIMATOR=MLR; MODEL: DV BY SDV1-SDV70 EDV1-EDV70; OUTPUT: STANDARDIZED SVALUES;
ANALYSIS: TYPE=COMPLEX; ESTIMATOR=MLR; MODEL: SDV BY SDV1-SDV70; EDV BY EDV1-EDV70; OUTPUT: STANDARDIZED;
ANALYSIS: CONVERGENCE=100000000; TYPE=COMPLEX; ESTIMATOR=MLR; MODEL: SDV BY SDV1-SDV70; EDV BY EDV1-EDV70; OUTPUT: STANDARDIZED TECH5;
Also in the syntax for the M10 model I included the starting values for the parameters obtained in the output for the M0 model. I did not list them here in the interest of conserving space, as there are many indicators of each factor
Is this the correct way to specify the M10 model for the one-factor vs. two-factor comparison that I am interested in?
I am new to MPlus (although not SEM). I am using CFA to try to compare a single-factor model with 16 items and a first-order model with four correlating factors. I followed the helpful instructions in previous posts and on Web Notes 12 to calculate the strictly positive Satorra-Bentler chi-square (as I am using MLR estimation and have a negative scaling correction). However, I am having difficulty with the M10 model, as I am not certain what to do with parameters that are not in both models. Here is the input for Model M0 and Model M1:
MODEL: F1 by T1PGI01-T1PGI16;
MODEL: RC by T1PGI02 T1PGI08 T1PGI11 T1PGI16; PL by T1PGI01 T1PGI03 T1PGI05 T1PGI10 T1PGI13; UR by T1PGI06 T1PGI12 T1PGI14; IB by T1PGI04 T1PGI07 T1PGI09 T1PGI15;
When creating M10, I know to copy the starting values for the parameters, but since I am using different factors, I am not certain how to do this correctly.
MODEL M10: (put in the starting values from M0 results) RC by T1PGI02* T1PGI08* T1PGI11* T1PGI16*; PL by T1PGI01* T1PGI03* T1PGI05* T1PGI10* T1PGI13*; UR by T1PGI06* T1PGI12* T1PGI14*; IB by T1PGI04* T1PGI07* T1PGI09* T1PGI15*; RC-IB with RC-IB@1; RC-IB@1;
MODEL M1: RC by T1PGI02*1 T1PGI08 T1PGI11 T1PGI16; PL by T1PGI01*1 T1PGI03 T1PGI05 T1PGI10 T1PGI13; UR by T1PGI06*1 T1PGI12 T1PGI14; IB by T1PGI04*1 T1PGI07 T1PGI09 T1PGI15; RC-IB@1;
I've run the models using your suggestions. M0 and M1 look good. Unfortunately, when I run M10, I receive the following error message:
THE STANDARD ERRORS OF THE MODEL PARAMETER ESTIMATES COULD NOT BE COMPUTED. THE MODEL MAY NOT BE IDENTIFIED. CHECK YOUR MODEL. PROBLEM INVOLVING THE FOLLOWING PARAMETER: Parameter 19, RC BY T1PGI02 THE CONDITION NUMBER IS -0.249D+00. THE ROBUST CHI-SQUARE COULD NOT BE COMPUTED.
I must not be specifying something correctly in the M10 syntax. (I did make sure to set the convergence to equal 100000000.) Do you know what might be causing this error message?
Ingrid you have done the inputs correctly and the outcome is not unexpected.
1. I calculated the regular SB chi-square using your output files and it is not negative. See http://statmodel.com/chidiff.shtml There is no point in computing the more complicated version given that result, particularly because of the fact the result is so decisive, chi-2 = 144 with 6 df, the more restricted model is rejected, and particularly because this result agrees with BIC and regular LRT.
2. The strictly positive SB chi-square assumes that the models are "close" to work properly. When the models are not close you can get a result as yours. Thus we would not recommend using the strictly positive SB test when the regular SB is not negative or in disagreement with other criteria.
Thank you Dr. Asparouhov! I must have calculated the original SB chi-square incorrectly. I apologize, as I double-checked the information I'd entered. I will look back at the link you've provided and work through it.
I appreciate the time you've taken to help me with this.
Somehow, I am still calculating a negative chi-square. I'm using the formula for calculating the difference based on MLR chi-square, and I wonder if I'm using the wrong correction factor when calculating cd. I'm using the "H0 Scaling Correction Factor for MLR" provided under Loglikelihood for both models. Should I be using the "Scaling Correction Factor for MLR" provided under Chi-Square Test of Model Fit instead?
Thank you Dr. Asparouhov! I have one question. According to the notation on http://statmodel.com/chidiff.shtml, the first number for TRd is -2, not 2. This would then make the result negative. Am I looking at this correctly?
Ingrid a very basic check one your computations is the following: if the correction factors for H0 and H1 are identical - so will be cd. In addition if this correction factor is 1 then the result should be identical to the regular LRT.
You can use either of this versions TRd = -2*(L0 - L1)/cd or TRd = 2*(L1 - L0)/cd
I am running into the same problem as some other users. I am trying to compare a two-level random-intercept with a random-slope model using the Satorra-Bentler strictly positive chi-square difference test. To estimate the M10 I used the following ANALYSIS-command: ANALYSIS: TYPE IS TWOLEVEL RANDOM; ALGORITHM=INTEGRATION; INTEGRATION=MONTECARLO (100); CONVERGENCE = 100000000; MITER = 1; In the output, I get the error message "THE MODEL ESTIMATION DID NOT TERMINATE NORMALLY DUE TO AN INSUFFICIENT NUMBER OF E STEPS. INCREASE THE NUMBER OF MITERATIONS. ESTIMATES CANNOT BE TRUSTED." No correction factor b10 is reported.
May I ask you to help me with this? What do I have to change to get the required coefficients for M10? Thank you!
It does not sound like the model converged. Please send the output and your license number to firstname.lastname@example.org.
Tao Yang posted on Friday, July 28, 2017 - 9:40 pm
My base model M0 is a 2-level model with random slopes of level 1 predictors. The alternative model M1 is to add level 2 predictors of the random slopes. Because the regular S-B test has negative value, I used the strictly positive S-B test and constructed the M10 model following web note 12, but the strictly positive S-B value is also negative.
The M10 model terminated normally, with a saddle point message (THE MODEL ESTIMATION HAS REACHED A SADDLE POINT OR A POINT WHERE THE OBSERVED AND THE EXPECTED INFORMATION MATRICES DO NOT MATCH. AN ADJUSTMENT TO THE ESTIMATION OF THE INFORMATION MATRIX HAS BEEN MADE).
Anything I can try to get a positive S-B value? Thanks!
Hello, I would like to make sure I understand how to use the strictly positive sattora-bentler difference test according to mplus note 12:
Specifically, this is what I did:
1)I ran model M0 with "svalues" output 2) then I ran model M1 as usual 3) finally, to create model M10, I copied the "svalues" output from model M0 and added it to model M1 (I did not erase model M1 specifications). I added the suffix *0 to the parameters that differ between M0 and M1, to make their starting values equal zero.
If I understand correctly, the only thing I need to take from the output of model M10 is the value of the Scaling Correction Factor, and then to compute the test written in note 12 manually, using the other values from the outputs of model M0 and M1. That is, I do not need to look at anything else in model M10. Is this correct? (I ask because many things turn out different between the output of M10 model and the outputs of the other two models).
Also, I understand that the log likelihood of M10 and M0 model should be the same. The log liklihood itself came out differently in my output, but the MAXIMUM LOG-LIKELIHOOD VALUE FOR THE UNRESTRICTED (H1) MODEL was the same. What does it mean?
The log-likelihood of M10 and M0 should be the same if it is setup correctly. If they are not the same the procedure should not be applied as it is not setup correctly. The additional parameters may not be set as *0. It depends on the model - see example 4 in the web note.
Thank you for your quick response! I looked now at footnote 1 and accordingly tried the MCONVERGENCE=100000000; ALGO=ODLL; or the MITER=1; command. I actually got the exact fit indices and log-likelihood as in the model M1. Also, looking at the tech5 output, it seems that a lot of iterations occurred. Thus, I assume it is not the right direction. I don't think that example 4 fits my model, as I am simply trying to compare between two (bifactor) CFA models (with no groups).
Is there anything else that I could try?
I attach the beginning of the input of model M10 (shortened due to length limitation)
Thank you very much
!new parameters from M1 curis9r with explr9r*0; poant9r with enths9r*0; negaf9r with sad9r*0;
!output from svalues in M0 na BY negaf9r*1.054; na BY frust9r*0.438; na BY anger9r*0.646; na BY sad9r*0.611; ........
There should be no iterations in tech5 and tech8. You can try the rest of the options that control that LOGCRITERION, RLOGCRITERION, CONVERGENCE, MCCONVERGENCE, MUCONVERGENCE, MCITERATIONS, MUITERATIONS, ITERATIONS. If nothing works for you send it to email@example.com