Chi-Square Difference Testing Using the Satorra-Bentler Scaled Chi-Square

Chi-square testing for continuous non-normal outcomes has been discussed in a series of papers by Satorra and Bentler. A popular test statistic is the Satorra-Bentler scaled (mean-adjusted) chi-square, where the usual normal-theory chi-square statistic is divided by a scaling correction to better approximate chi-square under non-normality.

A little-known fact, however, is that such a scaled chi-square cannot be used for chi-square difference testing of nested models because a difference between two scaled chi-squares for nested models is not distributed as chi-square. Mplus issues a warning about this.

In discussions with Albert Satorra, Bengt suggested that Albert might want to figure out how to get a chi-square difference test for the Satorra-Bentler scaled chi-square and he did, producing the following book chapter which can be downloaded as a working paper (in postscript format) from his web site at http://www.econ.upf.es/~satorra/.

Satorra, A. (2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In Heijmans, R.D.H., Pollock, D.S.G. & Satorra, A. (eds.), Innovations in multivariate statistical analysis. A Festschrift for Heinz Neudecker (pp.233-247). London: Kluwer Academic Publishers.

The formulas in the paper are, however, complex and subsequently Albert and Peter Bentler wrote a paper showing that simple hand calculations using output from nested runs can give the desired chi-square difference test of nested models using the scaled chi-square. This paper is available as number 260 from the UCLA Statistics series at http://www.stat.ucla.edu/papers/preprints/260/

Difference Testing Using Chi-Square

Following are the steps needed to compute a chi-square difference test in Mplus using the MLM (Satorra-Bentler) or MLR chi-square.

1. Estimate the nested and comparison models using both ML and MLM or MLR.
2. Compute scaling correction factors if they are not printed in the output using the 4 chi-square values obtained in step 1. c0 is the scaling correction factor for the nested model, c1 is the scaling correction factor for the comparison model; t0 and t1 are the regular chi-square values; and tr0 and tr1 are the scaled chi-square values. If using Mplus Version 2, the scaling correction factors can be found in the outputs when MLM is used. In this case, skip to step 3.

        T0  =    1593.85
        TR0 =    271.094
        T1  =   1582.359
        TR1 =    266.909
   
        c0 = T0/TR0 = 1593.85/271.094 = 5.8793 
   
        c1 = T1/TR1 = 1582.359/266.909 = 5.9285
   

3. Compute the difference test scaling correction where d0 is the degrees of freedom in the nested model and d1 is the degrees of freedom in the comparison model.

         cd = (d0 * c0 - d1*c1)/(d0 - d1) 
            = (90*5.8793 - 89*5.9285)/(90-89) = 1.5005
   

Note that the ML chi-square is equal to the MLM or MLR chi-square times the scaling correction factor.
4. Compute the Satorra-Bentler scaled chi-square difference test (TRd) as follows:

         TRd = (T0 - T1)/cd 
             = (1593.851-1582.359)/1.5005 = 7.6587
   

Difference Testing Using the Loglikelihood

Following are the steps needed to compute a chi-square difference test based on loglikelihood values and scaling correction factors obtained with the MLR estimator.

1. Estimate the nested and comparison models using MLR. The printout gives loglikelihood values L0 and L1 for the H0 and H1 models, respectively, as well as scaling correction factors c0 and c1 for the H0 and H1 models, respectively. For example,

   	L0 = -2,606, c0 = 1.450 with 39 parameters (p0 = 39)
   	L1 = -2,583, c1 = 1.546 with 47 parameters (p1 = 47)
   

2. Compute the difference test scaling correction where p0 is the number of parameters in the nested model and p1 is the number of parameters in the comparison model.

   	      cd = (p0 * c0 - p1*c1)/(p0 - p1) 
   	         = (39*1.450 - 47*1.546)/(39 - 47) = 2.014
   

3. Compute the chi-square difference test (TRd) as follows:

       	 TRd = -2*(L0 - L1)/cd 
             	     = -2*(-2606 + 2583)/2.014 = 22.840