LR test of nested models with count data PreviousNext
Mplus Discussion > Categorical Data Modeling >
Message/Author
 James W. Shaw posted on Wednesday, June 04, 2008 - 2:11 am
I am using CFA to compare nested one- and two-factor models for a questionnaire. Since the indicators are counts, I believe it would be appropriate to estimate loadings using Poisson or zero-inflated Poisson. I would prefer to fit the models using MLR instead of ML since the data may be heteroscedastic.

I want to perform a significance test to compare the models. Although a robust LR test can be formulated with MLR estimates when indicators are continuous, this is not possible with count indicators (see www.statmodel.com/chidiff.shtml). However, a post by Dr. Bengt Muthen suggests that the LR test can be performed using ML estimates with no Satorra adjustment to compare nested models with count indicators.

I see the following as my best options:

1. Use AIC to perform a qualitative comparison.
2. Perform an LR test using LR statistics derived with the ML (not MLR) estimator. Standard errors for loadings would be derived using the ML or MLR estimator. The latter would be inconsistent with the model LR test.
3. Treat the indicators as continuous and use the Satorra-adjusted LR test in conjunction with the MLR estimator to compare the models. This would probably be a more grievous error than treating the indicators as counts and using the ML estimator.

I would appreciate any suggestions you may have for how I should proceed.
 Linda K. Muthen posted on Wednesday, June 04, 2008 - 8:48 pm
MLR is available for a CFA with count factor indicators.
 James W. Shaw posted on Wednesday, June 04, 2008 - 10:14 pm
Dr. Muthen:

Thanks for your reply. I am sorry if I did not make myself clear.

My problem relates to the performance of model comparison tests in the context of CFA with count indicators. Chi-square difference tests are not feasible since Mplus does not report chi-square statistics when indicators are counts. (You have discussed this issue in other posts.)

I know that the MLR estimator can be used to perform CFA with count indicators. However, I do not believe that the Satorra adjustment cited in my earlier post can be used to facilitate LR model tests following MLR estimation when the indicators are counts. Specifically, I believe that the adjustment can only be used when indicators are continuous.

If I am correct in my understanding of the Satorra adjustment, then I would have to:

(1) perform the LR test (comparing 1-factor vs. 2-factor) using log-likelihoods derived with the ML estimator, OR
(2) treat the indicators as continuous and perform the estimation using MLR.
 Linda K. Muthen posted on Wednesday, June 04, 2008 - 10:36 pm
You can do difference testing using -2 times the loglikelihood difference. Scaling correction factors are given.
 James W. Shaw posted on Wednesday, June 04, 2008 - 10:50 pm
Dr. Muthen:

A discussion of the Satorra adjustment available at the Mplus website (see http://www.statmodel.com/chidiff.shtml), refers to "Chi-square testing for continuous non-normal outcomes." This led me to believe that I could not apply the Satorra adjustment when fitting models with MLR and count indicators.

Additionally, when I did apply the Satorra adjustment, my calculated cd was negative. Consequently, my scaled LL difference was negative. Since a negative chi-square statistic is not feasible, I presumed that the problem was due to an incompatibility of the Satorra adjustment with count indicators.
 Linda K. Muthen posted on Thursday, June 05, 2008 - 12:19 pm
The web page applies to MLM, MLR, and WLM. MLM is Satorra Bentler which is for continuous outcomes. We started out with MLM so this may be confusing because we did not change the introduction. You can use difference testing for MLR with count outcomes. One problem with Satorra Bentler and difference testing is that they sometimes don't work well as in your case. Try MODEL TEST which does a Wald test.
 James W. Shaw posted on Friday, June 06, 2008 - 11:30 am
Dr. Muthen:

I have read that the one-factor model can be nested within a two-factor model by constraining the correlation between factors in the latter to equal 1.0. I would expect the constrained two-factor model to yield the same fit derived when fitting a one-factor model. Given this, I believe that a simple Wald test of

H0: r_1,2 = 1 vs. H1: r_1,2 <> 1,

where r_1,2 represents the estimated correlation between the two factors in the two-factor model, would be a test of the appropriateness of a one-factor specification, i.e.,

H0: one-factor vs. H1: two-factor.

I am able to perform the Wald test after fitting the two-factor model without any problems. I obtain a significant result, which is what I would expect.

However, when fitting a two-factor model in which r_1,2 is constrained to 1.0, I obtain different log-likelihood and AIC values from those obtained when fitting a one-factor model. Are there any parameters that need to be constrained/manipulated other than r_1,2? I receive an error message about the covariance matrix not being positive definite due to the correlation between factors being >=1.0. This could be causing the problem.
 James W. Shaw posted on Friday, June 06, 2008 - 11:31 am
Dr. Muthen:

Additionally, an alternative way to compare the 2 models would involve fitting a one-factor model in which the correlation between the indicators for the second factor is estimated and tested against 0. Unfortunately, I do not believe this is possible. Mplus does not estimate residual variances for count indicators. Thus, I do not believe one can specify correlated indicators when they are counts.
 Linda K. Muthen posted on Friday, June 06, 2008 - 2:48 pm
Are you fixing the correlation to one or the covariance? If you are setting the metric of the factor by fixing the first factor loading to one which is the Mplus default, then you are fixing the covariance to one. The two-factor model should have no cross loadings. If you have further questions, please send the appropriate files and your license number to support@statmodel.com.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: