Message/Author 

Anonymous posted on Wednesday, January 28, 2004  6:36 am



When running a CFA on 8 vars I have i) model 1: a 1 factor model  all 8 vars load onto a single factor. ii) model 2: a 2 factor model  4 vars on 1 fac, 4 vars on the other. iii) model 3: a 3 factor orthogonal model; 3 vars on 1 factor, 3 on another, 2 on a third AND all corrs between factors set = 0. iv) model 4: 3 factor oblique model (as model 3 but corrs allowed between factors). v) model 5: a 2nd order 3 factor model  as iii) but the three factors then load onto a single factor. Am I right in saying that model iii) is nested within model iv), but there is no other nesting here? 


That sounds correct. 


I have two CFA models: 1) four primary factors that are allowed to correlate with each other; 2) four primary factors that all load onto a secondorder factor . Am I right to assume that the 2nd model is nested within the first (and thus, I can use the chisquare difference test). If the test is significant, the 1st model is better? Thank you! 


Yes and yes. 


With 7 observed variables, I have 2 models: 1) all 7 items load on 1 factor 2) 4 items load on factor A and 3 items on factor B. Factors A and B are not correlated (fixed at 0). Is it correct that the models are not nested (they have the same # of df's) and chisquare difference test is not allowed. If this is correct, can I still compare the other fit statistics (GFI etc) 


I don't believe these models are nested. I don't know of any ways to compare nonnested models using various fit statistics other than descriptively. 

Kerry Lee posted on Tuesday, November 30, 2010  12:41 am



Dear Dr Muthen, I have been going through the posts on the testing of nested CFA models and am getting a bit confused. The scenario that I am concerned with involves testing whether a number of observed variables fit a 1, 2, or 3 factor CFA model (similar to the example given in .../9/1611.html). In a reply to that post, it was stated that forcing the correlation between two factors to 1 violates a key assumption in testing nested model using chisquared. In ...9/344.html, similar advice was given: a 2 factor model in such scenarios is not nested under a 3 factor model. Yet, in post .../9/5167.html, an elegant solution seems to be available using the MODEL TEST command. Rather than fix the correlation between two factors to 1, a Wald test can be used to test whether the the correlation between two factors differ significantly from 1. I guess if the Wald test is not significant, one can then infer that the correlation in question does not differ from zero and adopt the model with fewer factors. Because the Wald test is applied to only one model, the question of "nestedness" does not technically arise. Can I please confirm whether this line of reasoning is correct and that using the Wald test in this way allows for the evaluation of a 2 versus 3 factor model? Sincerely, Kerry. 


Testing on the border of the admissible parameter space (corr=1 being one example, v=0 another) could affect both LRT chisquare testing and Wald testing (z testing in a univ setting). But the question remains how well or poorly it works in practice. Next, I give input for 2 simulation studies I did to explore testing 1 factor vs 2 CFA factors. You can run it and look at the good results for the Wald test. And then generalize to your setting. 


Here is Monte Carlo input for generating data from a 1factor model and analyzing with a 2factor CFA. The question is what the empirical pvalue is for the Wald test: How often do we reject that the 2 factors correlate one? The pvalue is 0.06 which is close to the correct 0.05, that is, we reject that the factor correlation is one at the correct low Type I error level. In other words, we correctly choose 1 factor. The pvalue 0.06 is also seen in the last power column for the New parameter diff. title: 1f 2f montecarlo: names = y1y8; nobs = 200; nreps = 500; model population: f by y1y8*.8; f*1; y1y8*.36; model: f1 by y1y4*.8; f2 by y5y8*.8; f1f2@1; f1 with f2*1 (corr); y1y8*.36; model constraint: new(diff*0); diff = 1corr; model test: 0 = 1corr; 


Here is Monte Carlo input for generating data from a 2factor CFA and analyzing with the same model. Here the empirical 5% pvalue and power estimate is 0.74 at n=200, which approaches good power to reject the false hypothesis that the factors correlate 1. Note that this is the case despite the true correlation being high, 0.95. title: 2f 2f montecarlo: names = y1y8; nobs = 200; nreps = 500; model population: f1 by y1y4*.8; f2 by y5y8*.8; f1f2*1; f1 with f2*.95; y1y8*.36; model: f1 by y1y4*.8; f2 by y5y8*.8; f1f2@1; f1 with f2*.95 (corr); y1y8*.36; model constraint: new(diff*.05); diff = 1corr; model test: 0 = 1corr; 

Kerry Lee posted on Wednesday, December 01, 2010  7:41 pm



Dear Dr. Muthen, Thank you for the detailed response. I will work through the examples as soon as I finish reading the Monte Carlo section in the Users' Guide! I have a related question. In my data set, one of the variables is censored. For this reason, I have been using the MLR estimator and using the SB scaled chisquared. Unfortunately, the data is such that I have to use the strictly positive version. Again, the focus of the analysis is whether a 1, 2, or 3 factor model better explains the data. I am running this analysis on both longitudinal and crosssectional samples, with a view to linking up the samples in an accelerated design/growth model later on. Is there any precaution that I should take in generalizing Monte Carlo findings regarding the Wald test to the SB strictly positive chisquared? Sincerely, Kerry. 


MLR is robust to nonnormality and this carries over to the Wald test. I don't know how much you can generalize these results. You would need to do a study for each situation to know for sure. 

Kerry Lee posted on Friday, December 03, 2010  2:59 am



Dear Dr. Muthen, Thanks for the reply. I hope you do not mind clarifying one more thing. I gather from posts elsewhere that the Wald test can legitimately be used with MLR to test the effect of constraining a parameter (rather than the SB scaled chi squared). I also read that the Wald and the chisquared difference tests are supposed to be asymptotically equivalent. If this is the case, how come the Wald is not affected by the same issues that render the chisquared unsuitable for analyses conducted with a MLR estimator? Sincerely, Kerry. 


It is true that these two methods are asymptotically equivalent. Methods that are asymptotically equivalent may have different sensitivities. If you did a Monte Carlo study of likelihood ratio chisquare difference testing for your situation, you can see if in your case this test is also not sensitive. However, both tests are theoretically not correct for testing parameters on the border of the admissible parameter space. 


Dear, I've been trying to understand an article by Schönberger in Psychiatry Research. The authors test the Hospital Anxiety & Depression Scale (HADS) to see whether the 1, 2 or 3factor structure fits the data best. Model 1 = 1 factor (14 items) Model 2 = 2 factors (depression = 7 items & anxiety = 7 items ) Model 3 = 3 factors (depress = 7 items, anxty = 3 items, neg affectivity = 4 items) + 2 arrows departing from affectivity factor to the anxiety and depression factors. Now the authors state that Model 1 is nested in Model2 & Model3 + Model2 is nested in the 3 factor model. This to do X² diff testing. However, if I want to do the same thing (difftest because I use WLSMV), MPlus indicates that the 1factor model is not nested within the 2factor model. Can you please clear this up because I do not understand this? A 2nd problem are the DFs. According to my outputs, model 1 has a X² of 111.5 and DF of 76, model 2 has a X² of 111.6 and DF of 75 and model 3 has a X² of 97.3 and DF of 73. The DF thus go down while the paper of Schonberger reports the DF to go up from 1 to 3 factors). If you would need the paper, the reference is: Schönberger & Ponsford(2009), 'The factor structure of the Hospital Anxiety and Depression Scale in individuals with traumatic brain injury' in Psychiatry Research, 179: 342349 Many thanks. 


The DFs should go down from 13 factors because the models are using progressively more parameters. Nestedness would seem to be at hand here, so please send the outputs of your runs to support@statmodel.com. Sending the data may also be helpful. 


Thanks. I have just sent all relevant information to the abovementioned emailaddress, because I still do not understand everything (my limitations I am affraid). Many thanks on beforehand, Edwin 


Dear Dr Muthen, For a set of 14 variables, I want to test whether there is one underlying factor. Whereas a simple onefactor model did not fit well, I successively added 4 (theoretically plausible) residual correlations to the model. I interpret the resulting model A as indicating onedimensionality with minor additional factors. I am wondering whether in order to better support this, I should compare model A to a model B specifying 5 correlated factors, the hypothesis being that in a chi2difference test model A fit is not significantly worse than that of the more liberal model B. Model A would look like this: ERA by item1 item2 item3 item4 item5 item6 item7 item8 item9 item10 item11 item12 item13 item14; item1 with item2; item3 with item4; item5 with item6; item7 with item8; Model B: ERA by item9 item10 item11 item12 item13 item14; FAC1 by item1 item2; FAC2 by item3 item4; FAC3 by item5 item6; FAC4 by item7 item8; Are these models nested? Thank you very much! 


I would just stay with Model A and look at the residual covariance significance. Model B muddles things, I think. 


My question relates to the Monte Carlo code Dr. Muthen posted here on 12/1/10. I'm trying to adapt that code to do a power analysis for a multitrait, multimethod CFA in which I have two correlated traits and two correlated methods. Specially, I want to test whether the correlation between the two traits and that between the two methods are significantly different from 1. The problem is that even with 10,000 obs, the power estimates for my code are in the range of .69.71 for diff1 and diff2, which seems impossible. I'm running out of space so will post my code in a new post. Any help you can offer would be greatly appreciated. 


Code for message above: montecarlo: names = y1y8; nobs = 10000; nreps = 2000; model population: fMethod1 by y1y4*.8; fMethod2 by y5y8*.8; fMethod1*1; fMethod2*1; fTrait1 by y1*.8 y3*.8 y5*.8 y7*.8; fTrait2 by y2*.8 y4*.8 y6*.8 y8*.8; fTrait1*1; fTrait2*1; y1y8*.36; fMethod1 with fMethod2*.80; fTrait1 with fTrait2*.80; analysis: model = nocovariances; model: fMethod1 by y1y4*.8; fMethod2 by y5y8*.8; fMethod@1 fMethod2@1; fTrait1 by y1*.8 y3*.8 y5*.8 y7*.8; fTrait2 by y2*.8 y4*.8 y6*.8 y8*.8; fTrait1@1 fTrait2@1; y1y8*.36; fMethod1 with fMethod2*.80 (corr1); fTrait1 with fTrait2*.80 (corr2); model constraint: new(diff1*.20); new(diff2*.2); diff1 = 1corr1; diff2 = 1corr2; model test: 0 = 1corr1; 0 = 1corr2; 

Emil Coman posted on Wednesday, August 24, 2011  6:23 am



Dear Michael, I took the liberty of running your syntax [thanks for posting, I saved and will use it in future analyses...]. After correcting a small typo on line fMethod@1 fMethod2@1; which is fMethod1@1 fMethod2@1; it ran well, and got, I think as expected, for the Wald Test of Parameter Constraints Proportions Percentiles Expected Observed Expected Observed 0.050 1.000 5.991 2357.723 If we tweak the instructions on p 362 in the guide, 'the column 1 value of 0.05 gives the probability that the Wald Test value exceeds the column 3 percentile value of 5.991. Columns 2 and 4 give the corresponding values observed in the Monte Carlo replications. Column 2 gives the proportion of replications for which the critical value is exceeded,' which here is 100%. So the Wald test concludes that both constraints 1corr1=0 and 1corr2=0 are rejected, powerfully so, right? The new parameters indicate that .8's are different than 1, and .2's are not different than 0. Is this what you expected? Am I right on this? Cheers, Emil 


Hi Emil, Thanks for your help. I apologize for the typo; it wasn't in the code I was running but got introduced when I was renaming the factors to make them more intelligible here. Anyway, my confusion is because the power estimate from the Wald Test of Parameter Constraints that you mention is so widely different from the power estimates given by the % Sig coeff column under MODEL RESULTS for the individual parameters diff and diff2, which are .697 for diff and .704 for diff2 when n = 10,000. I wouldn't expect that these would be identical since the Wald test is a global test of two constraints, but am surprised they are so different. Also, I'm interested in testing these individually. To make matters more confusing, the power given by the Wald test of Parameter Constraints for this model seems strikingly high even at small sample sizes (e.g., .993 when n = 100), so I'm wondering if there's something wrong with the factor specification. Or does this sound reasonable to other folks? Thanks, Michael 


I ran your setup with n=10000 and 500 replications. The output shows that only of 442 of the 500 reps converged without problems and Tech9 shows the various estimation problems in many replications. The coverage is poor and there is both parameter and SE estimate bias. The power estimates in the last column (%Sig) rely on these quantities being well estimated. So the results are not trustworthy. Not for Wald either. The model is probably not a stable one, which we know is the case for MTMM with few methods. 


Dear Dr. Muthen, I have a Set of 6 Indicators v1 v2 q1 q2 n1 n2 and want to compare 3 Models: MODEL 1: v by v1* (lv1) v2 (lv1); v@1; q by q1* (lq1) q2 (lq1); q@1; n by n1* (ln1) n2 (ln1); n@1; MODEL 2: like Model 1 plus: v with q@0 n@0; q with n@0; g by v* q n; g@1; MODEL 3: like MODEL 1 plus: g by v1 v2 q1 q2 n1 n2; g@1; g with v@0 q@0 n@0; v with q@0 n@0; q with n@0; My questions: 1. With intent of identification am I right to equalize the loadings by e.g. v by v1* (lv1) v2 (lv1)? 2. Is MODEL 2 nested within MODEL 1 and I can use chisquare difference testing for comparing them? 3. MODEL 3 is not nested within the other models. Am I correct? (chi2diff. testing is not allowed and I have to compare by using descriptive fit statistics?) Thanks for your help! 


1. You don't need to hold the loadings of the 2 indicators equal for identification in Model 1. You would need to do that if the factors are specific factors in a model with a general factor, as in Model 3. Note also that this is not how to hold them equal  you should say v by v1v2* (lv1); 2. Model 2 is equivalent with model 1 because you have only 3 factors as indicators. 3. I believe Model 2 is nested within Model 3 when you have enough indicators. For a discussion, see e.g. Yung, Thissen & McLeod (1999). On the relationship between higherorder factor model and the hierarchical factor model. Psychometrika, 64, 113128. 


Hi, I´ve got a questionnaire and two different CFAs (three factors each, but the factos consist of different variables) One CFA consists of 25 Items, the other one only contains of 21 of these 25 Items. I treated the items as ordered categorical. The data are not normally distributed. I used the WLSMVestimtor. Am I right that these two models are not nested? (So I can´t use chisquaredifference test). What can I do to test the difference between these two models? Thank you very much for your help in advance! 


At a minimum, to be nested, the models must have the same set of observed variables. 


Hi, thank you for your answer! Right, and because I dont have the same set the two models are not nested. But how can I compare them nevertheless? Can I use AIC or something like that? 


AIC and BIC are not available with the weighted least squares estimator. I don't believe there is a way to statistically test if one of these models fits better than the other. 


Ok, I didn´t know about that! What do you think about reestimating both CFAs in a new sample using the WLSMextimator or Maximumlikelihood. Then comparing them with AIC or BIC would that be possible (if I still wouldn´t use exactly the same set of obeserved variables)? Would that make sense at all? 


All the WLS estimators are weighted least squares. When you have a different set of observed variables, AIC and BIC are not in the same metric. 

Paula Vagos posted on Friday, December 07, 2012  7:59 am



Dear Dr. Muthen I am qoeking with a data set of 549 observations on 40 variables, and I wanted to try different CFA on possible organization of those variables. I tried 3 models: 1. 40 variables loading on factor A 2. 20 variables loading on factor A and 20 on factor B 3. 10 variables loagind on factor A, 10 on factor B, 10 on factor c, and 10 on factor D. I though theses were nested models, and so I cousl use the chisquare difference test (the scaled one, because my data is nonnormal) to compare de models. Is this correct? Thank you 


We do not believe these models are nested. You can compare them using BIC. 

Jane Wu posted on Tuesday, December 11, 2012  1:57 pm



I'm having trouble determining if Model A is nested within Models B and C. Model A: Has 3 firstorder factors subsumed by a single higherorder factor, and with a 4th firstorder factor that covaries with the higher order factor. Model B: Has 4 firstorder factors subsumed by a single higherorder factor. Model C: Has 4 firstorder factors all intercorrelated. Thanks! 


Do all 3 models fit differently? I don't see that A is nested within B or C. 

Jane Wu posted on Wednesday, December 12, 2012  10:36 pm



OK, now I see that Models A and C have the same df and can't be compared. Model A, however, does have 2 more df than Model B. As I understand it, models with all firstorder factors subsumed by one secondorder factor (e.g., Model B) are nested within models where all firstorder factors are intercorrelated (Model C). Using that rationale, I was wondering if the nesting might work like this:  3 of the 4 factors could be subsumed by a higherorder factor (in Model A)  And the correlations between the 4th factor and 2 remaining factors are fixed at zero  Note that I'm still estimating one correlation between the 4th factor and a 1st/2nd order factor Maybe this is a stretch? In which case, I guess comparing AIC and BIC is the way to go. Thanks, Bengt! 


Note that in Model A you have a justidentified part where you have 3 1storder factors influenced by 1 2ndorder factor. The way to find out if a model X is nested within a model Y is to see if you can restrict parameters of model Y to get model X (get it to fit the same as model X. So you can experiment along those lines. See also a 2010 BentlerSatorra article in Psych Methods on the topic of how to decide nestedness. 

JOEL WONG posted on Friday, August 09, 2013  7:30 am



I want to compare 2 measurement models  Model A is a bifactor model with a general factor and 2 group factors (general factor is orthogonal to the group factors and the group factors are orthogonal to each other). Model B only has two correlated group factors. Is Model B nested within Model A? I'm guessing not, but just want to be sure. I used MLMV and when I ran the chi square dff test, there was a message indicating that model B may not be nested in A. I read a study that used the chi square diff test to compare these 2 models, implying that they are nested. 


If you use EFA for the bifactor and 2factor models, they are nested. This is because the EFA bifactor model with 2 specific factors (2 group factors) has the same fit as a regular 3factor EFA and a regular 2factor EFA is nested within a regular 3factor EFA. For the CFA versions, I am not sure and would have to see the two outputs. A not nested message is given if the model with more parameters does not have a better loglikelihood (or WLSMV fitting function). Getting stuck in a local solution can be one explanation. 

JOEL WONG posted on Saturday, August 10, 2013  1:16 am



Bengt, thanks a lot. I used CFA. As you rightly predicted, the model with more parameters (bifactor model) has a poorer loglikelihood. I'm guessing this is because in my bifactor model, the 2 group factors are specified as orthogonal whereas in my 2factor model, the 2 group factors are allowed to correlate with each other. So,the 2factor model is NOT tested within the bifactor model, and I should be looking at the BIC values rather than the chi square difference to assess model fit. Am I right? 


If I were you, I would investigate this further. First, make sure your bifactor model doesn't get stuck in a local optimum  use STARTS=50 for example. Second, read the literature, such as Reise 2012 in MBR on bifactor modeling, including nestedness YungThissenMcLoud 1999 Psychometrika Rindskopf & Rose 1988 MBR MulaikQuartetti 1997 SEM etc 

Back to top 