Obtaining Additional Fit Indices PreviousNext
Mplus Discussion > Confirmatory Factor Analysis >
Message/Author
 David Kosson posted on Thursday, August 19, 2010 - 10:29 am
I am running a CFA with ordered categorical data using Mplus 5.1 and trying to test three-factor and four-factor solutions. I am finding generally acceptable (but not great) fit for the CFI, TLI, and RMSEA but poor fit for the WRMR -- one of my colleagues tells me I can also obtain the SRMR -- but I cannot figure out how to request the SRMR. It does not show up in my output --

here is a sample:
CFI/TLI
CFI 0.900
TLI 0.939
Number of Free Parameters 42
RMSEA (Root Mean Square Error Of Approximation)Estimate 0.084
WRMR (Weighted Root Mean Square Residual)
Value 1.571

Is there any way to request that the SRMR also be calculated?
Dave Kosson
 Linda K. Muthen posted on Thursday, August 19, 2010 - 1:03 pm
When SRMR is available it will be given automatically.
 David Kosson posted on Friday, August 20, 2010 - 2:46 pm
What I do not understand is why SRMR was available in the older version of Mplus but is not available in the current version -- that is, i am using Mplus 5.1 and I do not get the SRMR --

One of my colleagues has Mplus 4.21, and he gets it: here is his output for the 3-factor model:
CFI 0.922
TLI 0.955

Number of Free Parameters 16

RMSEA (Root Mean Square Error Of Approximation) Estimate .078

SRMR (Standardized Root Mean Square Residual) Value 0.060

WRMR (Weighted Root Mean Square Residual)
Value 1.553

Here is my output with version 5.1:

CFI 0.922
TLI 0.955

Number of Free Parameters 42

RMSEA (Root Mean Square Error Of Approximation) Estimate 0.078

WRMR (Weighted Root Mean Square Residual)
Value 1.345

Has the program changed in some way so that the SRMR is no longer appropriate but used to be considered appropriate?

I also noticed that the number of free parameters seems to be different -- it appears that now Mplus counts the number of items as part of the number of parameters?

Dave Kosson
 Linda K. Muthen posted on Friday, August 20, 2010 - 3:05 pm
Between the two versions you mention, TYPE=MEANSTRUCTURE became the default. SRMR is not available in this situation. Add MODEL=NOMEANSTRUCTURE to the ANALYSIS command to run the model without means.
 David Kosson posted on Monday, August 23, 2010 - 10:02 am
Linda,
Thanks. That may be what I need to do. However, it is not currently working -- here is the syntax i tried to use:
VARIABLES:
NAMES ARE r1pcl1 r1pcl2 r1pcl3 r1pcl4 r1pcl5 r1pcl6 r1pcl7 r1pcl8 r1pcl9 r1pcl10 r1pcl12 r1pcl13 r1pcl14 r1pcl15 r1pcl16 r1pcl18 r1pcl19 r1pcl20 misgrp;
MISSING = *;
USEOBSERVATIONS ARE (misgrp eq 0);

MODEL: f1 BY r1pcl1 r1pcl2 r1pcl4 r1pcl5;
f2 BY r1pcl6 r1pcl7 r1pcl8 r1pcl16;
f3 BY r1pcl3 r1pcl9 r1pcl13 r1pcl14 r1pcl15;

ANALYSIS: TYPE = GENERAL;
MODEL = NOMEANSTRUCTURE;


I was using this so that i could use the same dataset for my primary analyses that do not include cases with missing data and for supplementary analyses that include estimation of missing data.

However, it did not work -- here is the message I got:

*** WARNING in ANALYSIS command
MODEL=NOMEANSTRUCTURE is not allowed in conjuction with TYPE=MISSING.
Request for MODEL=NOMEANSTRUCTURE will be ignored.
1 WARNING(S) FOUND IN THE INPUT INSTRUCTIONS

Is there a way to do this without MEANSTRUCTURE?
Dave
 Linda K. Muthen posted on Monday, August 23, 2010 - 10:13 am
I think you need to also add LISTWISE=ON; to the DATA command.
 David Kosson posted on Friday, August 27, 2010 - 2:29 pm
That was a big help. I was able to complete all my analyses that did not include missing values using this approach.
Now, I am wondering if there is a way to run an analysis without using mean structures that does include missing values -- that is, my colleague has been able to run such an analysis in Mplus 4.21, but the following warning makes me think it may not be possible in Mplus 5.1 --

*** WARNING in ANALYSIS command
MODEL=NOMEANSTRUCTURE is not allowed in conjuction with TYPE=MISSING.
Request for MODEL=NOMEANSTRUCTURE will be ignored.

Dave
 Linda K. Muthen posted on Friday, August 27, 2010 - 3:27 pm
I don't think your colleague was able to to this in Version 4.21. TYPE=MISSING has always included meanstructure. Having unstructured means is the same as having no means.
 David Kosson posted on Monday, August 30, 2010 - 3:12 pm
Linda,
I think you are correct. My colleague now realizes that he was mistaken about this issue. I am guessing that this means that there is no way in Mplus to estimate missing values without using mean structures.
I have one additional question. The final step of my project is to conduct multi-group CFAs to compare the fit of the models across groups -- e.g., for one of these, we are comparing equivalence of the models in North American versus European samples. Once again, i would prefer to do these analyses without using mean structures, if possible. My reading suggests that it is possible to conduct multi-group CFAs that do not use mean structures, but I am having trouble getting Mplus to allow this -- even though I am limiting my multi-group analyses to cases with full data. Is there a way to do this in Mplus?
Dave
 Linda K. Muthen posted on Tuesday, August 31, 2010 - 9:06 am
You cannot exclude means from the analysis with TYPE-MISSING. However, if the means are unstructured, the fit will be the same as for a model without means.

With multiple group analysis, to obtain an unstructured mean model, relax the default equality of the intercepts across groups and fix the factor means to zero in all groups. This is the same as not having means in the model.
 David Kosson posted on Tuesday, August 31, 2010 - 9:25 am
With respect to the multiple group analysis, my impression is that with ordered categorical data, thresholds are modeled rather than intercepts -- so instead of relaxing the equality of the intercepts across groups, should i allow the thresholds to vary freely across groups?
(For my dataset, each indicator has three possible scores (0, 1, and 2), so I think I do this by mentioning the thresholds in a 2nd MODEL paragraph that is specific to the 2nd group -- e.g., if Europe is my 2nd group, I have a 2nd paragraph that says:

MODEL Europe (after my general model paragraph) and then specify:
[u1$1];
[u1$2];
for each indicator for which i want to let the thresholds vary --

Does that sound right?
 Linda K. Muthen posted on Tuesday, August 31, 2010 - 9:55 am
Correct. If you have ordered categorical variables, you need to free all of the thresholds as you show. You should check the results or TECH1 to be sure you have successfully achieved your goal.
 David Kosson posted on Tuesday, August 31, 2010 - 10:40 am
And if I fix the factor means to 0 in each group (as I did), does that mean that the multigroup CFA will not examine whether there are differences between the two groups in the average levels of the latent factors?

If so, then I think that means that the only difference between the model in which I allow the groups to differ on parameters (i.e., loadings and thresholds but not factor means) and the model in which i constrain the parameters to be the same is that, in the more constrained model, I require that the loadings be the same in the two groups -- but i still allow the thresholds to differ. Does that sound right?
 Linda K. Muthen posted on Tuesday, August 31, 2010 - 4:23 pm
Yes.

Yes.
 David Kosson posted on Wednesday, September 01, 2010 - 8:21 am
Thanks again. I have now run all the multiple group CFAs and the results suggest a lack of invariance. I am sorry to say that I still have two questions for you.
1) Is there any evidence that the software is overly sensitive to failures of invariance so that people should not worry too much about a small degree of variance -- e.g., in one case, the result of the chi-square difference test is fairly close to .05 --

Chi-Square Test for Difference Testing
Value 19.
Degrees of Freedom 9**
P-Value 0.0249


2) Given that I expect the mean levels of the latent factors to differ between the two groups, I am now wondering if it is possible to run the analysis without setting the means of the latent factors to 0 in both groups but allowing the program to estimate the means?

And if I do that, would any differences between the groups in the levels of the factor means lead to a lack of invariance?
 Linda K. Muthen posted on Wednesday, September 01, 2010 - 8:33 am
1. I don't of any formal study of this. You might do a literature search to see if you can find anything or ask on SEMNET.

2. To estimate factor means, thresholds must be held equal across classes. This is the Mplus default for multiple group modeling. Comparing means across groups requires threshold invariance.
 Leslie Rutkowski posted on Tuesday, October 26, 2010 - 1:55 pm
Hello Linda and Bengt,

Can you point me to the technical details behind why there is an incompatibility between missing data methods and no mean structure (e.g. NOMEANSTRUCTURE is not possible with TYPE=MISSING).

Thanks,
Leslie
 Bengt O. Muthen posted on Tuesday, October 26, 2010 - 5:42 pm
See

Muthén, B., Kaplan, D. & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 42, 431-462.

which you find on my UCLA web site. This article shows that ML estimation under MAR involves a mean structure which cannot be ignored if you want ML estimates.
 Claudia Kruzik posted on Monday, August 06, 2018 - 10:55 am
Hello,

I am running multilevel growth models with latent slopes (see syntax below):

Data:
Variable:
Names are SES U V W X Y Z CHID SCHID;
Missing are all (-9999);
Weight = weight;
Usevariables are SES U V W X Y Z CHID SCHID;
within = Y Z;
between = (CHLDID) U V W X;
cluster = SCHID CHLDID;
Analysis:
type = threelevel random;
Model:
%within%
ssch | W ON Y;
ssum | X ON Z;
%between CHLDID%
ssch on U (a1);
ssum on V (b1);
ssch on SES (a3);
ssum on SES (b3);
U on SES (a2);
V on SES (b2);
Model constraint:
new(asum asch tot);
asch = (a1*a2);
asum = (b1*b2);
tot = asum+asch+a3+b3;
Output: standardized modindices;

When I run these models, the only fit indices I get are AIC, BIC & adjusted BIC. I'm wondering why the output doesn't include a more comprehensive set of fit indices (TLI, CFI, Chi-square, SRMR, RMSEA), and whether there is a way to get these stats?
 Bengt O. Muthen posted on Monday, August 06, 2018 - 3:36 pm
With random slope models, the usual mean and covariance structure based testing in SEM is not available because those statistics are not sufficient data for such models - instead the full raw data is needed. You can compare neighboring models - models where restrictions are relaxed - and see if the extra parameters are significant.
 Cesar Daniel Costa Ball posted on Monday, May 27, 2019 - 8:02 am
Hello. Please, I would like to make you a question.

Perform a confirmatory Factor Analysis (CFA) with the following results:

CFA with categorical factor indicators

Chi square = 132
df = 35
p < .000
Chi square/df = 3.7
CFI = .96
TLI= .95
RMSEA = .085 (IC 90% 0.07 - 0.10)
RMSEA = 0.049

Hu and Bentler’s Two-Index Presentation Strategy (1999): TLI and SRMR (TLI of 0.96 or higher and an SRMR of .09 or lower) and CFI and SRMR (CFI of .96 or higher and a SRMR of 0.09 or lower) (as cited in Hooper, Coughlan, & Mullen, 2008).

Consider the above. With this poor RMSEA (0.085) value, can the data be considered to fit a one-dimensional model?
 Cesar Daniel Costa Ball posted on Monday, May 27, 2019 - 8:06 am
In the previous post I put an error the SRMR = 0.049

Hello. Please, I would like to make you a question.

Perform a confirmatory Factor Analysis (CFA) with the following results:

CFA with categorical factor indicators

Chi square = 132
df = 35
p < .000
Chi square/df = 3.7
CFI = .96
TLI= .95
RMSEA = .085 (IC 90% 0.07 - 0.10)
SRMR = 0.049

Hu and Bentler’s Two-Index Presentation Strategy (1999): TLI and SRMR (TLI of 0.96 or higher and an SRMR of .09 or lower) and CFI and SRMR (CFI of .96 or higher and a SRMR of 0.09 or lower) (as cited in Hooper, Coughlan, & Mullen, 2008).

Consider the above. With this poor RMSEA (0.085) value, can the data be considered to fit a one-dimensional model?
 Bengt O. Muthen posted on Wednesday, May 29, 2019 - 4:10 pm
This analysis question is suitable for SEMNET.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: