Fit indexes PreviousNext
Mplus Discussion > Missing Data Modeling >
Message/Author
 Anonymous posted on Tuesday, May 16, 2000 - 12:18 pm
Does Mplus provide RMSEA, CFI, NNFI etc? How about missing data situation?
Thanks.
 Linda K. Muthen posted on Tuesday, May 16, 2000 - 3:18 pm
RMSEA is currently provided when all outcomes are continuous. This includes the missing data situation.
 Anonymous posted on Thursday, May 16, 2002 - 8:31 am
When running separate analyses on two nested models using FIML to handle missing data, is it still legitimate to do a chi-square difference test?
 bmuthen posted on Thursday, May 16, 2002 - 9:39 am
Yes, but if you use MLR you need to use the chi-square difference testing procedure described in Chi-Square Difference Test for MLM under Special Analyses with Mplus.
 Yifu Chen posted on Monday, December 06, 2004 - 11:53 am
Hi, Dr. Muthen,

I was running a growth model analysis with three time-varying variables. I also used TYPE=MISSING to handle the missing data. I found that MPLUS 3.11 only reported limited model fit indices (AIC, BIC...). I wonder if I can get chi-square and other fit indices in for the fitted model.

Thanks
 bmuthen posted on Monday, December 06, 2004 - 3:04 pm
You get fit indices if you add H1, saying

Type = Missing H1;
 Anonymous posted on Sunday, January 23, 2005 - 9:58 am
Do I understand from the comments above that it is reasonable to do a Chi-Square difference test with Chi-Square values that result from MLR estimation if the correction procedure for MLM estimation is used?

Thanks.
 bmuthen posted on Sunday, January 23, 2005 - 3:32 pm
Yes.
 Jim Prisciandaro posted on Friday, November 03, 2006 - 8:32 am
Hello Drs. Muthen,

I am running a 1-factor CFA model with the following features:

TYPE IS MISSING H1 COMPLEX
ESTIMATOR IS MLR

My 3 dependent variables (indicators) are count variables (I have specified this, as well as my missing specification, weight strata and cluster variables in the "VARIABLE" section).

My problem is that I am only getting limited fit indices (AIC, BIC...). Is there a way for me to get more fit indices (RMSEA, CFI...)? If not, which specification above is preventing me from getting these indices?

Thanks,
Jim
 Linda K. Muthen posted on Friday, November 03, 2006 - 6:14 pm
With count variables, means, variances, and covariances are not sufficient statistics for model estimation. Therefore you will not obtain chi-square and other related test statistics.
 Alicia Merline posted on Wednesday, December 13, 2006 - 11:34 am
I am using Mplus to estimate a multiple group path model using FIML. Because it is a path model, the Chi square of the baseline (freely estimated) model is 0. So, when I constrain a parameter to be equal across groups how do I test to see the impact the constraint has on fit. Doing a chi square difference test between the baseline and a constrained model means comparing the constrained model to 0. Is this appropriate? Do you know of any references regarding how to test for group differences in path models?

Thanks,

Alicia
 Bengt O. Muthen posted on Thursday, December 14, 2006 - 5:42 am
The answer to your first question is yes. General SEM ref's apply.
 Sofie Wouters posted on Thursday, April 30, 2009 - 7:26 am
I was testing the path model below:

USEVAR ARE sex zacap zaca zach zscho zstu;
MISSING IS ALL (-99.00);
cluster = scho;
define: int1=zach*zscho;
int2=zach*zstu;
ANALYSIS: type=complex;
type=missing H1;
estimator =MLR;
MODEL: zaca ON sex zach zscho zstu;
zacap ON sex zaca zach zscho zstu;
model indirect: zacap IND zach;
OUTPUT: STANDARDIZED;

I already added the text "H1" to my syntax to get information about the overall fit of my model, but I only got this:

Chi-Square Test of Model Fit

Value 0.000*
Degrees of freedom 0
P-value 0.0000
Scaling Correction Factor 1.000
for MLR

Is this because I have a recursive/saturated model? And how do I report the overall model fit? Can I do this through chi square difference testing, i.e., comparing with a model with no predictors?
Thank you for your time!
 Linda K. Muthen posted on Thursday, April 30, 2009 - 8:29 am
You cannot test the fit of a model with zero degrees of freedom. The models would not be nested if you get rid of the covariates. You could try the following:

MODEL: zaca ON sex@0 zach@0 zscho@0 zstu@0;
zacap ON sex@0 zaca@0 zach@0 zscho@0 zstu@0;
 Elizabeth Oliva posted on Wednesday, July 29, 2009 - 4:56 pm
I had a similar problem as Sofie Wouters (April 30, 2009) with my freely estimated (full/saturated) path model:

USEVARIABLES ARE alc2 cn0 gp1 cn2 gp2;
CLUSTER = IDYRFAM;
ANALYSIS: TYPE = COMPLEX;
MODEL: gp1 gp2 cn2 alc2 ON cn0;
gp2 cn2 alc2 ON gp1;
alc2 ON gp2;
alc2 ON cn2;
gp2 WITH cn2;
OUTPUT: SAMPSTAT STANDARDIZED;
standardized mod(3.84);

Chi-Square Test of Model Fit
Value 0.000*
Degrees of freedom 0
P-value 0.0000
Scaling Correction Factor 1.000
for MLR

Using your suggestion to add @0, I was able to get a Chi-Square Test of Model Fit with actual values, but I am confused as to why I would want to do that. The UCLA Academic Technology Services explanation suggests that @0 sets the structural paths to 0. If I am using this as my full, comparison model with all paths being freely estimated and then trying to compare it to nested models with more constraints, how would I do that if the paths have been set to 0?

Thanks!
 Linda K. Muthen posted on Thursday, July 30, 2009 - 11:21 am
When you fix the parameters to zero, the chi-square you obtain is not a test of the saturated model. It is a chi-square difference test between the two models. There is no way to assess the fit of a saturated model.
 Abhijit Visaria posted on Thursday, December 16, 2010 - 9:50 am
Greetings!

The 2-factor model I am trying to fit is:

VARIABLE: NAMES = finemp decihealth deciexpl deciexps visitfam prenatal tt instdel immun;

MISSING = all(999);

CATEGORICAL ARE finemp decihealth deciexpl deciexps visitfam prenatal tt instdel immun;

ANALYSIS: ESTIMATOR =MLR; INTEGRATION=MONTECARLO;

MODEL: mch BY prenatal tt instdel immun ;

autonomy BY finemp deciexpl deciexps decihealth visitfam;

mch ON autonomy;

OUTPUT: STDYX ;

I am getting only loglikelihood, AIC, BIC and Chi-Square Test of Model Fit for the Binary and Ordered Categorical (Ordinal) Outcomes. My N=46304, and degrees of freedom reported under the chi-square tests is 489.

Is there any way for me to get CFI, TLI and RMSEA?

Thank you!
 Abhijit Visaria posted on Thursday, December 16, 2010 - 10:49 am
I had a follow-up:

1. If I run WLS or WLSMV as the estimator in the same model I noted above, then the model drops all cases where there is any missing data on x-variables. Is this normal? Is there a way to avoid this?

2. The standardized parameters across WLS and MLR results are extremely similar. With WLS, I even get CFI, TLI and RMSEA and they show very good model fit. Now if there were some way for WLS to run with all of my cases, and not do listwise deletion on my cases with missing data, then would I be better off just using WLS rather than MLR?

Thank you so much.
 Linda K. Muthen posted on Thursday, December 16, 2010 - 3:03 pm
The chi-square you get with maximum likelihood and categorical outcomes is not the chi-square for the H0 model. It is the chi-square that compares observed and expected frequencies of the categorical outcomes. Chi-square and related fit statistics are not available in this situation.

Yes, cases with missing on covariates are dropped with all estimators as the model is estimated conditioned on x.

I would suggest imputing data sets using multiple imputation and then using WLSMV.
 rahel grueninger posted on Wednesday, May 09, 2012 - 3:11 am
I have got questions to the following problem:

I conduct a path analysis with weighted least squares means and variance adjusted (WLSMV) estimation in Mplus, version 5.1.
The WLSMV estimator was chosen automatically, because the variable “processing depth” was indicated as categorical.

The information of 20 data sets (reached over imputation) was included to examine the model.

For each descriptive quality criterium the average fit indices and standard deviations over the 20 data sets were computed.

In the output of Mplus, the model fit indexes for descriptive quality criteria are indicated as means and standard deviations. I think, I must report these means of the fit indices to report the model fit?
(The fit indices of my path model are (standard deviation are in parentheses): CFI = 1.00 (.00), TLI = 1.39 (.02), RMSEA = .00 (.00), and WRMR= .26 (.02)).

I do not know, whether I must report the Chi-Square, because there is not the mean of chi square indicated.

Furthermore I read that TLI and CFI have rather low power to reject a model with binary outcomes, while WRMR works well. And I read that “Recent studies indicate that a value less than .90 indicates good fit for WRMR.” (Linda K. Muthen posted on Thursday, March 08, 2001 - 3:14 pm). Can you please give me the references to these studies?

Thank you very much for your help.
 Linda K. Muthen posted on Wednesday, May 09, 2012 - 12:18 pm
It sounds like you are using TYPE=IMPUTATION for your analysis. We provide means for the fit indices over the imputed data sets. It is a research topic of how fit statistics should be handled with multiple imputation. So it is not known how to interpret these.
 Eric Teman posted on Sunday, June 24, 2012 - 11:45 am
When using WLSMV with multiple imputation in Mplus, is the model fit chi-square valid?
 Linda K. Muthen posted on Sunday, June 24, 2012 - 5:07 pm
No, we give the average value.
 wei w posted on Monday, January 27, 2014 - 12:37 pm
I am wondering how the chi square test statistic is computed with FIML. Is it computed based on the log likelihood difference between tested and saturated model or one of the test statistics proposed in Yuan and Bentler(2000)?

If it is computed based on Yuan and Bentler(2000), which formula is used? The formula in equation 18 or 20.

Yuan, K., & Bentler, P. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological methodology, 30(1), 165-200.

Thanks!
 Tihomir Asparouhov posted on Tuesday, January 28, 2014 - 3:11 pm
If you use estimator=ML then it is based on the log-likelihood difference between tested and saturated model.

If you use estimator=MLR it is T2* from Yuan, K., & Bentler, P. (2000).
 Sabrina Ott posted on Friday, July 11, 2014 - 2:38 am
I have the following problem: I want to do a CFA with one latent variable and three observed variables:


MODEL: control BY m1hs1 m1hs2 m1hs3;
OUTPUT: STDYX;

But the output tells me that chi-square is zero. My observed variables are highly correlated though. This must be some kind of mistake:

Chi-Square Test of Model Fit

Value 0.000
Degrees of Freedom 0
P-Value 0.0000

RMSEA (Root Mean Square Error Of Approximation)

Estimate 0.000
90 Percent C.I. 0.000 0.000
Probability RMSEA <= .05 0.000

CFI/TLI

CFI 1.000
TLI 1.000

Chi-Square Test of Model Fit for the Baseline Model

Value 46.998
Degrees of Freedom 3
P-Value 0.0000
 Linda K. Muthen posted on Friday, July 11, 2014 - 5:48 am
You have zero degrees of freedom. Your model is just-identified. Model fit cannot be assessed.
 Eric Teman posted on Wednesday, June 10, 2015 - 6:49 am
As of now, when using WLSMV with multiple imputation in Mplus, is the model fit chi-square valid? Or is the average still given?
 Tihomir Asparouhov posted on Wednesday, June 10, 2015 - 1:15 pm
It is just the average.
 Dex  posted on Thursday, August 20, 2015 - 3:23 pm
Hello, When compute SRMR using FIML (with missing data).The model implied Covariance matrix can be obtained using the parameter estimated from the model output. I was wondering how does the population covariance matrix be estimated. Does it obtained by treating the estimation of each variance or pairwise covariance separately, and to use all the observations for which both variables have valid values? Could I get the (estimated) population covariance matrix by specifying
"savedata sample=...". Thanks for your help.
 Bengt O. Muthen posted on Friday, August 21, 2015 - 1:48 pm
You get the estimated covariance matrix when you request Residual in the Output command.
 Andrew Triplett posted on Saturday, May 07, 2016 - 8:58 pm
Hello,

I need help, as I am not getting any fit indices (e.g., RMSEA, CFI, etc.) for my model when I run it. I only get the following:

MODEL FIT INFORMATION
Number of Free Parameters 40
Loglikelihood
H0 Value -14893.531
Information Criteria
Akaike (AIC) 29867.061
Bayesian (BIC) 30093.522
Sample-Size Adjusted BIC 29966.438

How can I get the other fit indices? Here is my input syntax:

ALGORITHM = INTEGRATION;
ESTIMATOR = ML;
Model:
f1 BY LS1-LS4;
f2 BY FS1-FS4;
f3 BY Health1-Health4;
f3 ON f1 (b1) f2 (b2);
f1xf2 | f1 XWITH f2;
f3 ON f1xf2 (b3);
Output:
sampstat residual standardized TECH1 TECH8;


Thanks!!
 Linda K. Muthen posted on Sunday, May 08, 2016 - 4:01 pm
Chi-square and related fit statistics are not available with numerical integration.
 Andrew Triplett posted on Sunday, May 08, 2016 - 5:03 pm
Hi Linda,

Thank you for your quick response! What could I do to fix this? When I run it without the "ALGORITHM = INTEGRATION" it returns an error. Is there anything I can do to give me a fix index?

Thanks,
Andrew Triplett
 Linda K. Muthen posted on Monday, May 09, 2016 - 7:45 am
You cannot get an absolute fit index. You can compare nested models using -2 times the loglikelihood difference which is distributed as chi-square or non-nested models with the same set of dependent variables using BIC.
 Danyel A V Moosmann posted on Tuesday, June 14, 2016 - 9:54 am
Dr. Muthen,

I am running an SEM with one predictor, three mediators,and four latent constructs as outcomes. The model is estimation is terminated normally, but I'm getting very little fit information. It's giving me the Loglikelihood, AIC, BIC, and SSABIC. I'm wondering why it's not giving me Chi-Square, RMSEA and SRMR?

Thanks for your help in advance.

Best,
Danyel
 Bengt O. Muthen posted on Tuesday, June 14, 2016 - 10:15 am
It sounds like you have a categorical or count DV in which case the usual ML chi-square testing against an unrestricted mean vector and covariance matrix is not relevant.

If this doesn't help, send your output to Support along with your license number.
 Monti Cvert posted on Sunday, November 13, 2016 - 3:24 am
Tell me, can there be inconsistencies in the indexes due to the fact that I use a proxy server? I understand that it can and it is not related, but I have nothing else comes to mind.
 Bengt O. Muthen posted on Monday, November 14, 2016 - 7:48 am
The server has nothing to do with this. Model fit indices frequently disagree.
 Bryn Spielvogel posted on Wednesday, November 29, 2017 - 3:56 pm
I am running three versions of the same two-level path model - the only difference between the three models is the academic domain being assessed (math, reading, or language). All model specifications and covariates are parallel. All fit indices are provided for the math and language models, but only the Loglikelihood, AIC, BIC, and SSABIC are provided for the reading model. Any ideas on why this would happen?
 Bengt O. Muthen posted on Wednesday, November 29, 2017 - 4:19 pm
We need to see the output for e.g. math and for reading - send to Support along with your license number.
 Bryn Spielvogel posted on Tuesday, January 16, 2018 - 12:27 pm
I was able to figure this out by running the model on one dataset rather than on all imputed datasets. The former approach gave me a message that had previously not been visible. The message stated that

"THE H1 MODEL ESTIMATEION DID NOT CONVERGE. CHI-SQUARE TEST AND SAMPLE STATISTICS COULD NOT BE COMPUTED. INCREASE NUMBER OF H1ITERATIONS."
 Caroline Haselgren posted on Saturday, November 03, 2018 - 2:28 am
Dear Professors Muthén and Asparouhov,

I’m a beginner with SEM/MPlus and I’m trying to figure out how to deal with the following:

I’m currently estimating a structural model using the WLSMV estimator. Since my data is not MCAR, I have conducted multiple imputation and created 5 datasets. When I run the full structural model, the fit indices suggest that there are room for improvements. However, since I do not get any modification indices I’m not sure how to motivate the respecifications (even though they make a lot of sense in theory). I’m guessing that that might not be enough? Hence, I wonder what would be the best way to localize areas of misfit in this situation? I tried running the model with FIML (without imputation) and results were very similar. By looking at the modification indices from this model, some changes were suggested that make sense (and imporve model fit) also with the imputed data. Is this a reasonable approach to finding, and motivating, localized areas of strain?

Best regards, Caroline
 Bengt O. Muthen posted on Monday, November 05, 2018 - 4:03 pm
If you can do FIML that's the way to go here - no imputations are needed.
 Caroline Haselgren posted on Monday, November 05, 2018 - 4:23 pm
Ok, thanks. I’m not sure that FIML is appropriate though, given that most of my variables are ordinal scale and/or heavily skewed? Is there any other way to localize areas of strain with multiple imputation and WLSMV?
 Bengt O. Muthen posted on Monday, November 05, 2018 - 4:38 pm
FIML does not mean that the variables have to be continuous-normal; they can be ordinal and skewed. FIML simply means ML under the MAR missing data assumption. ML can be used also for ordinal and other non-normal outcomes.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: