Assessment of model with multinomial ...
Message/Author
 Tibor Zin posted on Tuesday, August 04, 2015 - 10:16 am
Hello,

I would like to ask a question about the appropriate approach for analysing model with multinomial dependent variable. I am straggling with this question for a long time, so every advice would be very helpful!
In my model are four independent observed variables, four latent mediators and one multinomial dependent variable.
My model is similar to the model below:

usevariables are X1 X2 M1 M2 Y1 Y2 Z;
Z is nominal;

Analysis:
estimator = MLR;
MITERATIONS = 1000;
M1 on X1(a);
M2 on X2 (b)
M2 on M1 (c);
Y1 on M1 (d);
Y2 on M2 (e)
Y2 on Y1 (f);
Z#1 on Y2 (g);
Z#2 on Y2 (i);
Z#2 on Y2 (j);

Model constraint:
NEW(ind_1 ind_2 ind_3 ind_4 ind_5 ind_6);
ind_1=b*e*g;
ind_2=b*e*i;
ind_3=b*e*j;
ind_4=a*c*e*g;
ind_5= a*c*e*i;
ind_6= a*c*e*j;

..................
My problem is that fit indices are not provided. I am considering, what is the common practice in this case. Should I provide fit indices of model without multinomial dependent variable? How can I find out, whether model with multinomial dependent variable fits the data?

Thank You very much for the advice,

Tibor
 Linda K. Muthen posted on Tuesday, August 04, 2015 - 1:55 pm
Chi-square and related fit statistics are not available when means, variances, and covariances are not sufficient statistics for model estimation. In this cases, nested models can be compared using -2 times the loglikelihood difference which is distributed as chi-square. There is no absolute fit statistic in your case.
 Tibor Zin posted on Friday, May 06, 2016 - 10:35 am
Thank You for the reply!

Is this also a case when I am using Bayesian estimator and my dependent variable is binomial? Although this analysis provides PPP, I am not sure whether I should use this statistic in a such a model I described before.

Many thanks!

Tibor
 Bengt O. Muthen posted on Monday, May 09, 2016 - 2:02 pm
Bayes gives PPP for categorical outcomes but not nominal outcomes. And the power is low for categorical outcomes (see our Bayes papers on our website where we document this).
 Tibor Zin posted on Tuesday, August 16, 2016 - 8:19 am
Thanks!

I would like to ask another question based on the feedback that occurred during the review process.

The suggestion is that a particular strength of MLR estimator is that it allows the estimation of models with interval, and categorical variables as long as dichotomous dependent variable is defined as such. Afterwards, the fit values are provided. What do you think about this approach? Should I define binomial variable as categorical instead of nominal and use MLR instead of Bayesian estimator?

Many thanks!

Tibor
 Bengt O. Muthen posted on Tuesday, August 16, 2016 - 5:28 pm
ML (or MLR) doesn't give overall fit values when you have dichotomous DVs. The WLSMV estimator does.
 Tibor Zin posted on Wednesday, August 17, 2016 - 12:19 am
Thanks!

You mentioned before that chi-square and related fit statistics are not available when means, variances, and covariances are not sufficient statistics for model estimation. Is it not true using WLSMV?

So in the case when in the model is dichotomous DV, should I use WLSMCV to obtain fit indices? Or should I maybe test model without DV?
 Bengt O. Muthen posted on Wednesday, August 17, 2016 - 2:00 pm
Q1. WLSMV fits the model to these quantities so that's why these fit measures are available. But it is not checking fit to the raw data.

Q2. Yes you can.

Q3. That would be an additional good check.
 Tibor Zin posted on Friday, September 02, 2016 - 12:56 am
Thanks for the clarification! I would like to ask some follow-up questions.

Which estimator would you suggest using in a model with dichotomous DV? Bayesian or WLSMV? I am also testing indirect effects. I think that Bayesian estimator is better at handling missing values but are there some other advantages?

My last question is how can be alternative models compared using Bayesian estimator. Is it possible using Mplus? I suppose that in the case when there are not available absolute model fit statistics, this is the only option how to test whether model represents data.

Thank you for all your help, it is greatly appreciated!
 Bengt O. Muthen posted on Friday, September 02, 2016 - 9:57 am
See our FAQ:

Estimator choices with categorical outcomes

DIC is a way to compare models using Bayes.
 Tibor Zin posted on Saturday, September 03, 2016 - 3:56 am
Thanks!

The problem is that the output does not provide DIC. I believe that it is due to dichotomous variable. The only provided fit statistic is Bayesian Posterior Predictive Checking using Chi-Square. In this case, is there a way how to compare alternative models?
 Bengt O. Muthen posted on Saturday, September 03, 2016 - 9:22 am
If you have a model that is nested in a second model, you can check if the new parameters of the second model are significant.
 Tibor Zin posted on Saturday, September 03, 2016 - 1:49 pm
Thanks for the prompt reply!

I thought that nested model testing in Bayes is not possible in MPlus, or at least I found this information on MPlus forum. Is there some other possibility than using DIFFTEST option?
 Bengt O. Muthen posted on Saturday, September 03, 2016 - 2:45 pm
There is not yet a Difftest or Model Test for Bayes but you can see if each parameter of the less restrictive model is significant.
 Tibor Zin posted on Saturday, September 03, 2016 - 4:46 pm
I am sorry but I am not sure how to proceed. Do you mean that a sufficient comparison of nested models is to see whether a path is significant or not?

For example, in my model when one path is restricted the second path is significant. But when I estimate the first path the second path is not significant anymore. In this case, does the second model fit the data better?

Please, do you know about a reference where I could see how to proceed? That would be very helpful!
 Bengt O. Muthen posted on Saturday, September 03, 2016 - 5:21 pm
Say that your more restricted model is

y on m c1 c2;
m on x1 x2 c1 c2;

then your less restrictive model could be

y on m x1 x2 c1 c2;
m on x1 x2 c1 c2;

The difference is that the less restrictive model adds x1, x2 as having direct effects on y. Therefore, you want to consider the significance of the slopes for x1 and x2 in the y regression of the less restrictive model. If either one is significant the more restrictive model is not appropriate.
 Tibor Zin posted on Monday, April 10, 2017 - 6:03 am
Dear professor Muthén,

you have mentioned papers that demonstrate that ppp values are unreliable for categorical outcomes:
"Bayes gives PPP for categorical outcomes but not nominal outcomes. And the power is low for categorical outcomes (see our Bayes papers on our website where we document this)."

Please, could you send me a link to the exact paper? I could not find it.
 Bengt O. Muthen posted on Monday, April 10, 2017 - 7:00 pm
See section 6.3 of the paper

Asparouhov, T. & Muthén, B. (2010). Bayesian analysis of latent variable models using Mplus. Technical Report. Version 4. Click here to view Mplus inputs, data, and outputs used in this paper.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: