Comparing item estimates in Multiple ... PreviousNext
Mplus Discussion > Confirmatory Factor Analysis >
Message/Author
 Dvora Shmulewitz posted on Tuesday, December 01, 2009 - 2:33 am
I am doing a multiple group IRT analysis, using "TYPE=MIXTURE" and "KNOWNCLASS" and I freed the loadings and thresholds for each group (while setting the factor means to zero). How can I determine if any of the specific item discriminations or difficulties are significantly different between the groups?
 Linda K. Muthen posted on Tuesday, December 01, 2009 - 7:28 am
If a difference test between the model where factors loadings and thresholds are free to the model where factors loadings and thresholds are constrained to be equal indicates that the constrained model worsens fit, you can look at modification indices to see where the misfit may be.
 Dvora Shmulewitz posted on Tuesday, December 01, 2009 - 10:51 am
Yes, the difference test indicates that the constrained model worsens fit, but when I asked for modification indices, I got a message that I couldn't get them using ALGORITHM=INTEGRATION, which I needed to use because I was using TYPE=MIXTURE and ESTIMATOR=MLR. Is there another estimator that I can use that does not require integration?
Thanks, Dvora
 Linda K. Muthen posted on Wednesday, December 02, 2009 - 9:47 am
Without modification indices, you would need to look at one parameter at a time or small sets of parameters.
 Paul A.Tiffin posted on Friday, March 12, 2010 - 6:21 am
Dear Team

I have a quicky query: Using WINSTEPS I have done a Rasch analyis on some binary items to obtain their relative difficulties. I have reproduced the model in MPlus by fixing the loadings to 1 and the factor variance to 1. I get item difficulties that correlate 100% with the ones derived using WINSTEPS but the are different in absolute value. Why might this be?
 Linda K. Muthen posted on Friday, March 12, 2010 - 7:13 am
I think you should hold the loadings equal and fix the factor variance to one.
 Paul A.Tiffin posted on Friday, March 12, 2010 - 8:07 am
Thanks

I have done that and the difficulty estimates remain different to the ones produced by WINSTEPS (although correlate).

Also, I was wondering if it is possible to run a Monte Carlo simulation on a 1-PL IRT model in Mplus by specifying the item difficulties?
 Linda K. Muthen posted on Friday, March 12, 2010 - 8:25 am
It sounds like you are using the default weighted least squares estimator in Mplus. Try ESTIMATOR=ML; If that is not it, you need to send the Mplus and WINSTEPS outputs and your license number to support@statmodel.com.

You would need to translate the difficulties into the Mplus parameterization. See the IRT technical appendix on the website.
 Paul A.Tiffin posted on Saturday, March 13, 2010 - 12:25 am
Following on from my question yesterday, is it possible to include thresholds in a Monte Carlo simulation of a CFA or IRT model? I have had difficulties with this:
My current (appended) input looks like this:
MONTECARLO:
NAMES ARE eb76-md39 ;

NOBS= 94;
NREPS=1000;
SEED=67689;

MODEL POPULATION:

G by eb76-md39@1;
G@1;
eb76*.5 eb77*.5 etc ;

MODEL:

G by eb76-md39@1 ;
G@1;

Analysis:
estimator=MLR;

This seems to work fine. However when I try and include thresholds (e.g. [eb76$1@0.2]; I get warnings such as "
*** ERROR
The following MODEL statements are ignored:
* Statements in the GENERAL group:
[ EB76$1@0.2 ] ".

Is there some correct syntax to define thresholds in the MC model?

Many thanks for your valuable help.
 Linda K. Muthen posted on Saturday, March 13, 2010 - 6:38 am
I would have to guess to answer you. Instead, please send your full output and your license number to support@statmodel.com. This will be much quicker.
 Paul A.Tiffin posted on Saturday, March 13, 2010 - 7:21 am
thanks
I have my licence number at work but will email the outputs to you with it next week
Best wishes
Paul
 Mike Linacre posted on Sunday, March 14, 2010 - 1:43 am
Paul, there are two ways that Mplus and Winsteps estimates can differ systematically:

1. By a mean difference. This is set by the choice of zero point for the estimates. The default in Winsteps is "the mean item difficulty = 0", but you can override this.

2. By a difference in dispersion. This is set by the estimation method, and the adjustment (if any) for estimation bias. For tests with more than 20 items, estimation differences are usually too small to have a practical impact. They are less than the standard errors of the estimates.

Paul, you can simulate a Rasch dataset using pre-set item difficulties and sample abilities in Winsteps, and then produce the estimates from Mplus and Winsteps, so that you can compare them with each other and with the pre-set generating values.

OK?

Mike Linacre
 Jillian Smith posted on Monday, April 11, 2011 - 10:21 am
I am doing a series of factorial invariance analyses on a categorical data set (using the WLSMV estimator) and when I got to strong invariance I received this error: “The following MODEL statements are ignored: *statements in the GENERAL group: followed by the list of my intercepts to be constrained.” What does this error mean? How can resolve the issue?
 Linda K. Muthen posted on Monday, April 11, 2011 - 10:47 am
With categorical data, you don't refer to intercepts but to thresholds. See the inputs under multiple group analysis in the Topic 2 course handout to see how to test for measurement invariance of factors with categorical factor indicators.
 Jillian Smith posted on Tuesday, April 12, 2011 - 12:05 pm
Thank you for your response. That is very helpful. I have a follow-up question. Some of my items have 3 categories and the rest have 5 categories. Do the items need to be designated with the same threshold? I just tried to run the analysis with different thresholds (2 and 4 for the respective items) and I received an error.
 Linda K. Muthen posted on Tuesday, April 12, 2011 - 12:27 pm
A five-category variable has four thresholds and a three-category two. If you want help with the message, send the output and your license number to support@statmodel.com.
 Ben Emmert-Aronson posted on Friday, May 18, 2012 - 9:34 am
Dear Dr. Muthen,

I am running a multiple group IRT analysis, and having difficulty finding the source of measurement invariance. While following the input in Topic 2 my analyses ran normally, and I got the modification indices for by and with statements, but not for item thresholds.

Should I be examining partial model invariance based upon the residuals, or is there a way to get modification indices similar to multiple group analyses with categorical outcomes? I've appended my edited input.

Thanks kindly for your time,
Ben Emmert-Aronson


VARIABLE: NAMES ARE ADIS ethnicr depress anhedon appetite sleep motor fatigue
guilt concen suicide;
CATEGORICAL ARE depress anhedon appetite sleep motor fatigue guilt concen suicide;
usevariables are depress anhedon appetite sleep motor fatigue guilt concen suicide;
grouping is ethnicr (1=Caucasian 2=African-American 3=Asian 4=Hispanic);

ANALYSIS:ESTIMATOR = WLSMV;
difftest = C:\Users\Ben\Desktop\Stats\Deprace\omnibusform.dat;

MODEL:f BY depress anhedon appetite sleep motor fatigue guilt concen suicide;
depress with anhedon;

OUTPUT: sampstat standardized modindices (3.84) residual;
 Linda K. Muthen posted on Friday, May 18, 2012 - 11:50 am
Try MODINDICES (ALL 0);
 P. Fleming posted on Friday, March 08, 2013 - 8:43 am
We are trying to do a multiple group IRT analysis with one factor. We can get the model to terminate normally, but it says that the model parameter estimates cannot be computed because of nonconvergence or nonidentified model. We are uncertain what we may, or may not, need to specify in the Analysis statement. Below is our code, could you please tell us what are we missing or doing wrong?

VARIABLE: NAMES ARE u1 u2 u3 u4 u5 u6;
USEVARIABLES ARE u1 u2 u3 u4 u5 u6 ;
CATEGORICAL ARE u1-u5;
GROUPING IS u6 (1=Brazil 2=Nepal 3=Senegal 4=Tanzania 5=Ghana);
Missing are all (99);

MODEL: f BY u1-u5*;
f@1;

Model Nepal: f BY u1-u5*;
f@1;
Model Senegal: f BY u1-u5*;
f@1;
Model Tanzania: f BY u1-u5*;
f@1;
Model Ghana: f BY u1-u5*;
f@1;


Thanks for any input you might have!
 Linda K. Muthen posted on Friday, March 08, 2013 - 9:33 am
I am assuming you are using the default estimator WLSMV. When factor loadings are free, scale factors must be fixed to one in all groups. Also, with binary indicators, the model with free factor loadings is not identified unless the thresholds are also free and factor means are fixed at zero.
 P. Fleming posted on Thursday, March 21, 2013 - 2:33 pm
Thank you so much for your response! I think I implemented your suggestions successfully because I got it to run using the following input:

VARIABLE: NAMES ARE u1 u2 u3 u4 u5 u6;
USEVARIABLES ARE u1 u2 u3 u4 u5 u6 ;
CATEGORICAL ARE u1-u5;
GROUPING IS u6 (1=Brazil 2=Nepal 3=Senegal 4=Tanzania 5=Ghana);
Missing are all (99);

ANALYSIS: ESTIMATOR = WLSMV;

MODEL: f BY u1-u5*;
[f@0];
f@1;

Model Nepal: f BY u1-u5*;
[f@0];
{u1-u5@1}
f@1;
Model Senegal: f BY u1-u5*;
[f@0];
{u1-u5@1}
f@1;
Model Tanzania: f BY u1-u5*;
[f@0];
{u1-u5@1}
f@1;
Model Ghana: f BY u1-u5*;
[f@0];
{u1-u5@1}
f@1;

I'm a bit confused about what the fit statistics for this model tell us. Is it giving the global fit across all five groups?
For measurement invariance, are these the unconstrained model fit statistics? Should we compare the constrained model (constraining the thresholds across groups) to these fit statistics using the DIFFTEST? Thanks for your help!
 Linda K. Muthen posted on Thursday, March 21, 2013 - 2:42 pm
You should get an overall chi-square and a chi-square for each group.

See multiple group analysis in the Topic 2 course handout on the website to see the inputs for testing measurement invariance using WLSMV. Yes, DIFFTEST should be used in this case.
 P. Fleming posted on Thursday, March 21, 2013 - 3:51 pm
Thanks for your response. Using that input above, I don't get a chi-square for each group. Do I have to write something in the analysis statement to get a chisq for each group?

Thanks for the reference to topic 2!
 Linda K. Muthen posted on Thursday, March 21, 2013 - 3:53 pm
It is automatic. You may be using an older version of Mplus.
 P. Fleming posted on Friday, March 22, 2013 - 2:12 pm
Thanks for your help! I was able to get a chi-sq for each group by specifying a DIFFTEST. That is then my unconstrained model.

For the constrained model, I am following the guide for Topic 2 (slide 168, bullet 3). Here is my code:

VARIABLE: NAMES ARE u1 u2 u3 u4 u5 u6;
USEVARIABLES ARE u1 u2 u3 u4 u5 u6 ;
CATEGORICAL ARE u1-u5;
GROUPING IS u6 (1=Brazil 2=Nepal 3=Senegal 4=Tanzania 5=Ghana);
Missing are all (99);

ANALYSIS: ESTIMATOR = WLSMV;
DIFFTEST is deriv.dat

MODEL: f BY u1-u5*;
[f@0];
{u1-u5@1}
f@1;

Model Nepal:
[f@0];
{u1-u5*}
f@1;
Model Senegal:
[f@0];
{u1-u5*}
f@1;
Model Tanzania:
[f@0];
{u1-u5*}
f@1;
Model Ghana:
[f@0];
{u1-u5*}
f@1;

The model is not converging. I can see in the outputs that the factor loadings and thresholds are equal across groups, factor means are fixed to 0 in 1st group, and scale factors are fixed to 1 in first group. But, since it doesnt converge I get no chisq stats or SE. What am I missing?
 Linda K. Muthen posted on Friday, March 22, 2013 - 2:45 pm
Please send the output and your license number to support@statmodel.com.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: