Xu, Man posted on Monday, March 18, 2013 - 8:22 am
I was wondering if it is still sensible to compare model fit such as TLI, CLI when the raw data has been altered slightly.
For example, in likert scale data with response style 1 2 3 4. Most response concentrate on categories 1 and 2, with fewer and fewer for categories 3 and 4. The graded response model for data with 4 categories doesn't fit well.
But, if I recode the data, combine categories 1 and 2 to just 1 category, so now the data has only 3 categories: 2 3 4. I fit graded response model again and this time the model gives better fit. I was wondering if this is evidence that it is better to model the data based on the recoded 3 categories?
I would hesitate to change the data to fit a model. Also, collapsing is typically done for categories with few subjects, not categories with many subjects - that would seem to lose information.
Xu, Man posted on Monday, March 18, 2013 - 10:19 am
Is there a more flexible model than the graded response model?
Based on my limited experience with IRT, in binary data, sometimes one can compare nested models from most restrictive Rasch model with equal loading for all item, to a free model that allows different ladings for all items.
Well, you can always use nominal. And there is work by Hedeker in the logistic regression context using models that allow (partial) relaxation of the proportional odds assumption behind the ordered polytomus logistic regression model (and the graded response model).
Xu, Man posted on Monday, March 18, 2013 - 1:27 pm
Re "use nominal", do you mean that under WLSMV, in CFA models, one can declare all items to be nominal? I tried that but the programme returned a warning for all items:
Nominal variables must be specified with a threshold number