Marginal Reliability in IRT PreviousNext
Mplus Discussion > Categorical Data Modeling >
Message/Author
 Glenn Walters posted on Wednesday, March 19, 2014 - 7:17 am
I was wondering if and how you can calculate (or at least estimate) the marginal reliabilities in IRT using MPlus. I am currently working with MPlus 5. Thank you.
 Bengt O. Muthen posted on Wednesday, March 19, 2014 - 4:20 pm
What is a marginal reliability?
 Glenn Walters posted on Wednesday, March 19, 2014 - 6:04 pm
Marginal reliability is based on the true score model (Lord & Novick, 1968) and is an estimate of the overall reliability of a test based on the average conditional standard errors. Often it is close in value to coefficient alpha (and sometimes it may even be identical). Several of the resources I have reviewed on this issue seem to suggest that alpha provides a lower estimate of marginal reliability. The formula used to define marginal reliability is reproduced below:

reliability = true score variance/ true score variance + error variance
 Bengt O. Muthen posted on Thursday, March 20, 2014 - 3:25 pm
You may want to consult the Raykov-Marcoulides book "Introduction to Psychometric Theory", which describes many connections between test theory and CFA using Mplus.
 Glenn Walters posted on Thursday, March 20, 2014 - 4:53 pm
Thank you. I am familiar with the book but do not remember it discussing marginal reliability with MPlus. I will contact Dr. Raykov directly to see if he has any ideas on this. One final question: how do I pull up a plot of the SEMs in an IRT analysis?
 Bengt O. Muthen posted on Friday, March 21, 2014 - 8:31 am
What is a plot of SEMs in IRT?
 Glenn Walters posted on Friday, March 21, 2014 - 11:47 am
I am sorry if it sounds like I am jumping from one thing to the next but I was told by a reviewer to provide a reliability estimate for my IRT analysis and the only IRT reliability estimate I was familiar with at the time was marginal reliability. In speaking with Dr. Raykov, however, it appears that marginal reliability is not used much any more (and so the plot of the IRT SEMs is no longer relevant). I have found a reference that shows how to calculate composite reliability from IRT data but I need the standardized factor loadings. My new questions are as follows: 1. are the discrimination parameter estimates in an IRT analysis the factor loadings (and if not, what is?). 2. are these estimates standardized and if not, what do I need to do to get standardized estimates?

Thank you for your time and persistence.
 Linda K. Muthen posted on Saturday, March 22, 2014 - 11:47 am
Ask for STANDARDIZED in the OUTPUT command. The standardized factor loadings are found in the BY statements.

No, factor loadings are not discrimination parameters. See the Topic 2 course handout and the IRT note on the website.
 Glenn Walters posted on Saturday, March 22, 2014 - 12:51 pm
That was very helpful. Thank you.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: