Reliability using a measurement model
Message/Author
 John B. Nezlek posted on Sunday, March 27, 2016 - 4:03 pm
Dear Mplussers,

Raudenbush and Bryk in their presentations of HLM describe how to use a measurement level to estimate the reliability of a scale. Items are nested within some upper level unit of analysis and the reliability of the intercept from an unconditional model is an estimate of the reliability of the items (as measures of a single construct). For example, participants complete a 4-item scale every day for two weeks. The data are structured as items nested within occasions of measurement (days) nested within persons. They present the reliability of the day level intercept as an estimate of the reliability of the four items, corrected for within- and between person variability. Is there a way to generate such an estimate using Mplus?

Many thanks for considering the question,

John
 Bengt O. Muthen posted on Sunday, March 27, 2016 - 5:59 pm
You can do that in a more general way in Mplus by taking a wide, single-level approach. With 4 items measured every day for 2 weeks you have 14*4 outcomes which are your data columns. Then you specify a 1-factor model at each of the 14 timepoints where the factors correlate over time. And before you estimate reliability you can test for measurement invariance across time.

Then you can compute reliability using the omega formula that for instance Raykov and Marcoulides have described Mplus scripts for.
 John B. Nezlek posted on Sunday, March 27, 2016 - 8:23 pm
Hi Bengt,

Thanks for that, but I am not certain that such an analysis would be appropriate if days were a random and not a fixed factor, i.e., if there was no way/reason to match days across persons.

The HLM reliability estimates rely on the ratio of true:total variance. Is there way to generate these estimates in Mplus?

John
 Bengt O. Muthen posted on Monday, March 28, 2016 - 6:35 pm
Can you point to a page where R & B does this so I can see the details? In principle, you should be able to do the same thing in Mplus.
 John B. Nezlek posted on Friday, June 03, 2016 - 7:35 am
Hi Bengt,
Sorry for the delay in responding to this, but I did not want to waste your time until I knew exactly what to ask for. Assume a three-level model in which responses to items on a scales are nested within occasions of measurement and occasions are nested within persons. I can get what I need by using the variance estimates from a three-level unconditional model. Using example data 9.20 from the users guide, I have conducted what I believe is an unconditional model:
MODEL: %WITHIN%
y;
%BETWEEN level2%
y;
%BETWEEN level3%
y;
In looking at the output, I see within level variance for y as 2.645, between level 2 variance as .733 (also for y), and between level 3 variance as .802 (also for y).

I assume that these three numbers represent the decomposition of the total variance of y into the three levels, similar to the three variance estimates provided by HLM.

I apologize if this is trivial question, but I had difficulty finding specific advice about how to run a null model in Mplus, and I don't want to embarrass myself in print.

Many thanks,
John
 Bengt O. Muthen posted on Sunday, June 05, 2016 - 11:53 am
You are looking at this correctly.