R2 for categorical DV (Snijders & Bos...
Message/Author
 Jorge Walter posted on Monday, August 26, 2013 - 1:12 pm
Dear Drs. Muthen,

Complying with a reviewer comment, I am trying to calculate explained variance (R2) for a two-level model with an ordered categorical dependent variable following the recommendations of Snijders & Bosker (1999: 225-234 (available at http://faculty1.ucmerced.edu/sdepaoli/docs/Snijders-Bosker%20Intro%20to%20HLM.pdf).

Now I'm having trouble understanding how to obtain the variance for the linear predictor (or fixed part) for Y as well as the intercept variance (both referenced on p. 225) from the MPlus output, which are needed to calculate their R2 in equation 14.21. Do you have any suggestions?

 Bengt O. Muthen posted on Tuesday, August 27, 2013 - 6:15 am
Mplus calculates R-2 in line with this reference. You should find this when requesting standardized solution. As the reference says, the residual variance on level 1 is 3.29 for logit and 1 for probit.
 Jorge Walter posted on Wednesday, August 28, 2013 - 10:21 am
Thanks for your answer. The reason I was trying to calculate this myself is that, in a different multilevel regression with a 1-7 Likert-type dependent variable, the Level-1 R2 I got from MPlus is substantially higher (.61 compared to .36) than the one I got from manually calculating the Level-1 R2 = (unrestricted level-1 error term – restricted level-1 error term) / unrestricted level-1 error term, with the unrestricted model containing only the dependent variable and level-1 random intercept, and the restricted model containing all level-1 and level-2 variables (Kreft and De Leeuw 1998).

Do you have any thoughts on why this discrepancy might exist?

Unfortunately, the formula above does not seem work for a categorical DV; when I try to run the unrestricted model, I get the following error message: "Variances for categorical outcomes are not allowed on the within level."

 Jorge Walter posted on Wednesday, August 28, 2013 - 5:24 pm
On a related note, which literature reference (and calculations) could I provide reviewers for MPlus's R2 calculations for multi-level models with ordinal-/interval-/ratio-scale DVs?

Thanks and best,

Jorge
 Bengt O. Muthen posted on Wednesday, August 28, 2013 - 6:37 pm
Mplus computes R-2 for each level using the same principle as with simple regression:

R-2 = 1 - standardized residual variance,

Here, the standardized residual variance is the residual variance for the particular level, divided by that level's DV variance.

There is a good discussion of this in the 2nd edition of the Snijders-Bosker multilevel book of 2012, see chapter 7.
 Jorge Walter posted on Wednesday, September 04, 2013 - 7:03 am
Thanks, Bengt, much appreciated!