Student 09 posted on Sunday, February 15, 2009 - 3:22 pm
sometimes it is of interest to inform readers/reviewers what has been termed a "residual ICC" - the ratio of between-group variance/total variance in a dependent variable which remains after covariates have been entered into the model.
However, I just noted that the (presumed) residual ICC of my dependent variable (I use Mlwin example data) does not differ from the according ICC I requested via 'multilevel basic', even though I added several significant group-level covariates. At the same time, the between-group R_square of the model with covariates clearly increased as compared to the null model without any between-group covariates.
Is it correct that the ICC for a dependent variable shown by Mplus ALWAYS relates to a null model without covariates?
Yes, you should instead compute your residual ICC from the model-estimated parameters.
Nidhi Kohli posted on Wednesday, October 05, 2011 - 12:38 pm
I have a two-level nested cross-sectional data where the dependent variable is nominal with 3 unordered categories. The dataset contains within- and between-level covariates. I want to compute ICC (between variance/total variance) for the twolevel model with and without covariates. I tried running the model with TYPE=TWOLEVEL BASIC, but the output did not contain the ICCs. Can you please help me to compute ICC in Mplus.
There is no within-level variance with a nominal outcome (this parameter does not exist) and therefore icc's are not defined. I would just see if the between-level variances for the random intercepts of the nominal variable are significant.
Thank you for your response. I have a follow-up question. I did not see the between-level variances for the random intercepts of the nominal variable in the output. How can I get the between-level variances for the random intercepts of the nominal variable?
I am sorry, I forgot to mention that I was not able to run TYPE=TWOLEVEL BASIC when I classified the dependent variable as nominal. Thus, I classified the dependent variable as categorical so that at least I can get something in the output, which obviously didn't work. What is the best way to get the between-level variances for the random intercepts of the nominal variable? I greatly appreciate your help.
To get the between-level variance of the random intercept of the nominal variable, you need to mention the variance in the between part of the model. If the nominal variable is u with three categories, say
Just to confirm, are you referring to model with TYPE=TWOLEVEL?
Another question that I have is: will the random intercepts of u#1 and u#2 be treated as continuous variables? If yes, then the regression coefficients at the second -level obtained from regressing u#1 and u#2 on the between-level covariates are the linear regression coefficients? Thanks.
One last question on this thread. I fitted a Twolevel model where I requested the program to estimate the between-level variance of the random intercept of the nominal variable by using the following command:
alt_dep#1 alt_dep#2 ON md_age md_female md_white img family panel_pts;
The output shows that the residual variances corresponding to alt_dep#1 and alt_dep#2 is non-significant (p-value>=0.05). Based on this information, is it reasonable to assume that a Twolevel model is not required. Alternatively, there is not much variability at the second-level, hence, a single-level model is appropriate for this dataset?
Li Lin posted on Monday, October 10, 2011 - 12:29 pm
Hello, I have a question about two-level model with dichotomous respones. For each observation and each endogenous variable, I'd like the residual (observed-predicted) for further analysis. Are there any output options I can specify to save these residuals?
Li Lin: No options are available for this. You would need to create the predicted score and the difference score in DEFINE.
Li Lin posted on Thursday, October 13, 2011 - 6:55 am
Hi Linda Thanks for your response. I have two more questions. 1. How to create the predicted score in Mplus? 2. I have dichotomous responses in the model. residuals = observed-predicted score? Does multivariate normality assumption hold for the residuals?
You need to use DEFINE to compute a log odds for each person:
logodds = -a + bx
where a is the threshold and b is the regression coefficient.
The probability is equal to
1 / (1 + exp (-logit)
where - logit is a - bx.
If a person has a probability greater than .5, assign a value of 1 as the estimated value. If a person has a probabiity less than or equal to .5, assign a value of 0 as the estimated value. Then take the difference between the observed and estimated values.
I am sorry for this basic question. I have been reading the discussion board and looking at the Topic 5 handout you suggest, but I am confused about how to calculate the overall ICC for my model so that I can calculate the design effect.
Here's what I am doing. Could you help me figure out what to do differently?
1) I estimate what in HLM would be the null model-- just my dependent variable-- using type=basic twolevel and specifying the dependent variable only on the within portion of the model-- i.e.
Analysis: Type=basic twolevel;
Model: %Within% engage by m_a3 m_a31 m_a60 m_a72 m_a74 m_a79 m_a80; m_a3 with m_a31 m_a60; m_a31 with m_a60;
In the output, I get ICCs for Y variables. But I am not sure how to combine them to get the total ICC. Do I average the ICCs for the Y variables to get the ICC for the null model? Do I use some other option to get the total residual variance for the within and between? Something else?
I am not familiar with the concept of "overall" or "total" icc. To me, icc is a variable-specific concept.
You can also computed an icc for a latent variable as we describe in Topic 7.
You say that you want to compute a design effect. I wonder why. Design effects are sometimes used to make an approximate correction of SEs of estimates. But there is no need to do that since you can simply use Type=Complex to get the right SEs.
It is not necessary to compute design effects. You should run the analysis without taking clustering into account and with TYPE=COMPLEX that takes clustering into account. Compare the standard errors. If they are very close, it is not necessary to take clustering into account.