Constrained 3-level Model vs. 2-level... PreviousNext
Mplus Discussion > Multilevel Data/Complex Sample >
Message/Author
 James Algina posted on Wednesday, April 02, 2014 - 7:21 pm
I have data in which classes are assigned from within schools to two treatments. Scores are child-level. I fit a latent means ANCOVA model. BIC and adjusted BIC are smallest for a model with no level-3 random effects:
MODEL:
%within%
ep_pst on ep_pre;
%between tchid%
TX_ep2|ep_pst on trt;
TX_ep1|ep_pre on trt;
ep_pst on ep_pre;
%between schid%
ep_pst on site;
ep_pre on site;
TX_ep1 on site;
TX_ep2 on site;
TX_ep1@0;
TX_ep2@0;
ep_pre@0;
ep_pst@0;
ep_pst with ep_pre@0;

I also fit a two-level model

MODEL:
%within%
ep_pst on ep_pre;
%between%
ep_pst on trt site TxS;
ep_pre on trt site TxS;
ep_pst on ep_pre;

-2LL is identical for the two models. There are two notable differences between results for the two models: the intercept and the site effect for ep_pst. The intercept difference is due to centering differences. But I do not understand the source of the site effect difference. Can you explain why the difference occurs?

Jamie
 Bengt O. Muthen posted on Thursday, April 03, 2014 - 8:01 am
What is the first level in your 3-level model? Is it time?

How do you declare the cluster variables in your two setups?
 James Algina posted on Thursday, April 03, 2014 - 10:00 am
Thank you for your response.

The design is child within teacher within school. Child is the first level.

In the 3-level program I use

cluster = schid tchid;
between (tchid) trt (schid) site;

In the two-level program I use
cluster = tchid ;
between trt site TxS;

Trt and Site are each coded -.5, .5 to represent levels of treatment and site factors. TxS is created in Define: as
TxS = trt*site;

Jamie
 Bengt O. Muthen posted on Thursday, April 03, 2014 - 3:48 pm
If you have across-school variation it seems that you can differences in results when you combine the teacher and school levels into a 2-level model.
 James Algina posted on Thursday, April 03, 2014 - 5:33 pm
Thank you for your response.

The following may be of interest: I have tried the two models with other variables from the same design and have three examples in which the fit statistics are different for the constrained 3-level model and the 2-level model. In each case it was necessary to use more than 500 miterations in the 3-level model to avoid this warning "AN INSUFFICENT NUMBER OF E STEP ITERATIONS MAY HAVE BEEN USED." In each example fit statistics are better for the 2-level model.

Jamie
 Bengt O. Muthen posted on Friday, April 04, 2014 - 8:34 pm
But if the 3-level model has significant level 3 parameters, that's an important finding.
 James Algina posted on Thursday, April 10, 2014 - 7:29 pm
Thank you for your response.

To review since I was not able to reply earlier this week: the constrained 3-level model does not have any level-3 variance or covariance parameters.

By "that's an important finding", do you mean the finding that fit statistics (AIC, BIC, and sample size adjusted BIC) are better for the 2-level model or do you mean the finding of significant level 3 parameter estimates?

There are parameters (a site effect and an intercept) with significant estimates at level 3, but these parameters also have significant estimates at level 2. And some parameters that have estimates that are non-significant at level 3 have estimates that are significant at level 2. Overall I have interpreted the better fit statistics for the 2-level model to mean I can interpret its results. Is this reasonable?

Jamie
 Bengt O. Muthen posted on Friday, April 11, 2014 - 11:40 am
Q1. I was referring to significant level 3 parameter estimates. I assume you have also ruled out level 3 variances.

Q2. Yes.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a private posting area. Only registered users and moderators may post messages here.
Password:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: