

Invariance testing sig value oddity 

Message/Author 


Hello, Drs. I am presently running a structural invariance test among two groups of a model something like D on A B C; C on A; B on A; i.e. two mediators. These are latent variables, and I've gone through measurement invariance testing and am now up to testing invariance of the path coefficients. When I constrain all paths to equality between the groups, the model fit does not go down much (chisq change is not significant at p = .05). /However/, among the two groups, when I run them separately, there are two paths for which the values for each group are on different sides of the critical value and while numerically small, they change the paths from significant to not; e.g., for group 1, path CA is .25** and for group 2 it is .18 (not significant). I'm confused about how to report this, as I think the invariance testing suggests the model is not different between the two groups, though looking at the models separately there are differences likely to be mentioned by reviewers, and also which would have shown up if we'd done all the sampling with either group, instead of both. Do you have any advice for reporting this trickiness? 


I would report it all. The limited sample size may have made .18 not significant but we don't know that for certain. 

CMP posted on Monday, October 14, 2013  11:03 pm



Hello, I am interested in individual differences in the impact of Y on X across 4 time points (year 1, 2, 3, 4). It is a multilevel design, with measures nested within persons. X is measued with 3 indicators only. From my understanding alpha does not reflect the reliability of the meaures. So I ran a SEM measurement model, and found a weak factorial model fits the data. Question: Do you think I can go ahead with my multilevel analysis? I am new to Mplus. I usually use HLM, and would think this is OK with the latter approach. Thank you for your assistance. 


It sounds like you are saying that you have tested your measurement model for X and found that metric invariance across time holds. If that has been done right, then you can go ahead with a 2level model as long as you don't compare factor means across time. 

CMP posted on Thursday, October 17, 2013  4:17 am



Yes, I tested for metric invariance of X. I believe this was done right: the only issues I had in the process was correlation of some of the residual errors, which I then accounted for. The resulting model had adequate fit. So, what I understand from your response is that weak factorial invariance is good enough to test for individual differences using a 2level multi level approach. Is that correct? I thank you so much for your help. 


When you say "individual differences in the impact of Y on X", I assume that you mean impact of X on Y and that you contemplate the model %Within% s  y ON x; %Between% s on w; where s is a random slope and w is a level2 covariate. If so, I believe you are fine with metric invariance since you are studying regressions. Personally, however, I would always prefer scalar invariance to be really convinced that you have invariant measurements. 

Back to top 

