Multiple Group Analysis
Message/Author
 learningmplus posted on Wednesday, April 25, 2012 - 6:23 pm
Hi!

I am running a model and want to see if the paths are moderated by sex (boys vs. girls). I get non-significant chi square difference, suggesting that the paths are not moderated by sex. When I look at the paths coefficients in the unconstrained model, one of the paths in my mediation model going from the mediator to the outcome is significant for girls but not for boys. I can make theoretical sense of it. However, what I don't understand is why the chi square shows the moderation but when I look at the individual paths for boys and girls the path for girls is significant and for boys is not. Since there is no moderation based on the chi square diff. test, do I report this path that is significant for girls but not boys? What would be the name of this effect (obviously not interaction effect). Will simple main effect for that particular group (girls) be accurate? Thanks in advance.
 Linda K. Muthen posted on Thursday, April 26, 2012 - 1:45 pm
The test in each group of a coefficient is a test that the parameter is significantly different from zero. A difference test is a test of whether or not the two parameters are equal to each other. You can get a z-test to test if the parameters are equal to each other using MODEL CONSTRAINT. It should agree with the chi-square difference test.

MODEL CONSTRAINT:
NEW (diff);
diff = p1 - p2;

whre p1 and p2 are labels in the MODEL command for the parameters to be tested.
 Francesca Pesola posted on Thursday, November 22, 2012 - 9:37 am
Hi Linda,

I have a similar situation - the Chi square (difftest) shows a significant difference between two groups on one moderator w. When I look at the path size (and p) for each group however, the two paths have a p>.05.

Additionally, when exploring a second moderator I obtain a significant Chi square but when I explore the paths for the two groups, I find that the path is significant for 1 group but not the other.

Is there any point reporting these results? especially, if the path is not significant for the groups of interest?

Or should I simply report and focus on the estimate of the path for each group and ignore the p value?

Thanks!
 Linda K. Muthen posted on Friday, November 23, 2012 - 8:53 am
The tests you are comparing don't test the same thing. In one case, you test if the coefficients are equal to each other. In the other case, you test if they are equal to zero.
 Francesca Pesola posted on Friday, November 23, 2012 - 10:39 am
Thanks Linda -

I am now working with imputed data so I was thinking about using the 'diff' approach as difftest does not work on imputed data

MODEL CONSTRAINT:
NEW (diff);
diff = p1 - p2;

Would that be OK? also can it be used when you have more than 3 groups to compare?

Thanks
 Linda K. Muthen posted on Friday, November 23, 2012 - 11:55 am
This tests if p1 and p2 are equal. It can be used with more than two groups.
 Francesca Pesola posted on Saturday, November 24, 2012 - 3:43 am
Having 3 groups, would I need to incorporate 3 difference scores for the combinations?

MODEL CONSTRAINT:
NEW (diff1 diff2 diff3);
diff1 = p1 - p2;
diff2 = p1 - p3;
diff3 = p2 - p3;

or can it be done in a more efficient way?
 Linda K. Muthen posted on Saturday, November 24, 2012 - 10:33 am
That looks okay.
 Francesca Pesola posted on Sunday, November 25, 2012 - 6:01 am
can I build an overall test in the style of ANOVA analyses:

MODEL CONSTRAINT:
NEW (overall diff1 diff2 diff3 );
overall = (p1^2+p2^2+p3^2)-(p1+p2+p3)^2;
diff1 = p1 - p2;
diff2 = p1 - p3;
diff3 = p2 - p3;
 Linda K. Muthen posted on Sunday, November 25, 2012 - 9:15 am
Do this in MODEL TEST where you will get a Wald test for the set of tests:

MODEL TEST;
0 = p1 - p2;
0 = p1 - p3
 Francesca Pesola posted on Monday, November 26, 2012 - 2:10 am
Thanks Linda;

Can I check with you that the command (implicitly) includes the third comparison (i.e, 0 = p2-p3)?

If this is the case, is this the equivalent to the difftest when comparing the free model with the model where all three groups are set to be equal?

Is there any reference I should be citing/reading?

Thanks again,
Francesca
 Francesca Pesola posted on Monday, November 26, 2012 - 3:27 am
one more thing. I found in previous threads that before multiple group testing, one needs to test for measurement invariance across groups when using latent variables.

Is that a necessary pre-requisite?
 Linda K. Muthen posted on Monday, November 26, 2012 - 8:29 am
Yes, it includes the third comparison.

Yes, it is equivalent to DIFFTEST in that case.

You must establish measurement invariance for comparisons of factors across groups to be valid.
 Francesca Pesola posted on Monday, November 26, 2012 - 9:10 am
Thanks Linda!