Andy Ross posted on Tuesday, October 24, 2006 - 1:53 pm
Dear Bengt and Linda
I am running a simple multi-group path model with missing data and the WLSMV estimator. I wish to test whether some of my regression coefficients are the same for my two groups by comparing the model fit for a restricted and less restricted model. In the manual you state the need to use the DIFFTEST when using WLSMV estimator for comparing restricted and less restricted models. However I cannot seem to get this function to work when I am also working with Type=missing. The Chi-Square Test for Difference Testing does not appear in the output. In fact i get no tests of model fit except something called a Weighted Root Mean Square Residual. Can you please advise.
On a more substantive note. The restrictions I have added relate only to those regression coefficients i am interested in. I have not requested the coefficients on a number of control variables to be equivalent. Is this an adequate modelling approach? Or would i need to also 'equivalise' the coefficients of the control variables if i am to test whether the two groups differ on the coefficients of interest? I hope this makes sense.
I think you need to add H1 to the TYPE option of the ANALYSIS command.
If your hypothesis involves only certain regression coefficients, then those are the ones you should test for.
Andy Ross posted on Wednesday, October 25, 2006 - 10:53 am
That's great! many thanks Linda
Andy Ross posted on Wednesday, October 25, 2006 - 12:29 pm
One further question...
What is the opinion on using this method for multi group analysis when working with large samples?
Using the DIFFTEST to compare a model in which i restricted a number of coefficients to be the same across two groups the result for the chi-square difference test was 15.680 with 6 d.f. (p=0.0156). Further, in another model it was 18.033 with 6 d.f. (p=0.0062). However my sample size is around 6200 for each group.
Is there a way of taking sample size into account? Is it acceptable to do so? Are there any references for this?
I don't know of any reason you should not use this for multiple group analysis when working with large samples.
I am sure there are articles related to the sensitivity of chi-square to reject the H0 model when sample size is large. I don't know of any specific references. Discussions of power will also address issues related to sample size.
Having a large sample size also gives you the opportunity to randomly split the sample so that you can cross validate your resluts.