Anonymous posted on Monday, October 03, 2005 - 3:07 pm
I'm trying to examine a basic multiple group model. However, I am having trouble getting Mplus to constrain all of the parameters of the model to develop the baseline model. I tried the method where a number is placed in parentheses. However, it doesn't appear to constrain the parameters.
I also let Mplus do this by default as in part of the model presented in 5.14. I guess I'm lost. Any suggestions as to what I should do to constrain these measures? Thanks in advance.
Are all parameters not being constrained or just certain ones? With some parameters, like factor means, the equality for the parameter must be given in the group-specific models for the first group since the special default is applied to the first group so specifying equalities in the overall group doesn't do anything to the parameter for the first group.
Anonymous posted on Wednesday, October 05, 2005 - 5:55 am
I have this issue resolved. HOWEVER, I can't seem to get different chi-square corrections when I free some of the parameters. For example, I have a structural model developed, but I am unable to test for difference. I can't perform the formula on the website because the data are weighted and ML will not run with weighted data. I tried the difftest option and it didn't work as well. I'm lost as to what I'm doing wrong.
Also, when I use my weights should be observations that N change?
bmuthen posted on Saturday, October 08, 2005 - 2:00 pm
You can use the MLR estimator with weighted data and use the same chi-square difference testing approach as is described on the web site for MLM.
Jeff Lee posted on Saturday, November 03, 2007 - 12:01 pm
I am testing measurement model invariance across countries A, B and C. The measurement model is an established four-factor model with at least 4 indicators measuring each fator. The test results of the unrestricted model shows that all the 7 factor loadings of the first factor in Country C are not significant, indicating the configural invariance does not hold (GFI=0.857, CFI=0.903, RMSEA=0.041).
My first question is£º Is it ok to procede and test the latent mean structures with the other 3 configurally invariant factors?
Byrne, Shavelson, and Muthen (1989) suggested that at leat 2 indicators' loadings should be set equal across groups if we want to test mean structures, given partial measurement invariance.
My second question is: If the factor loadings of 2 out of the 4 factors are completely non-invariant across the 3 countries, while the factor loadings of the other 2 factors are all invariant across the 3 countries, how should I test the invariance of factor variance and covariance and latent mean structures?
Although the measurement model is an established one, it may not be correct for your data given that for one factor the factor loadings are not significant for one factor for one country. I would look at each country separately in an EFA as a first step to establish that the same number of factors is best for each group. If so, I would test the measurement model for each country to see if it fits the data for each country. Only then would I attempt to establish measurement invariance across countries.
Once you have determined that you have measurement or partial measurement invariance, you can test the equality of the structural parameters using chi-square difference testing of the MODEL TEST command.
Jeff Lee posted on Friday, November 09, 2007 - 2:43 pm
I am trying to test latent mean structure invariance of a 5-factor model across 3 countries. Each construct is measured with at least 3 indicators. Partial configural invariance is established with only 1 non-significant factor loading in facor 1 in Country C. There are 3 indicators in factor 1. None of the 4 factor loadings of factor 3 is invariant while all the factor loadings for the other factors are invariant across 3 countries. Byrne, Shavelson, and Muthen (1989) suggested that at leat 2 indicators' loadings should be invariant across groups if we want to test latent mean structures, given partial measurement invariance. I have no problem with factor 1 because there are 2 invariant indicators across 3 countries and I can freely estimate the indicator that is not significant for Country C.
My first question is: How should I handle factor 3 in which no factor loadings are invariant across 3 countries when I test for the invariance of factor variance and covariance? Should I leave the variance and covariances associated with factor 3 freely estimated when I test the structural model invariance?
My second question is: Given all the loadings for factor 3 are not invariant, how should I test the latent mean structures? Should I leave the factor loadings, factor variance and covariances associated with factor 3 freely estimated when I test the mean structures?
I want to conduct multiple group SEM using data from a large sample (approx n = 27000). I have five groups (the smallest group has approx n = 800 and the largest group has approx n = 20000. I have the handout from the Day 1 training course. If I am following the handout correctly, I need to fit each factor model (of which I have 3) separately in each group, then fit the model in all groups allowing all parameters to be free, and then test for invariance of the factor loadings and intercepts? I have tried to fit the a priori factor models in each group but I have received an error message that indicate that the latent variable covariance matrix is not positive definite. I have also received error message that there are empty cells in the bivariate table of one of the models.
To overcome this problem, I was thinking of estimating the factor models using data from the entire sample and save the factor scores and then conduct a multiple group latent variable path model. Would this method seem in anyway plausible? Are there any references I could consult on this issue?
It sounds like you should start with an EFA. I don't think the approach using factor scores is a good idea. If you receive messages like report, you should send your input, data, output, and license number to firstname.lastname@example.org.
Hi Linda and Bengt and others: I have 2 questions to start regarding MG analysis. I am performing MG analysis for a sample N=1012. I have 1 exogenous latent factor and 3 endogenous latent factors (plus several observed variables, including 2 observed categorical DVs). Question 1: Writing syntax to set factor means to 0.I thought I had written the syntax properly, but when I ran tech 4, I found I was not getting the latent means equal to zero for step 1, and as I review the UG, I am unsure if I am writing the syntax re: the intercepts correctly as well. My syntax to set the factor means to 0 was: [a@0]; [b@0]; [c@0]; [d@0]; I then tried: [a@0b@0c@0d@0] and still did not get the factor means at 0 in tech4. Question 2: For freeing the intercepts of the factors in step 1, I was considering the observed DVs only for measurement invariance and my syntax was: [x y z]; [a b c]; [q r s]; is this correct? Or am I supposed to be setting the intercept of the factor mean to 0?
As always, so appreciative of your rapid feedback. Thanks for the slides-they are very helpful. I revised my syntax for step 1 to put the factor means set to zero in the MODEL command and not the g2 command (a la slide #205). I am including mean structure in my analysis. When I add tech4, my latent means are not = to zero. Is this correct? Thanks Linda, Sue
I tested invariance of loadings across two groups (preschool vs no preschool) in the CFA model with two constructs at two times. The results showed I cannot contrain loadings. Two questions:
- I would like to test the invariance of path coefficient in the structural model...can I do that even though the loadings across two groups cannot be constrained (I do have configural invariance)
- if I report the loadings for each group separately I should report undstd estimates, right (because std are not really comparable)? Are the unstd comparable across groups? E.g. preschool group has an unstd loading of 0,54 and no-preschool groups' same loading is 0,80 - can they be compared in terms of size?
Hi. I am trying to establish measurement invariance for a 7-item scale that is examined across two independent samples. The scale is made up of a mix of continuous and categorical items.
So my question is whether Mplus allows me to cross validate my CFA measurement model or check to see whether the model's invariant across the two samples? I would imagine that if it could be done, I'd have to use a multiple group CFA analysis to check for configural, weak, and strong factorial invariance. Though, I am unsure of how to use a multiple group analysis when there are 2 separate samples involved. Thanks.
Multiple group analysis is designed for independent samples where you want to test that certain parameters are equal across groups - see Chapter 13 in the UG for how to do this. It is not clear if your 2 samples constitute samples from the same population so that you are interested in cross validation or represent samples from different populations like male/female. Multiple group analysis is typically used for the latter. It doesn't matter if you have taken 2 independent samples or just 1 sample where you then divide into males/females - given independent observations, you still end up with independent samples.
Tom Cheong posted on Thursday, November 12, 2009 - 3:46 pm
I performed a MG analysis with two groups and referenced notes on Topic 1 and would like to ask the following questions:
1. When allowing all parameters to be free, is it correct that one has to fix the factor means of second group to be zeroes in Mplus while letting both the factor loadings and intercepts to vary? On slide 205, it indicates that only the factor loadings are freed? I gathered from Slide 206 that the df of the "together model -- males and females" (52) is equal to the sum of the two individual models (26 each) so the means of the second group should be fixed to zeroes? Correct? Actually, when I just freed the factor loadings and the intercepts, the model did not run. Then does it mean not all parameters are not free?
2. It is because the factor means for both groups have to be zero when looking at factor loading invariance, one would use the nomeanstructure statement on slide 207 ? I tried with the default (meanstructure) and set the factor loadings to be invariant and the factor means for the second group to be zero and got identical results.
3. It is correct that once the intercepts are fixed to be the same for the groups, the factor means are automatically set to be zero--this is what I got even though I did not fix the factor means to zero. I read from Slide 210 that the no. of factor means was taken into consideration when computing the Chi-square difference.
I think you are looking at an old version of the Topic 1 course handout because the slide numbers do not match what I have.
The models to be compared are given on page 399 of the most recent user's guide which is on the website. Inputs for each of these models are given in the Topic 1 course handout.
To answer your questions, you need to send the outputs you refer to along with the questions and your license number to support@statmodelcom. I can't follow your questions.
Tom Cheong posted on Thursday, November 12, 2009 - 7:28 pm
Thanks for the reply! Sorry for referring to an old version of the handout and my unclear questions. I looked at the page of the user's guide you referred to, applied them to the example data set I have, and was able to follow the procedure, get, and understand the results.
I would like to ask a question re. the factor mean with reference to the guideline on p. 399: Why are the factor means to be fixed at zero in Steps 1 and 2 and freed in Step 3? It seems that with intercepts VARIANT, the factor means have to be equal? Could one let the factor means vary in Step 1 as well?
Thanks a lot.
Tom Cheong posted on Thursday, November 12, 2009 - 8:06 pm
I forgot to mention what motivated the question. To look at population heterogeneity, one looks to see whether the factor means and covariances differ. My understanding of MG analysis is that it first starts with investigating measurement invariance and then proceeds to examining population heterogeneity--if this is correct, why would factor means be held invariant in the first and second steps during measurement invariance assessment? Thank you.
Factor means need to be fixed at zero in all groups when intercepts are free for the model to be identified. If you go forward in the handout, you will find a test of factor means being equal. To do this test factor means are zero in all groups versus factor means being zero in one group and free in the other groups.
Tom Cheong posted on Friday, November 13, 2009 - 12:06 pm
Thank you, Dr. Muthen.
Nicolas M posted on Saturday, June 26, 2010 - 2:25 am
Dear Professors Muthèn,
I have 5 indicators representing health. I tested the measurement invariance for the factor across 5 groups using the MULTIGROUP option and the constrained model (same thresholds and loadings across groups) was clearly rejected by the DIFFTEST procedure.
Actually, my hypothesis is that one of the indicator (a subjective evaluation of health) is not evaluated in the same way by the individuals in each group. Could I test this by doing a path analysis (multinomial regression) of this indicator on the others (that are objective measures of health)?
I did this by specifying a unconstrained regression model (with different thresholds for the dependent variable across groups) and then a constrained one (same thresholds across groups). Again, using DIFFTEST, the constrained model is rejected (p<0.05).
Does it sound like a reasonable approach to you and does it allow me to conclude that subjective health is evaluated in a different way in each group?
I wonder whether multiple group multilevel SEM (MG MSEM) can be applied to my data of three consecutive years. The data of each year have students' responses to same or similar background variables and IRT scaled math scores with same mean and SD, as well as principals' responses to same or similar school-level variables. The problem is that in each year students were sampled by two-stage stratified cluster method which is being used in PISA and TIMSS. So, students and schools sampled in each year are not the same. In my opinion, however, they are representative samples of national population of each year. Moreover, background variables and math scores of each year seem comparable. I was thinking that each year's students and schools can be formed as an independent group and the three groups can be campared using MG MSEM. I guess MG MSEM for my data isn't a good match. Then,what is your opinion? How can I statistically compare the three-year data using MSEM if MG MSEM isn't possible? Thanks in advance!
It sounds like you are interested in change over time. In this case, the same individuals must be measured at multiple time points using the same measurement instrument. In your case it sounds like both the sample and the measures change over time. I think it would be hard to make a case for making a comparison in this case.
jessy posted on Friday, November 26, 2010 - 10:34 pm
Dear Dr. Muthen,
I have a simple question. From a previous discussion and an answer provided by you and Bengt, I realize that ordinal variables, even if they are at 3 levels or more can be treated as continuous in the presence of MLR estimator. (these are independent variables by the way as my dependent variables are scale) However my reviewers want more justifications. Do you possibly have any article/s in mind I can read and quote that could provide me with such justification? OR any other information that would be helpful in this case? Sincerely, Jessica
In all regression books, you will see that observed exeogenous variables, covariates, are either continuous or binary and in all cases are treated as continuous in the estimation of the model. If you have a three-level ordinal variable, you can either treat it as continuous or create two dummy variables.
jessy posted on Saturday, November 27, 2010 - 10:46 am
I am running a path analysis and I want to see whether the coefficient of the variable ‘IP’ on the variable ‘rad’ is different for small versus large firms. I have run the model below (with default settings) and obtain a Chi-square value of 7.326 and 6 degrees of freedom.
If I force the coefficient of ‘IP’ on ‘rad’ to be equal across both groups (by adding (1) right after ‘rad ON IP’), then I obtain a Chi-square value of 7.885 and – again as in the unrestricted model - 6 degrees of freedom.
I order to compare the fit of both models, I would have to subtract the Chi-square values, and also the degrees of freedom. However, the degrees of freedom are the same for the restricted and default model. How do I deal with that?
Thank you in advance for your help,
DATA: FILE IS C:\CLEANED_verkort.dat;
VARIABLE: NAMES ARE pm04 pm01 sz IP intRD extRD hi rad lic class; USEVARIABLES ARE pm04 pm01 IP rad lic intRD ; CATEGORICAL IS lic IP;
GROUPING IS class (0 = small 1 = large); missing are .;
ANALYSIS: parameterization = theta;
MODEL: rad ON IP; rad ON lic intRD; lic ON IP; pm04 ON rad lic IP pm01 intRD; IP on intRD;
I found variance between 2 groups on two paths within a structural model so I ran a partially constrained model. this partially constrained model differs from the fully constrained model suggesting group moderation. I want to report on the standardized structural path coefficients for the partially constrained model. When I look at the output, it looks like the unstandardized path coefficients are the same across the two groups except for the paths I allowed to vary. However, when I look at the standardized path coefficients they all differ between groups even those that should have been constrained across groups. Is this accurate? And then what standardized path coefficients do I report for this multiple group analysis.
Standardized coefficients are functions of not only the slopes but also the variances of the variables. They are therefore not invariant unless the variances are. With group comparisons, it is better to report unstandardized coefficients.
anonymous posted on Sunday, July 07, 2013 - 3:04 pm
I'm testing a model predicting variables Y1-Y6 from variables X1 and X2 and interested in whether gender moderates these paths. I tested a multiple group model (male and female) and used the Satorra-Bentler scaled X2 difference test, testing each path individually because I really would expect based on theory that not all of the paths would be moderated by gender. However, the final model--which includes 5 paths that are different across gender-- has a poor SRMR (.191) with otherwise strong fit statistics. However, a multiple group model that fixes all of the paths to be equal across gender has a strong SRMR (.027) even though the Satorra-Bentler scaled X2 difference test suggested than more then half of the paths were not significantly different for males and females. Does it make sense for the Satorra-Bentler scaled X2 method to lead to a potentially worse fitting model? Does this suggest that perhaps moderation should be tested instead at the model level? I'm not sure what to make of the worsening of the SRMR in this case.
Does your model fit well in each group when tested separately?
anonymous posted on Monday, July 08, 2013 - 8:40 am
I apologize, in my post above I meant to say: "a multiple group model that allowed all paths to be freely estimated has a strong SRMR (.027) even though the Satorra-Bentler scaled X2 difference test suggested than more then half of the paths were not significantly different for males and females." So this was essentially a multiple group model without fixing any of the paths.
And yes, the model does fit well when each group is tested separately.
I don't think these findings contradict each other.
anonymous posted on Monday, July 08, 2013 - 1:03 pm
I'm sorry, I think I'm confused. Why would a final model that reflects which paths were and were not significantly different across gender according to the Satorra-Bentler scaled X2 difference test (some paths were found to differ based on gender using the Satorra-Bentler scaled X2 difference test and some were not so in the final model some of the paths were fixed and some were not) then have a poor SRMR value when a model that is inconsistent with the findings of the Satorra-Bentler scaled X2 difference tests has a strong SRMR?
When nothing is held equal across groups, it is the same as running each group separately. You said that the model fit well in each group so I am not surprised that a multiple group analysis with no constraints would fit well. And I'm not surprised that a model with constraints does not fit as well. This question may be more appropriate for a general discussion forum like SEMNET.
Drs Muthen, I have two MGROUP questions. I'm running a multiple group correlational analysis across four groups with all continuous variables to determine which groups I can collapse for further analysis.
1) Using the MGROUP command with ML estimator I constrained my paths of interest to be equal and freely estimated all remaining paths. The output gives a chi-square contribution from each group which I interpret to be the reduction in fit for each group given the constraints. Can I compare each chi square at 1 df to determine which groups are significantly different (and therefore can't be collapsed)? Modification indicies for each group are provided but I'm unclear what this is telling me?
2) I also ran a separate MGROUP models comparing group 1 to group 2, group 2 to group 3 etc. When I compare modification indicies to the stdyx correlations, the mod indicies are are sometimes large for small differences in correlation (e.g .08 or smaller) across groups and sometimes small (less than 3.84) for larger correlation differences across groups (e.g. .2 or more). Shouldn't there be a general trend?
Thanks so much in advance, this message board is amazingly helpful.
I am looking at how well variable X explains the variance in y not predicted by Z (all latent variables - X and Z are correlated).
I have three questions:
1) Is this correct? Y by a1-a15; X by b1-b15; Z by c1-c15; Y on Z; Y@0; Yres by; Y on Yres@1; Yres on X;
2) I have noticed that if I swap X and Z (i.e. Y on X in the first line and Yres on Z in the last) the unstandardised estimates for the relationship of X to Yres in this model are the same as those for X to Y in the original model, as are Z to Yres and Z to Y - however the standardized estimates are different. My question is why does this happen? If X and Z are correlated, then shouldn't whichever variable is regressed on Y explain more variance than when it is regressed on Yres (i.e. the variable regressed on Y should account for the part of Y that the common variance between X and Z explains, plus its unique variance?).
3) Testing for multi-group invariance in the above model. As I understand it, firstly I test for invariance of the measurement parameters (constrain equal factor loadings, thresholds, and factor means to zero) and then constrain equal the structural parameters in the next step. Is this correct? Can I continue to test equivalence of the structural paths if factor means are not invariant?
1) - 2) Don't you get the same results with this model as when you say:
Y by a1-a15; X by b1-b15; Z by c1-c15; Y on Z X;
? If so, use those standardized estimates.
3) With invariance you don't fix factor means to zero, but yes you test measurement invariance first with free structural parameters and then you can constrain the structurals. You need only the metric invariance for testing structural paths.
I am doing a multiple group analysis using Mplus. I have constrained all the structural paths to be equal across groups by using the overall model command. I did not use any group specific command. Still Mplus generated separate reports for each group and the path parameters of each group are different. Why did this happen? Shouldn't there be just one estimate for each path in both groups?
I am running a SEM model and I further intend to test for structural invariance across two groups - please note that I am using observed variables, so there is no measurement model in place.
I have tested for: 1) equal structural weights - found only partial invariance, with 5 out of 22 regression pathways being variant across groups
2) equal factor means- found only partial invariance, after relaxing the intercepts of 6 variables (either dependent or dependent and independent)
I would further like to know if the estimated means for those 6 variables are significantly different across groups. I have read about fixing the means to 0 in one group, but that is not quite my intention - I know some estimated means will be different from 0 but I want to know if they are different from the estimated mean for the other group, which are not 0.
I have looked into the Tech 4 output, and the t-test is significant for all variables - not only the 6 that I relaxed - so I take it this t-test is again in comparison with 0 as a reference value.
Could you please advice on how to determine the significance of difference between the estimated means of two groups?
Thank you very much! My kindest regards, Paula Vagos
Paula Vagos posted on Wednesday, June 21, 2017 - 3:54 pm
Thank you Dr. Muthen. I will look into that...
Paula Vagos posted on Thursday, June 22, 2017 - 4:45 am
Hello Professor Muthen, I have some follow-up questions, if I may:
1) the model test command will compare if the intercepts (and not the estimated means) are equal or the same, right?
2) when I include six model test (one for each intercept that was allowed to differ) I still get only one wald chi-square statistic with six degrees of freedom. Is there a way to get a wald test for each of the model test? So, in this case, six wald statistics with 2 degrees of freedom each?
3) when constraining the intercepts to be equal (either fully or partially), there are some estimated means - as seen in TECH 4 - that change direction (i.e., they were higher for one group when no intercept constraint was in place and then become higher for the other group when that constraint is placed upon the model). Is this explicable?
4) Would you advice using the estimated means and SE to compare means and cohens' d calculation?
Again, thank you very much. My kindest regards, Paula Vagos