Judy Black posted on Wednesday, October 31, 2012 - 4:11 am
I am using the new 3-step approach to examine latent class. After finishing the first two steps, we got 3 classes. We would like to examine how depression scores changed within each of the class. In SPSS, we could use repeated measures ANOVA to examine whether/how depression changed within every class. Is it possible to do ANOVA/t-test in Mplus 7? Or, is there any other analysis that can be used to examine the change scores of depression within every class?
Judy Black posted on Thursday, November 01, 2012 - 1:46 am
Hi Linda,thanks for your reply. Here is the syntax of my third step. What commands shall I use within the model part to have a t-test?
TITLE: latent class ana DATA: FILE IS test7.dat; VARIABLE: NAMES ARE T1CESD T1CONTR T2CESD T2CONTR T3CESD T3CONTR p1 p2 p3 n; MISSING IS ALL (99.000); USEVARIABLES are T1CESD T2CESD T3CESD n; ClASSES = C(3); nominal=n;
If your T1CESD T2CESD T3CESD variables are covariates, just say
AUXILIARY = (R3STEP) T1CESD T2CESD T3CESD;
That's all. This will give you the tests you need.
If they are not covariates, use
AUXILIARY = (D3STEP) T1CESD T2CESD T3CESD;
Lewina Lee posted on Friday, November 02, 2012 - 5:00 pm
Dear Drs. Muthen,
Could you please explain when we need to use the AUXILIARY statement in Step 3 of the new 3-Step approach to mixture modeling?
In your reply to Judy Black's questions, you suggested using AUXILIARY = (R3STEP) T1CESD T2CESD T3CESD;
In Webnotes No. 15 v. 5 (Appendix F), it does not specify an AUXILIARY statement. The code is: variable: Names are u1-u10 y x p1-p3 n; usevar are y x n; classes = c(3); nominal=n; data: le=man3step2.dat; Analysis: Type = Mixture; starts=0; Model: %overall% Y on X; (and so on..)
I would like to regress a distal outcome on the latent class variable (C) & several covariates in Step 3. Do I need to specify the outcome & covariates in an AUXILIARY statement?
Thank you very much for your help on this new technique.
There is a distinction between the automatic and the manual approach to 3-step.
The automatic approach use AUXILIARY and does therefore not need to use the [n#1@] type of statements. The automatic approach is suitable for exploration of covariates and distal outcomes in an LCA. An input example showing the covariate case is given in the Version 7 UG ex 7.3. See also the V7 UG pp. 553-554.
The manual approach does not use AUXILIARY and needs the [n#1@] type of statements. The manual approach is exemplified in Appendix F of Web note 15 for the case where the variables not included in the latent class formation represent a regression model.
If you want to regress a distal outcome on the latent class variable (C) & several covariates in Step 3, you need to take the manual approach.
Lewina Lee posted on Sunday, November 04, 2012 - 1:35 pm
Great -- clear to me now. Thank you very much for your explanation!
Lewina Lee posted on Wednesday, November 07, 2012 - 6:34 am
I tried the manual 3-step approach. In the third step, I received an error message when I tried to regress a binary distal outcome (CHD) on a binary latent class variable (c). Could you please help?
These are the model statements and error message I received. I also tried including the ALGORITHM=INTEGRATION statement to the ANALYSIS section (as suggested in the error message), but that did not help.
%OVERALL% CHD on T1AGE; c on T1AGE; CHD on c;
%C#1% [N#1@ 2.682732393]; CHD on T1AGE; c on T1AGE; CHD on c;
%C#2% [N#1@ -2.923583166]; CHD on T1AGE; c on T1AGE; CHD on c;
*** ERROR The following MODEL statements are ignored: * Statements in the OVERALL class: CHD ON C#1 * Statements in Class 1: C#1 ON T1AGE CHD ON C#1 * Statements in Class 2: C#1 ON T1AGE CHD ON C#1 *** ERROR One or more MODEL statements were ignored. Note that ON statements must appear in the OVERALL class before they can be modified in class-specific models. Some statements are only supported by ALGORITHM=INTEGRATION.
Thank you, Lewina
Judy Black posted on Wednesday, November 07, 2012 - 7:45 am
Thanks for your reply.
I tried the commands you suggested, but got an error. You may see the output here:
Mplus VERSION 7 MUTHEN & MUTHEN 11/07/2012 4:27 PM
*** WARNING Variable name contains more than 8 characters. Only the first 8 characters will be printed in the output. Variable: EDUCATION *** WARNING in MODEL command All variables are uncorrelated with all other variables within class. Check that this is what is intended. 2 WARNING(S) FOUND IN THE INPUT INSTRUCTIONS
latent class analysis Nov 7th
*** FATAL ERROR THERE IS NOT ENOUGH MEMORY SPACE TO RUN Mplus ON THE CURRENT INPUT FILE.
Lewina Lee posted on Wednesday, November 07, 2012 - 8:00 am
I think I resolved the issue I had earlier -- I am retracting my question above (Wednesday, November 07, 2012 - 6:34 am). Sorry for the inconvenience.
Hello, I am using the 3-step manual approach Mplus 7 to regress a distal outcome on the latent class variable and covariates. And I may be just completely overlooking something but- Is there any way to test the equality of means of the distal outcome across classes(like in the du3step) while also adjusting for covariates? My third step output gives my outcome (y) on covariates (x1-x8) and the intercept and residual variance for each class. I was hoping to get a comparison across classes, is there a way to request this or a manual approach?
Leslie Roos posted on Tuesday, February 05, 2013 - 11:42 am
I am using 3-step approach in MPlus 6 to regress a distal categorical outcome on latent class variable (complex stratified dataset), currently in the 3rd step following procedures outlined in Feingold, Tiberio & Capaldi (2013) and Asparouhov & Muthen (2012).
I am stuck at receiving the error: *** ERROR One or more MODEL statements were ignored. These statements may be incorrect.
Any advice would be highly appreciated! Leslie
ANALYSIS: TYPE = COMPLEX MIXTURE; ESTIMATOR = MLR; ALGORITHM = INTEGRATION; INTEGRATION = MONTECARLO; MODEL: %OVERALL% aax1or2 ON cGROUP; jail ON cGROUP;
Hi, I am performing the manual 3-step for an LCA model with covariates and three distal outcomes (two of which are binary and one is continuous). I am unable to run the final step with all three distals and covariates.
Two problems happen. First, when I ran the model with only the continuous distal, the covariate relationships were estimated, but the Wald test for the comparison of the distal outcome means (using model test) produced the following error: “WALD'S TEST COULD NOT BE COMPUTED BECAUSE OF A SINGULAR COVARIANCE MATRIX.”
Second, when I ran the model with a binary distal (which was treated as continuous), I received the following error message under the Sample Statistics section:
“THE ESTIMATED COVARIANCE MATRIX FOR THE Y VARIABLES IN CLASS 1 COULD NOT BE INVERTED. PROBLEM INVOLVING VARIABLE (F1EVERDO). COMPUTATION COULD NOT BE COMPLETED IN ITERATION 4. CHANGE YOUR MODEL AND/OR STARTING VALUES. THE MODEL ESTIMATION DID NOT TERMINATE NORMALLY DUE TO AN ERROR IN THE COMPUTATION. CHANGE YOUR MODEL AND/OR STARTING VALUES.”
First, note that Web Note 15 which describes the 3-step approach has recently been corrected. The corrected version can be found on our home page. This means that the manual approach has changed and that R3STEP, DE3STEP, and DU3STEP have errors. Corrections are included in Mplus Version 7.1 which we plan to release at the end of March. Results are however quite similar.
Second, these types of singularities may be due to an observed variable being constant within a class. Please check using the most likely class approach.
Third, when using a binary distal variable, DU3STEP should be replaced by DE3STEP.
Hi, I am using the calculations based on the revised Web Note. I also checked that there is variance in each of my classes for my three distals using the most likely class assignment. However, I still am not able to test the equality of means using "model test." Is there a different way for testing the equality of means in the context of the manual 3-step?
Hello, I conducted a mixture model analysis and then subsequently evaluated evaluated covariates and distal outcomes using the new approaches in version 7. I was asked to justify why I would use the Wald test over anova/manova for my distal outcome means comparison. Is there any specific resource you can direct me that would help validate the use of the Wald test for comparison?
anova/manova doesn't exist for mixture modeling and Wald does where Wald is essentially the same as anova if it were available for mixtures.
Lewina Lee posted on Thursday, March 14, 2013 - 6:16 am
I am running an LCA on data from a sample at Time 1, and using DU3STEP to compare classes on outcomes at Time 2. Only about 60-70% of my LCA sample has data on Time 2 variables. How can I obtain the class-specific Ns for each distal outcome I specify with DU3STEP?
Not sure what you mean by class-specific Ns - if you are referring to sample size or the means of the nominal N variable representing latent class. If you refer to sample size, you can get that by doing a Type=Basic run.
Note that the Version 7 DU3STEP has an error that is corrected in the upcoming version 7.1. See also Web Note 15 which shows how to do this manually.
Lewina Lee posted on Thursday, March 14, 2013 - 9:58 am
When I ran DU3STEP, the output reads: LPES06 Mean S.E.
Class 1 15.613 0.246 Class 2 14.222 0.394 Class 3 15.294 0.215
Overall test 9.092 0.011 Class 1 vs. 2 8.836 0.003 Class 1 vs. 3 0.702 0.402 Class 2 vs. 3 5.488 0.019
Because my LCA was ran on N=1076, and only a subset of the sample had data (N=649) on variable LPES06 (auxiliary distal outcome), I want to know the split of the N=649 into Classes 1, 2 & 3.
That is, in the above, what N is the Class 1 mean of 15.61 (SE=.246) based on?
Hi there, I get the following problem in 3rd step when using LC as a moderator.
*** WARNING in MODEL command At least one variable is uncorrelated with all other variables within class. *** ERROR The following MODEL statements are ignored: * Statements in Class 1: [ N#1 ] [ N#2 ] [ N#3 ] [ N#4 ]...
My command as follows: (I have double checked my way of calculating the [N#values], by recalculating those in the example of web note 15,v6):
variable: names are Nu E O A CO EAU AU4 CPROB1-CPROB5 n; usevar are EAU AU4 n; classes=c(5) NOMINAL=n;
DATA: FILE=unweight2.dat; ANALYSIS:TYPE=MIXTURE; starts=0; MODEL: %OVERALL% AU4 ON EAU;
I have an LCA model in which the covariates influence the measurement model. It is similar to UG example 7.12, except the covariates do not have direct effects on the indicators. The covariates are binary (e.g. gender). How do you output the mean of the covariate for each latent class? For example, I would like to know the proportion of people within each class that are female. Thank you!
If you include the x in the model, it is harder because the x parameters are not part of the model. Unless you bring them into the model, mentioning say their means, and then do Model Test on them. But that is a heavier model, particularly if x's have missing data.
Thanks for your reply. I would like to keep the covariates in the LCA measurement model. What if I used the manual 3-step procedure (Web note 15 v. 6) and ran the measurement model with covariates in step 1. Is step 3 flexible such that I could leave the covariate out of the LCA model, but specify it on an auxiliary statement instead using the (e) or (r3step) command?
No. And the (E) option is now more or less superseded by the DU/E3STEP and DCAT/DCON options (see the Version 7.1 Mplus Language Addendum).
But to me, if you have gender as a predictor of latent class in your model, it seems that this model tells you what you need already - the significance of the gender effect is reported (males have significantly different latent class logits than females). If you are instead interested in how significantly different the gender proportions are in the different classes, shouldn't the model be one where gender is a distal outcome instead of a predictor?
You are correct about already having what I need already in terms of gender differences across classes (the mean is estimated in the residual output, and significance test from the regression). Thanks for pointing that out. The remaining piece that I am unclear on is how to include gender in the measurement model, but still use R3STEP to examine the association between covariates and latent classes, while controlling for gender in the multinomial logistic regression. Is that possible using the "automatic" or manual r3step procedure?
So you have a model with say u1-u5 as latent class indicators and gender as an x variable with c on x. And then you want to check other potential covariates by having them predict the classes determined by the u1-u5 plus x information. If I am understanding you correctly, you can do that by manual R3STEP.
Wendy Liu posted on Thursday, June 13, 2013 - 5:12 pm
Dear Professor Muthen, I have used the manual 3-step approach (Web Notes: No. 15, V6) for an LCA model with one continuous covariates and one distal continuous outcome. The output file, under the section of “Model Results”, showed the following results
Jamie Taxer posted on Thursday, August 29, 2013 - 3:13 am
Dear Dr. Muthen,
I have conducted an LPA with a sample size of 260 and replicated the classes in a second sample with 193 participants. I would now like to see if the classes differ from one another on multiple distal outcomes. I was planning on using the 3-step approach, but I am not sure if my sample size is big enough for it. I noticed in presentation online that you recommend using the 3-step method with a sample size of 500 or more. What would you suggest, should I use the 3-step approach or the pseudo-class draw approach?
I am running 3 step LCA with distal outcome (web note15). I could run the program but I am not sure how I can run Wald TEST with outcome of each class. (my outcome is a continuous variable). When my understanding is correct, intercept of each group is mean of each group's outcome. I'd like to compare one group's (e.g., g1) outcome is significantly higher than another group(e.g., g4). Would you please tell me how to do it? I would very appreciate if you can indicate reference point or syntax for it.
I am implementing manual 3 step because I'd like to predict my outcomes at w2 by controlling previous outcomes (W1) to examine how my latent subgroups significantly related to my outcomes.
1. Does this idea fit in Mplus' manual 3 step LCA with distal outcome? (In other words, manual 3 step can test this model?)
2. I'd like to compare "intercept" of each class while controlling for previous outcomes (W1) because I can show how my latent classes are related to my dismal outcomes. 1) Is it OK to compare "intercept" to show how each class has different "mean" of dismal outcomes? If so, how can I compare "intercept" of each class?
Jamie Taxer posted on Thursday, October 17, 2013 - 3:15 am
Dear Dr. Muthen,
I am doing the automatic 3 step approach to test for differences between my latent classes. When testing for distal outcomes using the DU3STEP command I get the following message for several variables and classes:
PROBLEMS OCCURRED DURING THE ESTIMATION FOR THE DISTAL OUTCOME EMOEX. THE LATENT CLASS VARIABLE IN STEP 3 HAS MORE THAN 20% CLASSIFICATION ERROR RELATIVE TO STEP 1 IN CLASS 1.
For these variables I am also only getting values of 999.000 for the equality tests. What does this mean and is there some way that I can fix it?
This means that you should not use DU3STEP. Instead, try the new option of DCON/DCAT. These issues are described in web note 15.
Jamie Taxer posted on Thursday, October 17, 2013 - 7:06 am
Thank you! Does this mean that I should do two separate analyses? One time R3STEP to see which variables predict the latent classes and DCON to analyze the distal outcomes, since In Lanza's method latent class predictors should not be included in the model?
Drs. Muthen, I could run the manual 3-step approach. I'd like to test whether DV' mean of each class are significantly different or not. As your recommendation (web note 15),I ran Wald test. However, once I ran it, I can only assign the same variance of DV among groups. Please take a look of my syntax.
We ran a latent class analysis that settled on a 6-class solution, and used r3step to test predictors of class membership. Because of the high number of classes and because some of them look very similar, we were wondering if it would be possible to combine some of them and re-evaluate the influence of predictors. Is this possible with r3step or the manual 3-step method?
I created a new most-likely latent class variable, with the original six levels recoded into three.
Then, I re-calculated the classification uncertainty rates using the average posterior probabilities table, the proportions based on the estimated model, the proportions based on most-likely class membership, and the following formulas:
P(A or B) = P(A) + P(B) - P(A and B)
P(B|A) = P(A|B) * P(B) / P(A)
Finally, I calculated the logit table and used the values for step 3 of the 3-step method.
I am running an LPA with the automatic R3STEP and DCON procedures for covariates and distal outcomes. I have three separate data sets that I am combining and would like to account/control for this in the the actual profile development. Do you have any suggestions on ways to do this (or point me in the right direction in the literature)? Currently, I have just created a set of dummy codes and regressed c onto them (syntax below). Might this be an appropriate way to control for the data sets in the class development? Thank you for your time.
MODEL: %OVERALL% u1; u2; c ON dummy1 dummy2;
Jamie Taxer posted on Monday, November 25, 2013 - 9:20 am
I am using r3step to check for predictors of my latent profile groups and would like to know if it is possible to receive the class level means of each of the predictors. Next to whether or not the groups significantly differ from one another, I would like to report each classes mean and confidence interval for the predictors.
Your input says that the class probabilities can change over the data sets. But the item probs for each class will be the same, so measurement invariance is specified. To avoid MI you can explore direct effects from dummies to class indicators; we talk about that in our teaching of Topic 5, also giving a ref to Clogg work on multiple-group LCA in I think Soc Meth.
A more general Mplus approach than using dummy covariates is to let data set be represented by a Knownclass variable in which case you have great flexibility in checking data set invariance for the model parameters.
That does not come out automatically with R3STEP, so instead I would recommend using the Most likely class membership as an observed variable in a secondary analysis where you get the means and their SEs computed.
Thank you for your response. I understand I can explore MI across data sets with the knownclass command. There does seem to be some minor differences in estimates when I go that route.
To clarify your first point, would it be accurate to say that specifying direct effects from the dummy variables to the indicators controls for differences across the studies in the profile estimation?
Thanks for your help, Bengt. One other question if you don't mind: I am modeling distal outcomes of my profiles using the DCON command (given high classification error in some instances). However, due to listwise deletion of missing data, my sample size drops to 173 for some outcomes. Do you think this sample size is too low for reliable estimates? Also, is it possible to impute missing data when running this analysis? If so, would you be able to provide syntax in the analysis statement for that? I'm not sure how to include type = mixture AND missing. Thanks again for all your help.
I don't know the percentage missing, but I would do simple descriptive stats on the not missing variables in your model, comparing the sample with missing on the DCON variable to those not missing. So you see how selective the missingness is.
Missing data handling is not implemented for 3-step. Not sure if we do 3-step for imputed data using Type=Imputation data; try it.
Karen Kochel posted on Thursday, January 16, 2014 - 11:15 am
I am conducting LPA with the goal of regressing three distal outcomes on latent class variable (with 2 continuous indicators) and 1 covariate and then examining equality of means (via Wald Test) on the distal outcomes. I first estimated a joint model (i.e., I combined the latent class model and distal outcome model) and entropy was high (i.e., >.90), but the inclusion of the distal outcome resulted in a substantial change in latent class formation so that the latent variable lost its intended meaning. Having reviewed Web Notes 15 and discussion posts, it seems a 3-step approach is warranted; am I right? If so, and given that I have a distal outcome and covariate, will I need to use the manual 3-step approach? Or can DU3STEP and R3STEP be used in a single analysis to explore the distal outcome and covariate, respectively? Will each distal outcome need to be investigated within a separate analysis?
DU3STEP and R3STEP cannot be used together which means that if you have a covariate you would have to do the "manual approach" mentioned in the web note, but that would likely also change the class formation.
If you don't include the covariate, the DCON/DCAT approach is best.
Thank you for your help! I have two follow-up questions related to this four class LPA. (1) To evaluate the effect of latent profile on distal outcome while controlling for prior levels of the distal outcome (i.e., the covariate), I regressed the distal outcome on the covariate (per Linda's suggestion to another user). I would now like to evaluate the relation between profile membership and distal outcome. Is it acceptable to use model test to examine intercept differences in the distal outcome (because the mean is not provided in this case, correct)? For example:
(2) I used the 3-step manual approach (with guidance from Webnote 15 and Vermunt, 2010) to evaluate the abovementioned models. In a manuscript, should I be presenting statistics (e.g., a comparison of true values and estimates) as evidence that the 3-step procedure works well with my data? If so, what values do you recommend I present? Thank you!
So in your script here you are not doing 3-step but instead a regular LPA with a distal regressed on a covariate? Note that this may change the class formation relative to not including the distal in the model.
But apart from that, yes you can test intercept differences this way.
Not sure what you mean in (2); not enough information about what you did. And, what do you mean by "true values"?
Thanks for your response. To clarify: I followed the 3-step approach outlined in Web Note 15. In Step 1, I estimated my LPA and included the auxiliary option but did not specify a type (so that my auxiliary vars would not be used in estimation but rather saved in savedata file). I obtained support for a four class solution. In Step 2, I “computed” error for most likely class (in reality, used the “logits for the classification probabilities for most likely class” obtained in Step 1). In Step 3, I used most likely class as the indicator variable and fixed measurement errors at values in Step 1 output. Also in Step 3, I specified auxiliary model: I regressed the distal outcome on the covariate (i.e., prior levels of distal outcome). Am I correctly implementing the 3-step approach? And, I can now test intercept differences via the Wald test?
Re: question (2) in my prior post: in Web Note 15 you present an example in which you conduct the 3-step with an arbitrary 2nd model. You conclude that the procedure works well for your example because your estimates obtained in the final stage are close to true parameter values. Elsewhere, I’ve seen you suggest that when class separation in the LCA is good, then N is a good indicator of C. Should I be evaluating if the 3-step performs well with my data and, if so, how do I do this? In other words, do I need to justify why I am using this 3-step versus, say, the 1-step? Thank you again!
Regarding your first paragraph, you want to check that your latent class formation (percentage in the classes, same people in the classes) in Step 3 is close to that of Step 1 - often it isn't in which case the stepwise method loses its advantage/meaningfulness.
Regarding your second paragraph. I think the only justifications you need to make are
(1) substantive reason for wanting classes to be defined only by the LPA indicators, not the distal
(2) Step 1 and Step 3 classification agrees.
Kathleen posted on Wednesday, February 05, 2014 - 9:56 pm
I am trying to implement the 3-step approach for an LTA, although the entropy at each time point > than 0.87. I was looking at Webnote 15, and slides from a presentation at the Modern Modeling Methods Conference, UConn, May 21, 2013. From slides 7-9, it appears the values for the nominal variable are fixed at the log-odds of the results from the "Average Latent Class Probabilities for Most Likely Class Membership..." However, in the Webnote, the results from the "Logits for the Classification Probabilities..."are used. Which method should be used, and how can I relate the Average Latent Class Probs to the Logits? I see the formula for the logits described on page 4, but I haven't been able to replicate it.
I am using the new automatic auxiliary command, R3step, in an LPA. Is it reasonable to include some covariates in the model command (e.g., "c on age") and others in the auxiliary command ((R3Step) pred1, pred2))?