Message/Author 

Jon Heron posted on Tuesday, September 04, 2012  2:08 am



Hi Bengt/Tihomir thanks for a very inspiring few days in Utrecht. There was a brief discussion last thursday on the topic of how to deal with missing data amongst covariates when you get to the third step of the 3step approach. I meant to quiz you further on this but didn't get the chance. I am about to submit a paper where I dump my data into a chainedequation imputation model prior to the "modalML" third step as a way of filling in gaps in the covariates. This seemed to work quite well. I also have an applied paper in print where I tried imputation as the first step  this was a nightmare as I had to fit 100 mixture models and pool the results. My sense is that this is far from an answered question and that there are still many interesting avenues to explore  would you agree? perhaps you already sort of know the answer. My reason for asking is that i'm toying with the idea of making a PhD project around this, combining simulations and reallife data. all the best, Jon 


Good to see you there, Jon; great meeting. Our current thinking is that you do step 1 and 2 as usual and only deal with missing on x in step 3, doing the missing on x by Mplus multiple imputation (perhaps using other variables to impute). So step 3 will be a Type=Imputation run over the different ximputed data. It is a research topic as you say. 

Jon Heron posted on Tuesday, September 04, 2012  7:02 am



Thanks Bengt, I'll have a play around with that. On a semirelated note, I have just finished estimating a 3step (C on X) model in Mplus v6 and can report that the model based on modal assignment agrees perfectly with Latent Gold  no surprise there! I have also attempted to implement Vermunt's "Proportional ML" 3step model which uses the probability weights and an alternative classification matrix. Here the estimates are great, but the SE's are attenuated as Mplus' "weight" command is rescaling the sample size so my assignment probabilities will no longer sum to 1 within respondent. I think you said there was a trick for this? 


Try WTSCALE=UNSCALED in the VARIABLE command. 

Jon Heron posted on Tuesday, September 04, 2012  7:23 am



Needs a twolevel analysis apparently :( 

Jon Heron posted on Tuesday, September 04, 2012  8:33 am



Not to worry, I'm planning on sticking with modalclass anyway, just be nice to finish that off neatly. Last post (promise!). I set my self the challenge of 3step model where exposure and outcome are derived separately from mixture models (C on C). Turned out to be simple: usevar = mod_y mod_x; nominal = mod_x mod_y; classes = new_x(4) new_y(3); Model: %overall% new_y on new_x; model new_x: %new_x#1% [mod_x#1@13.785 mod_x#2@9.598 mod_x#3@9.634]; %new_x#2% [mod_x#1@1.236 mod_x#2@1.782 mod_x#3@0.507]; %new_x#3% [mod_x#1@1.851 mod_x#2@0.345 mod_x#3@2.545]; %new_x#4% [mod_x#1@13.619 mod_x#2@2.233 mod_x#3@2.247]; model new_y: %new_y#1% [mod_y#1@6.606 mod_y#2@3.949]; %new_y#2% [mod_y#1@1.210 mod_y#2@2.0707]; %new_y#3% [mod_y#1@2.322 mod_y#2@0.522]; 


Looks good. You are certainly an "early adopter". Be sure to take the misclassification errors from a separate analysis for each latent class variable and not the joint one. 

Jon Heron posted on Thursday, September 06, 2012  5:54 am



Thanks Bengt, good tip. These are both from separate alreadypublished LCA models. Something that comes out of Vermunt's paper that is a real surprise, though consistent with your own simulation study is that proportional assignment is actually *worse* than modalclass. We are having trouble convincing colleagues of this fact. If you calculate Vermunt's Dmatrix (the flipside of Mplus' classification table) for the two approaches then the one for modalclass is better in terms of the magnitude of the probabilities on the main diagonal compared with proportional assignment. I was initially shocked to discover this matrix was not actually the identity matrix when using proportional assignment, but you can't argue with the math. 

Jon Heron posted on Friday, September 14, 2012  7:31 am



Hi Bengt/Linda I've been having a play today  attempting to incorporate imputation prior to the final step of the 3step approach. Parameter estimates look believeable, and pleasingly similar to the complete case findings, however the pvalues look too high For instance, with a 4cat exposure and a binary outcome, I obtain 3 odds ratios: 1.30 [0.84, 2.00] 1.70 [1.11, 2.62] 1.89 [1.34, 2.66] but model test gives an omnibus pvalue of 0.0513. I have replicated the estimates in latent gold but obtained a lower pvalue (0.0001). I have also removed the fancy 3step bit from both programs and fitted simple logistic models  there is still a discrepancy in pvalues. Incidentally, without the imputation, both packages agree totally, so it's not a mistake in my modeltest statement. I know we need to be wary of significance testing when working with imutation  you can't pool LRtests  but it was my understanding that the Wald was OK. any ideas? thanks, Jon 


Could you send the data? 

Jon Heron posted on Friday, September 14, 2012  11:40 am



OK, I'll try and replicate with a small example so I can send. Of course that usually leads to a solution in itself 

Sar posted on Wednesday, September 26, 2012  4:21 am



Hi, I am trying to manually compute the error rate for N using the output "Average Latent Class Probabilities for Most Likely Latent Class Membership by Latent Class" in Step 2 of the 3Step Mixutre Modeling method. On slide 53 of the Mplus 7 version training handouts (New Developments in Mplus Version 7: Part 2) there is an example for a 2Class model and I was just wondering how to do this for a 4Class model. Also I just had a question about using AUXILIARY=x(R3STEP) with multiple predictors when you look at main effects for each predictors does this covary for other predictors? Thank you 


Going from 2 to more classes simply means that you have to compute more log odds of the kind shown on the next slide, using the values of the classification table ("Average Latent Class Probabilities...}. So compute log of the value for a certain class divided by the last class  with 4 classes this gives 3 such log odds. You do this for every row (every class). And your 3rd step will then have 4 classes instead of 2. The R3STEP procedure gives you a multinomial logistic regression for the covariates included in the auxiliary list. So in line with regular regression, each coefficient is a partial regression coefficient showing the effect of one covariate holding the other covariates constant. A revised version of the Mplus Web Note 15 with more extensive instructions will be posted shortly. 

Jon Heron posted on Thursday, September 27, 2012  11:58 pm



This 3step approach where we correct for the classification error in a categorical manifest variable has got me thinking about another potentially interesting application. I've recently published an LCA model of repeated measures of selfreported smoking frequency. (there are now a fair few of those!!). We've since acquired some measures of cotinine taken from blood samples. I'm wondering about the possibility of using this additional information to get an idea of the accuracy of the selfreport and then using this to adjust the manifest measure. Unlike the 3step which is based on nominal measures, this would be ordinal, but it feels to me like it might be a goer. Not really a question that, just an earlymorning ponderance. 


I like the idea of trying to get a better understanding of the measurement error. What would you say to a Devil's advocate who would want include the cotinine measures in the first step? 

Jon Heron posted on Friday, September 28, 2012  9:09 am



We don't have cotinine assessed at each wave and it may be a year or so before we have it for more than one wave. Hence I envisaged a onetime point comparison that would then inform the next stage of modelling, based on the (dubious?) assumption that selfreport accuracy is constant. My initial hope was we would have repeated pairs of selfreport and biological samples and to do a onestep. Is that enough our a reason not to? 

Sar posted on Wednesday, October 24, 2012  2:10 pm



Hi, I just wanted to confirm that I am correctly computing the error rate for N using the output "Average Latent Class Probabilities for Most Likely Latent Class Membership by Latent Class" in Step 2 of the 3Step Mixutre Modeling method. My main concern is what do I do when I am taking the log of 0, or in the special case where the last class has 0. Otherwise it makes sense, it is just in these cases I am unsure what to do. I have copied a section of my output and the calculations for the first 2 classes below. Thank you so much. Average Latent Class Probabilities for Most Likely Latent Class Membership (Row) by Latent Class (Column) 1 2 3 4 1 0.936 0.000 0.000 0.064 2 0.000 0.959 0.021 0.020 3 0.001 0.016 0.860 0.124 4 0.011 0.000 0.029 0.960 Log (0.936 /0.064) = 2.6827 Log (0.000/0.064)= 0 Log (0.000/0.064)= 0 Log (0.000/0.020)= 0 Log (0.959/0.020)= 3.870 Log (0.021/ 0.020)= .0488 %c#1% [Class4#1@2.6827 Class4#2@0 Class4#3@0]; %c#2% [Class4#1@0 Class4#2@3.870 Class4#3@.0488]; 


You should fix these values to 15, not 0. %c#1% [Class4#1@2.6827 Class4#2@15 Class4#3@15]; %c#2% [Class4#1@15 Class4#2@3.870 Class4#3@.0488]; Alternatively in your computations above replace the 0 probability with 0.0001. 

Lewina Lee posted on Sunday, April 07, 2013  8:25 pm



Drs. Muthen, I am trying to compare classes on multiple distal outcomes, which have significant missingness. I attempted multiple imputation and realized that MI only works with R3STEP but not DU3STEP. Is there a workaround to this? I noticed the earlier posts by Jon Heron addressing perhaps a similar issue. If the manual 3step can be implemented along with MI, at which step do I perform MI? Thank you, Lewina 


If you want to do this using multiple imputation, you must do it for each data set and then correctly combine the results. If you can use FIML, it would be a lot easier for you. 

Lewina Lee posted on Tuesday, April 09, 2013  7:31 am



Thanks, Linda. When you suggested using FIML to handle missingness in class predictors and distal outcomes, did you mean outputting the most likely class assignment for each person, and doing the analysis (e.g., C on X) with a regression approach? I have an entropy of .80. 


No, I meant instead of using multiple imputation to impute missing values to use TYPE=MISSING which is the default in Mplus. 

Lewina Lee posted on Tuesday, April 09, 2013  2:52 pm



Thank you, Linda. I can see that FIML can handle missingness on distal outcomes. When auxiliary variables used in R3STEP are missing, they are treated as exogenous variables and so FIML doesn't apply (i.e., listwise deletion on cases missing Xs). Is there a way to "trick" M+ to treat them as endogenous in the mixture model? 


You can bring them into the model by mentioning their variances. However, each variance would constitute one dimension of integration so this would not be practical. 


I have a similar issue as above. I have a few hundred missing cases on distal outcomes and 1 covariate I want to use in a latent profile analysis using the BCH method. If I use either the outcome or covariate in the analysis, there is listwise deletion of all cases missing on x or the distal outcome before fitting the model right? Or would FIML apply? If the only approach to retaining the hundreds of observations is either MI or lwdeletion, how would I go about imputing? You mentioned above that it would be a daunting task. Thanks! 

Jon Heron posted on Thursday, January 28, 2016  11:39 pm



In principle FIML would apply for your distal outcome, however, while I think FIML can be justifiable when considering missingness in one of many class indicators, I think the MAR assumption can be more questionable for a distal outcome subsequent to you latent class model. (Class indicators are excellent predictors of other class indicators). I've had a recent unsuccessful grant which had the aim of addressing the missing covariate problem and I'm currently reworking this for resubmission. There have been a number of pragmatic solutions adopted in the applied literature. These may satisfy reviewers but perhaps not the biaspolice. Ultimately the solution is probably Bayesian but this is likely to rule out the more complex problem, and the lesshardened analyst. Of course your adopted method should depend on your missing data assumptions. Have you considered sticking with the listwise deletion of covariates and employing IPW (probability weighting) to readjust your sample? best, Jon 

Back to top 