P value and CI contadicted PreviousNext
Mplus Discussion > Dynamic Structural Equation Modeling >
Message/Author
 Yasemin Kahya posted on Tuesday, May 21, 2019 - 1:05 am
Hi,

we conducted multilevel time series analyses via using DSEM.

Some of the findings showed that even if one tailed p value is significant (lower than .05), CI included zero.

How can we interpret these results? Are they marginal, or should we look at only CI and say this is a nonsignificant effect although CIs generally included zero between -.01 .13 for example.

Thank you in advance
 Tihomir Asparouhov posted on Wednesday, May 22, 2019 - 12:03 pm
CI will include 0 if and only if the one tailed p value is less than 0.025. The one sided p-value is smaller than 0.05 if and only if the credibility interval
[Lower 5%, Upper 5%] does not contain 0. You can obtain that using
output:cint;

You might find this helpful.

https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faq-what-are-the-differences-between-one-tailed-and-two-tailed-tests/
 Yasemin Kahya posted on Wednesday, June 12, 2019 - 4:55 am
Thank you for your answer,

I can see the explanation.


I am not sure but, if for example p = .07 and CI includes 0 in a DSEM model, can we interpret this as a marginal result since p is lower than .10 and CI is very narrow in this case?

In conventional results, we can interpret marginal effects when they are lower than .10.

Can we do the same thing here?
 Tihomir Asparouhov posted on Wednesday, June 12, 2019 - 11:06 am
Absolutely, but if the confidence interval is very narrow (meaning you probably have enough sample size) and it contains zero then much of the effect distribution is near zero. Under these circumstances the problem with the significance of the effects maybe the smaller problem. See
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3444174/
and also consider looking at the standardized estimates (output:stdyx)and the R2 contribution of that effect.
 Yasemin Kahya posted on Wednesday, June 12, 2019 - 3:08 pm
Dear Asparouhov,

I appreciate your help, thank you again.

This new info is extremely helpful,


Best Regards,
 Yasemin kAHYA posted on Sunday, September 08, 2019 - 2:07 am
Dear Asparouhov,

Would you please explain why DSEM uses one-tailed p-value for significance test?

Cut off = .025 in my models of intensive mother-infant interaction data, and I have marginal results when p = .03 and CI = -.01- .14, for example.

I estimated mother face self- and interactive contingency and infant face self- and interactive contingency with 56 dyads and 150 seconds coding for each dyad.

Could you please explain why one-tailed p value is preffered in computations.

It increases power, but at the same time increases Type II error, and so I wanted to make sure that I understand why a p-value is given in output with a Bayesian approach, and why this p value is one-tailed?

Thanks in advance
Best
 Tihomir Asparouhov posted on Monday, September 09, 2019 - 3:28 pm
We use the 95% confidence (symmetric credibility) interval for significance testing. If you want the two-sided p-value you can just multiply the one sided by 2. The one sided p-value in your case is computed as the posterior probability for the parameter to be positive.
 Yasemin Kahya posted on Tuesday, September 10, 2019 - 12:20 am
Dear Asparouhov,

Thanks a lot for explanations,

I am so sorry for taking your time with my questions,

I am a clinical psych Phd student, so the analysis we did is a little bit complex for me to understand,

As a last question, would you please share what you mean with the parameter to be positive? Is this parameter estimated for my independent variables?

Since I did not hypothesize my IVs will decrease or decrease DV, which is generally in one sided tests, so wanted to make sure what you mean by one sided-p value is computed for the posterior probability for the parameter to be positive?

Thanks for your time,
Best,
 Tihomir Asparouhov posted on Wednesday, September 11, 2019 - 1:03 pm
My mistake, I misread the CI values.

The one sided p-value in your case is computed as the posterior probability for the parameter to be negative. Since the CI is -0.01 to 0.14 that means that 2.5% of the distribution is below -0.01 and 3% of the distribution is below 0.
Back to top
Add Your Message Here
Post:
Username: Posting Information:
This is a public posting area. Enter your username and password if you have an account. Otherwise, enter your full name as your username and leave the password blank. Your e-mail address is optional.
Password:
E-mail:
Options: Enable HTML code in message
Automatically activate URLs in message
Action: