Message/Author |
|
|
Hi, we conducted multilevel time series analyses via using DSEM. Some of the findings showed that even if one tailed p value is significant (lower than .05), CI included zero. How can we interpret these results? Are they marginal, or should we look at only CI and say this is a nonsignificant effect although CIs generally included zero between -.01 .13 for example. Thank you in advance |
|
|
CI will include 0 if and only if the one tailed p value is less than 0.025. The one sided p-value is smaller than 0.05 if and only if the credibility interval [Lower 5%, Upper 5%] does not contain 0. You can obtain that using output:cint; You might find this helpful. https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faq-what-are-the-differences-between-one-tailed-and-two-tailed-tests/ |
|
|
Thank you for your answer, I can see the explanation. I am not sure but, if for example p = .07 and CI includes 0 in a DSEM model, can we interpret this as a marginal result since p is lower than .10 and CI is very narrow in this case? In conventional results, we can interpret marginal effects when they are lower than .10. Can we do the same thing here? |
|
|
Absolutely, but if the confidence interval is very narrow (meaning you probably have enough sample size) and it contains zero then much of the effect distribution is near zero. Under these circumstances the problem with the significance of the effects maybe the smaller problem. See https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3444174/ and also consider looking at the standardized estimates (output:stdyx)and the R2 contribution of that effect. |
|
|
Dear Asparouhov, I appreciate your help, thank you again. This new info is extremely helpful, Best Regards, |
|
|
Dear Asparouhov, Would you please explain why DSEM uses one-tailed p-value for significance test? Cut off = .025 in my models of intensive mother-infant interaction data, and I have marginal results when p = .03 and CI = -.01- .14, for example. I estimated mother face self- and interactive contingency and infant face self- and interactive contingency with 56 dyads and 150 seconds coding for each dyad. Could you please explain why one-tailed p value is preffered in computations. It increases power, but at the same time increases Type II error, and so I wanted to make sure that I understand why a p-value is given in output with a Bayesian approach, and why this p value is one-tailed? Thanks in advance Best |
|
|
We use the 95% confidence (symmetric credibility) interval for significance testing. If you want the two-sided p-value you can just multiply the one sided by 2. The one sided p-value in your case is computed as the posterior probability for the parameter to be positive. |
|
|
Dear Asparouhov, Thanks a lot for explanations, I am so sorry for taking your time with my questions, I am a clinical psych Phd student, so the analysis we did is a little bit complex for me to understand, As a last question, would you please share what you mean with the parameter to be positive? Is this parameter estimated for my independent variables? Since I did not hypothesize my IVs will decrease or decrease DV, which is generally in one sided tests, so wanted to make sure what you mean by one sided-p value is computed for the posterior probability for the parameter to be positive? Thanks for your time, Best, |
|
|
My mistake, I misread the CI values. The one sided p-value in your case is computed as the posterior probability for the parameter to be negative. Since the CI is -0.01 to 0.14 that means that 2.5% of the distribution is below -0.01 and 3% of the distribution is below 0. |
|
Back to top |