| Attachment | Size |
|---|---|
 RandomEffects.txt | 849 bytes |
Greetings,
I have tested a correlation, for which I have a small number of studies (two studies), using univariate random-effects model in metaSEM. The results (see attached) show a significant Q statistic, non-significant Tau2_1_1, and I2 of 99%. I was wondering if such results should be interpreted as a significant heterogeneity among my studies or not. In particular, how should I interpret the results when Q is significant, but Tau is not?
I appreciate any help,
Best,
/Hamed
Printer-friendly version
Hi Hamed,
The significant Q statistic means that the null hypothesis of equality of population correlations is rejected. Since there are only two studies, the estimated heterogeneity (Tau2_1_1) won't be accurate. It is also well known that the test on the heterogeneity variance is not very helpful. To summarize, I won't trust the Tau2_1_1, the test on it, and the I2.
Regards,
Mike
Hi Mike,
Thank you for your help on my previous question.
I tested another correlation-based relationship for which I have 11 studies. The purpose of my meta-analysis is to (1) show that there is a significant heterogeneity among studies for this relationship, and (2) test different moderators that can explain the heterogeneity. However, I am a bit confused as to which model of meta-analysis I should use for testing the moderation? Mixed-effects model or the three-level model? The moderator that I want to assess is the stage of analysis in each study, which is a between-study categorical moderator with three levels, coded as 1, 2, and 3. Conceptually, I think both mixed-effects meta-analysis model and three-level meta-analysis model can be used. When I test my data using mixed-effects model the coefficients for slopes are significant. However, the coefficient for the third level in three-level meta-analysis is not significant, when I test it. I believe the third-level in the three-level meta-analysis refers to the moderator. I have attached the results for both tests on the same data set. So, I am a bit confused as why one test shows significant moderation, while the other shows non-significant results; and which one I should trust. I am using your 2015 paper in Frontiers in Psychology, as the reference.
I will appreciate your input,
Thank you,
/Hamed
p.s.,
Cheung, M.W.-L. 2015. "Metasem: An R Package for Meta-Analysis Using Structural Equation Modeling," Frontiers in psychology (5:Article 1521), pp. 1-7.
Hi, Hamed.
The three-level meta-analysis is not appropriate here. It assumes that the effect sizes are nested within stage 1, 2, and 3. And we do not compare the means across these three stages.
As you have indicated, you would like to test the variable "stage" as a moderator. A mixed-effects meta-analysis seems to be a better choice.
Mike
Hi Mike,
Thank you for your help. Yes, for moderation testing I need to compare the means across the "stages". Therefore, as you indicated, mixed-effects is more appropriate.
Best,
/Hamed