Hi all, hi Mike,
I am asking a question in reference to this project (https://osf.io/awz2p/ [6]). I am revising a paper based on the project in response to reviewer comments.
For most analyses in this project, we fit multivariate meta-analytic models, estimating 11 quantities across all studies. Of course, not every study contributes an estimate of each quantity, so the between-studies variance-covariance matrix is constrained such that all variances are equal and all covariances are equal. This ensures that the model is identifiable.
In the first submission, we reported I2 for each meta-analytic quantity as reported by summary.meta(). One of our reviewers asked us to report tau2 as well as an absolute measure of heterogeneity. This seems reasonable.
However, over the course of thinking about heterogeneity, it occurs to me that it might not make sense to report separate heterogeneity indices for each of the 11 estimated meta-analytic quantities -- after all, all the between-studies variances are constrained to be equal.
Does it make sense to report separate heterogeneity indices given that the between-studies variances are constrained to be equal? If not, how would I obtain a single heterogeneity estimate? From Cheung (2008; https://www.statmodel.com/download/MCheung.pdf [7]), I'm guessing that I can use these formulas:
H2 = Q/Qdf
I2 = (H2-1)/H2
Assuming that mod is a model fit using meta() in R, I believe this would be:
H2 <- summary(mod)$Q.stat$Q/summary(mod)$Q.stat$Q.df
I2 <- (H2-1)/H2
(P.S.: Mike, your advice on this and other projects over the years has been exceptionally helpful! You'll see that you are acknowledged in the author notes of this project and your work is referenced extensively throughout the paper)