| Attachment | Size |
|---|---|
 meta-analysis syntax and output.txt | 1.77 KB |
Hi all,
I want to conduct a meta-analysis with multiple effect sizes in several studies. I have used the three-level approach (see for example: Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behavior research methods, 45(2), 576-594.) In my case, there are three different types of variances: sampling variance, variance between the effect sizes of one study (level 2 variance), and variance between the effect sizes across studies (level 3 variance). Tau2_2 is the level 2 variance and Tau3_2 is the level 3 variance, is this correct? And what is the sampling variance?
Furthermore, I understood that I can compare the three-level model with a two-level model by fixing the level three variance to zero ('RE3.constraints=0'). I was wondering if this is also correct in my case. According to Van den Noortgate, when I would use a two-level meta-analysis I would have only sampling variance and variance between the effect sizes across studies, so no variance between the effect sizes of one study. Therefore, I was thinking that I should fix the second level variance to zero ('RE2.constraints=0') if I want to compare the three-level model with the original two-level method. Is this correct?
My last question: could I also use the standard error in the syntax for the three-level meta-analysis instead of the variance?
Thanks in advance!
Printer-friendly version
Hi,
It is correct that Tau2_2 is the level 2 variance and Tau3_2 is the level 3 variance. The sampling variance (v) is the squared standard error (SE), i.e., v=SE^2.
Both of the followings are identical in the metaSEM package:
metatwo <- meta3(y=yi, v=vi, cluster=studyID, data=meta, RE3.constraints=0)
metatwo <- meta(y=yi, v=vi, data=meta)
If you use “RE2.constraints=0”, the effect sizes within a study are direct replicates of each others. There is no heterogeneity at level 2 except for the sampling variance. Cheung (2015, Chapter 6) gives more details on testing Tau2_2=0, Tau3_2=0, and Tau2_2=Tau3_2.
The followings sources provide more details on the three-level meta-analysis and its implementation in the metaSEM package.
Cheung, M. W.-L. (2015). Meta-analysis: A structural equation modeling approach. John Wiley & Sons. http://www.amazon.com/dp/1119993431
Cheung, M. W.-L. (2014). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. Psychological Methods, 19(2), 211–229. https://dl.dropboxusercontent.com/u/25182759/Modeling%20dependent%20effect%20sizes%20with%20three%20level%20meta%20analysis%20A%20structural%20equation%20modeling%20approach.pdf
Cheers,
Mike
Thank you for your reply!
I have one more question regarding the specification of the covariates. I want to include year of publication (variable is already centered) and type of control group (Control1, Control2, Control3) in one model. Is this the correct specification?
multivariate <- meta3(y=yi, v=vi, x=cbind(scale(pyear), Control2, Control3), cluster=studyID, data=meta)
When I only include year of publication (centered variable), this is the syntax:
year <- meta3(y=yi, v=vi, x=scale(pyear, scale=FALSE), cluster=studyID, data=meta)
So I was wondering, should I not specify 'scale=FALSE' in the first syntax?
Thanks, again!
Both seem to be correct.
scale(pyear) and scale(pyear, scale=FALSE) mean standardizing and centering pyear, respectively.
Please refer to help(scale) in R for more details.
Mike
Dear Mike,
In addition to my previous question, I have another question. I have tested whether there was significant variance at level 2 and 3, using likelihood ratio tests. However, I was wondering if I shouldn't devide the p-value by 2, because the test is an one sided test (H0: variance = 0; H1: variance > 0). Am I correct that I should half the p-value?
Thanks in advance.
Hi, Boterman.
It seems that you have to half the p values if you are testing it. However, many researchers prefer not to test the variance components. If you are going to generalize the findings to level 2 and level 3, you may want to keep them even though they are not statistically significant. Another issue is that the test may not be powerful enough when the number of studies is small.
Mike
Hi, Dr. Cheung:
Currently, I am working on the 3-level meta-analysis using metaSEM. I am wondering if it is necessary to consider whether or not effect sizes within a study is homogeneous before using meta3() function in metaSEM? Take the teacher-burnout scale for example, there are 3 sub-scales:EE, DP, & PA. The correlations between EE/DP and CMSE are negative,while the correlation between PA and CMSE is positive(See Aloe, Amo, & Shanaham, 2014). Therefore, there are three related effect sizes within each study. I used the subscale as a cluster in your program :
>summary(meta3(y=ES,v=Variance,cluster=Subscale, data=teacher_burnout_3_level))
The summary is shown below:
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept -0.0547680 0.2019398 -0.4505627 0.3410268 -0.2712 0.7862300
Tau2_2 0.0161386 0.0042756 0.0077587 0.0245186 3.7746 0.0001603 ***
Tau2_3 0.1211002 0.0998886 -0.0746779 0.3168782 1.2124 0.2253776
The final pooled estimate is -.05468(the balance-out result of the positive and negative values), which is far away from the three corresponding estimates that I calculated using the meta() function. That is why I deem it is necessary to test whether or not effect sizes within a study is homogeneous before using meta3() function in metaSEM to resolve dependence. Am I wrong?
Thanks for your insights in advance!!
Hi Fred,
The three-level meta-analysis does not assume that the effect sizes are homogeneous within a study. However, it assumes that the heterogeneity of the effect sizes within a study is introduced by a common level-2 random effect. If you know the sampling covariances among the effect sizes (as in the case for the teacher-burnout scale), a multivariate meta-analysis is preferred, e.g.,
meta1 <- meta(cbind(EE,DP,PA), cbind(V_EE, C_EE_DP, C_EE_PA, V_DP, C_DP_PA, V_PA), data=Aloe14)
summary(meta1)
As you have indicated in your question, the correlations among the sub-scales are quite different. If you still want to apply the three-level meta-analysis, you may need to create dummy variables to estimate the means of the three sub-scales.
Mike
Dear Dr. Cheung:
By the way, you mentioned that I can create dummy variables to estimate the means of the three sub-scales.
Could you give me a hint:how to get it done in your program of metaSEM?Do I have to create two dummy variables as a covariate in level 2 or level 3 using meta3X()function in the metaSEM?
The dummy variables are listed below:
> head(teacher_burnout_3_level)
Study Year Subscale ES Variance EE DP
1 Betoret 2009 1 -0.38 0.0016 1 0
2 Brouwers-Tomic 2000a 1 -0.40 0.0013 1 0
3 Bumen 2010 1 -0.31 0.0014 1 0
4 Chang 2009 1 -0.32 0.0021 1 0
5 Durr 2008 1 -0.47 0.0061 1 0
6 Evers-et-al 2002 1 -0.26 0.0093 1 0
Or use the following control statement:
summary(meta3(y=ES,v=Variance,cluster=Subscale, x=cbind(EE,DP),data=teacher_burnout_3_level))
By the way, waht is the main difference between meta3() and meta3X() in metaSEM? Are they equivalent in some way?
Your insight is greatly appreciated.
Fred Li
Hi, Fred.
You may try the following example:
library(metaSEM)
my.df <- data.frame(Study=1:16, Aloe14[, 3:8])
Reshape into long format for three-level meta-analysis
my.long <- reshape(my.df, idvar="Study", direction="long",
varying=list(c("EE","DP","PA"), c("V_EE","V_DP","V_PA")),
v.names=c("y","v"), times=c("EE","DP","PA"))
Create indicator variables
my.long$ind_EE <- ifelse(my.long$time=="EE", yes=1, no=0)
my.long$ind_DP <- ifelse(my.long$time=="DP", yes=1, no=0)
my.long$ind_PA <- ifelse(my.long$time=="PA", yes=1, no=0)
my.long
Three-level meta-analysis without the intercept
summary( meta3(y=y, v=v, cluster=Study, x=cbind(ind_EE, ind_DP, ind_PA),
intercept.constraints=0, data=my.long) )
Multivariate meta-analysis
summary( meta(cbind(EE,DP,PA), cbind(V_EE, C_EE_DP, C_EE_PA, V_DP, C_DP_PA, V_PA),
data=Aloe14) )
Chapter 6 of my book discusses some similarities and differences between the multivariate and three-level meta-anayses.
https://books.google.com.sg/books?id=sp3TBgAAQBAJ&pg=PA195&dq=Three-level+meta-analysis+as+a+special+case+of+the+multivariate+meta-analysis&hl=en&sa=X&ved=0CCcQ6AEwAGoVChMIvfOEpv-CxwIV4x2mCh3YFQ_a#v=onepage&q=Three-level%20meta-analysis%20as%20a%20special%20case%20of%20the%20multivariate%20meta-analysis&f=false
meta3X() handles missing predictors with maximum likelihood while meta3() deletes the missing data before running the analysis. However, meta3X() is likely less stable. meta3() is recommended if you don't have too many missing values in the predictors.
Mike
Dear Dr. Cheung:
Should I use meta3X()function, is it better to to include these dummy variabes in level 3?
I would suggest to use meta3().
Mike