Dear Mike and others,

I would like to conduct a fixed effect MASEM with clusters and I am stuck when I try to run TSSEM1. At first I tried to use a cluster variable that I labeled, and then I tried to use the cluster variable without labels (just "0" and "1" codes). I got different error messages each time.

I don't use R much and I don't know how to format the data in exactly the same way they are formatted in the examples on Mike's website (all data in a single file). I was able to run a random effects TSSEM1 and TSSEM2 the way my data were formatted, without clusters.

I have pasted the error messages below and also attached the R script and data files.

Thank you in advance for your help with this,

Mei Yi

> #Run fixed-effect TSSEM1 according to cluster labels

> cluster1 <- tssem1(vector, n, method="FEM",

+ cluster=mod$IPT1)

Error in if (!all(isPD)) warning(paste("Group ", (1:no.groups)[!isPD], :

missing value where TRUE/FALSE needed

In addition: Warning message:

Unknown or uninitialised column: 'IPT1'.

> #Run fixed-effect TSSEM1 according to cluster labels

> cluster1 <- tssem1(vector, n, method="FEM",

+ cluster=mod$IPT)

Error in tssem1FEM(data.cluster[[i]], n.cluster[[i]], cor.analysis = cor.analysis, :

Lengths of "df" and "n" are different.

In addition: Warning message:

In split.default(x = seq_len(nrow(x)), f = f, drop = drop, ...) :

data length is not a multiple of split variable

> summary(cluster1)

Error in summary(cluster1) : object 'cluster1' not found

Dear Mei Yi,

I am not able to read the data. For example, v_pre_post.txt is not present; n <- read.table("vectormaxn.txt") returns n as a data frame.

Could you please check the script whether the data and sample sizes can be imported?

Best,

Mike

Dear Mike,

Thanks for your quick response, and apologies for the errors in my script. I have checked the script and made a few changes, the revised script is attached (cluster.R).

v_pre_post.txt is changed to v_pre_posti.txt in the script and should be able to be imported (same data file that I have sent you)

n <- read.table("vectormaxn.txt") was what I used to run random effects analyses successfully, so I had assumed it would be ok for fixed effects analyses. In case it is helpful I have attached my random effects script, random.R, which uses this syntax without problems.

In case n being read as data frame was causing problems for fixed effects analyses, I have created another data file, nmod.txt that contains n and data for one moderator. The cluster.R script has been changed to read n using the syntax:

nmod <- read.table("nmod.txt", header=TRUE)

nmod$n

I tried to run the fixed effects analyses, without cluster, and then with cluster, and now I am getting the error message below for TSSEM1 in both analyses:

Error in if (!all(isPD)) warning(paste("Group ", (1:no.groups)[!isPD], :

missing value where TRUE/FALSE needed

Thank you for your help with this issue,

Mei Yi

Dear Mike,

Sorry to post so many messages, I have just realized that I may have gotten the above error message because I may have too much missing data in the correlation matrices. When I run is.pd I get TRUE for the matrices without missing data, and NA for the matrices with missing data. Thus I have rerun the fixed effects analyses without cluster, and then with cluster, and there were no longer any error messages, and openmx status was 0 for all analyses.

My questions now are:

Is there a way to run the fixed effects analyses with clusters and include the matrices with missing data?

For the fixed effects analyses with clusters that I have run, the Stage 1 fit is not good in one cluster or both clusters (depending on which dataset I use). However, Stage 2 fit is ok. I have already run random effects analyses (without clusters) and fit was good, but I wanted to see whether clusters could explain some of the heterogeneity. I also have hypotheses that the mediation model I was testing should apply in one cluster but not in the other--true enough, indirect effect was significant in one cluster but not in the other. Can I use this as evidence to support my hypotheses, or would that not be advisable given that the cluster stage 1 analyses showed poor fit?

In case it helps, I have attached the new script, correlation matrix data file, and sample size/moderator data file, where all matrices have no missing data.

Thanks so much,

Mei Yi

Dear Mei Yi,

The fixed-effects TSSEM is more sensitive to the presence of missing data. You may use the following code to conduct the fixed-effects model with missing data.

fixed1 <- tssem1(vector, nmod$n, method="REM", RE.type="Zero")

It does not work with the "cluster" argument yet. I hope to add it later.

Regards,

Mike

Thank you Mike for the syntax to run fixed-effects TSSEM when there is missing data. I look forward to hearing more about the syntax to work with missing data for the cluster argument.

Also, what are your thoughts on interpreting the Stage 2 cluster TSSEM results if Stage 1 fit is not good for one or both clusters? I have already run random effects analyses (without clusters) and fit was good, but I wanted to see whether clusters could explain some of the heterogeneity.

I also have hypotheses that the mediation model I was testing should apply in one cluster but not in the other--true enough, indirect effect was significant in one cluster but not in the other. Can I use this as evidence to support my hypotheses, or would that not be advisable given that the cluster stage 1 analyses showed poor fit?

Thank you in advance for your help!

Mei Yi

Hi Mei Yi,

If the model works in one cluster but not the other, it seems that there is evidence for the moderating effect. For the cluster that does not work, a different model may be required.

Best,

Mike

Thank you Mike! That makes sense. If I understand correctly, the fit needs to be good in both Stage 1 and Stage 2 of the cluster analyses to say that there is evidence that the data fit the mediation model. If Stage 1 AND Stage 2 fit are good for one cluster, but Stage 1 OR Stage 2 fit (or BOTH) are poor for the second cluster, then there is evidence that the data fit the mediation model in the first cluster but not in the second cluster. This suggests that there is moderated mediation by cluster, and a different model may be needed to fit the data in the second cluster better. If there is something that I misunderstood, I hope you can clarify. Thank you!

You are correct. If Stage 1 (fixed-effects model) does not fit in one cluster, it suggests that the studies are not homogeneous. You should not fit the Stage 2 analysis.

Hi Mei Yi,

The following code demonstrates how to apply the "cluster" argument to fixed-effects model. Hope it helps.

Regards,

Mike

Hi Mike,

Thank you for this sample syntax for applying cluster argument to fixed-effects model with missing data (that is, random effects model type zero). I was able to use it to conduct analyses with all my matrices, including those with missing data. I have a few follow-up questions about running my analyses with random effects model type zero that I would appreciate your help with.

In TSSEM 1 using method=REM, type=Zero, the output does not show any goodness-of-fit indices (TSSEM2 did show the fit indices). How do I tell whether the data fit the model for each cluster? If OpenMX=0 or 1, and the P-value of the Q statistic is >.05, then can the data be considered to fit the model in that cluster?

I noticed that your examples on your website and your papers only show TSSEM cluster analyses for fixed effects models. Is there a way to run TSSEM cluster analyses for random effects models without fixing the tau2 at 0 (i.e., method="REM", type="Diag" or type="Symm")? I tried the following syntax but did not get openmx=0 or 1.

## Stage 1 analysis

rand1 <- mapply(tssem1, Cov=data.ipt, n=n.ipt, method="REM",

RE.type="Diag", SIMPLIFY = FALSE)

lapply(rand1, summary)

## Type=Diag : Openmx=6 for CBT (cluster1), 5 for IPT (cluster2).

rand1 <- mapply(tssem1, Cov=data.ipt, n=n.ipt, method="REM",

RE.type="Symm", SIMPLIFY = FALSE)

lapply(rand1, summary)

## Type=Symm : Openmx=10 for CBT (cluster1), 5 for IPT (cluster2).

Thanks,

Mei Yi

Hi Mei Yi,

Yes, you may use the Q statistics to test whether the correlation matrices are homogeneous. Similar to other null hypothesis significance tests, its power depends on the number of studies.

In theory, you may use method="REM", type="Diag" or type="Symm". The issue is that there may not be enough studies to fit a random-effects model when you split the studies into clusters.

Best,

Mike

Thank you Mike! Is there a rule of thumb we can use to know whether there are enough studies for method="REM", type="Diag" or type="Symm"? I believe one of your TSSEM papers mentioned that 10 studies with at least 1000 participants total is ideal, for cluster analyses would the rule of thumb be 10 studies with 1000 participants per cluster? Or is there no way to know except to run the analyses and check whether openmx=0 or 1?

A related question--I do not have enough studies to run random effects TSSEM with clusters. So my choices are to conduct one fixed effects (random effects type zero) TSSEM with clusters, or two separate random effects TSSEM, one for each cluster. What are the pros and cons for each approach? Is one approach better than the other? I am fitting a mediation model and expect to see differences in the indirect effect between the two clusters. Thank you so much!

I don't have any concrete suggestions. A straightforward way is to recognize that the studies in a meta-analysis is similar to the sample size in a primary study. Suppose that you fit a model with 5 variables, the no. of correlation coefficients is 5*4/2=10. If you fit a random-effects model, you are estimating the means and variance-covariance matrix of these 10 correlation coefficients. Although we may constrain the off-diagonal elements in the variance-covariance matrix at 0, there are still many parameters to estimate. So, my point is that it is difficult to fit a random-effects model by splitting the data into clusters.