A fixed effect model is a more restricted version of a random effects model, when meta-analyzing correlations using tssem1(), correct? Is there a way to do a nested model test? I know the degrees of freedom for the difference test would be the number of random effects estimated. I'm tripped up because the fixed effect model gives many fit stats, whereas the random effects model only gives a -2 log likelihood.
Thank you!
Printer-friendly version
Hi Katie,
The answer is both yes and no. There are two versions of fixed-effects models implemented in tssem1(). When you specify the argument method="REM", it uses the multiple-group SEM approach (Cheung & Chan, 2005). This fixed-effects model is not nested within the random-effects model.
Another fixed-effects model is to fix the variance component of the random effects at zero by specifying both method="REM" and RE.type="Zero". This fixed-effects model is nested within the random-effects model (see the following R code). Please note that this is tested on the boundary of the parameter space. The likelihood ratio test is likely wrong without the adjustment.
Best,
Mike
Fixed-effects model by using a multiple-group SEM
fixed_no <- tssem1(Digman97$data, Digman97$n, method="FEM")
Non-nested models: Wrong! Don't do it!
anova(random, fixed_no)
Fixed-effects model by fixing the random-effects heterogeneity covariance matrix at zero
fixed_yes <- tssem1(Digman97$data, Digman97$n, method="REM", RE.type="Zero")
Random-effects model
random <- tssem1(Digman97$data, Digman97$n, method="REM", RE.type="Diag")
Nested models: testing on the boundary
anova(random, fixed_yes)
Thanks! I have done what you suggest, and I see in the anova() command that I get negative AIC values. I attach my code (code will also pull down data). In another dataset, in the -2LL column, the model with random effects specified has a negative -2LL and the fixed effects model, specified with method="REM" & RE.type="Zero" has a positive -2LL, which seems strange. Is there an error in my code?
Your analyses seem fine.
Different SEM packages produce different AIC (http://openmx.psyc.virginia.edu/thread/89). AIC is usually used for model comparisons. You may choose the model with the smallest AIC as long as you are using the same package.
There is nothing wrong with positive -2LL. For example, http://openmx.psyc.virginia.edu/thread/1831 and http://blog.stata.com/2011/02/16/positive-log-likelihood-values-happen/
OK - just to confirm - a negative AIC value (not when comparing models, just each model on its own) is OK?
Also - you might have missed my question above (threading is a bit confusing here):
Is it also the case that when you add a cluster variable to tssem1(), and then you sum the chi-square associated with each group, that the total chi-square can be compared to a tssem1() with no cluster as a nested model? The df of the difference here would be the number of parameters estimated in each group.
Is there a way to fix or free individual correlations in tssem1() across subgroups, or must the model be specified in tssem2()?
Thanks again! It's really amazing to be able to get your feedback.
Katie
Yes, the (negative) AIC is not very useful on its own.
Sorry, I missed that part. Yes, The chi-square and its dfs are based on the sum of the individual chi-squares and dfs. You may compare them against the model without the cluster variable.
I haven't implemented this in metaSEM though it can be done in OpenMx. Suzanne Jak (S.Jak@uu.nl) and I have written a paper on this topic. She is the corresponding author. If you write to her, I am sure that she is willing to share it.
Hi again!
Is it also the case that when you add a cluster variable to tssem1(), and then you sum the chi-square associated with each group, that the total chi-square can be compared to a tssem1() with no cluster as a nested model? The df of the difference here would be the number of parameters estimated in each group.
Is there a way to fix or free individual correlations in tssem1() across subgroups, or must the model be specified in tssem2()?
Thanks!
Katie