You are here

Inconsistency between global chi-square difference tests and t-tests

4 posts / 0 new
Last post
George Richardson's picture
Joined: 10/18/2018 - 13:26
Inconsistency between global chi-square difference tests and t-tests

Hi,
I've specified a multivariate ACE model and found that the z/t-tests produce substantially different p-values than the chi-square difference test per parameter constrained to zero. My recollection is that these tests should produce equivalent results but I recall reading somewhere that at relatively small n (here, n = 260 pairs), the global test may have more power. Does anyone know if this is likely correct or if there are methods papers address it? Could the difference be due to correlations among the z/t-tests? Not sure what's going on here.

Thanks in advance!
George

AdminRobK's picture
Online
Joined: 01/24/2014 - 12:15
asymptotics

The tests are only asymptotically equivalent, meaning that they will only match closely in large samples. Additionally, the Wald z-test is sensitive to the choice of parameterization, meaning that in finite samples, for example, a divided by its standard error will not equal a² divided by its standard error. In contrast, the likelihood-ratio chi-square test is invariant under change-of-parameter.

Note that the Wald test is regarded as a poor choice for testing hypotheses about variance-covariance parameters in modest-sized samples, since the sampling distributions of such "second-order" parameters' estimates converge slowly to normality.

BTW, if you want to calculate a Wald test statistic for multiple parameters, you'll need to use the inverse of the repeated-sampling covariance matrix of those parameters' point estimates. See Wikipedia for details.

George Richardson's picture
Joined: 10/18/2018 - 13:26
Thanks you!

This is a huge help. Thank you so much!

AdminNeale's picture
Online
Joined: 03/01/2013 - 14:09
Related article

For a simple account of how t statistics change when calculated from estimates of, e.g., a versus a^2 in an equivalent model, see this paper: Neale, M.C., Heath, A.C., Hewitt, J.K., Eaves, L.J. Fulker, D.W. (1989) Fitting genetic models with LISREL: hypothesis testing. Behavior Genetics 19: 37-49.