using t-values as input data in SEM

Posted on
No user picture. hpar Joined: 03/30/2021
Hi all,

I have a more statistics-related question regarding the type of data we can use to run SEMs. I recently conducted a study looking at the heritability of functional brain activity, and extracted t-values from our region-of-interest, and took those values to heritability analyses (using OpenMx). We conducted it this way as previous studies looking at similar research questions used this methodology (i.e., using t-values, rather than parameter estimates/beta weights; e.g., Blokland et al., 2011; Li et al., 2019). However, a reviewer has commented that this is wrong, and we should not be using t-values as they are just 'statistical indices'.

I was wondering if any experts could shed a light on whether there is an issue using t-values in SEM (or any regression models) - any help would be much appreciated, thank you.

Replied on Mon, 04/05/2021 - 10:37
Picture of user. Ryne Joined: 07/31/2009

SEM isn't special in this regard: whatever answer we give should be equally applicable to other types of regression. A t-statistic is just a difference in means, adjusted for (a) variability, as measured by a pooled standard deviation, and (b) sample size or n. If n is relatively constant, you're just adjusting mean differences by the variance in each region or voxel. This is very similar to previous use of the coefficient of variation as an outcome, or previous work that uses individual standard deviations, with the caveat that same size requirements for ISDs are higher than typically expected.

So I think you can walk a reviewer through this if you walk through the t-statistic. You're presumably looking for differences across some grouping variable, and makes sense under your hypothesis to adjust those differences in variation/precision rather than look at raw totals. In my limited imaging experience, you could make this argument around different regions/voxels measured with varying levels of precision, and the variance in signal in each region is a good measure of measurement precision, thus you should adjust for it.

However, this would be totally indefensible if you were moving to t-statistics as a dependent variable in an attempt to p-hack after not seeing the results you like with a simpler unadjusted DV. If it's the same both ways, you get to say something like "we ran the analysis with multiple treatments of the DV per reviewer request, and saw the same pattern of results and significance tests in all analyses."

Good luck,
Ryne

Replied on Mon, 04/05/2021 - 21:48
Picture of user. Mike Cheung Joined: 10/08/2009

I am not familiar with heritability analysis; my suggestions may be irrelevant to your context.

It is questionable to use t-values as variables in analyses because t-values are sample-size dependent. More importantly, t-values are not estimating any meaningful population parameters. That is, what are the t-values are estimating when the sample size tends to infinity?

If your concerns are mean differences, as Ryne mentioned, you may consider either an unstandardized mean difference or a standardized mean difference. This is a standard topic in meta-analysis. You may fit regression or SEM on the mean differences by properly taking the sampling variances into account.