You are here

Multiple testing

4 posts / 0 new
Last post
JBosdriesz's picture
Offline
Joined: 03/07/2017 - 09:59
Multiple testing

One of my Phd students recently received reviewer comments for a submitted paper in which she presented 9 univariate ACE models on separate outcome measures. One of the reviewers asked: “Please indicate how correction for multiple comparisons was handled for the genetic modeling. Given that several brain regions were being analyzed, what statistical threshold was used?”
First of all, is any such correction really needed for 9 outcomes?
Second, how would we go about performing such a correction in openmx? What output I'm used to seeing reported in papers often includes the p-value only for selection of the best fitting model, and the path loadings obtained from the model, but no p-values for these path loadings.
Thanks.

AdminRobK's picture
Offline
Joined: 01/24/2014 - 12:15
Let me make sure I understand

Let me make sure I understand the situation. Your student reported results of 9 monophenotype ACE models, for some metric on 9 brain regions, correct? How many different models did she fit for each phenotype? If she fit more than one model per trait, how did she select the "best" model?

AdminNeale's picture
Offline
Joined: 03/01/2013 - 14:09
How was it decided which models to fit?

To further RobK's comment, was there a fixed set of models to be fit for each phenotype, or did the number and type of models vary depending on what was found from the modeling itself?

Possibly, false discovery rate correction could be used. On the whole though, with genetic model estimates I don't think that these procedures are very helpful. By analogy, it is as though one has decided that 20,000 feet is a significantly high part of planet earth, so one draws a map that has a few contours around the Himalayas and the Andes, but is blank white otherwise. For the purposes of global navigation, the map is largely useless. It answers the question of whether there are significantly high bits of the Earth, but whether there are significantly highly heritable regions is rarely the question in structural MRI studies. For practical purposes, it is typically better to have a global map (recognizing its errors and all) than a map of where the big mountains are.

JBosdriesz's picture
Offline
Joined: 03/07/2017 - 09:59
She compared monophenotype

She compared monophenotype ACE vs AE vs CE vs E for each of the 9 brain regions, selecting the 'best' model based on log likelihood and AIC.

Log in or register to post comments