Hello,
I have been trying to figure out the "parametric bootstrap" when ‘boot=TRUE’. I read the document for mxCompare as well as studying its source code. However, I would appreciate if you could answer the following questions:
1- Does mxCompare generate data from both fitted base and comparison models, fit the base comparison models, calculated LRT, and compute p-values?
2- According to the document for mxComparison,
The Monte Carlo SE of a proportion for B replications is √(p*(1-p)/B), but this will be zero if p is zero, which is nonsense. Note that a parametric-bootstrap p-value of zero must be interpreted as p < 1/B, which, depending on B and the desired Type I error rate, may not be "statistically significant."
I am not sure if I understand the part about the p-value < 1/B. Any reference or clarification is appreciated.
Thanks,
Davood
Printer-friendly version
Not exactly. Instead, for each replication, it (1) generates random multivariate-normal data under the null (more-constrained) model, (2) fits the null model to the generated data, (3) fits the alternative (less-constrained) model to the generated data, and (4) calculates the LRT statistic for that replication as the difference in -2logL between the null and alternative models. Then, the p-value is the proportion of LRTs from the simulated data that exceed the LRT from the real data. I'm not sure exactly what you're trying to do, but you might want to look at the somewhat simpler
mxParametricBootstrap(), which only fits the alternative model to each randomly generated dataset.Suppose I am using a Type I error rate of 0.01. Further suppose I use only 50 replications, and that none of the LRTs from the bootstrap distribution exceeds the real-data LRT. That would give a p-value of zero. But, due to the large Monte Carlo sampling error resulting from the small number of replications, I cannot be certain that the p-value truly is smaller than my Type I error rate. I can only be sure that it is some value smaller than 1/50=0.02.
Edit: 0.02, not 0.2.
Fantastic! thank you.