Error message for mxCompare() with boot = T
I want to compare a 2-class growth mixture model to a latent growth curve model in individually-varying time points framework (i.e., definition variables), and here is the code and error message and OpenMx version:
> mxCompare(Weight2, Weight1)
base comparison ep
1 Growth Mixture Model, fixed knots
2 Growth Mixture Model, fixed knots Estimate a fixed knot 15
minus2LL df AIC diffLL diffdf p
1 4775.720 1464 1847.720 NA NA NA
2 4859.904 1480 1899.904 84.18401 16 2.906173e-11
> mxCompare(Weight2, Weight1, boot = T)
> mxCompare(Weight2, Weight1, boot = T, replications = 10)
Error in collectStatistics1(otherStats, ref, other, bootPair) :
Fewer than 3 replications are available.
> mxVersion()
OpenMx version: 2.9.9.248 [GIT v2.9.9-248-gd602399]
R version: R version 3.5.1 (2018-07-02)
Platform: x86_64-apple-darwin15.6.0
MacOS: 10.13.6
Default optimiser: CSOLNP
NPSOL-enabled?: Yes
OpenMP-enabled?: Yes
May I know how to address this issue? Thank you in advance!
hm
mxCompare(Weight2, Weight1, boot = T)
mxCompare(Weight2, Weight1, boot = T, replications = 10)
Why? And why only 10 replications the second time?
Log in or register to post comments
In reply to hm by jpritikin
Thanks
Thanks for your reply. The default setting of replication is 400 and I let it run overnight but I got nothing. So for the second number, I just tried a smaller replication number to see if I can get something.
Log in or register to post comments
In reply to Thanks by Veronica_echo
previousRun
Log in or register to post comments
In reply to previousRun by jpritikin
I'm curious if this was ever
tmp <- mxCompare(m1, m2, boot=TRUE)
Error in collectStatistics1(otherStats, ref, other, bootPair) :
Fewer than 3 replications are available.
Log in or register to post comments
Fewer than 3 replications are available
Log in or register to post comments
In reply to Fewer than 3 replications are available by jpritikin
Thank you for responding.
Log in or register to post comments
In reply to Thank you for responding. by cjvanlissa
mxOption 'Status OK'
> options()$mxOptions$`Status OK`
[1] OK OK/green
attr(,"mxFactor")
[1] TRUE
10 Levels: OK < OK/green < infeasible linear constraint < infeasible non-linear constraint < ... < infeasible start
> levels(options()$mxOptions$`Status OK`)
[1] "OK" "OK/green" "infeasible linear constraint"
[4] "infeasible non-linear constraint" "iteration limit/blue" "not convex/red"
[7] "nonzero gradient/red" "bad deriv" "internal error"
[10] "infeasible start"
For example, if your models involve thresholds, you might want to set the option as follows:
mxOption(NULL,"Status OK",c("OK","OK/green","not convex/red","nonzero gradient/red"))
Note that this is only a fix if your models are failing to converge in most of the bootstrap replications (which is possible, even if they do converge when fitted to your real dataset).
Log in or register to post comments
don't stop
Gah, that's super annoying. I removed that code. Can you install v2.19.5-26-g8a688cd29 ?
Log in or register to post comments
In reply to don't stop by jpritikin
Thank you sincerely. I am
Log in or register to post comments
I think I know why this fails
Log in or register to post comments
In reply to I think I know why this fails by cjvanlissa
both mixture components should not have the same starting values
```
classes = lapply(classes, mxAutoStart, type = "ULS")
coef(classes[[1]]) - coef(classes[[2]])
```
When you start both mixtures at the same point in the parameter space, the optimizer has trouble getting anywhere because it isn't clear that there is more than 1 mixture component.
Log in or register to post comments
In reply to both mixture components should not have the same starting values by jpritikin
Dear Joshua, I respectfully
1) The model does arrive at the correct solution for the 2-class mixture, despite having the same starting values for both classes. (Correct solution as validated by Mplus and Mclust).
2) When I manually provide the same starting values for both mixtures, but also put a mxData() object inside both classes, mxCompare(boot = TRUE) also fails. Moreover, when I reverse the order of the models, mxCompare succeeds, but I notice that the number of degrees of freedom is incorrect. Cases from all the datasets are added up.
3) When I manually provide the same starting values for both mixtures, and do not put a mxData() object inside both classes, mxCompare(boot = TRUE) works. Thus proving that putting mxData() in the submodels breaks mxCompare(boot = TRUE)
See attached script for a demonstration.
Log in or register to post comments
In reply to Dear Joshua, I respectfully by cjvanlissa
Additionally, manually
fit1$class1$data <- NULL
fit2$class1$data <- NULL
fit2$class2$data <- NULL
Log in or register to post comments
In reply to Additionally, manually by cjvanlissa
oh!
I don't think there is a bug here. That's how it works by design.
Log in or register to post comments
In reply to oh! by jpritikin
Bootstrap LRT
Remember, mixture models were the whole motivation behind implementing the bootstrap LRT in OpenMx
Log in or register to post comments
In reply to Bootstrap LRT by AdminRobK
on mxGenerateData
Yeah, but that's not an important distinction. I agree that my diagnosis was not quite correct.
If you run `mxGenerateData` you will see that it only finds at the top-level mxData and ignores the mxData in the mixture components. If the mixture components contain an mxData then they will use it instead of traversing up the model tree to the top-level mxData. So the result is similar to what I suggested before, the mixture components never see the bootstrapped datasets and just fit against the same datasets over and over again.
Log in or register to post comments
In reply to on mxGenerateData by jpritikin
Thank you for the detailed
Secondly, I wonder how to set starting values for a mixture model using mxAutoStart() without adding mxData() to the individual classes. Do you think mxAutoStart() should be able to handle such cases where mxData() is only present in the top-level mxModel()?
Log in or register to post comments
In other words
Edit: To be clear, both the null and alternative models need to have valid expectation objects.
Log in or register to post comments
In reply to In other words by AdminRobK
mxCompare
Log in or register to post comments
In reply to mxCompare by jpritikin
no MxData needed
I'm looking over the code for `mxGenerateData()`, and yeah, I'm pretty sure that it can work without the models containing any MxData objects. It doesn't need raw data, just the expected sufficient statistics under the null model. Of course, what exactly `mxGenerateData()` does depends upon the definition of method `genericGenerateData()` for the relevant expectation class...which I notice only has definitions for MxExpectationNormal, MxExpectationLISREL, MxExpectationRAM, and MxExpectationStateSpace. There is no definition for MxExpectationMixture.
Log in or register to post comments
In reply to no MxData needed by AdminRobK
if no mxData() is needed for
Log in or register to post comments