Hi all,
It appears my 95% CI problems for a series of ordinal ACE models I'm running is not solved after all. Specifically, while CIs for my AE and CE submodels appear to be fine, the ACE CIs return an error (shown in bold below).
confidence intervals:
lbound estimate ubound note
oneACEo.VC[1,1] NA 0.59633060 NA !!!
oneACEo.VC[1,2] 1.000507e-08 0.02077049 NA !!!
oneACEo.VC[1,3] NA 0.38289892 0.5185679 !!!
CI details:
diagnostic statusCode
1 alpha level not reached infeasible start
2 active box constraint infeasible start
3 success infeasible start
4 active box constraint infeasible start
5 active box constraint infeasible start
6 success infeasible start
I should mention I am using this model, which I think is appropriate given these data. I am very new to the OpenMX environment and am struggling with these models. Any advice would be very much appreciated.
The first thing I think you should try is to get rid of the lower bounds on the ACE path coefficients. You can do that either by simply not using argument
lbound
tomxMatrix()
when you create the objects with R symbols 'pathA', 'pathC', and 'pathE', or by appropriately usingomxSetParameters()
on your assembled MxModel object.You could also try using a different optimizer. In your case, I'd suggest trying SLSQP. To switch to SLSQP, run this,
, before you run
mxRun()
.I you're using OpenMx v2.7.11 or newer, you could get bootstrap confidence intervals instead. See the help page for
mxBootstrap()
. The default coverage probability of bootstrap CIs is 50%. If you increase it. to 95% (via argumentboot.quantile
tosummary()
) definitely increase the number of replications to at least 1000.Thanks much for your reply, Rob. I tried each of your suggestions. First, I attempted eliminating lbounds on path coefficients and/or using the optimizer you suggested. This fills in some gaps in CIs for the ACE model:
confidence intervals:
To investigate missing CIs, run summary() again, with verbose=T, to see CI details.
The bootstrapped CIs seem off and when I use mxBootstrap() command I get the following error:
Warning message:Only 5% of the bootstrap replications converged. Accuracy is much less than the 1000 replications requested
Thanks Rob. I had this exact problem.
For anyone interested.
I used:
To (I hope) unbound the confidence intervals. But it didn't work.
Changing the optimiser from NPSOL to SLSQP worked for me.
I put this just before mxRun.
Thanks!
This line, I think, will have no effect.
To remove an lbound, if there is one (and if it needs removing), you'd want to add
lbound = NA
The bound is there because estimating negative
a
makes no sense in an ACE model, as A = a %*% t(a), i.e., squared, which is always positive), so you could look at the upper bound, see if it suggests the lower bound is bumping up against 0 and report zero if it is).PS: umx has a memorable function to switch optimizers: e.g. umx_set_optimizer("NPSOL")
With no input, it will list the current optimizer and available options.
PPS: I made a suggestion that we warn users when functions have no effect (If I'm correct and that is the case)
https://github.com/OpenMx/OpenMx/issues/143
I'm curious to see the details, from
summary()
with argumentverbose=T
. I'm guessing that the lower limit for the shared-environmental component is also approximately zero, as it is for the additive-genetic component.How large is your sample, anyhow? And how often does the least-common level of the ordinal variable occur the sample?
Hmm. Since you're analyzing an ordinal variable, you might need to tell OpenMx that status Red is OK for bootstrap replications, with
BTW, if you decide to go with bootstrap CIs instead of profile-likelihood CIs, CSOLNP or NPSOL might be a better choice of optimizer than SLSQP. If you want, it's possible to use one optimizer to find point estimates, and a different one to find profile-likelihood CIs, for example, as in the syntax in one of my previous posts.
Tiny tiny sample,
n=334/2=167 pairs. 29 DZ and 138 MZ pairs. Original data collection effort.
Here is verbose output (forgive the lack of formatting):
CI details:
c11 e11 method diagnostic statusCode
1 7.450660e-01 0.6669908 neale-miller-1997 success OK
2 -2.959168e-10 0.5262822 neale-miller-1997 success OK
3 -1.084544e-15 0.6253159 neale-miller-1997 alpha level not reached infeasible non-linear constraint
4 7.698171e-01 0.6382561 neale-miller-1997 success infeasible non-linear constraint
5 2.761231e-05 0.5262812 neale-miller-1997 success infeasible non-linear constraint
6 2.189199e-01 0.7190757 neale-miller-1997 success infeasible non-linear constraint
Tiny tiny sample, n=334/2=167 pairs. 29 DZ and 138 MZ pairs. Original data collection effort.
OK, then yes, I guess you can just report the lower limit for the C variance component as zero. The point estimate was near-zero anyhow, and since the variance component is the square of a path coefficient, it can't go any less than zero.
Was this with SLSQP, BTW?
Thanks again, Rob--I appreciate it.
I am running into a similar issue with the CIs of an ADE model. I am using OpenMx version 2.7.18, R version 3.4.2 and a 64 bit MacOS. I used the omxRunCI function.
I removed the bounds on the path coefficients and am using SLSQP. I get some NAs and three exclamation points next to the confidence intervals:
Confidence intervals:
lbound estimate ubound note
a11 NA 6.398189 7.465746 !!!
d11 NA 6.529459 7.587030 !!!
e11 4.889512 5.491900 NA !!!
The verbose output shows:
CI details:
parameter value side fit method diagnostic status Code
1 a11 6.300771 lower 31642.74 neale-miller-1997 alpha level not reached infeasible non-linear constraint
2 a11 7.465746 upper 29780.27 neale-miller-1997 success infeasible non-linear constraint
3 d11 6.178623 lower NA neale-miller-1997 alpha level not reached infeasible non-linear constraint
4 d11 7.587030 upper 29780.27 neale-miller-1997 success infeasible non-linear constraint
5 e11 4.889512 lower 29780.27 neale-miller-1997 success infeasible non-linear constraint
6 e11 5.327989 upper NA neale-miller-1997 alpha level not reached infeasible non-linear constraint
Whereas the upper bound does not look reasonable for the e parameter, the rest of the values appear to be somewhat reasonable. Is there a way to specify the convergence criteria or do you have any additional recommendations? Thanks in advance.
It's hard to diagnose what's wrong without seeing the actual script you're using, along with the
summary()
output. Nonetheless, something is clearly wrong with the lower limit for a11 (fit value seems way off), the lower limit for d11 (fit value ofNA
), and the upper limit for e11 (on the wrong side of the point estimate, plus a fit value ofNA
).One way to validate dubious CI results is to use
omxSetParameters()
or an MxConstraint to fix the reference quantity to the value of the confidence limit, re-run the model, and see if the fit worsens by the target value (approximately 3.841 for a 95% CI).