Optimizers w/ numerical vs. analytical derivatives
Posted on
falkcarl
Joined: 10/29/2015
Forums
I'm experimenting with some novel item models for use with OpenMx's item factor analysis capabilities. I was wondering what optimization methods are currently available (e.g., for the M-step when doing Bock-Aitkin EM), and which one(s) can make use of analytical derivatives (which tend to be faster, at least in my experience). I've previously used my own hand-rolled Newton-Raphson code and discovered that N-R has trouble with some of these item models and usually requires reducing the step size or some backtracking, and sometimes ridging the hessian when early in the E-M cycles. I've been looking at mxComputeNewtonRaphson, but don't see in the documentation whether it just tries to take the full step each time or does anything else.
I also see mxComputeGradientDescent, which optionally uses the analytical gradient?
Did I miss any other optimization options?
Many thanks,
Carl
optimizers
No, I don't think so.
> mxComputeGradientDescent, which optionally uses the analytical gradient?
Yeah, it can use an analytic gradient. The use of an analytic gradient is implemented for NPSOL and could be implemented for SLSQP very easily.
> discovered that N-R has trouble with some of these item models and usually requires reducing the step size or some backtracking, and sometimes ridging the hessian when early in the E-M cycles. I've been looking at mxComputeNewtonRaphson, but don't see in the documentation whether it just tries to take the full step each time or does anything else.
I wrote the N-R optimizer. It includes some heuristics, but I'm not sure whether it will perform adequately on your model. Certainly we can add whatever heuristics you need. You can enable some diagnostics by passing verbose=2 or verbose=3. I recommend you try it out. If you can find a reproducible example of poor performance and suggest how to fix it then I can investigate.
Log in or register to post comments
In reply to optimizers by jpritikin
Thank you! So far I have it
Log in or register to post comments
In reply to Thank you! So far I have it by falkcarl
priors
Log in or register to post comments
In reply to priors by jpritikin
What is sometimes done is the
Mislevy, R. J. (1986). Bayes modal estimation in item response models. Psychometrika, 51(2), 177-195.
I believe certain other software programs do this when calculating -2LL (and can provide an example if requested), though I cannot say for sure whether it is conventional across all other programs.
P.S. in rpf.rparam() when using the dichotomous response model and a 1-dimensional model,
such as:
rpf.param(rpf.drm(1))
should the second element returned be a "c" instead of "b"? I ask b/c documentation lists slope/intercept parameterization and uses "c", but the output is labeled as a "b".
P.P.S. Do you want future questions regarding IFA in OpenMx to be posted to this main forum, or Categorical Outcomes, or...?
Log in or register to post comments
In reply to What is sometimes done is the by falkcarl
various
Hm .. well, to return separate values for the likelihood and prior, we'd have to distinguish them in the backend. We don't do that now. With respect to estimation, we just optimize the whole posterior. If you set up a multigroup model with the prior in one group and the likelihood in the other then it is pretty easy to pull out the separate values. There is an example in OpenMx/inst/models/passing/fm-example2-2.R but I admit that it is a bit cluttered and not easy to read.
> should the second element returned be a "c" instead of "b"? I ask b/c documentation lists slope/intercept parameterization and uses "c", but the output is labeled as a "b".
Yeah, the parameter labels do not match convention. I guess I'll accept a patch to fix the label. All the current models use slope/intercept parameterization because it is more consistent between single and multidimensional models.
> Do you want future questions regarding IFA in OpenMx to be posted to this main forum, or Categorical Outcomes, or...?
It doesn't really matter. We don't have enough traffic to make it inconvenient to monitor all the forums together.
Log in or register to post comments
In reply to various by jpritikin
Yeah, as long as I at least
Log in or register to post comments