Program gets stuck
Posted on
dtofighi
Joined: 10/12/2009
Attachment | Size |
---|---|
My simulation file | 1.93 KB |
sessionInfo file | 582 bytes |
Forums
Hello,
I am running an 'embarrassingly parallel' simulation study using your excellent OpenMx software. The simulation study generates data, fits an unconstrained and constrained mediation model, where the constraint is a non-linear function of the model parameters. The problem is it appears that the program (optimizer) gets stuck-- when I check the CPU usage, it comes back down to a minimum usage as if the the program were terminated. To replicate the problem, I am attaching my code (with a parallel random generator seed) as well as sessionInfo() of the R environment on my computer. I would greatly appreciate your response.
works for me
Log in or register to post comments
I tried your script
rep
reduced from 200 to 20. The script ran very quickly on both systems.How long should I expect it it take to run 200 replications, anyhow?
Under Windows, I'm running:
OpenMx version: 2.7.11 [GIT v2.7.11-dirty]
R version: R version 3.4.0 (2017-04-21)
Platform: i386-w64-mingw32
Default optimiser: CSOLNP
Under Linux/GNU:
OpenMx version: 2.7.11.59 [GIT v2.7.11-59-g5d1b3b3-dirty]
R version: R version 3.4.0 (2017-04-21)
Platform: x86_64-pc-linux-gnu
Default optimiser: CSOLNP
If the issue is with OpenMx and not with doParallel, the only suggestions that come to my mind are to try using a different optimizer, or to provide each path coefficient with a nontrivial upper and lower bound with the
lbound
andubound
arguments tomxPath()
. In fact, since your MxModel uses an MxConstraint, using SLSQP instead of CSOLNP is advisable in any event.Log in or register to post comments
In reply to I tried your script by AdminRobK
I tried again
mxOption(NULL,"Default optimizer","SLSQP")
after OpenMx is loaded. The script runs OK under Linux, but seems to hang under Windows, with only one of the three child processes still using the CPU. I can't tell if it's "stuck," or if that process's share of models to fit are just taking a long time to optimize.I would not be surprised if doParallel has some Windows-specific bugs, as Joshua suggested.
Log in or register to post comments
re-runs
I would like to thank you for all your advice. I modified my code per your suggestions using SLSQP optimizer as well. I ran the code using both optimizers on Mac, W10, and Ubuntu. For W10, I used doSNOW instead of doParallel because it would allow me to use a progress bar in an interactive session. The results of all my runs were the same: the program did not stop when all the calculations were done. This can clearly be seen when running R on W10 because of the progress bar.
I attached my Windows 10 scripts as well as the ones for Mac. Mac and Ubuntu scripts were the same. I am also attaching all my sessionInfo files.
Log in or register to post comments
In reply to re-runs by dtofighi
w10 only?
Log in or register to post comments
In reply to w10 only? by jpritikin
It appears on all platforms.
Log in or register to post comments
In reply to It appears on all platforms. by dtofighi
works for me
problemSLSQPMac.R
on Debian Linux and it seems to run fine in less than 5 minutes. It creates output file resultsSLSQMac.csv with 200 lines plus a header line. I am not able to reproduce the hang.Log in or register to post comments
Thanks for all your comments.
Below, I am pasting the output from one of the the simulation runs when CSLONP got stuck:
[0] MxComputeGradientDescent: engine CSOLNP (ID 1) #P=7 gradient=central tol=6.3e-012 constraints=1
[0] resultForTT
[0] 0.005102
[0] resultForTT
[0] 0.000012
[0] resultForTT
[0] 0.000011
[0] resultForTT
[0] 0.000043
[0] MxComputeGradientDescent: engine CSOLNP done, iter=1288 inform=10
[0] MxComputeGradientDescent: engine CSOLNP (ID 1) #P=7 gradient=central tol=6.3e-012 constraints=1
[0] resultForTT
[0] 0.013429
[0] resultForTT
[0] -0.000013
[0] resultForTT
[0] 0.000625
[0] resultForTT
[0] -0.016426
[0] resultForTT
[0] -0.000001
Log in or register to post comments
In reply to Thanks for all your comments. by dtofighi
a few more comments
The different optimizers have their strengths and weaknesses, but I'm VERY surprised that you're getting better results with CSOLNP than with SLSQP. By what criteria are they better, and are you sure? Our testing has consistently indicated that SLSQP is the best of the three gradient-descent optimizers at handling nonlinear constraints. CSOLNP's biggest strength is with in analyses involving ordinal data, for which, relative to the other two, it can sometimes reach a lower fitfunction value and/or reach the minimum in fewer function evaluations.
Status code 10 means that the start values were infeasible. If the start values violate constraints, the optimizer is supposed to try to find a feasible point before beginning its algorithm in full swing. The fact that the attempt ended with status code 10 indicates either (1) that the optimizer was not able to find an initial feasible point, or (2) that the start values are completely outside the parameter space, i.e. the fitfunction evaluates to
NaN
orInf
or something like that at the start values.Have you tried using NPSOL as the optimizer? It's a proprietary library, so you would need to install OpenMx from our repository, rather than from CRAN.
On a completely different topic...since you're doing a simulation involving an MxConstraint, you can probably speed up optimization of the constrained model if you provide an analytic Jacobian for the constraint function. Only NPSOL and SLSQP can use one. If you're interested, I could post some suggestions on how to modify your script for that purpose.
Log in or register to post comments