OpenMx Goes Multicore

OpenMx includes support for parallel processing by using the R package snowfall. Snowfall is an optional package for the library. If snowfall is loaded at runtime then independent models will be executed in parallel. Many thanks go to Greg Carey for writing the sequential version of the following simulation on the forums. The parallel version of the script can be found at https://openmx.ssri.psu.edu/repoview/1/trunk/demo/BootstrapParallel.R.

As of revision 1063 in the subversion repository, the following clock times have been observed on a system with 8 logical processors. The performance of the current binary release is significantly worse. Either compile from the trunk, or wait until the next binary release.

5000 submodels ('nReps' variable in script)
1 cpu - 362.3 secs
2 cpu - 205.958 secs
4 cpu - 125.727 secs
8 cpu - 72.622 secs

Getting Started

To gain any benefit from the snowfall library, you must declare independent submodels in your script. A model is declared independent by using the argument 'independent=TRUE' in the function mxModel(). An independent model and all of its dependent children are executed in a separate optimization environment. An independent model shares NO free parameters with either its sibling models or its parent model. An independent model may NOT refer to matrices or algebras in either its sibling models or its parent model. A parent model may access the final results of optimization from an independent child model.

To use the snowfall library, you must start your R environment with the following commands:

library(OpenMx)
library(snowfall)
sfInit(parallel = TRUE, cpus = ???)
sfLibrary(OpenMx)

'sfInit' will initialize the snowfall cluster. You must specify either the number of CPUs on your machine or the cluster environment (see snowfall package documentation). 'sfLibrary' exports the OpenMx library to the client nodes in the cluster. At the end of your script, use the command:

sfStop()

To Improve Performance

Any sequential portions of your script will quickly become the performance bottleneck (Amdahl's Law). Avoid iteration over large data structures. Use omxLapply() and omxSapply() instead. These functions will invoke the snowfall sfLapply() and sfSapply() if the snowfall library has been loaded. Otherwise they will invoke lapply() and sapply(). Run your script with multicore settings enabled, and use Rprof to profile a reasonable size test case. Ignore the calls to sfLapply() and sfSapply() in the results of profiling. Any other time-consuming calls represent potential sequential bottlenecks.

Sadly, some of the functions provided by the OpenMx library can be sequential bottlenecks. Iterative use of the mxModel() function in order to add submodels can be time consuming. Use the following unsafe idiom to improve performance:

submodels <- # a list of independent submodels
names(submodels) <- omxExtractNames(submodels)
topModel@submodels <- submodels

For Developers Only

Model flattening (omxFlattenModel) is another sequential bottleneck. I have rewritten mxRun() to avoid model flattening if possible. I am not 100% positive, but I think the conversion from RAM + raw data == FIML requires model flattening. Also model freezing (the step performed after independent submodels have been executed) does not currently alter free parameters. This is bad. I'm not sure what is the correct algorithm: either eliminate the free parameter names entirely (but then they aren't available in the output), or convert free parameters to fixed parameters (but then free parameters with the same name in the parent will throw errors).