Login

News

Can you give me simple, self contained working examples for testing parallel OpenMX

Posted on
No user picture. pjohnson Joined: 09/19/2009
Forums
OpenMx General Help
I'm not an OpenMx user, but I am trying to help somebody who is. We want to parallelize OpenMX models, sending them across many compute nodes in a Linux cluster. We can run with Rmpi or SNOW. (I see you support snowfall's approach to multicore, but I mean parallelization across many computers. We can run programs using snow, but not snowfall is not allowed (incompatible with OpenMPI)).

So I'm writing to ask if you can give me some completely self contained R programs using OpenMX that are known to work. The examples in your help pages are not as helpful because they don't actually run.

You know what I mean? Something that fabricates some data and uses it, like:

x <- rnorm(100)
y <- 3 + 4 * x + rnorm(100, sd=10)
dat <- data.frame(x,y)
rm(x,y)
mod <- lm(y~x, data=dat)

I need this because we have had some trouble parallelizing the user's OpenMX model. I can't tell if the hangup is in the user's code or OpenMX itself. So I need some "certified by the factory" working examples from you.

PJ

Replied on Sat, 08/14/2010 - 13:43
Picture of user. mspiegel Joined: 07/31/2009

Hi. We are in the process of developing a mechanism for parallelism across machines in a cluster. In the meantime, you could play around with some directions I found on running a task with snow using MPI. The directions are over here: http://biowulf.nih.gov/apps/R.html. In theory, since the snowfall package uses snow to facilitate parallelism, this technique might work with independent submodels that are executed in parallel by OpenMx using snowfall. Or it might not work, I'm not sure it still needs to be tested.
Log in or register to post comments
Replied on Mon, 08/16/2010 - 12:45
No user picture. pjohnson Joined: 09/19/2009

I can handle the parallel work. I just need some examples of OpenMx code that is supposed to to work. If you can supply me with some examples, some of which are supposed to be fast, and some which take longer, I can get to work.

As it is, I can't even get started because there are no working examplesin OpenMx itself.

Log in or register to post comments
Replied on Mon, 08/16/2010 - 13:13
Picture of user. mspiegel Joined: 07/31/2009

Oh, I misunderstood the question, sorry about that. The bootstrap parallel demo is a working script that uses snowfall to take advantage of multiple cores on a single CPU. The version of the demo that we use in our nightly build system is over here: http://openmx.psyc.virginia.edu/repoview/1/trunk/models/nightly/BootstrapParallelNightly.R. You'll see that the script enables the snowfall library at the beginning of the script and pins the # of cpus to 8. And at the end of the script, the snowfall library is detached so that the remainder of our tests will run sequentially.
Log in or register to post comments
Replied on Mon, 08/16/2010 - 13:28
Picture of user. tbrick Joined: 07/31/2009

If you're looking for examples that aren't already parallelized, the models and demo directories in the trunk have a variety of working examples.

Specifically, http://openmx.psyc.virginia.edu/repoview/1/trunk/models/passing and http://openmx.psyc.virginia.edu/repoview/1/trunk/demo have a series of very short-running examples (up to a minute or two). http://openmx.psyc.virginia.edu/repoview/1/trunk/models/nightly has a series of examples that are more time-consuming, and http://openmx.psyc.virginia.edu/repoview/1/trunk/models/enourmous (sic) has some fairly long ones.

You can get these by downloading them from the SVN repository, as described on http://openmx.psyc.virginia.edu/wiki/howto-build-openmx-source-repository.

These are all part of our regular testing and benchmarking suite, so they are still on the scale of minutes-to-hours of runtime, but most are written to be scalable if you wanted to make a longer-running example. We're also constantly adding examples to these as we continue to develop our testing and benchmarking suite.

Log in or register to post comments
  Login

News