snow and snowfall now optional

Posted on
Picture of user. mspiegel Joined: 07/31/2009
As of revision 808 in the subversion repository, snow and snowfall are now optional packages. If they have been loaded prior to running a model, then independent submodels will be executed in parallel. Otherwise sequential execution occurs. We can update the installer in the next closed beta release (0.1.4) to not download these packages.
Replied on Thu, 09/10/2009 - 19:16
Picture of user. tbates Joined: 07/31/2009

In the kind of setup we have with a supermodel whose objective is the sum of its sub-model's objectives, and submodels which do algebra on matrices in the supermodel, is there any scope for parallelisation?
Replied on Fri, 09/11/2009 - 23:02
Picture of user. neale Joined: 07/31/2009

In reply to by tbates

If I understand you correctly, there should be some scope but at a finer granularity than if the submodels (aka groups in Mx1 parlance) are independent. So, it would be possible to evaluate the submodels separately for each set of parameter estimates, and to combine the threads on the pass back to the optimizer with the sum of the fit functions of each submodel. Indeed, this could significantly speed up single-group jobs simply by cutting the dataset into as many subsets as there are processors, and creating a separate group for each.
Replied on Sat, 09/12/2009 - 11:56
Picture of user. tbates Joined: 07/31/2009

In reply to by neale

Yes, that's what I meant.
Certainly having each group automatically evaluate on a separate core if available passing the fit functions to the optimizer where they need to be aggregated would be a massive speed up.

Certainly seeing just 7 of 8 cores running at 0% is silicon sadness :-)