Parallel programming with foreach and snow

One of the design goals of the foreach construct for parallel programming was to create a syntax that would allow R users to specify a loop whose iterations could be run in parallel without having to worry about the mechanics of how it was done. In fact, foreach supports several different backend architectures for distributing code in parallel: we've already discussed multicore (for single multiprocessor or multicore workstations), NetworkSpaces (for fault-tolerant heterogeneous clusters -- currently only available with REvolution R Enterprise), and sequential programming (for testing and for single-processor machines). All the user needs to do is to "register" their preferred backend, and the foreach loop will use the selected method to achieve the parallelism.

Now there's a fourth parallel backend: snow. Snow, or "simple network of workstations" is an R package from Luke Tierney that allows you to connect multiple workstations into a parallel cluster. If you're already using snow, it's trivial to use foreach for parallel programming. Simply install the doSNOW package (just released to CRAN by the REvolution Computing development team), and then call:

> registerDoSNOW(clustobj)

where clustobj is your snow object defining your cluster. From then on, foreach will use that cluster to parallelize its iterations. 

Comments

[Tal Galili]

Hi there.

I have a question -
I wish to use the foreach with snow, to allow multicore on windows XP.

could you please give an example on how that might be done using the doSNOW ?

Thanks,
Tal

[Steve Weston]

The trick to using doSNOW on Windows is to properly configure snow. Once you've done that, you can just create a socket cluster object with:

clusterobj <- makeSOCKcluster(3)

and then register it as in David's blog. You could try to create an MPI cluster, but I believe that that would be even harder to setup.

I don't use Windows much, and I've never used snow on Windows, but I believe that the main trick is to get ssh installed. I think you need to install both the ssh client and server, but it's possible that newer versions of snow don't require that.

Sorry I can't help you more. I primarily use Linux and Mac OS X.