By Andrie de Vries
Note by the editor after publication:
In the original post we neglected to give a shout out to Steve Weston, who continues to be the prime driver of new functionality for foreach, iterators and their backends.
Earlier this month Rich Calaway, programme manager at Microsoft and maintainer of the foreach package, published some updates to the foreach suite of packages, including:
Most of the changes were cosmetic, or to conform to CRAN policy. However, the last two packages (doParallel and doSNOW) had some functional changes.
The doSNOW package is a foreach parallel adaptor for the 'snow' Package. Thus it provides a parallel backend for the %dopar% function using Luke Tierney's snow package. (The snow package itself enables a "Simple Network of Workstations", i.e. support for simple parallel computing in R.)
The functional changes to doSNOW were the addition of support for user-defined progress bars.
This means that you easily enable progress bars when setting up a parallel job with doSNOW.
You can try it out with this code:
You can get other examples here.
The doParallel package provides a parallel backend for the %dopar% function using the parallel package (part of base R).
In doParallel, the most important change (change log) was a bug fix to stopImplicitCluster functionality, courtesy of Dan Tenenbaum.
Previous blog posts
We have previously written about the foreach package and parallel processing:
- Tutorial: Parallel programming with foreach
- Creating progress bars with foreach parallel processing
- Monitoring progress of a foreach parallel job
To get started with foreach, take a look at the vignette at https://cran.r-project.org/web/packages/foreach/vignettes/foreach.pdf
To get started with parallel programming with foreach and doParallel, the vignette is a great a resource: https://cran.r-project.org/web/packages/doParallel/vignettes/gettingstartedParallel.pdf