Updates to the foreach package and its friends

By Andrie de Vries

Note by the editor after publication:

In the original post we neglected to give a shout out to Steve Weston, who continues to be the prime driver of new functionality for foreach, iterators and their backends.

The new progress bar functionality as described in this post is all the work of Steve Weston (StackOverflow profile).

Earlier this month Rich Calaway, programme manager at Microsoft and maintainer of the foreach package, published some updates to the foreach suite of packages, including:

• foreach
• iterators
• doMC
• doParallel
• doSNOW

Most of the changes were cosmetic, or to conform to CRAN policy. However, the last two packages (doParallel and doSNOW) had some functional changes.

doSnow

The doSNOW package is a foreach parallel adaptor for the 'snow' Package. Thus it provides a parallel backend for the %dopar% function using Luke Tierney's snow package. (The snow package itself enables a "Simple Network of Workstations", i.e. support for simple parallel computing in R.)

The functional changes to doSNOW were the addition of support for user-defined progress bars.

This means that you easily enable progress bars when setting up a parallel job with doSNOW.

Easy progress bar with foreach and doSNOW

You can try it out with this code:

You can get other examples here.

doParallel

The doParallel package provides a parallel backend for the %dopar% function using the parallel package (part of base R).

In doParallel, the most important change (change log) was a bug fix to stopImplicitCluster functionality, courtesy of Dan Tenenbaum.

 

Previous blog posts

We have previously written about the foreach package and parallel processing:

• Tutorial: Parallel programming with foreach
• Creating progress bars with foreach parallel processing
• Monitoring progress of a foreach parallel job

Resources

To get started with foreach, take a look at the vignette at https://cran.r-project.org/web/packages/foreach/vignettes/foreach.pdf

To get started with parallel programming with foreach and doParallel, the vignette is a great a resource: https://cran.r-project.org/web/packages/doParallel/vignettes/gettingstartedParallel.pdf

 

 

Comments

[teramonagi]

Instad of foreach package, "pforeach" package(https://github.com/hoxo-m/pforeach) acts as a good wrapper package.