by Andrie de Vries
In my previous post, I demonstrated how to get some status of running jobs on a parallel back end. However, I stopped short of actually demonstrating progress bars. In this post I demonstrate how to do this.
The StackOverflow question How do you create a progress bar when using the “foreach()” function in R? () asks the question how to create progress bars with foreach.
This question attracted some helpful answers, but ultimately didn’t answer the question. In particular, Steve Weston said this approach is flawed, since
It won't work well with doParallel because doParallel only calls the combine function after all of the results have been returned, since it is implemented by calling the parallel clusterApplyLB function. This technique only with works well with backends that call the combine function on-the-fly, like doRedis, doMPI, doNWS, and (defunct?) doSMP.
My experiments
However, this set me thinking. Could it be that one could trigger a tcltk progress bar in each worker?
Thus I started to experiment, and came up with the following code. On my windows machine, using doParallel, this seems to work reliably (but I could not get this approach to work on Linux using doMC).
When you run this code, you should see synchronous progress bars, one for each worker:
Conclusion
Although it takes some work, and there isn’t a single solution for all cases, it is possible to get progress bars for at least some parallel back ends.
In this post I demonstrated how to create progress bars using doParallel on Windows, but failed to replicate on Linux.
Can you do better?
Comments
You can follow this conversation by subscribing to the comment feed for this post.