REvolution R Community 3.2 now available

REvolution R Community, REvolution's free distribution based on R from the R Project, has been updated to version 3.2 and is now available for download for Windows and MacOS. Some features of this release include:

Upgraded R engine. This release is based on R 2.10.1, the latest release (as of this writing). This brings many new features to the R language as detailed in the NEWS file (or you can read the highlights here and here).

Multi-threaded math libraries. The Windows version is compiled to link with the Intel MKL (Math Kernel Libraries). This means that many computations in R -- especially linear algebra functions like matrix multiply, inverse, and decompositions -- have been dramatically optimized for performance. (Optimizations are tuned for Intel chipsets, but improvements are apparent on AMD systems, too.) This is really noticeable on multi-core and/or multi-processor systems, where multi-threaded code uses all available cores for computations. (R for Windows typically only uses one core. Conversely, the Mac version has always used multi-threaded libraries.) For example, matrix multiply runs about 6x faster (32x on a quad-core box), and principal components analysis (PCA) runs about 4x faster (9x on a quad-core). (See this page for the specific benchmarks.) You don't need to change any code to benefit from these speedups -- it all happens automatically when you use those standard R functions that linear-algebra computations. On the other hand, it doesn't help with general R code that doesn't make use of math libraries, but that's where parallel programming comes in (see below).

New parallel backend for Windows. This release includes new open-source libraries from REvolution Computing to support symmetric multicore processing (SMP) on Windows machines, which enables you to speed up loops in R code running iterations in parallel on a multi-core or multi-processor machine. This is similar to using the doMC parallel backend for foreach on Mac or Linux. The new doSMP package acts as a replacement for doMC on Windows. For example, on a 4-core box, you'd register the doSMP backend like this:

require(doSMP)
workers <- startWorkers(4)
registerDoSMP(workers)

and from then on, foreach loops would run four iterations in parallel (one for each core) for up to a 4x speedup. (You get the most benefit when the body of the loop is performing time-consuming operations.) There's more info in the ParallelR Lite User's Guide, included with REvolution R Community 3.2.

Improved Installer: We've streamlined the install process to make it faster and require fewer clicks. For one thing, all of the added REvolution components are now open-source, so there's no longer any click-through license. (Of course, you're still bound by the terms of the respective open-source licenses, including the GPL for R itself.)

REvolution R Community 3.2 is available now from the link below.

REvolution Computing: Download REvolution R

Comments

[Tal Galili]

Hi David,
Is it possible to use doSMP package without installing the full installation exe ?

[David Smith]

Tal, I don't think so, no. It's bundled and tested with the REvolution distribution.

[Tal Galili]

Hi David,

I just made a simple code to compare foreach+doSMP on my dual core computer. And the performance was (much) slower then a simple for loop.
(Although looking at the CPU, I could see that doSMP did use both CPU's, which is cool)

Do you have any example showing superiority of doSMP with two cores, over the basic R code?
(since the difference might have had something to do with pre loading time of some sort)

Thanks,
Tal

[Tal Galili]

Now that I am home, here is the code:

require(doSMP)
workers <- startWorkers(2)
registerDoSMP(workers)

sme <- matrix(rnorm(100), 10,10)
solve(sme)

check <-function(n) {
#rnorm(n)
solve(sme)
}

times <- 100
system.time(x <- foreach(j=1:times ) %dopar% check(j))
system.time(for(j in 1:times ) x <- check(j))

[Jay Emerson]

Tal -

I haven't tried doSMP myself, yet, but I agree it will be a nice addition to the do* family.

Parallel computing isn't always faster. It's best for cases where the work being done at each step is substantial, so it isn't overwhelmed by the communication overhead. So, for example, inside your
check() function, you might do the solve 1000 times on 1000 random matrices.

Jay

[Tal Galili]

Hi Jay,

I tried your suggestion - and it indeed works better with dopar.

Thanks,
Tal

[etn]

David, is there any hope doSMP will eventually make it to CRAN?

[David Smith]

Hi etn, I expect it will eventually make its way to r-forge or CRAN, but for now it's exclusively available in the REvolution R distribution.

[David Smith]

Hi Tal,

I suspect the issue here is that solve() is using multi-threaded computations, and you are *also* parallelizing the large-grain computations with doSMP. In effect, you are running 4 "threads" on 2 processors, which will lead to inefficiencies.

Try running

setMKLthreads(1)

before you run the foreach loop with doSMP. This will constrain the solve() computations to one thread (processor), allowing each iteration of the foreach loop full use of one processor.