« New R User Group in Dallas | Main | Because it's Friday: Pixels invade New York »

April 09, 2010

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a010534b1db25970b0133ec92cfd9970b

Listed below are links to weblogs that reference REvolution R Community 3.2 now available:

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Hi David,
Is it possible to use doSMP package without installing the full installation exe ?

Tal, I don't think so, no. It's bundled and tested with the REvolution distribution.

Hi David,

I just made a simple code to compare foreach+doSMP on my dual core computer. And the performance was (much) slower then a simple for loop.
(Although looking at the CPU, I could see that doSMP did use both CPU's, which is cool)

Do you have any example showing superiority of doSMP with two cores, over the basic R code?
(since the difference might have had something to do with pre loading time of some sort)

Thanks,
Tal

Now that I am home, here is the code:

require(doSMP)
workers <- startWorkers(2)
registerDoSMP(workers)


sme <- matrix(rnorm(100), 10,10)
solve(sme)

check <-function(n) {
#rnorm(n)
solve(sme)
}

times <- 100
system.time(x <- foreach(j=1:times ) %dopar% check(j))
system.time(for(j in 1:times ) x <- check(j))


Tal -

I haven't tried doSMP myself, yet, but I agree it will be a nice addition to the do* family.

Parallel computing isn't always faster. It's best for cases where the work being done at each step is substantial, so it isn't overwhelmed by the communication overhead. So, for example, inside your
check() function, you might do the solve 1000 times on 1000 random matrices.

Jay

Hi Jay,

I tried your suggestion - and it indeed works better with dopar.

Thanks,
Tal

David, is there any hope doSMP will eventually make it to CRAN?

Hi etn, I expect it will eventually make its way to r-forge or CRAN, but for now it's exclusively available in the REvolution R distribution.

Hi Tal,

I suspect the issue here is that solve() is using multi-threaded computations, and you are *also* parallelizing the large-grain computations with doSMP. In effect, you are running 4 "threads" on 2 processors, which will lead to inefficiencies.

Try running

setMKLthreads(1)

before you run the foreach loop with doSMP. This will constrain the solve() computations to one thread (processor), allowing each iteration of the foreach loop full use of one processor.

The comments to this entry are closed.


R for the Enterprise

Got comments or suggestions for the blog editor?
Email David Smith.
Follow revodavid on Twitter Follow David on Twitter: @revodavid

Search Revolutions Blog