Belated thoughts on DSC 2009

John Maindonald has just posted a great collection of notes about his experiences at the two recent R conferences in Europe, UseR! and DSC. Especially if you weren't able to attend, his notes give a great sense of the atmosphere and focus areas of the two conferences.

It also reminds me that I hadn't yet posted my own thoughts about the DSC conference in Copenhagen. I've just returned home from all my conference travels (sadly, I can't make JSM) and am just now tying up some loose ends. So let me correct that omission.

For those that haven't attended before, DSC is quite a different conference than UseR. Unlike UseR which is focused on how R is used, DSC is focused mainly on R itself: how it works, and how it can be improved. DSC is also quite a bit smaller: about 60 attendees compared to almost 500 for UseR. But that's also its great strength, providing an intimate and social atmosphere to get to know the people behind R, and to have in-depth discussions about R and its place in the open-source, programming, and statistical world.

The format of the conference also allows for some deeper explorations into R issues. Save the opening talk (this year, an excellent history of computing from Tue Tjur), all talks are contributed, and all are 35 minutes in length (compared to 15 minutes for a contributed UseR talk). There are also few parallel sessions, meaning that it's easy to see all the talks you're interested in.

Some of the most memorable talks for me were:

• John Chambers talking about the background and motivations of the S3 and S4 class systems in R -- with some suggestions of future developments from the audience
• Deepayan Sarkar and Michael Lawrence talking about a new, faster graphics subsystem linking to Qt for interactive graphics
• Uwe Ligges, giving insight to some of the practical implications of how CRAN checks submitted packages 
• Karim Chine demonstrating how the Biocep front-end can use remote R sessions transparently, including R running on Amazon EC2 
• Nicholas Lewin-Koh discussing a real-life success story at Genentech, delivering statistical applications to scientists (but not R users) with Rserve
• Doug Bates talking about some of the slick computational internals of the lme4 package, making R the only package able to fit mixed models with very large numbers of parameters 
• Stefan Theußl describing the workings of R-Forge, now the development repository and build tool for hundreds of R projects
• Simon Urbanek demonstrating iPlots Extreme, an amazingly fast and beautiful linked-graphics system coming soon for R 
• and Hadley Wickham, discarding his planned talk in the wee hours of the morning to propose a new way of thinking about object-oriented programming in R and prompt some fascinating discussion from the audience. 

My own talk was on a new Integrated Development Environment (IDE) for R that the REvolution Computing is sponsoring and developing. I talked about the forthcoming changes to R (mainly by Robert Gentleman and Duncan Murdoch) that will allow step-debugging of R code from the command-line, how REvolution Computing's developers are integrating these features into Microsoft's Visual Studio Shell. My slides are available for download (1.3Mb PDF). There will also be a proceedings volume from all the talks published next year.

All in all, it was a great conference -- congratulations to the organizers (and not least for such a wonderful conference dinner!). I look forward to the next meeting in 2011.

Comments

[Kevin Wright]

David,

I would like to correct your comment that lme4 is "the only [R] package able to fit mixed models with very large numbers of parameters". It is certainly able to fit models with large numbers of parameters, but it is not the only package able to do so.

I would like very much to use nlme or lme4 since they are open source, but nlme is too slow for large datasets, and lme4 cannot fit some complex variance-structure models.

I believe that only asreml-r has this feature set:
* fast (on par with lmer)
* can handle large datasets / number of parameters
* stable and mature (more than 10 years, I think)
* fits complex variance structures
* convenient tools for making very complex model predictions

asreml-r is closed-binary-source and requires a license, so is not for everyone, but for certain users it is the only option.

asreml-r manual here:
http://www.vsni.co.uk/resources/doc/asreml-R.pdf

useR conference slides
http://www.r-project.org/conferences/useR-2007/program/presentations/butler.pdf

More information
http://www.vsni.co.uk/products/asreml/platforms.php

Kevin Wright

[David Smith]

Thanks for the links, Kevin. Doug's claim was that R was the only software that could fit such models (my paraphrasing was poor); you'll have to duke it out with him on which *package* does it best. :)

[Kevin Wright]

Here is a simple, illustrative example.

dat2 is a data.frame with
H - 51 levels
L - 6101 levels
23923 observations

Using asreml-r:

m0 <- asreml(y~1, data=dat2, sparse=~L, random=~H)

m0$varcomp

           component std.error   z.ratio

H           42.27893  8.616926  4.906498

Residual   101.09580  1.072468 94.264655

Using lmer:

m1 <- lmer(y~1+L+(1|H), data=dat2)

Error: cannot allocate vector of size 1.1 Gb

Kevin Wright