Revolutions

R Prominent at Predictive Analytics World

The new Predictive Analytics World conference focuses on data mining and predictive models to predict customer behavior in business environments, so it was gratifying to see R mentioned prominently in many talks. Keynote speaker John Elder included R in a list of data analysis tools he uses. Natasha Balac described how she uses R to fit models to detect fraud in Medicaid claims. Dean Abbott mentioned it was possible to use R to fit ensemble models. (An aside: ensemble models in general are a hot topic in this crowd.) Anne Milley, in the sponsored presentation from SAS, even took the time to mention the New York TImes article on R and the fact that integration to R is now available from SAS/IML. (It's pleasing to see SAS is warming to R: Anne also mentioned there are plans to extend integration to base SAS and JMP, too.) There may well have been more mentions -- I couldn't go to all of the parallel-track sessions, and I expect R will feature in more talks on the second day of the conference.

In fact, R was highly visible right from the outset. The Bay Area UseR! Group meeting was a prominent part of the agenda during the opening remarks from Eric Siegel, and was held in the same venue as PAW directly after the evening reception. The UseR! group itself was a great success, with more than 60 attendees and outstanding case studies and discussion from Bo Cowgill of Google and Itamar Rosenn from Facebook on their use of R.  Jim Porzak also gave a great overview of R itself.  My thanks go out to Jim and Mike Driscoll for inviting me to be such of a great panel. The panel discussion itself was taped and should be available online soon -- I'll link to it when it is. 

Reminder: Bay Area UseR Group meeting tonight

To those in the San Francisco area, a reminder that I'll be a participant in the panel discussion at the Bay Area User's Group meeting tonight, along with participants from Google, Facebook and The Generations Network.  If you haven't already RSVP'd, please do so here, to get a badge to enter the Predictive Analytics World venue.  I'll be at PAW all day today and tomorrow, so if you're there and want to meet up, send me an email.

Defending boxplots

Andrew Gelman thinks boxplots are horrible (he prefers horizontally-jittered dot plots). Some of his commenters leap to their defense, leading Andrew to issue this challenge:

I have never ever seen an example where I've felt a boxplot was appropriate. I'm open to being convinced, but I don't think you'll be able to convince me. Bring on the examples!

Personally, I think boxplots are especially useful when you need to compare multiple distributions: looking at changes in trend or variability over time, or between many samples. (Note the plural though: you'd probably prefer a histogram to a single boxplot.) It's a lot easier to compare the five statistics and look at the outliers than it is to try and see the differences in dozens of histograms lined up side-by-side -- if you can even fit them all on the page.

I think this example of microarray data from one of the commenters (which, I'd add, was created in R) is a great use of boxplots. Anyone got any other great examples in defense of boxplots?

R graphics in the media

As sophisticated quantitative graphics become more commonplace in the mainstream media, I sometimes see charts on the Web that might have been created in R.  I know the charts at pollster.com are done in R, and a correspondent (thanks, LA) suggests some in the Economist might have been done with R as well. The ones in the Economist aren't stock-standard R graphics, but R's graphics system is powerful enough that if you can design a chart, you can easily write a script to create it.

This led me to the question: when you see a quantitative chart on the Web, is there any way to tell if it's done with R?  Sadly, for the JPG and PNG graphics formats commonly seen on the web, the answer is no. Although the standards support the inclusion of metadata that might be used to identify R as the creator of the graphic, R's jpg() and png() drivers don't create any such identifying metadata. (Of course, as open-source software R could be modified to do so, but it doesn't today.) R doesn't create GIF files, so unless they've been converted from one of the formats that R does create, any GIF files you see aren't from R. 

In the case of PDF and PostScript files, though, R does leave an identifying signature, which you can see by looking at the .ps or .pdf file in a text editor.  PDF files contain the lines:

/Title (R Graphics Output)
/Producer (R 2.8.0)
/Creator (R)

(the Producer line depends on the version of R used), and PostScript files contain:

%%Title: R Graphics Output
%%Creator: R Software

It's possible to change the Title element with the title= argument to postscript() or pdf(), but not many people do that. You can actually search for the default "R Graphics Output" title with an advanced Google search, which makes it easy to find sites that use R for PDF/PS graphs. (Here's the Google search that finds PostScript and PDF files with the default title.) Looking through the results, the usual suspects appeared (NOAA, EPA, journals, academic sites) but no big surprises -- but I'd expect most "mainstream" sites to be using PNG or JPG graphics, anyway.

Know of any other popular websites that use R to create graphics?  Let us know in the comments.

How to get IT to accept, and love, R

IT departments in large organizations have a reputation for being conservative. Despite the successes and widespread deployment of projects like Linux, Apache and MySQL, it can still sometimes be difficult to get IT to accept and support open-source applications.

Daniel Viar had such an experience, and asked on the r-help mailing list, "How do I get my IT department to 'bless' R?":

I currently use R at work "under the radar", but there's a chance I could lose that access.  I'd like to get our company to feel comfortable with open source and R in particular.  Does anyone have any experience with their company's IT department and management that they would be willing to share?

The question prompted more than 30 replies with suggestions, which Daniel helpfully summarized in a followup post.  Below, I've selected some of the best arguments and information you can use to convince IT not only to accept R, but to understand its many benefits compared to other statistical software and even, perchance, to love it.

Find information about R with Rseek.org

One of the great things about R is that there's a wealth of information about it available on the web.  On the downside, though, it can sometimes be tough to find. For one thing, "R" is a search term found on many websites, many not related to R the language. (Google actually handles this better than you might think: the first 5 sites currently returned by a Google search for R alone are all relevant to R.  The sixth is for the R subway line in New York City.)  For another, information about R can be found in many different places: The official R project site, CRAN (and its mirrors), the archives of the R mailing lists, the documentation of R itself, and even this blog.  As a result, searching for relevant information about R can sometimes be a chore.

Rseek.org makes it easy to search for information about R while filtering out those sites (like those for the New York Subway) that match "R" but don't contain the information you're looking for. It's just like a Google search, but restricts the search just to those sites known to contain information about R.  The list of sites searched is maintained by Rseek.org creator Sasha Goodman and a cadre of volunteers.

R Google Search: Rseek.org

How to choose a random number in R

As a language for statistical analysis, R has a comprehensive library of functions for generating random numbers from various statistical distributions.  In this post, I want to focus on the simplest of questions: How do I generate a random number?

The answer depends on what kind of random number you want to generate.  Let's illustrate by example.

Generate a random number between 5.0 and 7.5

R2jags: Bayesian Hierarchical Modeling in R

I've just discovered an alternative to BUGS for Bayesian modeling: JAGS (Just Another Gibbs Sampler). According to author Martyn Plummer, JAGS is "not wholly unlike BUGS" and has the following goals:

• To have an engine for the BUGS language that runs on Unix
• To be extensible, allowing users to write their own functions, distributions and samplers.
• To be a plaftorm for experimentation with ideas in Bayesian modelling

There's also an interface to JAGS from R in the package R2jags by Yu-Sung Su, available from CRAN. I haven't used JAGS or R2jags myself, but it looks interesting.  I'd be interested to learn how it compares to OpenBUGS. Anyone out there used it? Let us know in the comments. 

Statistical Modeling Blog: Our new R package: R2jags

Update 13:05: Corrected authorship of packages.

Four recession stories, graphed

Displaying data graphically is a way of telling a story.  The story you tell depends on which data you select, and how you present it.  As a result, a simple question like: "How do the job losses in the current US recession compare to previous recessions?" can become contentious, depending on which recessions you choose to compare it to, what time point you compare job losses to (what is "peak employment"?), and how you align the comparisons in time (when did each recession start?).

Flowing Data presents four different stories about the current recession, in graphical form.  Which one is correct?  It depends on the story you're trying to tell.

UseR! abstract submission deadline: February 27 2009

As a reminder for anyone planning to attend UseR! 2009 in Rennes, France July 8-10, the submission deadline for abstracts is fast approaching.  All abstracts must be submitted by February 27.  This is also the last day for early registration.

More details: UseR! 2009 Website