Revolutions

R IDE and debugger now available for 64-bit Windows; Webinar Tuesday

We've just upgraded REvolution R Enterprise to version 3.1 and expanded the available platforms to include 64-bit Windows. (REvolution R Enterprise is our subscription-based distribution of R.) This means that you can now create R programs on Windows that use all of your available memory, instead of being constrained by the 3Gb limit imposed by 32-bit versions of Windows. And you can create and run such programs using REvolution's R Productivity Environment, with all of the features of a modern development environment including breakpoints and a step-debugger.

If you'd like to learn more, I'll be hosting a live webinar this coming Tuesday, February 23 to highlight the programming and debugging capabilities of REvolution R Enterprise. If you attended our previous webinar 7 Ways to Increase Your R Productivity this one will be similar, but with additional focus on the capabilities to analyze large data sets with the 64-bit version. Click the link below for more details and to register.

REvolution Computing: Webinar: 7 Ways to Increase Your R Productivity

SPSS Co-Founder Tex Hull Joins REvolution Computing

We're proud to announce that Tex Hull, who together with REvolution CEO Norman Nie created the first version of SPSS, has joined the REvolution team. Tex will be working with Norman and our CTO David Champagne to take REvolution R Enterprise to the next level, specifically to improve its scalability to handle very large data sets.

You can read more about Tex's appointment in the press release issued this morning.

REvolution Computing: SPSS Co-Founder “Tex” Hull Joins REvolution Computing

R project named in Intelligent Enterprise 2010 Editor's Choice Awards

Intelligent Enterprise has announced its 2010 "Editors Choice" Awards, and the R project is included as one of twelve "Companies to Watch" in the Business Intelligence category.

R Project is an open-source statistical programming environment that is winning broad praise and accelerating uptake as a language for in-database analytics. The likes of SAS, SPSS and Information Builders are even using it to extend their proprietary suites.

It might seem surprising that an open-source project is named as a "Company" to watch, but this reflects the notice that commercial companies are taking of R in recent years. As a company dedicated to the support and evolution of R, we at REvolution Computing are thrilled to see such recognition afforded to R.

Intelligent Enterprise: Intelligent Enterprise Editors' Choice Awards 2010

How to make a mosaic plot in R

Mosaic plots (aka treemaps) are a great way to visualize hierarchical data. A collection of rectangles represents all the elements to be visualized (customers, news items, blog posts), with the size and color of the rectangles coding attribute. But what makes this chart unique is the arrangement of the elements: where there is hierarchy (customer segments, news topics, post categories) those elements are collected and labelled together, perhaps even with subcategories. It's easier to show than explain: you might have seen this mosaic plot of stories in Google News before, where the stories are arranged by topic area (news, sports, etc.) and sized by the number of mentions. You can create your own mosaic plot yourself in R, too: FlowingData explains how, using their own blog posts (arranged by category) as data.

 

Click the link below for the simple five-step process, using the map.market function from the portfolio package (with some final touch-up in Adobe Illustrator).

FlowingData: An Easy Way to Make a Treemap

Bay Area meet-up Tuesday: John Chambers speaks, cocktail reception

At tomorrow's meeting of the Bay Area R User's Group, ACM Software Systems award winner and R Core Group member John Chambers will give a keynote talk on the R language. The talk, titled Interfaces and Paradigms, will explore choices for computing with data with visits along the way to the history of R, reasons for the growth of R, and the importance of different concepts for the computational tasks.

Before the meeting REvolution Computing is hosting a cocktail reception where you can mingle over drinks with fellow R users. The reception is part of the Predictive Analytics World conference, but it's free to attend if you're coming for the user group meeting. The reception will be held starting at 6:30PM in the Gold Ballroom of the Palace Hotel (right next to the venue for the R user group meeting). RSVPs are requested at the link below.

Bay Area UseR Group: Cocktail Hour & John Chambers on Interfaces and Paradigms at PAW Conference

Making publication-ready tables with xtable

When you use R at the command-line, the textual output is limited by the medium: one monospaced font, with no typesetting of any kind. That's great when you're doing exploratory analysis, but what about when you want to include R output in a report or publication? In other words, what if you want to convert this Analysis of Variance table:

 
       Df Sum Sq Mean Sq F value  Pr(>F)  
sex      1  75.4  75.37 0.3793 0.539478  
ethnicty   3 2572.1 857.38 4.3147 0.006781 **
sex:ethnicty 2  298.4 149.22 0.7509 0.474767  
Residuals  93 18480.0 198.71 

to this:

 

or this:

	Df	Sum Sq	Mean Sq	F value	Pr(> F)
sex	1	75.37	75.37	0.38	0.5395
ethnicty	3	2572.15	857.38	4.31	0.0068
sex:ethnicty	2	298.43	149.22	0.75	0.4748
Residuals	93	18480.04	198.71

With the xtable package, you can. It's a handy tool for converting the output of many of R's statistical functions into a presentation-ready table in LaTeX or HTML format. For example, to create the table above, I simply did the following:

> fm2 <- lm(tlimth ~ sex * ethnicty, data = tli)
> print(xtable(anova(fm2)), type="html")

and then pasted the HTML it generated straight into this blog post. I also tweaked the border="1" table directive to border="0" -- if you have a decent Web editor you could pretty the table up further to your heart's desire.

You can find more examples of xtable in action in the package vignette.

xtable package: vignette
 

Future of Open Source Survey

Our good friends at North Bridge Venture Partners have just opened the 2010 Future of Open Source Survey, an annual look at the state of open source technology and business models, the driving factors in rising adoption of open source products and how the market for open-source software is evolving. Anyone who uses open-source software in the commercial world, whether as a user or vendor of open-source products, should participate in the survey and have their voice heard. Creator Michael Skok describes the background of the survey in this blog post:

Well over a decade ago, in my operating career, Open Source presented itself as a solution to a critical development deadline for one of my engineering teams. I was inspired. Little did I know at the time how significant a moment that was.

Shortly thereafter, upon entering the Venture world, I was determined to support Open Source Software development by seeking out and investing in the best and brightest potential people behind it.  And yet as I began that work, I quickly found a deep gulf between those who were driving the OSS  movement and those attempting to learn from and work with it.

So I arranged “The Great Open Source Debate” to bring this into focus. See pictures below of some of the OSS leaders and leading CIOs of the day. It was a fun start and further encouraged me to create an enduring program, and to literally open it up to the community at large.

Now 5 years later, North Bridge is once again proud to be sponsoring the 4th annual Future of Open Source survey in conjunction with InfoWorld, and with more than 25 leading OSS industry collaborators and thousands of participants.

As usual, the results of the survey will be presented at the Open Source Business Conference, held this year March 17-18 in San Francisco. The results are made available to the public: here are the results from the 2008 and 2009 surveys.

2010 Open Source Survey: Take the Survey

Video: What is R?

By popular demand, we've made the video of our 30-minute webcast "The R Project" available on YouTube so that everyone can easily watch it. If you (or a friend!) have ever wondered what this R thing is all about, this is the video for you. Here's the first part:

Because of YouTube restrictions it's split up into four parts, but you can find the other three parts at the link below. There you'll also find also a download link for the slides in PDF format, with lots of useful links to R resources like tutorials, blogs, books and more.

REvolution Computing: The R Project: Data Analysis and Statistical Graphics for the Enterprise

Speeding up simulations with Amazon EC2

Over at Cerebral Mastication, JD Long tells a characteristically entertaining and informative story about how he uses R to run stochastic simulations of insurance portfolios and reinsurance treaties. A typical job involves 10,000 simulations, and when each estimate takes over 20 seconds you're talking some serious time to get the job done. 

Fortunately, this is the kind of problem that lends itself to parallelization: with a multiprocessor machine running R, you can run as many simulations as you have processors at the same time, and cut the total computation time by the same factor. JD describes how he boots up an 8-core instance running R on Amazon EC2, and then uses the multicore package on Linux to reduce the computation time by a factor of 8.

Sadly for Windows users, multicore doesn't run on Windows. But as Boris Shor points out in a comment to JD's post, REvolution R Enterprise allows you to do parallel programming like this (using the foreach operator) on a multiprocessor Windows machine (or even a cluster of many machines).

Cerebral Mastication: Using the R multicore package in Linux with wild an passionate abandon

Frank Harrell to teach Regression Modeling Strategies short course

If you're using regression models but want really hone your regression-fu this short course on Regression Modeling Strategies by Frank Harrell looks really interesting. Frank is the author of the book Regression Modeling Strategies which is my go-to reference whenever I'm doing regression of any kind in R, so it's definitely worth a trip to Nashville to if you want to learn how to make regression really work. Registration closes March 1, and the 3-day course begins March 31.

By the way, if you haven't tried them already check out Frank's rms package (formerly the Design package, a package of regression tools to accompany the book) and also his Hmisc package. Hmisc has a bunch of useful utilities (PDF) to extend the R language in various ways -- the tools for creating attractive tables in LaTeX are especially useful. 

Department of Biostatistics, Vanderbilt University: 2010 Upcoming short courses in Regression Modeling Strategies