Revolutions

R 2.11.1 released

It's official: R 2.11.1 is out. Source code and binaries for Windows and MacOS are available at the master CRAN mirror, and will be available for download from your local mirror soon.

As anticipated, this is an update release focussing mainly on bugfixes and with just one new feature. According to R core team member Peter Dalgaard one fix in particular may be significant if you work with large quantities of dates: a problem with format.POSIXlt that could cause a crash when working with long date vectors has been fixed. You can see the complete list of changes in the current NEWS file. 

R-announce mailing list: R 2.11.1 is released

Because it's Friday: The dating equation

According to internet lore, there's a mathematical equation that governs the lower bound for the socially acceptable age of a potential dating partner: half your age plus 7, or, in mathematical terms, if x is your age then the lower bound is f(x) = x/2 + 7.

Seems simple, right? if you're 20, then the minimum socially acceptable age for a date is 17. But it was The Ragbag who pointed out that this rule has a duality: for the date to be socially acceptable, the rule must be adhered to by your date as well. In other words, there's a socially acceptable maximum too, given by inverting the equation:

It was only yesterday that i realised that the rule of thumb for dating people of different ages (the “half your age plus 7” rule) determines not only the lower bounds for dating but the upper bounds as well—that for each ½x + 7, there is a corresponding 2(x-7). For the last 15 years of my life, i have been ignoring an entire market segment, namely those of the genus cougar.

The Ragbag handily plotted out the socially acceptable upper and lower bounds for dates at various ages in the chart below:

Notably, the chart also implies that there is a socially acceptable lower bound for your own age: below the age of 14, by this rule, you shouldn't be dating at all.

Incidentally, if you're wondering how well is this rule adhered to in practice, the dating website okcupid.com has done an extensive analysis of age preferences amongst their members. A really interesting analysis (and quite sophisticated - maybe done in R?), especially for heterosexual males looking for the ideal age-ranges for potential dates. 

The Ragbag: The half your age plus 7 rule

Map-reduce in R with Amazon EMR

There's new video at the Video Rchive[*] of a JD Long presentation: Amazon Elastic Map-Reduce (So Easy an Economist Can Do It). I'm on a plane right now and the in-flight wi-fi isn't quite up to playing the 30-minute video, but judging from the slides it looks like a nice, practical demonstration of getting a map-reduce computation programmed in R running on the Amazon cloud system. The code also appears quite complete: in a clever innovation, the commands to launch the Amazon instances run directly from the R code, which should eliminate a lot of the tedious command-line monkeying that typically accompanies an Amazon EC2 session. I look forward to trying it out when I'm landside.

[* Revolution Analytics sponsors the Video Rchive, which is provided as a service for the community by Drew Conway. Thanks, Drew! You can donate to the Video Rchive too, see the link in the right column of Drew's blog.]

VCASMO/JD Long: Map-reduce in R with Amazon EMR

How to map your Twitter social network

Ever wondered which Twitterers you and a friend share? Using R and the twitteR package, there's an easy way to find out. Cornelius Puschmann hacked together some R code to do just that for the Humanities and Technology Camp and it seems to work pretty well. Just replace 'coffee001' with the your Twitter username, 'mypassword' with your Twitter password, and 'Cornelius' with your friend's Twitter name, and the code does the rest. The only problem I had was that my graph was very cluttered, and despite the comments indicating you can set n to a smaller number, it didn't work for me. (The code is very clearly commented, by the way.) I had to add the following two lines to make a readable chart:

friends.object <- friends.object[1:25]
followers.object <- followers.object[1:25]

The chart of followers I (@revodavid) share with the RevolutionR twitter account is below.

Although to be honest, looking at the chart now, I'm not actually sure this is our shared followers. I'll have to look at the code more closely to figure out what's going on; in any case it's a neat example of integrating R with the Twitter API.

Cornelius Puschmann's Blog: Code and brief instruction for graphing Twitter with R

German Tanks, Statistical Intelligence

In World War II, the Allies had a problem: German tanks were often captured, but how many more did the Nazis have in reserve? Allied intelligence estimated around 1400 Panther tanks were being produced a month: a formidable arsenal, and perhaps an insurmountable one given the much smaller numbers being captured or destroyed. But those captured tanks provided exactly the clue needed to get a more realistic assessment of the German production rates: serial numbers. Assuming that the serial numbers on the tanks were assigned in sequence, and looking at the serial numbers of those captured, Allied statisticians came up with simple arithmetic formula  to come up with a less daunting estimate of the production rate: 256 per month. (Production data revealed after the war revealed the true figure to be 255 per month.)

Suppose N tanks were actually produced, and k were captured with the highest serial number observed being m. The formula (rather intuitively) estimates the actual number of tanks as the maximum serial number plus the average gap between serial numbers observed, or in formula form, m + m/k - 1. But how well does this work in practice for different values of N (actual tanks)? In a statistical version of a World War II reenactment, the Statistics Blog has recreated the German Tank Problem using R, simulating captured serial numbers for various values of N, and comparing this "true" value with the estimate obtained. The process was repeated for various values of N, and the results plotted against each other. Here are the results:

As evidenced by history, the formula is startlingly accurate, even though k, the number of captured tanks, is set at just 20! I'd never come across the German Tank Problem before, so thanks to Statistics Blog for providing such a great illustration of it. I also liked the commentary on Maximum Likelihood Estimation, as this provides one example where the MLE fails:

The MLE for the number of German tanks is the highest serial number observed. This is because MLE works backwards, finding the parameter which makes our observation most likely in terms of joint conditional probability. As a result, the MLE for this problem is not only biased (since it will always be less than or equal to the true number of tanks), but dumb as well.

Statistics Blog: How many tanks? MC testing the GTP

Webinar: R analytics and Business Intelligence dashboards

On Wednesday next week, Revolution will be hosting a joint webinar with open-source Business Intelligence software maker Jaspersoft and open-source Business Intelligence services company OpenBI. Together, we'll be talking about making BI dashboards even more powerful by integrating custom visualizations and advanced predictive models from R. If you create advanced analytics in R and are looking for a way to make the results available to business users in an easy-to-use dashboard environment, this is a great webinar to check out. The complete agenda is below, and you can register for free for the live event at 9AM (Pacific Time) on June 2.

SUPERCHARGE BI AND DASHBOARDS WITH PREDICTIVE ANALYTICS WEBINAR
Powered by Jaspersoft, Revolution Analytics, and OpenBI

Date:    Wednesday, June 2, 2010
Time:    9am - 10am Pacific

Presenters:
David Smith, Vice President of Marketing, Revolution Analytics
Steve Miller, President, OpenBI, LLC
Andrew Lampitt, ISV/Technology Alliances, Senior Director, Jaspersoft

Business intelligence is about providing reporting and analysis solutions that show business users what happened and why. On the other hand, advanced analytics solutions deliver deeper insight into what might happen in the future, based upon high volumes of historical data and sophisticated modeling techniques. Traditionally, advanced analytics has been reserved for a highly technical audience in fields such as life sciences and academia. However, with the explosion in data in nearly every sector, advanced analytics is now becoming useful to more mainstream business users. This webinar introduces commercial open source business intelligence solutions from Jaspersoft, advanced analytics solutions with the popular open source project R and Revolution Analytics, and how those separate products mesh together in a demonstration by OpenBI, the expert commercial open source system integrator.

Target Audience:

• BI implementors (including Jaspersoft users) seeking to integrate predictive analytics into BI dashboards
• R users / predictive analytics developers seeking to distribute advanced analytics to business users
• Business Users in general seeking to improve their BI outcomes

Level: No background knowledge required.

Revolution Analytics: Supercharge BI and Dashboards with Predictive Analytics

Because it's Friday: Ash

I lived for 10 years within sight (on a clear day, anyway) of Mount St. Helens, and had seen and heard a lot about the devastation caused by the eruption and pyroclastic flow 30 years ago. But I'd heard relatively little about the effects of the ash cloud settling on land. I was surprised to learn about the effect it had on the city of Spokane: basically immobilizing everyone there and every vehicle for days. If you thought the effect of the Eyjafjallajökull eruption on air traffic was serious, imagine what would happen if the ash cloud settled on a city like London or Paris. Check out the slide show below for what happened to Spokane, and extrapolate.

Spokane Spokesman-Review: Mount St. Helens

R 2.11.1 scheduled for May 31

As announced by the R Core Team, the next update to R will be 2.11.1 to be released on May 31. Despite being a minor-minor version increment, this release is expected to sport at least one new feature: BIC (in package stats4) will work with multiple fitted models, like AIC does. There will also be some improvements to the installation process, and a few bug fixes (mainly for some pretty obscure cases: the level of detail these fixes go to is a testament to the quality of R). 

Look for the updated sources on May 31, with binaries to follow soon after.

R-announce mailing list: R 2.11.1 Scheduled for May 31

Tip of the day: Keep the console active in R Productivity Environment

Jared Lander on Twitter asks:

When I alt-tab into [Revolution] R Enterprise how do I make the #rstats [R] console the active window by default. Now it goes to solution explorer.

Revolution's crack Support engineer Stephen Weller offers this solution:

The best way to do this is to right-click on the R-Console window and make it a 'Tabbed Document'. If you then make the 'R-Console' window the active window and exit Revolution R, it will remain the active window the next time you start the program (or press 'Alt-Tab').

In in addition to the R console, the  R Productivity Environment includes a smart R code editor and interactive debugger. It's included with Revolution R Enterprise for Windows, available by subscription for businesses, and for free download for academic users.

Calling all T-shirt designers

Got design skills as well as R skills? The organizers of the useR! 2010 conference are looking for a design to be used on the conference T-shirts. Mango Solutions (who are sponsoring the T-shirt) will select the winner, but personally I'd give extra points for designs that use R itself -- it's been done before!

R-statistics blog: useR-2010 is looking for a T-shirt design