Revolutions

Top 3 R resources for beginners

The community team at Revolution Analytics has just updated this list of resources to learn about R on the Web. Included is this list of the top 3 resources for absolute beginners getting started with R:

• An Introduction to R – The free, “official” CRAN R Manual
• Try R – a short course that lets you jump right in
• Computing for Data Analysis – 4 weeks worth of videos from a popular Coursera course

Other resources listed on that page include R community resources, books on R, tools for finding R packages, resources for getting help with R, and R training courses. Check them all out at the link below.

Revolution Analytics: Additional R Resources

In case you missed it: April 2013 Roundup

In case you missed them, here are some articles from April of particular interest to R users:

A critique of a SAS whitepaper comparing the performance of SAS, R and Mahout.

A video presentation from statistician Tess Nesbitt at UpStream, who uses GAM survival models in R for marketing attribution analysis.

The April edition of the Revolution Analytics newsletter.

The "lahman" package provides access to baseball statistics from R.

New features in Revolution R Enterprise 6.2, now available.

Videos from Jeff Leek's Coursera course on data analysis with R.

Revolution Analytics named to CRN Big Data 100.

An informative discussion amongst data scientists.

Why you should use R instead of Excel for statistics.

An early preview of Revolution R Enterprise 7.

A handy cheat sheet for symbols you can use in equations in R graphics.

How to discretize data and perform stepwise regression on big data with the RevoScaleR package.

Some useful lists for data scientists and statisticians (including 17 R-related sites).

R user group presentations from Dublin, Milan and Cambridge.

The R code behind Rodrigo Zamith's NCAA Tournament Visualizer.

The new bigvis package for visualizing large data sets.

Using the RevoScaleR package to visualize big data with hex bin charts.

A look at SAS from an R programmer's perspective.

R version 3 is released, now supports big vectors.

New features in the RevoScaleR package.

Ten new R user groups around the world.

Some non-R stories in the past month included: handy tech tips, dogs that play ball with themselves, how open source software can reform government, the PeerIndex Big Data 100, quadrocopter vs Tesla coil and a funny video about quirky programming languages. 

As always, thanks for the comments and please send any suggestions to me at david@revolutionanalytics.com. Don't forget you can follow the blog using an RSS reader, or by following me on Twitter (I'm @revodavid). You can find roundups of previous months here.

Because it's Friday: A visual history of meteorites

At the aptly named website bolid.es you can see this nifty animation of all the meteorites that have been observed to land on Earth, from 861 to this February's spectacular event in Russia:

The animation is flashy to be sure, but it certainly gets the point across: while more people are seeing meteors as the population grows, impacts have been a constant part of our history, and will continue to be so in the future. You can drill down into individual impacts by clicking the blue bars.

Keep your eyes on the skies this weekend, and have a good one!

A guide to speeding up R code

Noam Ross recently shared a very useful guide to speeding up your R code. 

• Get a bigger computer (for example, renting an instance on the Amazon cloud for a few cents an hour)
• Use parallel programming techniques
• Using the R byte-compiler
• Profiling and benchmarking your code
• Using high-performance packages (like xts, for time series)
• And lastly, rewriting your code to use more efficient constructs

One other tip that can have some great performance benefits is linking R to parallel BLAS libraries (Revolution R does this by default). For more details on how to speed up your R code read Noam's excellent guide, linked below.

Noam Ross: FasteR! HigheR! StrongeR! - A Guide to Speeding Up R Code for Busy People

Webinar on Data Science, May 14

A quick note that I'll be giving my take on the question, "What is a Data Science" along with Carla Gentry (Analytical Solutions, @data_nerd on Twitter) and Gregory Piatetsky (KDNuggets, @kdnuggets on Twitter) in a free webinar this Tuesday, May 14. You can register here, and I've included the webinar description below. Please join us for what promises to be a lively discussion.

Data Scientist: Your Must-Have Business Investment NOW

Data science has been called the sexiest job of the 21^st century by the Harvard Business Review, and for good reason. Big Data and analytics are growing trends that require experts in interpreting the vast amounts of data being collected and transforming it into actionable insights. Those experts are Data Scientists, and we are lucky to be hosting a few of them for a one hour webinar.

Please join us on May 14th at 3pm EST as we host, Carla Gentry, Data Scientist at Analytical Solution, David Smith, Data Scientist at Revolution Analytics, Gregory Piatetsky, Editor, KDnuggets; co-founder KDD and ACM SIGKDD, Eric Kavanagh, DM Radio Host at Information Management Magazine, and Darren Peirce, CTO at Kalido.

Some of the topics we will explore in this open panel will include:

• The current state of data science and emerging trends
• What’s new on the tricks and tools landscape
• The role (and shortfalls) of traditional data warehouses
• Best practices in deploying data science to production

Register here to sign up for the webinar

Many thanks to Kalido for putting the panel together.

Update May 2: Now available: A roundup of my presentation at the webinar, and the webinar replay

Trevor Hastie presents glmnet: lasso and elastic-net regularization in R

by Joseph Rickert

Even a casual glance at the R Community Calendar shows an impressive amount of R user group activity throughout the world: 45 events in April and 31 scheduled so far for May. New groups formed last month in Knoxville, Tennessee (The Knoxville R User Group: KRUG) and Sheffield in the UK (The Sheffield R Users). An this activity seems to be cumulative. This month, the Bay Area R User’s Group (BARUG) expects to hold its 52nd and 53rd meet ups while the Sydney Users of R Forum (SURF) will hold its 50th. Everywhere R user groups are sponsoring high quality presentations and making them available online, but the Orange County R User Group is pushing the envelope with respect to sophistication and reach. Last Friday, I attended a webinar organized by this group where Professor Trevor Hastie of Stanford University presented Sparse Linear Models with demonstrations using GLMNET. This was a world-class presentation and quite a coup for Orange County to have Professor Hastie present.

The glmnet package written Jerome Friedman, Trevor Hastie and Rob Tibshirani contains very efficient procedures for fitting lasso or elastic-net regularization paths for generalized linear models. So far the glmnet function can fit gaussian and multiresponse gaussian models, logistic regression, poisson regression, multinomial and grouped multinomial models and the Cox model. The efficiency of the glmnet algorithm comes from using cyclical coordinate descent in the optimization process and from Jerome Friedman's underlying Fortran code.

Although Professor Hastie’s presentation was primarily concerned with fitting models for the wide problem (the number of explanatory variables is much larger than the number of observations) the lasso and elastic-net algorithms are just as applicable to data sets with large numbers of observations. It is likely that in the future we will see glmnet implementations for variable selection on datasets with thousands of variables and hundreds of millions of observations. The following graph shows the regularization paths for the coefficients of a model fit the HIV data from one Professor Hastie’s examples.

Each curve represents a coefficient in the model. The x axis is a function of lambda, the regularization penalty parameter. The y axis gives the value of the coefficient. The graph shows how the coefficients “enter the model” (become non-zero) as lambda changes. The following code, based on an example from the webinar, produces the plot and also shows how easy it is to perform cross-validation.

library(glmnet)	    # load the package	
load("hiv.rda")	    # HIV data
class(hiv.train)    # The data are stored as a list 
names(hiv.train)    # The names of the list elements are x and y   
dim(hiv.train$x)    # The explanatory data consists of 704 observations of
		    # 208 binary mutation variables
head(hiv.train[[1]])	# Look at the explanatory data
head(hiv.train[[2]])    # Look at the response data: changes in susceptibility to antiviral drugs
fit=glmnet(hiv.train$x,hiv.train$y)	# fit the model
plot(fit,xvar="lambda", main="HIV model coefficient paths") # Plot the paths for the fit
fit		    # look at the fit for each coefficient
#
cv.fit=cv.glmnet(hiv.train$x,hiv.train$y)	# Perform cross validation on the fited model
plot(cv.fit)	# Plot the mean sq error for the cross validated fit as a function
                # of lambda the shrinkage parameter
		# First vertical line indicates minimal mse
		# Second vertical line is one sd from mse: indicates a smaller model
		# is "almost as good" as the minimal mse model
tpred=predict(fit,hiv.test$x)	# Predictions on the test data
mte=apply((tpred-hiv.test$y)^2,2,mean)	# Compute mse for the predictions
points(log(fit$lambda),mte,col="blue",pch="*")	# overlay the mse predictions on the plot
legend("topleft",legend=c("10 fold CV","Test"),pch="*",col=c("red","blue"))

Created by Pretty R at inside-R.org

Don’t be content with this partial example. Professor Hastie and The Orange County R User Group have graciously made the slides, code and data available at this link. The webinar is well worth watching in its entirety.

 

As you might expect, Professor Hastie gives a masterful presentation: lucid, clear and succinct. This is inspite of the fact that Professor Hastie begins the presentation by commenting that it was his first webinar ever and that he was a little uncomfortable talking to his screen. (I think anyone who has ever given a webinar can relate to this: you talk to the screen and no energy from the audience comes back. Nothing is more disruptive to efforts to be enthusiastic than silence.) Nevertheless, Professor Hastie presents a difficult topic with a clarity that carries his audience along, and he is completely unphased by the inevitable glitch. Watch how he handles the upside down slide. You can download his slides, R scripts and data from the link below.

Trevor Hastie: Sparse Linear Models, with demonstrations using GLMNET

What's new in Revolution R Enterprise 6.2 (video)

If you missed last week's webinar, product manager Thomas Dinsmore shared details of the new features in Revolution R Enterprise 6.2 in the video below:

 

You can also download slides of the presentation at the link below.

Revolution Analytics webinars: What's New in Revolution R Enterprise 6.2

SAS Big Data Analytics Benchmark (Part Two)

by Thomas Dinsmore

On April 26, SAS published on its website an undated Technical Paper entitled Big Data Analytics: Benchmarking SAS, R and Mahout.  In the paper, the authors (Allison J. Ames, Ralph Abbey and Wayne Thompson) describe a recent project to compare model quality, product completeness and ease of use for two SAS products together with open source R and Apache Mahout.

This is the second post in a two-part series.  Last week, I covered simple mistakes and errors. this week's post will cover the authors' methodology and findings.

I have contacted the authors and asked if they plan to release their data to the community.  To date, I have received no response. 

Methodology

The methodological problems in the SAS benchmark paper are too numerous to cite comprehensively, but I'll review the most glaring issues.

Choice of Products to Test:  Although the authors compiled the list of algorithms to be tested through a survey of SAS users, no single SAS product supports all three.  R supports all three, as does Mahout (with a workaround). 

The choice of products for comparison is strangely "apples and oranges".  Mahout is an early-stage analytics library; R is a mature analytic programming environment; SAS Rapid Predictive Modeler is a semi-automated ensemble modeling framework, and SAS High Performance Analytics Server is a high-end analytics platform designed for use in an MPP computing environment.  Comparing the two open source products with two high-end SAS applications on “ease of use” is a straw man comparison.  One would expect the two high-end SAS applications to be easier to use; that’s why SAS charges a premium for them.    

Oddly enough, SAS chose to omit its own product SAS/STAT from the benchmark. As an analytic programming language, SAS/STAT offers a more apt comparison to R, and it is far more widely used than either of the two high-end SAS products.

Analytic Workflow:  In the real world, data resides in external sources, and must be loaded into each of the respective platforms; analysts have choices when marshalling and loading data, and can do so in a way that best suits the application. The authors used SAS to prepare the data for use in all four products, starting with structured data sets loaded into Hadoop. This biases the "ease of use" assessment toward SAS.  

Structured data stored in Hadoop is not "native" to Hadoop but structured externally and loaded into the Hadoop file system; the authors did not assess the effort needed to do this, which biases the "ease-of-use" assessment towards SAS High Performance Analytics Server, which cannot operate directly on raw data. It also biases the assessment against Mahout, since the principal advantage of Mahout is its ability to work with data stored natively in Hadoop.  

The authors' decision to split the data into training and validation sets in SAS rather than using native splitting for each tool is puzzling. The stated reason for using this approach — that random seeds may differ among the tools — is fallacious; random samples of sufficient size are generalizable to one another even when produced with different random seeds.

Computing Environments: The computing environments used in this benchmark are vastly different. This is attributable in part to the widely different requirements of the products compared, but nevertheless there are some unexplained anomalies. There is no justification for using machines with very different memory profiles to host open source R and SAS Enterprise Miner; the two machines can and should be identical. By the same token, a five node Hadoop cluster is not equivalent to a sixty node MPP Greenplum appliance.  

Assumptions About User Skill: The authors tested the open source tools and the SAS high-end applications using a naïve "hands-off" approach that depends on "out of the box" functionality.  This is a trivial comparison that simply proves the obvious. When Deep Blue beats Kasparov at chess, it's news; when Deep Blue beats a three-year-old at chess, it's not news.  

Findings

The authors propose five main findings:

(1) The types and depth of classification models that could be run in SAS products and R outnumbered the options for Mahout. 

(2) The effort required by individual modelers to prepare a representative table of results was much greater in both R and Mahout than it was in SAS High-Performance Analytics Server and SAS Rapid Predictive Modeler for SAS Enterprise Miner.

(3) The object-oriented programming in R led to memory-management problems that did not occur in SAS products and Mahout; thus, the size of customer data that could be analyzed in R was considerably limited. 

(4) Overall accuracy was "comparable" across software in "most instances". 

(5) When the customer modeling scenarios were evaluated based on event precision and rank ordering, SAS HighPerformance Analytics Server models were more accurate. 

The first two points are true but trivial. Anyone with a smattering of knowledge understands that Mahout is an early stage project with limited functionality. The two SAS tools are high-end applications with a number of automated features. That's fine, there's nothing wrong with that for users willing to pay the license fees.  A reasonably competent R user can produce a comparable table of results with little effort; moreover, the table can be easily customized, which is not so easy to do with "out-of-the-box" tools.

The third point is silly. The authors attribute memory management issues encountered with R to its object-oriented nature. A more obvious explanation is that the authors deployed R on a machine with much less memory than the machine they used for SAS.

Points four and five relate to model quality, and deserve some analysis.

According to the authors, the overall misclassification rate (the inverse of overall accuracy) achieved by the four products is "comparable in most instances." They do not disclose the overall accuracy statistics, which leaves the reader in the dark about what they mean by "comparable" and which instances the overall accuracy was not comparable.

However, the SAS products produced models with higher precision and lift in the first decile. The authors choose to highlight this finding, reporting only the precision and lift statistics in the summary tables, while failing to report overall accuracy statistics.

If the models produced by SAS are more precise but not more accurate, it means that they were produced using techniques that minimize false positives at the expense of false negatives (since the overall error rate is the same). There are situations where this is a good thing — for example, in marketing campaigns where the goal is to optimize campaign ROI, the analytic goal is to target a list as precisely as possible. On the other hand, there are circumstances where the analyst must also be concerned about false negatives — for example, in a medical diagnostic situation where the cost of a false positive is "patient receives unnecessary dose" and the cost of a false negative is "patient dies."

While it is generally best to evaluate models using a range of measures and over the entire range of possible predictions (through a gains chart or ROC analysis), in the absence of information about the cost of errors, accuracy is the best single measure. And, as the authors admit, all of the products produced models of comparable accuracy.

But let's stipulate that increased precision is a good thing where models are equally accurate. How did the SAS tools produce the improved precision? As noted in the Discussion section on page 11, the SAS products accomplished this through use of well-known techniques:

• Oversampling the target event;
• Modification of model priors;
• Missing value imputation;
• Variable selection.

All of these methods are available in R; the authors simply chose not to use them. Which brings us back to a basic flaw in the authors' approach: it's news if SAS' automated tooling can beat a trained and reasonably competent analyst using R; it's not news if SAS' automated tooling can beat a passive and untrained analyst.

The reader should take note that SAS' automated tooling would also outperform a passive and untrained analyst using SAS/STAT. While all of the techniques detailed above can be implemented in SAS/STAT through the SAS Programming Language — a point recognized by the authors — they are not "automatic."

While there is a role for semi-automated model building tools in analytics, evidence from the marketplace suggests that working analysts in the real world prefer the flexibility and control offered by an analytic programming environment over "black-boxy" applications. This is true in the SAS user community as well as the broader community of analysts; users of SAS/STAT and the SAS Programming Language far outnumber users of the high end SAS applications included in this "benchmark."

Derek Norton, Andrie de Vries, Joseph Rickert, Bill Jacobs, Mario Inchiosa, Lee Edlefsen and David Smith all contributed to this post. 

Explaining real-time predictive analytics with big data (video)

In my presentation to the Strata Santa Clara 2013 conference earlier this year, my goal was to give a succinct (under 20 minutes!) explanation of three terms that are two often used as mere buzzwords: predictive analytics, real time, and big data.

 

You can download the slides for my presentation, Real-time Big Data Analytics: From Deployment to Production, from SlideShare.

In the talk I referenced the example UpStream Software's marketing attribution model. Last week I posted some details of how they used R to create and deploy the model for clients like Williams Sonoma, so follow that link if you're interested in the details.

I didn't have much time to get into why I believe that the R language is the ideal environment for creating such models, but I go into more depth in this longer version of the presentation.

Because it's Friday: Looking for Love in all the Right Places

Craigslist, the popular bulletin-board service, has long had a "missed connections" personals section. This is the place were lovestruck singles (or at least, the situationally single) can reach out to the unidentified partner of their dreams, armed only with a description of their fleeting encounter and the hope that the object of their affections is also looking on Craigslist for their missed connection. Here's a recent typical example, from a supermarket in northern California:

Trader Joe's on Cleveland - m4w - 41 (santa rosa)

Well, I feared this would happen. I'd see you at Trader Joe's, try to muster up the courage to say hello & ask for you digits, and then, having failed, wind up here in CL. You're maybe 30, and gorgeous. Blonde hair, wearing dark flip flops, and maybe your shirt was white? I was wearing a baseball hat. (Tell me what color.) We checked out at the same time at adjacent cashiers, at about 1pm, maybe a bit before then. You drive a small car, I think gray. (For some reason, I apparently don't remember colors too well.) Are you out there??

Dorothy Gambrell realised that these "missed connection" posts could give insights into where around the country "love at first sight" strikes the most often. By mining the most frequent locations mentioned in 100 posts in each state back in February, she created the following "love map" of the USA (via Flowing Data):

If you're looking for love this weekend, maybe this map will help you find it. (WalMart is certainly a popular choice in many parts of the country.) Whatever you get up to this weekend, enjoy! We'll be back on Monday.