Revolutions

Revolution Analytics named Startup to Watch in 2012

While we were out over the holiday break, Silicon Angle selected five open-source startups that are commercializing and contributing to open source projects as "Startups to Watch in 2012". Revolution Analytics was #2 on the list:

This was a busy year for Revolution. The company released Revolution R 5.0, which included Hadoop integration, allowing users to create map/reduce jobs using R. Also in 2011, Revolution added the ability to import data files from both SAS and SPSS, announced partnerships with IBM Netezza and Cloudera, and added a new predictive analytics algorithm.

Revolution has already landed a few big name customers, including Bank of America and Pfizer. Also, the open source nature R and Revolution’s liberal licensing enable data-as-a-service applications that the licensing agreements of some of Revolution’s competitors prohibit. This means companies can build new types of services based on their own data sets and Revolution’s algorithms.

You can read the complete profile, along with those of the other four nominees (Puppet Labs, Joyent, Opscode and Talend), at the link below.

Services Angle: 5 Open Source Startups to Watch in 2012 # 2: Revolution Analytics

Forbes: Big Data needs to earn its keep

Dan Woods at Forbes makes a salient point about Big Data: it's not good enough merely to spend a ton of money to collect and store data -- it needs to earn its keep via data analysis:

Let’s say you buy a huge Hadoop cluster along with Revolution Analytics, a productization of the R language that has been adapted to make it work with Hadoop and more easy to use in other ways. Cloudera and Revolution Analytics stand ready to sell you their products.

But who do you turn to to increase your analytic capacity?

Dan suggests -- and I agree -- that there also needs to be investment in what he calls "Agile Big Data technologies" to take advantage of the intelligence locked up in the data. Read more in Dan's article linked below.

Forbes: Curing the Big Data Storage Fetish

"Anyone planning to work with Big Data ought to learn Hadoop and R"

Dan Woods at Forbes interviewed LinkedIn's Daniel Tunkelang about the rise of data science and on building data science teams. When asked how students today should prepare themselves to be data scientists, Tunkelang gives some good advice:

When we built the data science team at LinkedIn a few years ago, we looked for raw talent, assuming that smart people could pick up the needed technical skills on the job. Now that the field has matured, it’s a good idea to learn some of those technical skills in school. Anyone planning to work with big data ought to learn Hadoop and R, the two open-source tools most used by data scientists. It’s also a good idea to take courses in statistics in machine learning. Beyond that, find every opportunity to work with real data sets. Struggling with the warts of real data is a key part of a data scientist’s job — in fact, some would say that the struggle is our “day job.”

(Emphasis mine.) Any student thinking about working with Hadoop and R should check out the RHadoop project, a collection of R packages that make it easy to write map-reduce jobs for Hadoop data stores in the R langauge.

Forbes: LinkedIn's Daniel Tunkelang On "What Is a Data Scientist?"

NYT on Big Data and R

In the New York Times' "Bits" blog today, Quentin Hardy offers recollections on Big Data talks at the recent Web 2.0 Summit. He begins with a definition of Big Data:

Big Data is really about ... the benefits we will gain by cleverly sifting through it to find and exploit new patterns and relationships. You see it now in things like Facebook ads, which are put in front of you because the posts you have read and contributed to (which Facebook’s algorithms get to examine as the price of this “free” service) indicate you might be ready to buy the advertised good.

The article includes applications of big data analytics at various companies: ad placement at Google; credit card transaction analysis (according the CEO of TrialPay, the value of the transaction data exceeds the transaction fee credit companies like Visa charge merchants); and inferring information from the semantic web at search start-up Domo. (By the way, here's a great presentation on using R to mine the semantic web, from Chris Davis and Alfredas Chmieliauskas of the Amsterdam R Users Group.)

Speaking of R, the NYT article ended with a mention of new tools for Big Data analyyics: MapReduce, NoSQL and R:

There are an uncountable number of data-mining start-ups in the field: MapReduce and NoSQL for managing the stuff; and the open-source R statistical programming language, for making predictions about what is likely to happen next, based on what has happened before. 

For example, R is used at social networking sites like Foursquare and OMGPOP to make predictions based on user transaction data.

Bits Blog: The Big Business of ‘Big Data’

Oracle's Big Data Appliance to include R

At the Oracle OpenWorld conference in San Francisco today, Oracle announced the new Oracle Big Data Appliance, "a new engineered system that includes an open source distribution of Apache™ Hadoop™, Oracle NoSQL Database, Oracle Data Integrator Application Adapter for Hadoop, Oracle Loader for Hadoop, and an open source distribution of R." Oracle's foray into the Hadoop and NoSQL spaces has captured most of the attention, but it's the inclusion of open source R that I find most interesting. Oracle is clearly finding demand from its customers for advanced analytics capabilities for big data, and is looking to R to fill that gap. I agree with Ed Dumbill's assessment:

Big data isn't much use until you can make sense of it, and the inclusion of R in Oracle's big data appliance bears this out. It also sets up R as a new industry standard for analytics: something that will raise serious concern among vendors of established statistical and analytical solutions SAS and SPSS.

Of course, this isn't the first time that R has been embedded into a data warehousing appliance. IBM Netezza's iClass device integrates with Revolution R, and AsterData, the Teradata Data Warehouse Appliance, and Greenplum all provide connections to R as well. Here at Revolution Analytics, we think that such enterprise-level integrations with R serve to grow the R ecosystem and serve as validation of R as a key platform for advanced analytics. As CEO Norman Nie said to GigaOm this weekend, 

"Oracle’s announcement to embed R demonstrates validation for the leading statistics language and offers further evidence that R is a key weapon in advanced analytics today", said Dr. Nie. "As the Enterprise R company, Revolution Analytics has seen an enormous demand for R solutions, so it’s no surprise that Oracle and other companies are looking to further evangelize and distribute R among the enterprise."

In today's announcement, Oracle also announced another new component, "Oracle R Enterprise". Details are sparse on what exactly this is, so far, other than it allows you to "run existing R applications and use the R client directly against data stored in Oracle Database", presumably thanks to an R package created by Oracle. Timothy Prickett Morgan at The Register has a couple of extra details:

The Big Data Appliance also includes the R programming language, a popular open source statistical-analysis tool. This R engine will integrate with 11g R2, so presumably if you want to do statistical analysis on unstructured data stored in and chewed by Hadoop, you will have to move it to Oracle after the chewing has subsided.

This approach to R-Hadoop integration is different from that announced last week between Revolution Analytics, the so-called Red Hat for stats that is extending and commercializing the R language and its engine, and Cloudera, which sells a commercial Hadoop setup called CDH3 and which was one of the early companies to offer support for Hadoop.

We look forward to hearing more about support for R from Oracle and other enterprise vendors. As the R ecosystem continues to grow into the enterprise space, this can only mean more investment in the R project as a whole.

Oracle Press Releases: Oracle Unveils the Oracle Big Data Appliance

Unlocking Big Data with R

I have an article out this week on ReadWriteHack: Unlocking Big Data with R. My thanks to the folks at ReadWriteWeb for giving us the opportunity to showcase some of the many real-world Big Data applications of R. Here are some additional links about the applications mentioned in the article:

New York Times: Destruction of the Haiti earthquake;  2010 U.S. election; Netflix rental habits; Mariano Rivera’s pitch chart

Orbitz: How Orbitz uses Hadoop and R to optimize hotel search

OkCupid: How OkCupid Demystifies Dating With Big Data

ReadWriteHack: Unlocking Big Data with R

FastCompany on telling stories with data

An article in FastCompany today profiles two websites that excel in telling engaging stories with data: the FlowingData and OkTrends blogs. Both sites often use R for data analysis and data visualization, and have been featured here on this blog. FlowingData regularly posts guides on how to use R to recreate many of its featured visualizations with bubble charts, time series, heat maps, mosaic plots and more. And the folks at OkCupid behind the charts in OkTrends were featured in a GigaOm article on how they use R to tell stories about dating data.

FastCompany: Visualizing Data And Telling Compelling Stories With OkCupid And FlowingData

Silicon Angle profiles Revolution Analytics

Jeff Kelly, Principal Research Contributor at the open-source analyst site Wikibon.org, provides an in-depth profile of Revolution Analytics at Silicon Angle. The article focuses on the Big Data aspects of Revolution R Enterprise, the new integration with IBM Netezza, and our forthcoming integration with Hadoop. The article concludes thus:

On the whole, I like Revolution’s approach. Data volumes are growing like never before, and the popularity/need for Big Data approaches like Hadoop and MPP data warehousing are likewise on the rise. If Revolution can use its value-add technology and support services become the de facto Big Data advanced analytics platform/tool, it has a real chance to give SAS and SPSS a run for their money.

Check out the complete profile at the link below.

Silicon Angle: You Say You Want a (Big Data) Revolution?

 

InformationWeek on low-cost statistical software

Doug Henschen of InformationWeek published an article looking at low-cost alternatives to legacy statistical software, including SAS. The second page of the article looks at the growth of R, and the offerings of Revolution Analytics:

Four-year-old Revolution Analytics provides commercial support for the R programming language as well as tools, integration, consulting and training aimed at making an open-source product enterprise ready.

Riding R's coattails is a good idea. It's used by 43% of data miners, according to the Rexer's Annual Data Mining Survey.

InformationWeek: Low-Cost Options For Predictive Analytics Challenge SAS, IBM

IBM Netezza: Embrace open source analytics

Earlier this month Thomas Dinsmore, solutions architect for IBM Netezza’s Advanced Analytics team, had a great blog post on why companies should embrace R as an analytics platform. He says:

There are three main reasons R should be part of your enterprise analytics architecture:

• R has capabilities not available in commercial analytics software
• Usage of R by analysts is growing rapidly
• R’s total cost of ownership is attractive

He also mentioned a great example of why companies are turning to R software for access to the statistical techniques they need:

A customer we’re working with in the life sciences industry wants to apply four new methods to their analytic toolkit.  This customer spends almost a billion dollars each year to run hundreds of thousands of experiments; very small improvements in precision directly impact this customer’s bottom line.  Right now, all of these new methods are available in R, and none are available in commercial packages.

The post also mentions the partnership between IBM Netezza and Revolution Analytics, which will soon bring in-database analytics based on R to the Netezza appliance.

Netezza Blogs: Embrace Open Source Analytics