Revolutions

Merry Christmas from Revolution Analytics

Today's the Christmas holiday in the US and many other places around the world. Wherever you may be, have a happy and safe holiday season. Best wishes go to our customers, partners and the entire R community from the team at Revolution Analytics. 

Some of the Revolution Analytics team (and significant others) photographed at our holiday party at the Chihuly Garden and Glass Museum in Seattle.

Here at the blog, we'll be taking a short break for the holiday season, with posts resuming on January 2. In the meantime, have a very Merry Christmas and a very Happy New Year!

24 Christmas Gifts from is.R

The is.R blog has been on a roll in December with their Advent CalendaR feature: daily tips about R to unwrap each day leading up to Christmas. If you haven't been following it, start with today's post and scroll down. Sadly there isn't a tag to collect all these great posts together, but here are a few highlights:

• December 4: Reading and writing Stata data files with R
• December 6: Calculating the outline surrounding a collection of points
• December 8: Ranking popularity of terms on Twitter
• December 14: A network of countries, grouped by flag similarity (chart below)
• December 16: Text analysis with random-effects models
• December 21: Beautiful network diagrams (like the one below)

You should check out all of the posts. All include the R code so that you can re-creating the charts, tables and analysis for yourself. What a lovely Christmas gift! The first post in the series is linked below.

is.R: Using XML to grab tables from the web

Because it's Friday: 360-degree landmark photography

You might recall back in June I posted a music video with some stunning shots of San Francisco, including this circumnavigation of Sutro Tower:

I wondered at the time how you could make such a video, and the creators have now revealed their secret. It turns out the process is pretty simple:

• On a map, draw a circle centered on the target landmark
• Using Google Street View, find locations along the circle where the landmark is visible. You'll need around 20 locations, fairly equally spaced around the circle
• At each location, take a couple of photos spaced a few feet apart. Each photo should align the landmark with a reference mark in the viewfinder. Using two photos allows the foreground to move in parallax, giving a sense of motion.
• Stitch the photos together using PhotoShop. Align the landmark position so that it rotates in place, and add movement blur to the foreground to give a sense of motion.

The whole process is explained in the video below, and demonstrated with Toronto's CN Tower.

 

PetaPixel: How to Shoot Photos for a ‘Space-Lapse’ Animation Around a Tall Building

Create optical illusions with R

I love optical illusions (like this and this and these), not just because they're fun, but also because they give us insights into how the brain processes sensory information. Apparently so does Kohske Takahashi, who conducts research on cognitive psychology and cognitive neuroscience at the University of Tokyo's Research Center for Advanced Science and Technology. He's re-created some classic optical illusions using R, as part of an upcoming workshop on illusions. Here are some of his examples:

The horizontal lines in this grid are all parallel (which you can verify by looking at the R code that generated it):

Though the stripes appear offset diaglonally, these symbols are arranged in perfect horizontal/vertical tiles, and appear to move as you scroll the page:

The small squares here are all exactly the same shade of grey:

That last one, by the way, is a variant of my favorite illusion of all time. In the diagram below, the squares labelled A and B are also the exact same shade of grey: 

Each of Kohske's illusions requires only a few lines of R code, which goes to show that even simple constructs can play havok with how our brain makes sense of visual signals. You can find the R code for these illusions and more (featuring some very elegant uses of the "grid" graphics package), at the document Kohske Takahashi published to RPubs, linked below.

RPubs: Illusions in R / Rで錯視 (via Greg Linch) 

R Journal Volume 4, Issue 2

The latest issue of the bi-annual, peer-reviewed journal about R, the R Journal, is now available for download.

This issue includes three articles on graphics from R-core member and R Graphics author Paul Murrell. He writes about accessing individual elements of an R chart by the component names, drawing complex symbols with the polypath function (useful for map icons, for example), and for developers of chart functions in R, debugging grid graphics. 

There are also several articles on statistical analysis in R, with papers on frailty models, influence in mixed effects models, nonparametric regression splines, and robust linear model estimation. There's also a tutorial on creating connected graphs for use in Markov models.

This final issue of the year also brings changes to the editorial board of the Journal. Thanks to to Heather Turner, Bill Venables and Editor-in-Chief Martyn Plummer who are relinquishing their editorial roles to Bettina Grün and new editor Hadley Wickham.

The R Journal: Volume 4/2, December 2012

Coursera's Data Analysis with R course starts Jan 22

Following on from Coursera's popular course introducing the R language, a new course on data analysis with R starts on January 22. The simply-titled Data Analysis course will provide practically-oriented instruction on how to plan, carry out, and communicate analyses of real data sets with R. Here's the course trailer from instructor Jeff Leek:

 

Jeff Leek is the co-author of the excellent Simply Statistics blog. His co-author Roger Peng is, in turn, the presenter of Coursera's Computing for Data Analysis course, which if you missed it last time, will be starting again on January 2 — a perfect introduction to the R language ahead of the Data Analysis course. Roger Peng and Jeff Leek shared their motivations for providing these courses in the MOOC (massive open on-line course) format in a blog post back in August.

As with all Coursera courses you can attend both online, free of charge. You can sign up for the Data Analysis course at the link below.

Coursera: Data Analysis

The most influential data scientists on Twitter

Twitter is emerging as an important medium for determining influence in many fields. Social ranking sites like Klout and Traackr include Twitter as a heavily-weighted component of their ranking algorithms, for example. Twitter isn't representative of the members of any field, but in areas where the members primarily engage online, it can be a useful proxy. SocialFlow's Gilad Lotan has used an analysis of social networks on Twitter to rank influences in such fields, including the communities of python users and data scientists. Here's the network chart for data scientists:

Each circle represents a data scientists on Twitter (anyone with Data Science, Data Scientist, Machine Learning, Data Strategy or the like in their bio). The circle size is proportional to their influence during a week this past October. If you're active in the R or Data Science communities on Twitter, you'll recognize many of the names: Hilary Mason (@hmason) from bitly, Pete Skomoroch (@peteskomoroch) from LinkedIn; DJ Patil (@dpatil), author of Building Data Science Teams, and #rstats regulars Ryan Rosario (@datajunkie), John Myles White (@johnmyleswhite) and Mark Alen (@siah). (I'm down there in the bottom right corner: @revodavid.)

The colors represent connected clusters within the social graph, detected autonomously. With Hilary Mason's help, Gilad assigns meaning to these clusters:

Purple seems to be a mix of east coast and academics, while the dark blue is the west coast data drinking crew. Yellow looks like west coast social network folks while green have been doing it for a while. Although @BigDataBorat is identified within that segment… hmmm… The orange cluster is harder to nail down. Perhaps more academic, applied math and less tech-scene?

The analysis of the social network code was performed with Python, and the visualization was created with Gephi. You can find the details of how it was done, including slides and Python code, at the link below.

Gilad Lotan:  Mapping Twitter’s Python and Data Science Communities (via Quora)

How to select and merge R data frames with SQL

The R language provides many features in the language for selecting data from data frames: the "[" operator, logical functions, and utility functions like "subset". But if you know SQL (the query language ubiquitous in database systems), none of this is necessary. With the sqldf package, you can just pretend that your data frame is a database, and use SQL directly. 

The sqldf function supports the full richness of the SQL language, but applied to data frames in R's memory. This includes:

• SELECT ... WHERE statements to select rows and columns according to logical criteria
• CASE clauses, for queries with special cases
• ORDER BY statements, to sort the resulting data according to specified columns
• LEFT JOIN and INNER JOIN statements for merging data frames

The sqldf package uses its own internal database engine, so there's no special database configuration you need to do. Just enter the following in R:

install.packages("sqldf")
library(sqldf) 

and you should be good to go. The SQLDF FAQ is a good resource for getting started, and this sqldf video tutorial from Keystone Solutions shows the sqldf package in action with examples of SQL queries from simple to complex.

Google code: sqldf: SQL select on R data frames

Because it's Friday: What is fire made of?

Ever wondered what fire is made of? It can be considered to be a plasma: neither solid, liquid nor gas, but a rather a collection of energetic, ionized particles. Stripped of their electrons, these ionized particles give fire some interesting electrical properties:

 

Enjoy your weekend, and if you're spending it sitting by an open fire, you may be looking at it in a new way.

Revolution Newsletter: December 2012

The most recent edition of the Revolution Newsletter is out. The news section is below, and you can read the full December edition (with highlights from this blog and community events) online. You can subscribe to the Revolution Newsletter to get it monthly via email.

Tell us what you're looking for in R training. 2013 is the International Year of Statistics, and the Revolution Analytics training department is seeking inputs on course topics in for 2013. What data science methods, R programming, Big Data, and Business Application training topics are you looking for? Let us know by taking our five-minute survey and enter to win a $100 Amazon Gift Card!.

Revolution R Enterprise 6.1 is now available. New features include big-data tree models, a Hadoop HDFS connection, and new security controls. Butler Analytics calls Revolution R Enterprise "the most powerful analytical tool on the planet with support and training" in this five-star review.

Validus Re uses Revolution R Enterprise for risk management. "What takes a day to do in Excel can be done in 15 minutes by writing a few lines of R code in Revolution R Enterprise," says pricing actuary Jamie Botelho in this new case study.

Calling all Teradata users! Revolution Analytics is developing a high-speed connection for Revolution R Enterprise to Teradata appliances and we're looking for beta testers.  If you are running R in a Teradata environment, then we'd love to get you on board. The beta program begins in early 2013. Please email [email protected] if you are interested.

Real-Time Predictive Analytics with Big Data. Learn how Revolution R Enterprise fits into the predictive analytics stack for production applications — and the meaning of "big data" and "real time" — in the replay of our latest webinar from David Smith.

John Deere uses R to accelerate manufacturing. See how the farming equipment manufacturer optimizes the production line and forecasts crop yields in this webinar replay from forecast analytics manager Derek Hoffman.

Revolution R Enterprise in the news. Read about unraveling the mystery of multiple sclerosis with big data in InfoWorld.

Revolution Analytics: Newsletter Archive