Revolutions

Global Migration, animated with R

The animation below, by Shanghai University professor Guy Abel, shows migration within and between regions of the world from 1960 to 2015. The data and the methodology behind the chart is described in this paper. The curved bars around the outside represent the peak migrant flows for each region; globally, migration peaked during the 2005-2010 period and the declined in 2010-2015, the latest data available.

This animated chord chart was created entirely using the R language. The chord plot showing the flows between regions was created using the circlize package; the tweenr package created the smooth transitions between time periods, and the magick package created the animated GIF you see above. You can find a tutorial on making this animation, including the complete R code, at the link below.

Guy Abel: Animated Directional Chord Diagrams (via Cal Carrie)

The Financial Times and BBC use R for publication graphics

While graphics guru Edward Tufte recently claimed that "R coders and users just can't do words on graphics and typography" and need additonal tools to make graphics that aren't "clunky", data journalists at major publications beg to differ. The BBC has been creating graphics "purely in R" for some time, with a typography style matching that of the BBC website. Senior BBC Data Journalist Christine Jeavans offers several examples, including this chart of life expectancy differences between men and women:

... and this chart on gender pay gaps at large British banks:

Meanwhile, the chart below was made for the Financial Times using just R and the ggplot2 package, "down to the custom FT font and the white bar in the top left", according to data journalist John Burn-Murdoch.

There are also entire collections devoted to recreating Tufte's own visualizations in R, presumably meeting his typography standards. Tufte later clarified saying "Problem is not code, problem is published practice", which is true of any programming environment, which is why it was strange that he'd call out R in particular.

Make a sculpture in LEGO from a photo, with R

The entrance to our office in Redmond in is adorned with this sculpture of our department logo, rendered in LEGO:

We had fun with LEGO bricks at work this week. APEX is our internal team name, this was fun. Oh and we're hiring for all roles in Azure! pic.twitter.com/VlNNaTexA5

— Jeff Sandquist (@jeffsand) March 30, 2017

Our team, the Cloud Developer Advocates, has a logo as well, created by the multitalented Ashley Macnamara. (The mascot's name is Bit: he's a raccoon because, like developers, he's into everything.) It would be nice to have a LEGO rendition of Bit for the wall as well, but converting an image into LEGO bricks isn't easy ... until now.

This R script by Ryan Timpe provides everything you need render an image in LEGO. It will downscale the image to a size that meets your bricks budget, convert the colors to those available as LEGO bricks, and divide the image up into LEGO-sized pieces, ready to lay out on a flat tray. The script is super easy to use: just source a file of utility functions and then:

(You can also use readJPEG to read in JPG images; I just loaded in the png package and used readPNG which works just as well.) Here's what the output looks like. (Click to see the original, for comparison.)

The script also provides a shopping list of the bricks you need by color and size: this particular project will require 1842 LEGO bricks in 19 different colors to create the 48x48 image. It will even provide a series of step-by-step instructions showing how the project will look in various stages of completion:

The R script is available on GitHub, here, and works with any recent version of R and with up-to-date tidyverse installation. (I used R 3.5.0.) You can find a complete walkthrough of using the scripts in the blog post at the link below.

Ryan Timple: How To: LEGO mosaics from photos using R & the tidyverse

Uber overtakes taxis in New York City

In an update to his analysis of taxi and ride-share trips, Todd Schnieder reports that the number of daily Uber rides exceeds the number of taxi rides in New York City, as of November 2017.

The data is provided by the New York Taxi and Limousine Commission, and the analysis and graphics were created using the open-source R language. (The code and data for the analysis are available on Github.) You can find a wealth of detailed analysis of rides in New York, including some gorgeous maps of pick-up and drop-off locations, at the link below.

Todd W. Schneider: Analyzing 1.1 Billion NYC Taxi and Uber Trips, with a Vengeance

 

Mathematical art in R

Who says there's no art in mathematics? I've long admired the generative art that Thomas Lin Pedersen occasionally posts (and that you can see on Instagram), and though he's a prolific R user I'm not quite sure how he makes his art. Marcus Volz has another beautiful portfolio of generative art, and has also created an R package you can use to create your own designs: the mathart package. 

Generative art uses mathematical equations and standard graphical rendering tools (point and lines, color and transparency) to create designs. The mathart package provides a number of R functions to create some interesting designs from just a few equations. Complex designs emerge from just a few trigonometric functions, like this shell:

Or this abstract harmonograph:

Amazingly, the image above, and an infinite collection of images similar to it, is generated by just two equations implemented in R:

  x = A1*sin(t*f1+p1)*exp(-d1*t) + A2*sin(t*f2+p2)*exp(-d2*t),
  y = A3*sin(t*f3+p3)*exp(-d3*t) + A4*sin(t*f4+p4)*exp(-d4*t)

You can have a lot of fun playing around with the parameters to the harmonograph function to see what other interesting designs you can find. You can find that function, and functions for designs of birds, butterflies, hearts, and more in the mathart package available on Github and linked below.

Github (marcusvolz): mathart

A fresh look for base graphics

While ggplot2 (and its various extensions) is often the go-to package for graphics in R these days, if you need to step outside the boundaries of what ggplot2 can do, you can always step back to base R graphics (and the built-in lattice package) and customize to your hearts content.

The problem is that (unlike for ggplot2) the default look for base graphics is kinda ... meh. That being said, the base graphics system offers almost unlimited flexibility, both via function options and via the par system for modifying layouts and graphic defaults. As Colin Gallespie explains in a recent blog post, you can take a scatterplot that looks, by default, like this:

and with a little extra code, make it look like this:

Find out how in Colin's blog post, linked below.

The Jumping Rivers Blog: Styling Base R Graphics

DataExplorer: Fast Data Exploration With Minimum Code

by Boxuan Cui, Data Scientist at Smarter Travel

Once upon a time, there was a joke:

In Data Science, 80% of time spent prepare data, 20% of time spent complain about need for prepare data.

— Big Data Borat (@BigDataBorat) February 27, 2013

According to a Forbes article, cleaning and organizing data is the most time-consuming and least enjoyable data science task. Of all the resources out there, DataExplorer is one of them, with its sole mission to minimize the 80%, and make it enjoyable. As a result, one fundamental design principle is to be extremely user-friendly. Most of the time, one function call is all you need.

Data manipulation is powered by data.table, so tasks involving big datasets usually complete in a few seconds. In addition, the package is flexible enough with input data classes, so you should be able to throw in any data.frame-like objects. However, certain functions require a data.table class object as input due to the update-by-reference feature, which I will cover in later part of the post.

Now enough said and let's look at some code, shall we?

---

Take the BostonHousing dataset from the mlbench library:

library(mlbench)
data("BostonHousing", package = "mlbench")

Initial Visualization

Without knowing anything about the data, my first 3 tasks are almost always:

library(DataExplorer)
plot_missing(BostonHousing) ## Are there missing values, and what is the missing data profile?
plot_bar(BostonHousing) ## How does the categorical frequency for each discrete variable look like?
plot_histogram(BostonHousing) ## What is the distribution of each continuous variable?

While there are not many interesting insights from plot_missing and plot_bar, below is the output from plot_histogram.

Analysis of Trump's State of the Union Speech, with R

President Trump's State of the Union speech was last night, and it seemed to me it dragged on a bit. That's because it was apparently the slowest SOTU speech in history, based on average number of words spoken per minute. The chart below, based on these data, was created by Josh Katz. (The R code for the chart is here. Note the use of his needs package to load the required R packages.)

Peter Aldhous at Buzzfeed analyzed last night's State of the Union speech as well. Not only was the speech delivered slowly, the script was at a 9-year-old's reading level, following a long-term decline in the complexity of State of the Union speeches.

You can find more analysis of this and past SOTU speeches at the link below. The R code behind the analysis can be found here. (You may also want to check out this analysis of Trump's Twitter activity and the associated R code.)

Buzzfeed: “I Have The Best Words.” How Trump’s First SOTU Compares To All The Others.

 

Visualize your Strava routes with R

Strava is a fitness app that records your activities, including the routes of your walks, rides and runs. The service also provides an API that allows you to extract all of your data for analysis. University of Melbourne research fellow Marcus Volz created an R package to download and visualize Strava data, and created a chart to visualize all of his runs over six years as a small multiple.

Inspired by his work (and the availability of the R package he created), others also visualized bike rides, activity calendars, and aggregated route maps with elevation data. (You can see several examples in the Twitter moment embedded below.) If you'd like to download your own Strava data, all you need a Strava access token, a recent version of R (3.4.3 or later) and the strava package found on Github.

ASA Police Data Challenge student visualization contest winners

The winners of the American Statistical Association Police Data Challenge have been announced. The ASA teamed up with the Police Data Initiative, which provides open data from local law enforcement agencies in the US, to create a competition for high school and college students to analyze crime data from Baltimore, Seattle and Cincinnati. In this post, we take a look at the winners of the visualization category.

The winners of the Best Visualization for college students were Julia Nguyen, Katherine Qian, Youbeen Shim, Catherine Sun from University of Virginia. Their entry included several visualizations of crime data in Baltimore, including the crime density map shown below. The team used R for all of the data cleaning, manipulation, and visualizations. The tidyverse suite of packages was used for data pipelining (including stringr and lubridate for merging data on latitude/longitude and date), and the ggmap package for visualization.

The winners of the Best Visualization for high school students were Alex Lapuente, Ana Kenefick and Sara Kenefick from Charlotte Latin School (Charlotte, N.C.). They used Microsoft Excel to look at overall trends Seattle crime data, the impact of employment and poverty on crime, and this visualization of the frequency of traffic-related incidents (note the "pedestrian violation" segment — I can attest from experience that jaywalking is strictly enforced in Seattle!):

For more on the Police Data Challenge and the winners in the Overall and Best Use of External Data categories, follow the link below.

This is Statistics: Police Data Challenge: Congratulations to our Winners!