Revolutions

Visualizing networks in R: arc diagrams and hive plots

Arc diagrams are an alternate way of representing two-dimensional graphs. Rather than scattering the nodes across the page connected by straight edges, you can instead arrange the nodes along a one-dimensional axis, and replace the straight edges with arcs between the nodes. While an arc diagram might not give as good a sense of the connections between the nodes as a traditional graph layout, judicious ordering of the nodes can help identify clusters. It's also a useful format (especially in the vertical orientation) when you want to label each of the notes with other quantities in a table-like format.

Thanks to Gaston Sanchez, you can now create arc diagrams in R with the arcdiagram package. Gaston created the arc diagram below to visualize the characters in the Victor Hugo classic (and now a major motion picture) Les Miserables. Each character is connected by an arc if they appear together in the same chapter; the wider the arc, the more the characters appeared in chapters together. The ordering (and colour) of the nodes identify groups of characters that appear in the novel together.

You can find the complete code to create the arc diagram above, along with details on how to install the arcdiagream package, at Gaston's blog, Data Analysis Visually Enforced.

An extension to arc diagrams is the hive plot, where instead of the nodes being laid out along a single one-dimensional axis they are laid out along multiple axes. This can help reveal more complex clusters (if the nodes represent connected people, imagine for example laying out nodes along axes of both "income" and "enthicity"), and is a particularly useful way of visualizing graphs with many nodes and edges that look like a dense "hairball" using traditional graph layouts. Here's an example of a hive plot:

The above plot comes from the Hive Plots homepage, and shows the connections between similar genes (nodes) in three related genomes (SL, BA and SN). You can create hive plots in R using the hiveR package.

Visualizing MLB Hall of Fame votes with R

Carlos Scheidegger and Kenny Shirley created this visualization of votes for the Major League Baseball hall of fame: 

They describe the chart as follows:

The main figure above is a plot of BBWAA Hall of Fame voting by year for all 1,070 players who have appeared on the ballot since Hall of Fame voting began in 1936. The circular points represent each player's vote percentage in his final year on the BBWAA ballot, and the lines represent his vote percentage in prior years if he appeared on the ballot multiple times.

R was used to develop the original version of the visualization above, but if you click it you can see the D3-based interactive version which allows you highlight individual platers, filter by position and see various stats of the players. In addition to the interactive chart, you can find more information about the data behind this visualization and how it was created at the link below.

cscheid.net: MLB Hall of Fame Voting Trajectories

Chances of making an NFL field goal

If the scores are tied in this weekend's Super Bowl and the game rests on the outcome of a field goal attempt, the chart below will tell you the chances of the goal being made. All you need to know is the distance of the kick:

What this says that if the kick is longer than 50 yards, there's pretty much no chance. The chart was created by Decision Science News (using the R statistical programming language) by tabulating field goal successes for all 10,750 attempts by all NFL teams since 2002. It would be interesting to the stats for just the Ravens and the 49ers; perhaps an R user would use the source data to make the comparison?

Decision Science News: Football geeks, your 10,705 field goals are ready (via Dan Goldstein).

Life expectancy and retirement age in USA

Data Scientist John Myles White used R to compare life expectancy and retirement age in the USA. While male (red) and female (green) life expectancies are rising, average retirement age is going down. What this means for the future of benefit programs like Social Security is the topic of an interesting series of comments on the post.

 

How to make your own 3-D sculpture with R

We've mentioned before how you can use R to design 3-D objects. Now, thanks to the latest version of the rgl package, you can produce real-world 3-D objects with R as well.

The rgl package has long made it possible to create virtual 3-D objects in R, and export them as animations like this:

 

But now, package author Duncan Murdoch has added the ability to export such 3-D objects in formats used by 3-D printers (such as STL, WebGL, PLY, and OBJ). Duncan used this feature to create the corresponding real-world sculpture below:

Duncan explained how he obtained the 3-D print via email:

I ordered it from shapeways.com, a service bureau that does 3D printing in a variety of materials. (They're in NYC, and were shut down for a few days by Hurricane Sandy, but still managed to get my print to me a day earlier than they promised to ship it!)  Including shipping to Canada, the cost was $22, and I had the print 11 days after I wrote the software.

The 3-D object file (in STL format) was created using just a few R commands and some elegant mathematics:

## Install rgl from R-forge with
## install.packages("rgl", repos="http://R-Forge.R-project.org")
 
library(rgl)
 
theta <- seq(0, 6*pi, len=2048)
sawtooth <- function(x) sin(x) + sin(2*x)/2
dsawtooth <- function(x) cos(x) + cos(2*x)
 
fb <- 21+1/3
x1 <- sawtooth(fb*theta)
dx1 <- dsawtooth(fb*theta)
 
offset <- 3*pi/3
y1 <- sawtooth(fb*theta - offset )
dy1 <- dsawtooth(fb*theta - offset)
 
r1 <- 0.1
r2 <- 0.4
 
k <- cylinder3d(cbind(sin(theta)+2*sin(2*theta), 2*sin(3*theta), cos(theta)-2*cos(2*theta)),
                      e1=cbind(cos(theta)+4*cos(2*theta), 6*cos(3*theta), sin(theta)+4*sin(2*theta)),
                      radius=0.8, closed=TRUE, keepVars=TRUE)
 
knot <- attr(k, "vars")
center <- cbind(0, x1, y1)*r2
e1 <- cbind(1, dx1, dy1)*r2
e2 <- cbind(rep(0, length(theta)), 1, 0)
for (i in 1:length(theta)) {
   trans <- cbind(knot$e1[i,], knot$e2[i,], knot$e3[i,])
   center[i,] <- knot$center[i,] + trans %*% center[i,]
   e1[i,] <- trans %*% e1[i,]
   e2[i,] <- trans %*% e2[i,]
}
 
braid <- addNormals(cylinder3d(center, e1=e1, e2=e2, radius=r1))
shade3d(braid, col="white")
writeSTL("braid.stl") 

Created by Pretty R at inside-R.org

Try it yourself! The package installed easily for me on a Mac running R 2.15.2, and I could also rotate the 3-D object interatively on-screen using the shade3d function. For more info, visit the rgl progect page linked below.

r-forge: rgl project page

Create optical illusions with R

I love optical illusions (like this and this and these), not just because they're fun, but also because they give us insights into how the brain processes sensory information. Apparently so does Kohske Takahashi, who conducts research on cognitive psychology and cognitive neuroscience at the University of Tokyo's Research Center for Advanced Science and Technology. He's re-created some classic optical illusions using R, as part of an upcoming workshop on illusions. Here are some of his examples:

The horizontal lines in this grid are all parallel (which you can verify by looking at the R code that generated it):

Though the stripes appear offset diaglonally, these symbols are arranged in perfect horizontal/vertical tiles, and appear to move as you scroll the page:

The small squares here are all exactly the same shade of grey:

That last one, by the way, is a variant of my favorite illusion of all time. In the diagram below, the squares labelled A and B are also the exact same shade of grey: 

Each of Kohske's illusions requires only a few lines of R code, which goes to show that even simple constructs can play havok with how our brain makes sense of visual signals. You can find the R code for these illusions and more (featuring some very elegant uses of the "grid" graphics package), at the document Kohske Takahashi published to RPubs, linked below.

RPubs: Illusions in R / Rで錯視 (via Greg Linch) 

Political revolutions on Twitter, visualized with R

Twitter has become a powerful medium for organizing and communicating with factions during popular uprisings: the crisis in Egypt, the uprising in Syria, the revolution in Iran, and other conflicts all around the world. Twitter's effectiveness relies on its ability for the various factions to self-organize and to fight the information battle in social media.

Esteban Moro Egido, a mathematics professor at Universidad Carlos III in Madrid, puts this battle into stark relief with a video depicting Twitter activity around Spain's general strike in March this year. Esteban has used the R language for years to understand complex networks with applications in areas such as telecommunications and social media, and has put those skills to great use analyzing all of the tweets, retweets and mentions related to the strike. Here's the video:

 

Each point in the animation represents a twitter user, each colour-coded according to their faction in the debate (pro-strike, anti-strike, or somewhere in between). He used community-finiding algorithms to automatically assign Twitter users to factions, and Esteban described the factions in an email:

The communities where identified using one of the community-finding algorithms in R. Specifically the walk trap algorithm run over the static graph of (weighted) RT graph between twitter accounts. So each Twitter account is assigned to a particular community right from the beginning. 

What we did afterwards was to check what these communities were talking about (tweets and RTs). We found that the tweets and RTs in the orange community were in favor of the general strike and the reasons behind it, while the tweets and RTs in the dark blue community were against the unions and the reasons behind the general strike. It is interesting to see that the communities found in the structural analysis of the RT graph also correspond to opinion communities, the reason being that there are not many RTs between groups which have such different opinions.  

There are other two communities in the video (light blue and green) which correspond to news media and, a bot network automatically tweeting about the general strike.

The animation itself was created entirely with R using the igraph package, and encoded to video using ffmpeg. You can create similar videos yourself for other dynamic political discussions on Twitter: Esteban has kindly provided a tutorial on how to create such animations, with R code. 

Implicit None: Temporal network of information diffusion in Twitter

 

Tutorial: How to make NYT-style bar charts with R

New York Times columnist Charles Blow needed a chart to accompany his op-ed piece Lincoln, Liberty and Two Americas (about one-party control in state legislatures). So he turned to resident graphic editor Kevin Quealy, who found the source data and used R to create the chart below:  

If you'd like to create similar charts yourself, Kevin provides a useful tutorial on his behind-the-scenes blog, chartsnthings. It's just a few lines of base graphics in R, beginning with line plots and then moving on to bar plots. The abline function is used to create the gridlines. You can also use the same technique to create "waffle charts" (the display technique often used by the NYT, where small blocks represent data elements). You can find the full tutorial at the link below.

chartsnthings: R tutorial: Simple charts

Another look at ideology of the US congress

In response to last week's post on the rapidly increasing ideology of the US Republican Party, Mike Lawrence suggested another way of looking at the DW-NOMINATE ideology data. Rather than simply looking at boxplots of the congress scores by party over time, we could fit a smooth curve to get a better sense of the trends over time. Mike provided R code to do exactly that, using a generalized additive model to fit the smooth curve to the data since the two-party system began, and using his ez package to visualize the result. I modified Mike's code slightly to match the traditional party colours and to put year (rather than congress number) on the horizontal axis:

In this chart, higher means more rightward-leaning. While the Democratic party has drifted leftwards over the past 50 years, you can see a sudden turn to the right beginning in 2005. And in this version of the chart you can clearly see the march of the Republican party to the far right, beginning in 1975.

Watch Obama and Romney criss-cross the US

The Washington Post has an interactive graphic showing the rate at which the US presidential candidates Barack Obama and Mitt Romney have visited the various states for campaign rallies and fundraisers. Here's how it looks today:

You can clearly see the focus on key swing states like Florida and Ohio, as well as non-competitive (but donor-rich) states like California and Texas. But this aggregate view doesn't give a sense of the sheer logistics of the candidates (plus their entourages and press pool) travelling across the country from event to event. 

So statistician Jerzy Wieczorek used data on the invididual campaign stops and dates to create this animated visualization of Obama's and Romney's itineraries over the past few months:

Now that's a hectic travel schedule! And that's not even all the travel: this animation shows only the campaign stops, not other trips unrelated to the campaign (such as returning to Washington DC for the President, or Romney returning to his Boston HQ). It's a bit of a shame they're not collecting frequent flyer miles in the process.

Jerzy used the R language to create the animation. This involved extracting the campaign stops in JSON format, geocoding the campaign stop locations into latitudes and longitudes (for mapping), and breaking down each day into 24 one-hour blocks so the animation runs smoothly in accelerated time. He then used the code behind this historic shipping animation to create the video above. R programmers can find all the technical details at Jerzy's blog post, linked below.

Civil Statistician: Animated map of 2012 US election campaigning, with R and ffmpeg