Revolutions

Where to find the worst weather in the US

Which US city has the worst weather? To answer that question, data analyst Taras Kaduk counted the number of pleasant days in each city and ranked them accordingly. For this analysis, a "pleasant" day is one where the average temperature was in the range 55°F-75°F, the maximum was in the range 60°-90°, the minimum was in the range 40°-70°, and there was no significant rain or snow. (The weather data was provided by the NOAA Global Surface Summary of the Day dataset and downloaded to R with the rnoaa package.)

With those criteria and focusing just on the metro regions with more than 1 million people, the cities with the fewest pleasant days are:

1. San Juan / Carolina / Caguas, Puerto Rico (hot year-round)
2. Rochester, NY (cold in the winter, rain in the summer)
3. Detroit / Warren / Dearborn, MI (cold in the winter, rain in the summer)

You can see the top (bottom?) 25 cities in this list in the elegant chart below (also by Tara Kaduk), which shows each city as a polar bar chart and with one ring for each of the 6 years of data analyzed. 

And if you're wondering which cities have the best weather, here's the corresponding chart for the 25 cities with the most pleasant days. San Diego / Carlsbad (CA) tops that list.

You can find the R code behind the analysis and charts in this Github repository. (The polar charts above required a surprisingly small amount of code: it's a simple transformation of a regular bar chart with the ggplot2 coord_polar transformation — quite appropriate given the annual cycle of weather data.) And for the full description of the analysis including some other nice graphical representations of the data, check out the blog post linked below.

Taras Kaduk: Where are the places with the best (and the worst) weather in the United States

Create data visualizations like BBC News with the BBC's R Cookbook

If you're looking for a guide to making publication-ready data visualizations in R, check out the BBC Visual and Data Journalism cookbook for R graphics. Announced in a BBC blog post this week, it provides scripts for making line charts, bar charts, and other visualizations like those below used in the BBC's data journalism. 

The cookbook is based around the bbplot package (available on Github), which has been in use at the BBC since March 2018 for creating graphics in the BBC News graphics style. Rather than drafting graphics in R and then relying another graphics tool or a design team for publication, R has been a "game changer" for the BBC. It offers more creativity and control while saving time and effort, especially when the chart needs to be reproduced every time the underlying data change. The BBC offers a six-week course for its data journalists internally, and you can learn as well with the free cookbook published by the BBC, linked below.

Github (BBC): BBC Visual and Data Journalism cookbook for R graphics

 

 

Simulating dinosaur populations, with R

So it turns out that the 1990 Michael Crichton novel Jurassic Park is, indeed, a work of fiction. (Personal note: despite the snark to follow, the book is one of my all-time favorites — I clearly remember devouring it in 24 hours straight while ill in a hostel in France.) If the monsters and melodrama didn't give it away, then this chart seals the deal:

Not the most glaring issue with #JurassicPark, but this plot from Michael Crichton's original 1990 novel looks pretty darn weird. /1 pic.twitter.com/8dQujwIVNf

— David Gerard (@EmpiricalDave) November 29, 2018

That chart seemed like a clever plot point at the time. The smooth bell curve was evidence for our protagonists that the dinosaurs were not, in fact, a controlled population produced in a lab by scientists: they were breeding. But besides the fact that the chart suggested half of a Procompsognathid was somehow roaming the Costa Rican forest, it was too smooth. David Gerard simulated a population of 29 dinosaurs (drawing their heights from a Normal distribution, and plotting it in R), and the resulting chart was invariably much more jagged:

The problem with this is that the density estimate still looks too clean. I made a few density plots in R (using the default settings) from data generated from a Gaussian. /13 pic.twitter.com/hb4uoRkCv4

— David Gerard (@EmpiricalDave) November 29, 2018

Check out the whole Twitter thread: not only is it an interesting plot-check for a great book, it's a great Statistics lesson soon.

Twitter: David Gerard's Jurassic Park thread

When the numbers don't tell the whole story

Anscombe's Quartet is a famous collection of four small data sets — just 11 (x,y) pairs each — that was developed in the 1970s to emphasize the fact that sometimes, numerical summaries of data aren't enough. (For a modern take on this idea, see also the Datasaurus Dozen.) In this case, it takes visualizing the data to realize that the for data sets are qualitatively very different, even though the means, variances, and regression coefficients are all the same. In the video below for Guy in a Cube, Buck Woody uses R to summarize the data (which is conveniently built into R) and visualize it using an R script in Power BI.

The video also makes for a nice introduction to R for those new to statistics, and also demonstrates using R code to generate graphics in Power BI.

XKCD "Curve Fitting", in R

You probably saw this XKCD last week, which brought a grimace of recognition to statisticians everywhere:

It's so realistic, that Barry Rowlingson was able to reproduce all but two of the "charts" above with a simple R function (and a little help from the xkcd ggplot2 theme):

And now for @revodavid et al, with the xkcd package and font! (still two more to do...) pic.twitter.com/3aVHis23Gl

— Barry Rowlingson (@geospacedman) September 21, 2018

You can find the R code behind Barry's reproductions here, and I'm sure he'd welcome contributions for the remaining two charts :). 

 

3-D shadow maps in R: the rayshader package

Data scientists often work with geographic data that needs to be visualized on a map, and sometimes the maps themselves are the data. The data is often located in two-dimensional space (latitude and longitude), but for some applications we have a third dimension as well: elevation. We could represent the elevations using contours, color, or 3-D perspective, but with the new rayshader package for R by Tyler Morgan-Wall, it's easy to visualize such maps as 3-D relief maps complete with shadows, perspective and depth of field:

👌 Dead-simple 3D surface plotting in the next version of rayshader! Apply your hillshade (or any image) to a 3D surface map. Video preview with rayshader's built-in palettes. #rstats

Code:

elmat %>%
sphere_shade() %>%
add_shadow(ray_shade(elmat)) %>%
plot_3d(elmat) pic.twitter.com/FCKQ9OSKpj

— Tyler Morgan-Wall (@tylermorganwall) July 2, 2018

Tyler describes the rayshader package in a gorgeous blog post: his goal was to generate 3-D representations of landscape data that "looked like a paper weight". (Incidentally, you can use this package to produce actual paper weights with 3-D printing.) To this end, he went beyond simply visualizing a 3-D surface in rgl and added a rectangular "base" to the surface as well as shadows cast by the geographic features. He also added support for detecting (or specifying) a water level: useful for representing lakes or oceans (like the map of the Monterey submarine canyon shown below) and for visualizing the effect of changing water levels like this animation of draining Lake Mead.

The rayshader package is implemented as an independent R package; it doesn't require any external 3-D graphics software to work. Not only does that make it easy to install and use, but it also means that the underlying computations are available for specialized data analysis tasks. For example, research analyst David Waldran used a LIDAR scan of downtown Indianapolis to create (with the lidR package) a 3-D map of the buildings, and then used the ray_shade function to simulate the shadows cast by the buildings at various times during a winter's day. Averaging those shadows yields this map of the shadiest winter spots in Indianapolis:

The rayshader package is available for download now from your local CRAN mirror. You can also find an overview, and the latest version of the package at the Github repository linked below.

Github (tylermorganwall): rayshader, R Package for Producing and Visualizing Hillshaded Maps from Elevation Matrices, in 2D and 3D

 

The Chartmaker Directory: Data visualizations in every tool

Working with a new data visualization tool, and wondering how to create a specific type of chart? The Chartmaker Directory (designed by Andy Kirk) indexes more than 35 tools and over 50 charts, and provides links to examples from each combination. Here's an intentionally small detail from the index, where each hollow dot in a column represents a sample chart created a tool (the rightmost column is R, for example), and solid dots also include code. 

The visualization tools include applications like Excel, Power BI and Tableau; languages and libraries including R, Stata, and Python's matplotlib); and frameworks like D3. The data visualizations range from the standard to the esoteric, and follow the taxonomy of the book Data Visualisation (also by Andy Kirk). The chart categories are color coded by row: categorical (including bar charts, dot plots); hierarchical (donut charts, treemaps); relational (scatterplots, sankey diagrams); temporal (line charts, stream graphs) and spatial (choropleths, cartograms). 

Follow the link below to explore the indexed tools and visualization. If you know of an example or solution that isn't listed, you can contribute your own links by clicking the plus sign in the top left of the index.

Visualizing Data: The Chartmaker Directory

Using gganimate to illustrate the luminance illusion

Many illusions are based on the fact that our perceptions of color or brightness of an object are highly dependent on the background surrounding the object. For example, in this image (an example of the Cornsweet illusion) the upper and lower blocks are exactly the same color, according to the pixels displayed on your screen.

Mind = blown. These two blocks are exactly the same shade of grey. Hold your finger over the seam and check. pic.twitter.com/OqAnforGqs

— David Smith (@revodavid) December 5, 2013

Here's another simpler representation of the principle, created by Colin Fay (in response to this video made with colored paper). In the animation below, the rectangle moving to the left to right remains the same color throughout (a middling gray). But as the background around it changes, our perception of the color changes as well.

Colin created this animation in R using the gganimate package (available on GitHub from author Thomas Lin Pederson), and the process is delightfully simple. It begins with a chart of 10 "points", each being the same grey square equally spaced across the shaded background. Then, a simple command animates the transitions from one point to the next, and interpolates between them smoothly:

library(gganimate)
gg_animated <- gg + 
  transition_time(t) + 
  ease_aes('linear')

You can find the complete R source code behind the animation at the blog post linked below, along with an interesting discussion of luminance and how you should consider it when choosing color scales for your data visualizations.

RTask: Remaking ‘Luminance-gradient-dependent lightness illusion’ with R

aRt with code

Looking for something original to decorate your wall? Art With Code, created by Harvard University bioinformatician Jean Fan, provides a collection of R scripts to generate artistic images in the style of famous artworks, for example this randomly-generated piece in the style of Mondrian:

Other art generators include "Tunnel" (rotated and scaled designs in the style of Päivi Julin), Pointillism (sylizes photographs as a dots), and Pixel (an interesting multi-scale filter for photos). The transformations using only the built-in R grid and lattice packages, which made it easy for me to try Pixel on a photo of my own dog, Max:

You can find examples of all of these styles and the R scripts used to generate them at the link below.

Jean Fan: Art With Code  (via the author)

A hex sticker wall, created with R

Bill Venables, member of the R Foundation, co-author of the Introduction to R manual, R package developer, and one of the smartest and nicest (a rare combo!) people you will ever meet, received some gifts from the user!2018 conference committee in recognition of his longtime service to the R community. He received a book of reminiscences from those whose lives he has touched, and this blanket:

This morning we took care of some unfinished #user2018 business. 😊 @visnut @tslumley @_jessie_roberts pic.twitter.com/gW0wX2PLF2

— Miles McBain (@MilesMcBain) July 19, 2018

The design on the blanket is the "Hex Sticker Wall" that was on design during the conference, created by Mitchell O'Hara-Wild:

The #useR2018 #hexwall has been revealed! Read about how it was created in #rstats on this blog post: https://t.co/krYYOQ3N84

A huge thanks to everyone who has submitted stickers and provided feedback. I hope you enjoy the end result as much as I have had creating it! 🎉 pic.twitter.com/GnG9m2cZme

— Mitchell O'Hara-Wild (@mitchoharawild) July 11, 2018

The design for the wall was created — naturally — using R. Mitchell created an R script that will arrange a folder of hexagon images according to a specified layout. He then used a map of Australia to lay out the hex sticker coordinates within the map boundaries and plotted that to create the Hexwall. If you have a similar collection of hex images you could use the same technique to arrange them into any shape you like. The details are provided in the blog post linked below.

Mitchell O'Hara-Wild: UseR! 2018 Feature Wall