The FBI's aerial surveillance program, visualized with R

Buzzfeed's Peter Aldhous and Charles Seife broke a major news story last week: the US Federal Bureau of Investigation and Department of Homeland Security operate more than 200 small aircraft (mainly Cessnas and some helicopters) which routinely circle various sites near US cities, presumably to gather data with onboard cameras and electonic equipment. The data behind the story weren't provided from the government; rather, the data was culled from public flight-tracking service FlightRadar24 and cross-referenced against FAA registrations for planes with transponders registered to those departments. The story includes an interactive map where you can navigate to a location of your choice and see all of the surveillance flights during the survey period of August 17 to December 31, 2015. (DHS flights are shown in red blue; FBI flights are red.)

All of the data gathering, analysis and visualization was done with the R language. The authors have provided a complete writeup of the analysis, including data and code, so that anyone can replicate the results. The analysis also includes details not included in the Buzzfeed story, such as this analysis of flights by hour of day. (This appears to be mostly a nine-to-five operation; there were also very few flights on weekends.)

For those interested in the details of the R code, much of the spatial data analysis was handled by the rgdal and rgeos packages, and the data preparation by the dplyr package (with extensive use of the magrittr %>% pipe). Data graphs (other than maps) were mostly handled by the ggplot2 package, and the interactive data tables were generated with the DT package. And as noted by story author Peter Aldhous in the comments below, the maps made with CartoDB, after the data was processed in R.

All in all, it's a magnificent piece of data journalism. I hightly recommend reading the full story and the accompanying detailed analysis at the links below.

Updated April 11 to correct the color assignments and to note use of CartoDB.

Buzzfeed: Spies in the Sky (feature); Spies in the Sky (detailed analysis)

Comments

[Peter Aldhous]

Hi David,

Small correction: FBI flights are red, DHS blue!

Also, maps made with CartoDB, after data processed in R.

Thanks,

Peter

[David Smith]

Thanks for the note Peter. Updated the post above.

[Alastair Muir]

Best variable choice ever

# create data frame
feds <- data_frame()