Revolutions

Because it's New Year's Eve: The Sun's Path in 2010

An analemma is a composite photograph of the sun's position, taken from the same place at the same time of day (ignoring Daylight Savings) at regular intervals. The photo below traces the sun's path in the sky over 2010, with each image of the sun separated by about 10 days:

In the Summer, the sun rises high in the sky; in the Winter it barely makes it over the horizon (and, if you live above the Arctic Circle or in Antarctica, it never rises at all). The elegant figure-eight shape is caused by the tilted angle of the Earth's axis as it revolves around the Sun -- think of it like a warped bicycle wheel floating in space, viewed from edge on.

This image comes from the National Geographic gallery linked below; follow the link for other pictures of analemmas, especially the first analemma ever made, from 1979.

Happy New Year, everyone!

National Geographic: Sun Pictures: A Full Year in a Single Frame

Revolutions blog: 2010 statistics

Since it's the end of the year, and since this is a statistics blog, I thought I'd pull some data from the blog server and run some number on the blog itself.

Overall, the blog has doubled the average number of daily visitors and pageviews compared to 2009. The number of pageviews varies quite a lot, as you can see from the kernel density estimate of daily values below. The median daily pageview count was just shy of 1400. I couldn't find an easy way to count the number of posts published in 2010, but there's been at least one every weekday (and often more than one).

By the way, some hints on exporting data from Google Analytics for analysis in R R. It's best to export using the TSV (tab-separated format) for ease of import. You'll need to delete or skip the first 9 rows of the export file to remove the comments Google Analytics prepends to the data. And finally, you'll need to read in the pageview data as text and then strip the commas from the numbers (which are exported as "1,234" instead of "1234") before you convert them to numeric in R. I used the following commands:

df <- read.table("pageviews.tsv", header=TRUE, skip=9, sep="\t", as.is=TRUE)
df$Pageviews <- as.numeric(gsub(",","",df$Pageviews))

The remaining stats I pulled directly from Google Analytics.

Top 10 Posts of 2010

The top 10 posts, as measured by direct traffic to each page, are listed below:

1. Why Learn R? It's the language of Statistics (by Joseph Rickert)
2. Facebook's Social Network Graph
3. Video: Hadley Wickham gives a short course on graphics with R
4. Charting the World Cup
5. How to make a heat map in R
6. A free book on probability and statistics with R
7. Because it's Friday: The dating equation
8. O'Reilly at OSBC: The future's in the data
9. An analysis of the Wikileaks data with R
10. How to animate Google Earth with R

This doesn't include data from posts syndicated at other sites (such as r-bloggers.com), and views to the blog homepage (which is how many people read the blog, rather than post-by-post). Interestingly, the #3 post from 2009, How to choose a random number in R, would have been the #1 post in 2010 -- thanks to its Google Search traffic -- if I hadn't limited the contenders to posts published this year. 

Top 5 Browsers of 2010

It's interesting to look at the top Web browsers used to visit this blog, as measured by the number of pages server to each browser:

• Firefox (44.45%)
• Internet Explorer (22.04%)
• Chrome (20.08%)
• Safari (11.41%)
• Opera (2.02%)

Chrome is the big change here: it represented just 8.4% of traffic in 2009. In the Web as a whole, Internet Explorer represents a nearly 50% share of Web traffic, so clearly the readers of this blog prefer Firefox disproportionately. I use both Chrome and Firefox myself.

Top 5 Operating Systems of 2010

Google Analytics also reports the operating system used to visit the blog, and the top five are:

• Windows 64.99%
• Macintosh 20.69%
• Linux 11.96%
• iPhone 1.32%
• iPad 1.05%

This makes Firefox's dominance in the browsers even more remarkable, given IE's built-in advantage on Windows. Regarding the operating systems themselves, the order is unchanged from 2009, and the ratios are about the same too (Windows was 59% in 2009, Mac at 28%). The only change is that the iPad replaced the iPod for the #5 slot.

Looking back...

Well, that's about it for 2010 - it's been a great year, and a lot of fun writing the blog and getting the word out about all the awesome things the community is doing with R. Thanks to everyone who sent in suggestions for articles, made comments, or just read the blog in 2010. We'll be back again next year with more news about R, statistics and the world of open source. Happy New Year to all of our readers, and we'll see you in 2011!

R Packages for Social Search

Jesse Bridgewater works on "social search awesomeness" for the Bing search engine, and is setting up his dev environment with the necessary tools including python, vim, and R. Jesse has shared a handy script he uses to install all the specialty packages he uses for his data analysis. This is a handy script to modify for your own purposes, but it's also interesting to review the packages Jesse lists as part of his standard toolkit:

Literate Programming: R2HTML, sweave, Rpad

Data Visualization: ggplot2, YaleToolkit

Data Mining / Machine Learning / Natural Language Processing: ElemStatLearn, gbm, bayesm, RWeka, lsa, tm

Graphs and Networks: igraph

Statistics: survival, Hmisc, ICSNP, zipfR

What R packages do you consider to be part of your standard toolkit?

Jesse S.A. Bridgewater: My Favorite R Packages (Installed With One Command)

Analysis of Facebook status updates

The Facebook Data Team has published an analysis of the status updates of Facebook users, by categorizing words according to the 68 categories of the Linguistic Inquiry and Word Count Dictionary, and tabulating the frequencies of their use. It's fairly interesting to see this kind of analysis applied to Facebook, but unfortunately doesn't reveal much in the way of "A-ha!" insights. For example, this chart of the frequency of six categories of words expressed throughout the day shows pretty much what you'd expect:

(I do wonder whether "Hour of Day" is in local time or all mapped to the same timezone, though: I'd expect the variation to be more pronounced if the analysis were based on local time, and if not I'd have to assume this represents US-based Facebook users only.)

Notable for R users is that all of the charts in the Facebook post were clearly created with Hadley Wickham's ggplot2 package. If you haven't tried ggplot2 yet, check out the ggplot2 website for resources for creating beautiful graphics with R. If you're already a ggplot2 user, you might have missed that it was updated just before Christmas with improvements to legends and axes and several bug fixes. Read the update announcement for the full details.

Facebook Data Team: What’s on your mind? 

 

The Complete Joy of Stats

First there was the trailer, then there were clips, and now the complete 1-hour BBC4 programme The Joy of Stats featuring Gapminder's Hans Rosling is available on YouTube!

 

GapMinder: The Joy of Stats

Travel grants and prizes for R/Finance 2011

If you've been thinking about heading to Chicago in April for the R/Finance conference, here's another reason to go: posting for the committee, Dirk Eddelbuettel announced last week that thanks to a favourable response from sponsors[*], the conference organizers can now offer:

• a competition for best paper, which given the focus of the conference will award for both an 'academic' paper and an 'industry' paper
• availability of travel grants for up to two graduate students provided suitable papers were accepted for presentations

Full details are in the updated call-for-papers. Note that the submission deadline is February 15, 2011.

[*] Revolution Analytics is a proud sponsor of R/Finance 2011.

R/Finance 2011: Call For Papers

Merry Christmas from Revolution Analytics!

We're taking a little break for the Christmas holiday, but in the meantime please enjoy this festive music produced live entirely on iPads and iPhones -- live:

 

We'll be back on Tuesday. Merry Christmas to all!

Did you feel that?

There was a small earthquake in northern England on Tuesday. Barry Rowlingson felt the quake (it rattled the photographs on his wall), but didn't know how big of a quake it was because he didn't know how close he was to the epicentre. The British Geological Survey hadn't yet announced the quake, but did give access to seismograph readings, which indeed featured spikes marking the earthquake. So Barry read the times of the spikes from the charts, did a little triangulation based on the speed of wave propagation, crunched the numbers in R, and came up with this location for the epicentre (in blue):

The actual epicentre, as calculated by the USGS, is marked in red. Professional seismologists also use R to measure and locate earthquakes, but this is pretty good for a self-proclaimed quick-and-dirty calculation!

Barry Rowlingson's GeoSpatial Blog: Mapping The Kendal Mint Quake

Citizen Data Journalism: Mexico Homicides

I've recently praised some mainstream media outlets like the New York Times and New Scientist for leading the charge on data journalism. But you don't need to be a large organization to find news in data. With open data sources, and open-source data analysis tools, individuals can make newsworthy discoveries.

Diego Valle-Jones has been investigating the impact of the Drug War in Mexico for a couple of years now, by using R to analyze the homicide statistics reported by the local municipalities. But Diego has noticed some anomalies in the data: many murders are not reported as homicides at all, but instead as accidental deaths. For example, data including the Acteal Massacre of 1997 (where 45 Tzotzil Indigenous people were killed by paramilitaries) shows a spike in accidents, not homicides:

(Diego provides the R code that generated this plot.) A data error, or falsification? Further investigation is needed to decide, but without the availability of open data for data journalism, and the possibility of citizen data journalism such as that by Diego, we might never know.

Diego Valle-Jones: Some problems with the Mexican mortality database

Forbes: R is a name you need to know in 2011

The December 20 issue of Forbes magazine, on newsstands now, includes a column about R on page 128 as part of the "Name You Need to Know in 2011" feature. It's basically an excerpt from this blog post by Steve McNally and its comments, and includes quotes from Norman Nie of Revolution Analytics, Bill Alpert of Barron's, and Brandon Whitcher of GlaxoSmithKline. There's no online version of the column, but it's worth checking out in the printed magazine.