With the data from the 2010 US Census to be published early next year, here's a cautionary tale from the 2000 Census. If you take a look at the ratio of numbers of men to women in the 5-Percent "PUMS" sample from the 2000 census over various ages, you'll see an odd spike near age 65:
What causes this strange anomaly in the data? In the video below, Sue Ranney (VP of Product Management at Revolution Analytics) visualizes facets of the data and explores how this anomaly came to be. (Watch on YouTube for a larger, high-def version.)
You can download a report on this analysis, including the R script files, from the link below. The R script makes use of the RevoScaleR package for big-data analysis in R, available to Revolution R Enterprise subscribers and free to academics.
Update: Reader Charlie in the comments points out this Freakonomics post, with more details about the problems with over-65s in the 2000 census data.
Revolution Analytics White Papers: Visualizing Huge Data Sets with R: An Old Wives Tale from the U.S. Census
Hey David,
Looking forward to watching the video. In the meantime, be sure to check out this Freakonomics post as an intro to some of the data problems with the 2000 Census PUM for those aged 65 and older.
Charlie
Posted by: Charlie Gibbons | October 19, 2010 at 14:19
That's a great article -- thanks for the pointer, Charlie!
Posted by: David Smith | October 19, 2010 at 15:29