The current edition of The Economist includes a "special report on managing information", targeting the issue of the information explosion / data deluge / whatever you want to call it these days. It includes the usual attributes of the problem: data is being collected faster than we can store it, astronomers are creating petabytes of data daily, the usual. But unusually, the Economist looks at some of the implications of all of this data being collected and analyzed:
“Revolutions in science have often been preceded by revolutions in measurement,” says Sinan Aral, a business professor at New York University. Just as the microscope transformed biology by exposing germs, and the electron microscope changed physics, all these data are turning the social sciences upside down, he explains. Researchers are now able to understand human behaviour at the population level rather than the individual level.
What happens when the data deluge evolves from a flood into a source of information hydro-power? We'll be finding out, sooner rather than later I suspect.
The Economist: Data, data everywhere


From the special report, page 8: "A free programming language called R lets companies examine and present big data sets, and free software called Hadoop now allows ordinary PCs to analyse huge quantities of data that previously required a supercomputer."
Posted by: Steve Bagley | March 01, 2010 at 18:29
Thats Steve, I didn't know about that R reference. I'll have to pick up a dead-tree copy and check it out.
Posted by: David Smith | March 02, 2010 at 09:28