Open Source software's opportunity to reform government

The results from the 2013 Future of Open Source Survey are in — thanks to everyone who contributed by completing the survey. You can read an overview of the results here, or see the detailed breakdowns in the slides at the end of this post.

For me, one of the most interesting nuggets from the survey is that a plurarity of respondents believed that of all sectors, the public sector will be impacted the most by open source software:

There is huge opportunity to improve the ways that we govern ourselves through the use of open source software, principles and communities. Federal, state and local governments have access to rich and large streams of data, but thus far government hasn’t been as quick as the business sector in building powerful data-based applications and services. But governments have one advantage that the business sector does not: givernment data is, by definition, public. Here are three ways that government can capitalize on rich sources of public data:

Democratize data. By making public data available on the internet (where practical and legal), governments can draw on the expertise of the data scientist community to unlock its value. At the Federal level, the Open Government Initiative is a good start, although many of the data sets made available at data.gov are over-aggregated and too small for truly innovative Big Data applications. Some local governments are making good progress here as well. New York City’s open data policy and new data transparency law has fostered a culture of civic “hacking” on city data, and created handy tools for locals like this iPhone app to find local WiFi-enabled venues suitable for getting out of the house when working “from home”.

Crowdsource data science talent with open tools. The data science community has created powerful open-source tools and resources for the analysis of Big Data. The open-source R language is designed to analyze data common to government applications (like economic, environmental and social data) and is used today by government departments such as NIST, FDA, NOAA, and the CIA. Crowdsourcing platforms and contests are a great way of getting data scientists engaged with public data, as NASA found in this Kaggle contest to find the location of dark matter from radio astronomy data.

Deploy data scientists within departments. Not all government can be made public, of course. There may be privacy issues (think IRS data), or national security issues (CIA, NSA) to consider. So it’s up to data science teams in civil service to work on sensitive data in a secure environment. It’s very encouraging to see departments like the Consumer Protection Bureau openly embrace big-data platforms and open-source tools for their teams. The growing drive towards evidence-based policy making, and the use of statistical experiments to improve government effectiveness, is another very positive sign.

It's great to see that the respondents to the 2013 Future of Open Source Survey recognize this opportunity to revolutionize government. It's up to us, as citizens, to press governments to capitalize on the opportunity, so make your voice heard.

Michael J Skok: 2013 Future of Open Source - 7th Annual Survey results

Comments

[nick]

Here we go again...Revolution lecturing about open source. When are you guys going to open source your proprietary bits? I am sure the R community can benefit a lot from that. More than that, I am sure they will make it better. Given that your product rests on the thankless efforts of so many, I think it's time you guys put your money where your mouth is. Either go open source, or go build something from scratch.

[David Smith]

@Nick, Here are just some of Revolution Analytics' open source contributions: the RHadoop project, NetworkSpaces, foreach and iterators, doMC, doParallel. This doesn't include open-source projects created by third parties funded by Revolution Analytics, e.g. bigvis.