Revolutions

R analysis shows how UK health system could save £200m

According to an analysis by Prescribing Analytics (a joint venture of technologists and doctors in the UK), Britain's cash-strapped National Health Service (NHS) is overspending on prescription drugs. While cheaper (but equally effective) generic drugs are widely available for many treatments, some doctors continue to prescribe patented drugs which can cost 10 times as much — and often much more. The problem varies regionally: for example, in the administrative region including the city of York nearly 30% of prescriptions for statins (drugs used to lower cholesterol levels) are for expensive non-generics:

Looked at another way, a nearly 70% generic prescribing rate seems pretty good. But the cost differential in generics vs patented drugs is the key here: generic Simvastatin (in blue below) still represents a small fraction of the costs for prescribed statins, compared to non-generic drugs containing Atorvastatin and Rosuvastatin.  According to the study:

Guidelines for that period said that the generic Simvastatin should be a default option for patients. Wherever the percentage of branded items is high, it represents potential to make big savings by switching. For example, if two thirds of the proprietary drugs had switched to the generic forms of the same drugs in the year to June 2012, this would have saved £200 million pounds. Any savings could be used to help other patients.

The analysis was done using R statistics software (and the R code is available for download at github). The prescription data is available thanks to the open data practices of the NHS, but unfortunately a precise estimate of cost savings isn't currently possible because prescription drug price data isn't public.

Prescription Analytics: NHS efficiency savings: the role of prescribing analytics (via Ben Goldacre)

Update: Corrected the description of statins in the first paragraph.

Population health management with RevoScaleR

This guest post is by Douglas McNair MD PhD, Engineering Fellow & President, Cerner Math Inc. -- ed.

RevoScaleR scaling big-data modeling performance for real-time health data analysis at Cerner

The size of data sets is increasing much more rapidly than the speed of cores, of RAM, and of disk drives. This is particularly true of electronic health records (EHRs) and other health care data, where multi-billion-row relational tables with dozens to thousands of columns are common in data warehouses derived from EHRs. Analyzing a data set that contains N observations on P variables frequently entails operating on several terabyte arrays — too large for main memory, beyond the statistical packages’ typical limit of a maximum vector length of 2³¹ – 1 (around two billion cells), and too large even for pagefile virtual memory on a single machine.

Statistical software able to use multi-core processors is a necessary, but not sufficient requirement to deal with data of this size. For statistical analysis of large data, you also need:

1. parallelized external memory algorithms (PEMAs) that do not require all data to be in RAM, and
2. efficient multi-threading and inter-process communication across massively parallel clusters of computers.

Like other statistical packages, the R system is memory-limited. Revolution Analytics' RevoScaleR and bigmemory and a few other big-data packages overcome the memory limitation of R by providing a backing file for processing blocks of data at a time or by providing an ODBC or other streaming service for processing blocks of data.

Since cardinality N is large, especially for our pharmacovigilance (drug safety) and other use-cases that involve modeling low-eventrate low-prevalence outcomes, only tasks with complexity that is approximately O(N) are feasible.  But not all big-memory and parallelized architectures offer comparable performance or scalability.

For example, Lumley’s biganalytics package in R provides a method for passing big.matrix objects to the biglm function, which computes ordinary least squares regressions on large datasets, like one of Cerner’s cohorts of Type 2 diabetics:

library(biglm)
library(foreach)
biglm.big.matrix( los.outlier ~ gender + age.grp + medicaid + last.HbA1c +
 max.creat + min.glu + nbr.adm24mo + nbr.comorbid_dx + nosocomial.inf,
 family=binomial(), data="type2_diabetic" )

And Revolution R Enterprise can do the same with RevoScaleR:

rxLogit( los.outlier ~ gender + age.grp + medicaid + last.HbA1c +
  max.creat + min.glu + nbr.adm24mo + nbr.comorbid_dx + nosocomial.inf,
  data="type2_diabetic.xdf" )

Happily, the biglm regression takes less than 20 seconds to run on this cohort of 2 million diabetic patients and 10 variables.  But even more remarkably, the rxLogit regression takes less than 0.5 seconds to run on this dataset. (Both analyses ran on a Windows 7, 64-bit quadcore 2.5 GHz laptop with 8 MB of RAM.) And the RevoScaleR regression runs far faster than this on a multi-node cluster.

The difference might seem inconsequential if our use-case involved a one-time execution of a regression at model-development-time. However, a majority of our run-time applications instead involve near-realtime kmeans ‘clustering’ or ‘classification’ or ‘prediction’ use-cases with high cardinality and dimensionality, where close matches to a particular patient and her attributes are sought within very large cohorts, and this task is repeated for successive different patients every second. 

For high-value, high-performance applications like these, RevoScaleR and related tools are fast becoming an indispensable part of Cerner’s armamentarium for addressing many use-cases in health services and population health management.

Douglas McNair, MD, PhD, is president of Cerner Math, Inc., and one of three Cerner Engineering Fellows and is responsible for innovations in decision support and very-large-scale datamining. McNair joined Cerner in 1986, first as VP of Cerner’s Knowledge Systems engineering department; then as VP of Regulatory Affairs; then as General Manager for Cerner’s Detroit and Kansas City branches. Subsequently, he was Chief Research Officer, responsible for Cerner’s clinical research operations. In 1987, McNair was co-inventor and co-developer of Discern Expert®, a decision-support engine that today is used in more than 2,000 health care facilities around the world. Between 1977 and 1986, McNair was a faculty member of Baylor College of Medicine in the Departments of Medicine and Pathology. He is a diplomate of the American Board of Pathology and the American Board of Internal Medicine.

FDA: R OK for drug trials

In a poster (PDF) presented at the UseR 2012 conference, FDA biostatistician Jae Brodsky reiterated the FDA policy regarding software used to prepare submissions for drug approvals with clinical trials: Sponsors may use R in their submissions.

The FDA does not endorse or require any particular software to be used for clinical trial submissions, and there are no regulations that restrict the use of open source software (including R) at the FDA. Nonetheless, any software (R included) used to prepare data analysis from clinical trials must comply with the various FDA regulations and guidances. The R Foundation helpfully provides a guidance document for the use of R in regulated clinical trial environments, which provides details of the specific FDA regulations and how R complies with them.

Despite some mistaken conceptions in the pharmaceutical industry, SAS is not required to be used for clinical trials. This origin of this fallacy is probably related to the fact that data must be submitted in the XPT "transport format" (which was originally created by SAS). This data format is now an open standard: XPT files can be read into R with the standard read.xport function, and exported from R with the write.xport function in the SASxport package. (And if you have legacy data in other SAS formats, there's a handy SAS macro to export XPT files.)

R is widely used at the FDA. (Full disclosure: the FDA is a Revolution Analytics customer.) The poster describes how FDA offices "use R on a daily basis", and that FDA scientists have written R packages for use by other scientists (within and outside the FDA) involved in the submissions process. FDA-contributed packages include popPK, MAVTgsa and QT, which was written specifically for the FDA's QT Interdisciplinary Review Team.

Jae Brodsky also provides some personal views on the use of R at the FDA. She notes:

• All results from data should be reproducible independent of software used.
• In submissions, FDA reviewers would like to know: which R functions were accessed; how they were accessed; and that the results are accurate, reliable and consistent.
• The FDA has used R for its own reports. (Interesting side-note: as a government agency, its reports must be Section 508 compliant, one part of which demands that reports be accessible to readers with limited color vision. Jae mentioned to me that she uses R's Rcolorbrewer package to generate 508-compliant pallettes for charts in FDA reports, as the poster excerpt below demonstrates.)

In Jae's experience, "R use at the FDA is completely acceptable and has not caused any problems." So as more and more companies adopt R for statistical analysis, we can expect to see more FDA clinical trial submissions using R for statistical analysis, reporting, and data visualization.

Thursday: Tweet-chat on Multiple Sclerosis research

The story about the great work that SUNY Buffalo has been doing to find a cure for Multiple Sclerosis with Revolution R Enterprise and IBM Netezza has generated a lot of attention, with stories in Forbes, InformationWeek and eWeek (amongst others). To continue the discussion, IBM has put together a panel for a "Tweet Chat" on Thursday (May 10) at noon Eastern time. I'll be there with the lead researcher Murali Ramanathan, Shawn Dolley from IBM and Tim Coetzee from the National MS Society. We'll be chatting about the search for a cure for MS, Big Data and analytics, in snippets of 140 characters or less. Join the conversation by following the #IBMDataChat hashtag at twitter.com or in your favourite Twitter client.

When: Thursday, May 10, 2012 from 12:00-1:00 p.m. ET (find your local time)
Where: Twitter hashtag #IBMDataChat
Who: 

• Shawn Dolley, IBM VP of Big Data Healthcare & Life Sciences (Moderator), @shawndolley
• Dr. Murali Ramanathan, SUNY Buffalo Professor Pharmaceutical Sciences and Neurology, Director of Graduate Studies & Co-Director, Data Intensive Discovery Initiative,@M_Ramanathan
• David Smith, Revolution Analytics R Evangelist & VP of Marketing, @revodavid
• Tim Coetzee, National MS Society Chief Research Officer, @tim_coetzee

Big Data statistics in the search for a cure for MS

Multiple Sclerosis (MS) is a debilitating and complex disease with an unknown cause — and for which there is currently no cure. The SUNY Buffalo is home to one of the leading multiple sclerosis (MS) research centers in the world, and as reported in Healthcare IT News, the research team is using IBM Netezza and Revolution R Enterprise to accelerate its study of more than 2,000 genetic and environmental factors that may contribute to MS symptoms:

Shawn Dolley, vice president and GM of global healthcare and life science at IBM Big Data (he worked for Marlborough, Mass.-based Netezza before IBM's acquisition of that company in 2010), remembers when Harvard Medical School researchers told him not long ago that, when it came to solving problems with big data analytics, "We have four questions we can ask per month." Why four? "Well, there's four weekends per month," he recalls them saying. "That's how long it takes us to run our jobs."

Now, using an IBM Netezza analytics appliance, in conjunction with software from Revolution Analytics, researchers can analyze disparate data in a matter of minutes instead of days, regardless of what type or size it is, say IBM officials. The technology automatically consumes and analyzes the data, and makes the results available for further analysis, leaving researchers more time to analyze trends. 

As described in this case study, SUNY Buffalo researchers were able to accelerate their research by reducing computation times more than 100-fold in the search for significant interactions between thousands of genetic and environmental factors. The R language itself also accelerates the research process, by making it easier to include new kinds of variables such as discrete dependent, Poisson dependent or continuous normally-distributed variables by simply adding a few lines of code. (Using another language would have required a re-writte of the entire algorithm.)

From an analytics perspective, the researchers were able to write a version of their software tools in in Revolution R Enterprise for IBM Netezza, so that all their reporting as well as our analysis is all consolidated in one place. This prevents the need to move very large amounts of data in and out of Netezza, which would cause further delays. They were also able to use a wider variety of data sets. 

The SUNY researchers were able to: 

• Use the new algorithms and add multiple variables that before, were nearly impossible to achieve. 
• Reduce the time required to conduct analysis from 27.2 hours without Netezza to 11.7 minutes with it. 
• Carry out their research with little to no database administration (Unlike other HPC platforms or databases available, Netezza was designed to require a minimum amount of maintenance)
• Publish multiple articles in scientific journals, with more in process.
• Proceed with studies based on ‘vector phenotypes’—a more complex variable that will further push the Netezza platform.

You can learn more about the research by the SUNY Buffalo team in this IBM Netezza blog post, and in the IBM press release at the link below.

IBM Press Releases: Leading Global Multiple Sclerosis Research Center Taps IBM Analytics to Improve Patient Care

Bad Science at Strata 2012

Ben Goldacre, the physician and  biostatistician behind the always-excellent Bad Science column in the Guardian, gave a barnburner of a talk at Strata 2012 yesterday, "The Information Architecture of Medicine is Broken". For anyone not aware of the problems caused by publication bias in clinical trials (for example, ineffective drugs with a wide variety of side-effects coming to market), his talk is a must-watch. There's also a great explanation of meta-analysis (using the logo of the Cochrane Collaboration as a backdrop).

Goldacre is attempting to shed light on the issue of negative results in clinical trials going unreported, by matching the register of planned clinical trials with those that are actually published. At around 12:00 in the video below, learn how you can help with this important project. You'll also spot an R chart or two during the presentation.

 

Strata 2012: The Information Architecture of Medicine is Broken

Why you should care about reproducible research

This week's Economist has an in-depth article on the consequences of failures reproducible research, adding more detail to the report in the New York Times in July. Errors in data analysis by researchers at Duke University led to patients in clinical trials being assigned the wrong drug:

Dr Potti and his colleagues had mislabelled the cell lines they used to derive their chemotherapy prediction model, describing those that were sensitive as resistant, and vice versa. This meant that even if the predictive method the team at Duke were describing did work, which Dr Baggerly and Dr Coombes now seriously doubted, patients whose doctors relied on this paper would end up being given a drug they were less likely to benefit from instead of more likely.

One startling aspect of this saga is that the story only came to prominent attention after revelations that Dr Potti falsely claimed to be a Rhodes Scholar on his resume, despite repeated attempts by Baggerly (who presented the problems at the R BioConductor conference more than 2 years ago) to raise awareness of this serious problem.

[Baggerly] noted that in addition to a lack of unfettered access to the computer code and consistent raw data on which the work was based, journals that had readily published Dr Potti’s papers were reluctant to publish his letters critical of the work. Nature Medicine published one letter, with a rebuttal from the team at Duke, but rejected further comments when problems continued. Other journals that had carried subsequent high-profile papers from Dr Potti behaved in similar ways. ... Eventually, the two researchers resorted to publishing their criticisms in a statistical journal, which would be unlikely to reach the same audience as a medical journal.

Said Dr Baggerly: “I find it ironic that we have been yelling for three years about the science, which has the potential to be very damaging to patients, but that was not what has started things rolling.”

The Economist: An array of errors

useR! 2011 roundup

As I stand[*] here at Heathrow waiting for my flight back to the States, I thought I'd dash off a few quick reflections of the userR! 2011 conference at University Warwick. It was an outstanding event. There's something about a conference of just a few hundred attendees (there were about 450) that creates a sense of camaraderie and common purpose you just don't get at larger conferences. It was wonderful to re-connect with colleagues, meet long-standing collaborators previously only known via email, and meet many new friends. The event was tremendously well-run, and as a community-run conference proceeded much more smoothly than many professionally-managed conferences I've been to. A big thank-you to the useR 2011 organizing committee (John Aston, Julia Brettschneider, David Firth, Ashley Ford, Ioannis Kosmidis, Tom Nichols, Elke Thönnes and Heather Turner) for such a fantastic conference, and we at Revolution Analytics were proud to have been a sponsor. (Photo courtesy useR! 2011.)

There was some great information shared in the tutorials and invited and contributed sessions, too. Here are a few quick nuggets compiled from my notes:

From Max Kuhn's tutorial:

• The caret package is a powerful yet easy-to-use front end to more than 122 different kinds of predictive models in R. It provides a consident user interface to all models, and makes it easy to tune and compare models to select the one with the best predictive power for your data.
• The partykit package provides some beautiful visualizations of tree models.

From Ulrike Gromping's talk

• The FrF2 function (from the eponymous package) is a complete solution for industrial experimental design.
• George Box (of the Box-Cox transformation) is R.A. Fisher's son-in-law.

From Jonty Rougier's talk

• Nomograms, the lost art of representing equations as combined axes, are pretty cool.
• Moroccan donkeys are adorable, and English donkeys are the biggest charity recipients in the UK.

From Pairach Piboonrungroj's talk:

• Structural Equation Modeling is just a combination of factor modelling and regression
• R packages for SEM include sem, OpenMx and lavaan

From Fabrizio Ortolani's talk:

• The forecast function is a simple yet powerful time series forecasting tool, and out-performs similar functionality in SAS and SAP.

From Patrick Burns' talk:

• Modern portfolio optimization is computationally intensive, and is basically a knapsack problem with many constraints
• The packages testthat and RUnit, and the functions do.call and tryCatch, are useful for testing R code
• Generating random inputs to functions is a good way of testing, especially of error-detecting code which often goes untested.

From Ian Cook's talk:

• R has been validated R for use in the pharmaceutical industry many times
• FDA officers have spoken about the use of R for clinical research: Sue Bell in 2006 (although this link now seems to be broken -- anyone have a copy?) and Matt Soukup in 2007.

From Alexander Kowarik's talk:

• The sparkTable package combines official statistics tables with sparklines
• A cool application of sparklines is win-loss records for football teams

From Olaf Mersmann's talk:

• The microbenchmark package provides nanosecond-scale benchmarking of R functions on some hardware systems.

From Paul Murrell's keynote:

• You can use R to read a PDF file, and extract the vector representation of the images and the text for analysis.
• The R graphics engine now has support for complex shapes (e.g. polygons with holes), which can be drawn with the grid.path and polypath functions
• The gridSVG package will soon make it possible to create interative SVG graphics (e.g. high-quality clickable maps) with R.

From Simon Urbanek's keynote:

• The forthcoming ix package will soon allow for interactive, cross-linked exploratory graphics in R that supports millions of points with high performance.

Anyway, I have to board my plane now. You can see more commentary from useR 2011 at the #user2011 hashtag on Twitter.

[*] Literally, stand. The only power outlets are conveniently placed out of cable reach of chairs. Thanks, BAA!

IBM Netezza: Embrace open source analytics

Earlier this month Thomas Dinsmore, solutions architect for IBM Netezza’s Advanced Analytics team, had a great blog post on why companies should embrace R as an analytics platform. He says:

There are three main reasons R should be part of your enterprise analytics architecture:

• R has capabilities not available in commercial analytics software
• Usage of R by analysts is growing rapidly
• R’s total cost of ownership is attractive

He also mentioned a great example of why companies are turning to R software for access to the statistical techniques they need:

A customer we’re working with in the life sciences industry wants to apply four new methods to their analytic toolkit.  This customer spends almost a billion dollars each year to run hundreds of thousands of experiments; very small improvements in precision directly impact this customer’s bottom line.  Right now, all of these new methods are available in R, and none are available in commercial packages.

The post also mentions the partnership between IBM Netezza and Revolution Analytics, which will soon bring in-database analytics based on R to the Netezza appliance.

Netezza Blogs: Embrace Open Source Analytics

NYT on the importance of reproducible research

Yesterday's New York Times includes a great article on the failure of some genetic tests for cancer detection, and the flaws in the research that led to them. The article features quotes from Keith Baggerly of MD Anderson Cancer Center, and includes a photo of him and colleague Kevin Coombes in front of a page of R code:

The article highlights the importance of reproducible research: unless others can have access to the data and code that backs up the statistical conclusions in the paper, many such errors are likely to continue to go undiscovered. I saw Keith give an amazing talk about reproducible research and data forensics at the BioConductor conference a couple of years ago (I'm pretty sure that's one of the slides from the talk in the image above):

The talk concerned a published article where the results seemed odd. Baggerly tried in vain to get hold of the source data to reproduce the analysis, but the article authors didn't cooperate. So in an amazing feat of data forensics, he managed to recreate the data by matching public sources to measurements from the printed graphs, and figured out that there were gross data errors in the article: labels transposed, data duplicated, that kind of thing. The conclusions were completely bunk, but the journal refused to print a correction, despite the fact that it meant actual patients were being trialled on inappropriate drugs.

I'm glad to see this important issue is getting some wider media attention -- check out the Times article at the link below for the story.

New York Times: How Bright Promise in Cancer Testing Fell Apart