Analyzing clinical trial data for FDA submissions with R

R is well-known for its use in pre-clinical data analysis in drug research. In the relatively unregulated phases before human subjects are exposed to new compounds, researchers have the opportunity to explore data with a variety of techniques. R's flexibility and breadth of standard and novel methods for data analysis and graphics makes it an ideal choice to make new discoveries and guide the direction of subsequent clinical trials.

It's in the clinical phases, where human subjects are exposed to prospective new drugs for the first time, where R is less well-known. SAS is currently the dominant software tool for data preparation and statistical analysis in this domain. There is a misconception that SAS is the preferred software of the FDA, the regulatory agency that oversees clinical trials, but in fact the FDA does not endorse or require use of any specific software. 

I'm often asked whether R is "validated" for use in clinical trials, but in fact neither SAS nor R can be considered validated in isolation.  That's because it's not software that's validated, it's the entire process of creating the submission to the FDA. Validation is defined by the FDA as:

Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes.

Clearly, the software used for the data analysis portion of this process must adhere to these same standards, so how can you established the required documented evidence for R?  To that end, the R Foundation has created the document R: Regulatory Compliance and Validation Issues. Described as "a guidance document for the use of R in regulated clinical trial environments", the purpose of this document is to demonstrate that R, when used in a qualified fashion, can support the various regulatory requirements for validated systems, most notably 21 CFR Part 11 and the relevant "good practice" documents (GxP) and industry guidance documents. 

The document describes the rigorous software development life cycle for R. The software for the core R engine, plus the base and recommended packages included in the R distribution, is all managed under a secure source code management system accessible only by the R Core Group (a widely recognized, international team of experts) and the changes between half-yearly releases are clearly documented. To verify that R works as designed and documented, a suite of validation tests compares the output of R against known data and known results.

Of course, the fact that R itself is maintained with a compliant development process doesn't complete the whole validation process, since R is always part of a larger process to collect, analyze, and report clinical trial data. (The data collection process usually happens independently of the statistical analysis process, and must usually be validated independently, so I won't touch on it further here.)

When any software is installed for analysis, it must be validated to meet IQ/OQ/PQ criteria. IQ refers to Installation Qualification: has the software been installed correctly? OQ refers to Operational Qualification: is the software operating correctly? Finally, PQ refers to Performance Qualification: is the data analysis process effective and reproducible? (Reproducible analysis in clinical trials is an active area for the R community -- see this presentation from Frank Harrell outlining the tools available, and here's an example of a report created with R using a reproducible analysis process.)

REvolution Computing offers a distribution of R designed to comply with established IQ/OQ/PQ criteria, and provides process documentation and services to support the validation process. This enables our clients in the pharmaceutical sector validate their installation of REvolution R, and if required any associated scripts and in-house or third-party packages being used, within the clinical analysis process.

Clear guidance from the R Foundation, sophisticated tools to create reproducible reports, and the support of vendors like REvolution Computing to validate the entire process, have resulted in growing use of R for clinical trial analysis. FDA employees have presented at statistical conferences on the use of open-source software for clinical trial analysis, have used R for internal FDA analyses, and have described the use of R in the FDA submission review process. Statisticians at companies like Novartis have described using R for pharmaceutical development. And with pharmaceutical companies embracing modern statistical methods and graphical techniques in support of FDA submissions, the use of R is only likely to grow further.

(Note: Special thanks go to Marc Schwartz for this r-help post, which was the source for many of the resources linked in this article.)

Comments

[Martin Holt]

"Guidance for Industry - Computerized Systems Used in Clinical Investigations (2007)", the FDA document: more use could have been made of this. No matter which software you're talking about, this document states that its use should be validated by the end-user, and quotes examples of at least 10 SOPs that should be written. The need for large resources to do this is emphasized in the last slide of (I think) the Novartis link. End user validation is the guidance from the FDA, and I wonder what this means for small organisations, and for NHS statisticians ? Is R better than other software in facilitating this process ? There's the steep learning curve, and validation will often be done by someone who understands IQ/OQ/OP working with the statistician. Comments have been made that the documentation is....to the point...maybe part of that learning curve rather than helping to overcome it. The very thing that makes R so attractive to those who are able (to look at the code and change it to meet their needs) might ironically present a regulatory hurdle. You would not want ad hoc changes being made, and if changes are planned they would need change control (one of the SOPs listed by FDA). But I don't want to be accused of FUD: this can be controlled, allowing R to shine. My concerns have shifted away from R (barring yesterday's comment) to the situation overall for small-scale users of any software and the validity of their findings, given that the onus for obtaining and demonstrating accurate use of their software rests with them. Marc Schwartz contributed twice to the MedStats discussion. His last posting 26/03/2009 15:22 unfortunately has my name on the posting (I forwarded it on), but I recommend it to any statistician, and especially those in a small organisation. It can be found at
http://groups.google.com/group/MedStats
This (Analyzing Clinical Trial Data....) is a very useful article, and I recommend reading the links, which serve to support the enthusiastic message whilst honestly including the qualifications.

[clinical research associate]

Your IQ/OQ/PQ criteria is very nice. In any company the documented should be maintained perfectly and It should not be shared with others. So you are doing everything perfectly including other methods you are following.

We are Speid & Associates partners with pharmaceutical companies to provide regulatory affairs
and drug development services to progress new chemical and other entities through the drug development process.

www.sndtm.com

[Martin Holt]

In the link to "Presentation from Frank Harrell", he says, "Classical software “validation” has little to do with quality, as most errors are committed when deriving variables, manipulating and analyzing data. Validation should be directed towards checking the analysis at hand." I hope this means that he is supporting the IQ/OQ/PQ approach. OQ and PQ address the issues he raises. I see software suppliers supporting their customers with IQ and OQ (so am somewhat puzzled by the Speid posting above) and end-users engaging in PQ ("checking the analysis at hand.") As the Novartis link says in the summary slide, this will involve considerable resources on behalf of the company doing the validation, usually one which expects FDA inspection. A software company will gain a considerable commercial advantage if it is known to be helpful in this respect.

[Peng]

hi friends,
I am new to R.I would like to know R-PLUS.Does any know where can I get the free training for R-PLUS.

Regards,
Peng.

[Sergiy]

There is no R-Plus. S-Plus (TIBCO Spotfire S+ now) is a commercial statistical programming package. You can find information about available training on Spotfire website: http://support.spotfire.com/training/pathways/s+pathway.asp . However it looks like new owner of S+ is more interested in Finance applications rather than in Life Science now.

[Ashritha [email protected]]

My colleagues and I have just concluded one of India's most in-depth studies on "Clinical Trials in India". We have looked at various areas such as:
- Market Trends
- Growth Drivers
- Regulatory Bodies and Framework
- Major Players.
-Etc.
We interviewed over 200 individuals and firms to collect the data in what we believe is one of the most detailed study on the subject in India. If you are interested in a copy, you may email me at [email protected]. It is a paid report.

Ashritha
[email protected]