Revolutions

Portfolio optimization in R

Diethelm Wuertz provides a handy overview of the current state-of-the-art in portfolio optimization in R in a recent posting on the r-sig-finance mailing list.

The fPortfolio package (part of the Rmetrics suite) is available for download from CRAN, and handles standard portfolio optimization problems. As Diethelm Wuertz explains, the package can calculate:

Mean-Variance portfolio optimization with a quadratic objective and linear constraints using the "quadprog" solver from R for quadratic programming problems. Note, to this class of problems do not belong the problems of maximizing the return for a given risk, and also not problems with quadratic or nonlinear constraints like portfolios with covariance risk budgeting, or 130/30 portfolios. These portfolios are more complex and require solvers which can handle quadratic and/or nonlinear constraints.

Mean-CVaR portfolios which belong to the class of scenario optimization problems with a linear objective function and linear constraints. The default solver for this kind of portfolio is the solver from the package "Rglpk". Again you cannot solve more complex problems which add quadratic or non-linear  constraints.

There is also ongoing development work in Rmetrics to solve more complex portfolio models, and for functions to perform rolling backtesting, rolling benchmarking, and rolling performance analysis. These functions are available in r-forge with the usual caveats for in-development code -- full details are available in the aforementioned r-sig-finance post.

Diethelm Wuertz: fPortfolio - Status - New Functionalities

Keep continuous data continuous

If you've collected data on a continuous variable (age, for example), there may be a temptation to bin the data before analysis as an attempt to account for the fact that the relationship between the variable and the response may not be linear or otherwise easily explained using traditional regression methods. You might consider using regression splines instead, for flexibility in the response without the associated loss of power.  Stephan Kolassa in this r-help discussion about categorical variables notes that Frank Harrell has an excellent page listing the problems associated with categorizing continuous variables.

Of course, if the data were collected in bins (as is common with survey data), you'll just have to make do with what you have and choose appropriate contrasts for the levels of the categorical variable.

(Update 11:44PST: Added link to r-help discussion.)

Course: Statistical Learning and Data Mining III

The following course was announced on r-help yesterday.  It will be presented by frequent R contributors Trevor Hastie and Robert Tibshirani, and runs March 16-17 in Palo Alto, California. 

Short course: Statistical Learning and Data Mining III:
Ten Hot Ideas for Learning from Data

Trevor Hastie and Robert Tibshirani, Stanford University

  Sheraton Hotel
  Palo Alto, CA
  March 16-17, 2009
This two-day course gives a detailed overview of statistical models   for data mining, inference and prediction. With the rapid developments in internet technology, genomics, financial risk modeling, and other high-tech industries, we rely increasingly more on data analysis and statistical models to exploit the vast amounts of data at our fingertips.

In this course we emphasize the tools useful for tackling modern-day data analysis problems. From the vast array of tools available, we have selected what we consider are the most relevant and exciting. Our top-ten list of topics are:

Regression and Logistic Regression (two golden oldies),
Lasso and Related Methods,
Support Vector and Kernel Methodology,
Principal Components (SVD) and Variations: sparse SVD, supervised PCA, Multidimensional Scaling and Isomap, Nonnegative Matrix Factorization, and Local Linear Embedding,
Boosting, Random Forests and Ensemble Methods,
Rule based methods (PRIM),
Graphical Models,
Cross-Validation,
Bootstrap,
Feature Selection, False Discovery Rates and Permutation Tests.  

The lectures will consist of video-projected presentations and discussion. Go to the site http://www-stat.stanford.edu/~hastie/sldm.html for more information and online registration.

Study: Statisticians have awesome jobs

According to a study recently reported in the Wall Street Journal, being a statistician is one of the best jobs available in the U.S., lagging only behind two other quantitative jobs: mathematician (#1) and actuary (#2).  "Best" is judged in terms of work environment comfort and safety, income, physical demands, and career outlook (full criteria here).  It seems being able to use world-class open-source software wasn't on the list, which surely would have bumped statisticians to #1.

Wall Street Journal: Doing the Math to Find the Good Jobs

Analyzing clinical trial data for FDA submissions with R

R is well-known for its use in pre-clinical data analysis in drug research. In the relatively unregulated phases before human subjects are exposed to new compounds, researchers have the opportunity to explore data with a variety of techniques. R's flexibility and breadth of standard and novel methods for data analysis and graphics makes it an ideal choice to make new discoveries and guide the direction of subsequent clinical trials.

It's in the clinical phases, where human subjects are exposed to prospective new drugs for the first time, where R is less well-known. SAS is currently the dominant software tool for data preparation and statistical analysis in this domain. There is a misconception that SAS is the preferred software of the FDA, the regulatory agency that oversees clinical trials, but in fact the FDA does not endorse or require use of any specific software. 

I'm often asked whether R is "validated" for use in clinical trials, but in fact neither SAS nor R can be considered validated in isolation.  That's because it's not software that's validated, it's the entire process of creating the submission to the FDA. Validation is defined by the FDA as:

Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes.

Clearly, the software used for the data analysis portion of this process must adhere to these same standards, so how can you established the required documented evidence for R?  To that end, the R Foundation has created the document R: Regulatory Compliance and Validation Issues. Described as "a guidance document for the use of R in regulated clinical trial environments", the purpose of this document is to demonstrate that R, when used in a qualified fashion, can support the various regulatory requirements for validated systems, most notably 21 CFR Part 11 and the relevant "good practice" documents (GxP) and industry guidance documents. 

The document describes the rigorous software development life cycle for R. The software for the core R engine, plus the base and recommended packages included in the R distribution, is all managed under a secure source code management system accessible only by the R Core Group (a widely recognized, international team of experts) and the changes between half-yearly releases are clearly documented. To verify that R works as designed and documented, a suite of validation tests compares the output of R against known data and known results.

Of course, the fact that R itself is maintained with a compliant development process doesn't complete the whole validation process, since R is always part of a larger process to collect, analyze, and report clinical trial data. (The data collection process usually happens independently of the statistical analysis process, and must usually be validated independently, so I won't touch on it further here.)

When any software is installed for analysis, it must be validated to meet IQ/OQ/PQ criteria. IQ refers to Installation Qualification: has the software been installed correctly? OQ refers to Operational Qualification: is the software operating correctly? Finally, PQ refers to Performance Qualification: is the data analysis process effective and reproducible? (Reproducible analysis in clinical trials is an active area for the R community -- see this presentation from Frank Harrell outlining the tools available, and here's an example of a report created with R using a reproducible analysis process.)

REvolution Computing offers a distribution of R designed to comply with established IQ/OQ/PQ criteria, and provides process documentation and services to support the validation process. This enables our clients in the pharmaceutical sector validate their installation of REvolution R, and if required any associated scripts and in-house or third-party packages being used, within the clinical analysis process.

Clear guidance from the R Foundation, sophisticated tools to create reproducible reports, and the support of vendors like REvolution Computing to validate the entire process, have resulted in growing use of R for clinical trial analysis. FDA employees have presented at statistical conferences on the use of open-source software for clinical trial analysis, have used R for internal FDA analyses, and have described the use of R in the FDA submission review process. Statisticians at companies like Novartis have described using R for pharmaceutical development. And with pharmaceutical companies embracing modern statistical methods and graphical techniques in support of FDA submissions, the use of R is only likely to grow further.

(Note: Special thanks go to Marc Schwartz for this r-help post, which was the source for many of the resources linked in this article.)

Using R as a scripting language with Rscript

In a recent posting to r-devel, Prof. Brian Ripley of R-Core announced an interesting development (pun intended) for future versions of R.  Several of the core utility scripts, such as INSTALL and REMOVE, have been converted from sh scripts or Perl into native R code.

You might not think of R as a utility scripting language, but it has many useful features that make it an excellent candidate for such a task, not least powerful string-processing tools and functions for manipulating file names. In fact, if you're familiar with programming in R, but (like me) can never remember the exact syntax for a loop in bash, you might be tempted to create your own command-line tools using R as the scripting language.

In Unix-like systems (including MacOS X), this is easy to do.  Create an ordinary text file where the first line is #!/usr/bin/Rscript (replacing /usr/bin with your actual R executable installation directory).  Then on the following lines include ordinary R commands.  Set the executable bit of the file (with chmod +x), put it in your PATH, and you can then use it as an ordinary command-line tool.

Rscript is a noninteractive variant of the standard R command, designed for just this use.  Here's an example (from the Rscript documentation) that allows you to install packages from the shell prompt, without having to invoke interactive R:

#! /usr/bin/Rscript --vanilla --default-packages=utils
args <- commandArgs(TRUE)
res <- try(install.packages(args))
if(inherits(res, "try-error")) q(status=1) else q()

If you have some tasks that you need to automate from the command-line, Rscript might be a useful tool to make use of your R programming skills, without having to dive for the bash documentation. And as a bonus for scripts you intend to distribute to other R users, your script will be platform-independent, easy to maintain, and may even run faster than alternative scripting languages on platforms like Windows.

Resources for learning R quickly

A reader of Andrew Gelman's blog asked a pertinent question: How can busy economists and political scientists learn R quickly?  Several useful resources were posted in the comments section.  I'll review the various suggestions in postings over the next few days, but for now check out the links for some handy tutorials and manuals relevant to any beginning R user, not just the economic types.

Of packages, and quality

A quote from SAS's Anne Milley (director of technology product marketing) in the recent NYT article about R has stirred up a bit of controversy on open-source blogs and mailing lists.  The quote in question is this:

“We have customers who build engines for aircraft. I am happy they are not using freeware when I get on a jet.”

Many commenters pointed out that "freeware" isn't an appropriate description of R, which is in fact open source software. The defining feature of R isn't that it is available without cost (in fact, some vendors sell supported versions of R for a subscription fee), it is that it is open software whose source code is freely available to all to inspect, verify, and modify.  (The GNU foundation provides an essay making the distinction between free and open source software better than I can.) In response to the controversy, Milley posted a followup on the SAScom magazine blog:

My remark reflects a key difference between R and SAS, that of support, reliability, and validation. Customers value SAS for many things, including our extensive testing, documentation, 24/7 support, and training. In contrast, the quality of proliferating R packages is varied and uneven, especially in complex analytical modules. Mistakes in these packages can lead to misleading results, even for experienced users.

REvolution Computing provides support and validation services for R so that it can be used in environments where reliability and quality must be assured -- I'll have more to say about that in a subsequent post.  But for now I'd like to focus on the narrow aspect of the aforementioned "proliferating R packages".

This really does seem like a specious argument to me. Just as R has many packages contributed by users and available for download, there are thousands of sites where one can download SAS code and procedures for different kinds of analyses.  In either case, whether one trusts the results is a function of one's trust in the source of the code, and the collective evaluations of those who have inspected the code and the results.  The number of packages available for R has no bearing on the quality of R, in the same way that contributed code for SAS has no bearing on the quality of SAS itself.

To be fair, R's nomenclature does confuse the issue a little. In R, there are a number of standard packages (the "base" and "recommended" packages) which, as with R itself, are part of the official R distribution and are under the oversight of the R core group.  (These are of a different class than the contributed packages which are downloaded separately from CRAN.)  These standard packages have undergone the same degree of rigorous scrutiny as the core of R itself, and by statistical community has repeatedly verified the results of these packages against the statistical literature.

But is this "community standard" enough for R to be used in validated environments, such as the analysis of clinical trials where SAS is often used?  Since R already is used in such environments the answer is self-evident, but that's a topic I'll pick up in more detail in another post.

R Video Tutorial from Decision Science News

The website Decision Science News has published two introductory tutorials about R.  Part 1 is a simple demonstration of downloading and installing R on Windows, and shows how to run simple mathematical and graphics commands.  Part 2 focuses more on reading in data and data analysis.  The sound quality isn't great, but if you have 20 minutes to spare these videos make for a clear and concise introduction to R for new users. 

Course: Intermediate Bioconductor: Analyzing Microarray and Sequencing Experiments

The Fred Hutchinson Cancer Research Center in Seattle is hosting an intermediate Bioconductor course January 21-23. This course will demonstrate how the Bioconductor project provides a comprehensive framework for analyzing mircoarray and sequencing data.

For more information and to register: Intermediate Bioconductor: Analyzing Microarray and Sequencing Experiments.