Revolutions

The R-Files: Paul Teetor

"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.

 

Name: Paul Teetor Profession: Quantitative developer (freelance) Nationality: American Years Using R: 7 Known for: Author of R Cookbook (O’Reilly Media, 2011)

An active member of the R community, Paul Teetor is a quantitative developer and statistical consultant based in the Chicago area. He’s been using R for seven years, during which time his contributions to the community have been significant -- particularly in the field of finance. He’s currently a freelance consultant largely focused on time series analysis. Teetor is also the author of the popular R Cookbook, which was published by O’Reilly Media this past March and offers new users over 200 “recipes” for performing more efficient data analysis with R.

He was first drawn to R for the flexibility it offered him in developing trading systems. Citing his own background in software engineering and the need to perform sophisticated statistical analysis in a programmable -- and cost-effective -- environment, Teetor said that R emerged as the perfect fit for him. Since then, he has performed the majority of his financial analyses in R and has also emerged as a leading evangelist for the community. He gradually collected a catalog of tricks and techniques for R, many of which were compiled into the R Cookbook. He's been a participant at conferences such as the Joint Statistical Meetings and the R/Finance Conference where he evangelized the role of R in quantitative finance. Some of those talks and papers are available on his website.

“Prior to R, I did most of my statistical analysis in Excel -- and occasionally SAS,” said Teetor. “However, performing statistical analyses for financial tables in either was extremely tedious and puts you in a specific box. R is a modern, programmable language, so I can make it do what I need it to do in a timely manner. It’s been a pleasure to be able to take what I’ve learned from R and share it with other community members - and to continue learning new tips and tricks from them as well.”

Teetor uses R for the majority of his finance work because, as he puts it, it does things other languages “simply cannot do.” He cited the example of hedge ratio calculations which benefit from the flexibility of R, a topic on which he gave a lightning talk at R/Finance this past summer. He was also quick to credit fellow R user Jeff Ryan (whom we profiled here earlier this year) as an influential member of the R community, citing his finance packages as particularly useful. “I use nearly every finance package he’s written, they’re incredibly helpful and greatly streamline the process of R-based financial analysis.”

When asked about the relationship between financial analysis and the rise of the data science movement, Teetor noted, “People in data science are experiencing what financial analysts have experienced for years: out of the box data analysis is not realistic. You need to incorporate a heavy amount of custom statistics, something that’s not easy to do with a commercial product where you can’t get to the source code. Data scientists need a way to construct custom analyses and R gives them that opportunity. Nothing else on the horizon that can compete with that, in terms of finance or the wider field of data science.”

Looking ahead, Teetor sees a bright future for the continued evolution of R. Since there is no real alternative on the market, he argues, R’s potential for future growth is nearly unlimited. He did, however, cite R’s capacity (or lack thereof) for software engineering as one possible area of improvement. “When R was originally envisioned, it wasn’t thought of as a vehicle for software engineering. Nobody expected people to keep their scripts as opposed to just throwing them away. As it’s grown though, people are building larger, more complex systems with longer lifetimes.” It’s an area that Teetor cites as one of the main struggles with R today, but also one which he cites as a great opportunity on which to innovate.

The R-Files: Martyn Plummer

"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.

 

Name: Martyn Plummer Occupation: Statistician at International Agency for Research on Cancer Nationality: British Years Using R: 16 Known for: Member of R core group; member of R Journal editorial board

Martyn Plummer is a longtime contributor to the R community and a member of the R core group, which consists of 20 members that help oversee the continued evolution of the project. Plummer also serves on the editorial board of the R Journal, the official journal of the R project. By day, he serves as a Statistician and Epidemiologist at the International Agency for Research on Cancer (IARC), based in Lyon, France.

Plummer, who has been using R since 1995, has developed or contributed to a number of popular packages, including coda for analyzing Markov Chain Monte Carlo output, JAGS, a clone of the popular WinBUGS software Bayesian analysis and Epi, which provides functions for epidemiologists and accompanies an annual course that aims to introduce epidemiologists to R.

He has also incorporated R into his work at IARC, where he works in the Infection and Cancer Epidemiology group. Much of the work of this group is focused on human papillomavirus (HPV), which causes half a million cases of cervical cancer per year worldwide.  Plummer and his colleagues use R (including his own Epi package) to analyze epidemiological studies of HPV infection and try to tease out some aspects of HPV natural history that are difficult to understand without statistical modeling, such as whether different HPV types interact with each other. He also relies heavily on R’s graphical capabilities for visualizing data in scientific publications.

Prior to R, Plummer worked primarily with S+ for analyzing data. He had been working in Cambridge, United Kingdom in the Biostatistics Unit at the Medical Research Council when he was offered a position at the IARC in Lyon. He recalls the transition, and how his new position introduced an entirely different computing environment. Soon after moving, he was introduced to the recently-formed R project by his colleague David Clayton.

“From the beginning, I saw enormous potential in R,” says Plummer. “While I was accustomed to S+, it wasn’t long before I completely switched over to R. It was and continues to be unparalleled in its flexibility in terms of data analysis.”

Plummer also points to R’s extensible nature as one of its defining features. As a modern language, R is able to effectively adapt to the changing nature of data analysis in an era of increasingly large, unstructured data sets. “One of the most important features of R is that it’s built around the data; it’s designed for programming with data, so it can take these developments in stride,” he says.

He went on to describe a recent article in the R journal that analyzed 18 months’ worth of text from the R mailing lists and identified relationships between prominent members of the R community based on the topics they discussed. Plummer cites it as an example of R’s ability to keep up with the ever-changing notion of “data.”

“10 years ago, I would have never called such an amalgamation of text a ‘data set,’” he says. “Today, though, we find ourselves in a situation where we can elicit structure from large and complex data sets and glean meaning from it.”

When asked about how he sees the R project evolving in coming years, Plummer speaks of a delicate yet effective balance. “R manages a difficult equilibrium; it’s partly a frontier for innovation in statistical computing, yet it’s also a stable platform for data analysis. It’s unique in this regard and I don’t see it facing serious competition for quite some time.”

He sees the current situation being maintained at least over the next few years, though one challenge for R users is to navigate the increasing number of contributed packages.  While there’s incredible innovation being done for a diverse range of functions, Plummer says, there are also opportunities for the community as a whole to pool and share their work.

“One of the most important and oft-overlooked values of the R community is its interdisciplinary nature,” he says. “It’s remarkable to be able to collaborate with so many talented people from a diverse range of fields. We’re all statisticians, but statistics has a terrible tendency to fragment by subject matter. R gives us all a common platform and brings us together to encourage innovation.”

The R-Files: Jeff Ryan

"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.

 

Name: Jeff Ryan Profession: Owner/Principal at Lemnica; Committee Member at R/Finance Nationality: American Years Using R: 8 Known for: R/Finance Conference, quantmod and xts packages

Jeffrey Ryan is a Chicago-based quantitative software analyst and avid R user. He is perhaps best known in the R community as one of the primary organizers of the annual R/Finance conference. By day, he’s Principal at Lemnica, a firm that specalizes in developing quantitative software systems for financial market trading. In addition to his efforts on the R/Finance organizing committee (of which Dirk Eddelbuettel, whom we profiled here earlier this year, is also a member), he has also developed a number of popular R packages for financial analysis -- most notably quantmod and xts.

Ryan first started using R in 2002 as an undergraduate at the University of Illinois - Chicago, where he studied economics and finance. He described his frustration with cumbersome proprietary tools and recalls his search for a more extensible programming language with which to work. "At that point, I did most of my work in perl and Excel/VBA," he says. "R was not particularly well known at that point, but I recognized the great potential it had for adding deeper statistical insights into my analyses."

After receiving his Bachelor’s degree from UIC, Ryan went to work for a Chicago-based hedge fund as a floor trader on the options exchange. While he wasn’t actively using R in his work at that point, he continued to experiment in his free time by building trading tools and models in R.

From there, Ryan began using R more frequently and became an increasingly active participant on the R-SIG-Finance mailing list. He recalls being particularly impressed by the work of Diethelm Wuertz, whom he credits as one of his major influences from the R community. Over time, Ryan took the lead on developing an all-in-one R package for quantiative trading models: quantmod.

The quantmod package is designed to assist quantitative traders in developing, testing and deploying trading models and has been adopted by big and small traders around the world. Says Ryan of quantmod, "I come from a quantitative trading background. When I was in school, there was no single package for building and testing a trading model. I saw the chance to do something like that with R, which is where quantmod was born."

"The package has evolved concurrently with R and is one of the more popular packages today – with the website serving as a gateway for new users to R and finance. I’m particularly honored to see the traction quantmod and other tools have gained amongst professional quants in the community, as well as how instrumental it has been in bringing new users into R."

In addition to quantmod, Ryan has developed numerous R packages and has been a contributor to many more. Highlights include xts, or eXtensible Time Series which is used to manage large financial time-series, and IBrokers, which is geared towards real time trading. He is also working on a new collection of packages called “indexing”. According to Ryan, indexing is an abstraction of data.frames that allow fast queries on data that do not fit in memory.

When asked to comment on the significant role he’s played in developing R’s capabilities for quantitative finance, Ryan is quick to credit other members of the R community. In addition to Diethelm Wuertz and Dirk Eddelbuettel, Ryan specifically mentioned Josh Urlich (TTR, LSPM) and Brian Peterson and Peter Carl (PerformanceAnalytics) for the work they’ve done in developing finance-focused packages in R, as well as their key contributions to the success of the R/Finance conferences.

The 2011 R/Finance conference was held this past April 29-30, drawing more than 250 attendees from across the globe. The 2012 conference has already been confirmed for May 17-19 of next year – once again to be held in Ryan’s hometown of Chicago. 

The R-Files: Jeroen Ooms

"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.

 

Name: Jeroen Ooms Background: Ph.D. Candidate, Statistics, UCLA Nationality: Netherlands Years Using R: 3 1/2 Known for: Developing web applications for popular R packages including ggplot2, lme4, stockplot and irttool

Jeroen Ooms is a statistical consultant and R enthusiast currently pursuing a Ph.D. in Statistics from UCLA. Jeroen earned a B.S. in Psychology from Utrecht University in the Netherlands in 2007. It was during that time that he made his first serious foray into the world of data science, taking several statistics courses as an undergraduate. He ultimately decided to pursue a degree in that field, earning a Master's Cum Laude in Statistics and Methodology in 2009, also from Utrecht University. 

It was during his Master's studies that Jeroen first started working with R. "Prior to learning R, I had done most of my programming work with Java," he says. "R is a very different environment and I was a bit skeptical at first. Initially, R looked to me like an advanced calculator that lacked important features found in most programming languages, like object oriented programming and direct memory reference. Eventually, though, I came to recognize R's simplicity as one of its greatest strengths: it is easily picked up by students, professionals and researchers and is supported by a lively community of developers and fellow users. Furthermore, R is unique in that it is both suitable as a development platform and a production platform for innovatory methods. After seeing how R is being used in a diverse range of fields and applications, I started to really appreciate the software."

After completing his M.S. at Utrecht, Jeroen looked to diversify his academic background. To that end, he spent six months as a visiting scholar at the UCLA Statistics Department from 2009-2010, studying under the renowned Jan de Leeuw. R has become the primary statistical software package used at UCLA, and plays an important role in many courses and research projects. Jeroen's work at UCLA focused primarily on developing R-based web applications for statistical education. During this time, Jeroen was accepted into the Ph.D. program at the UCLA Department of Statistics, which he began in September 2010.

Outside the classroom, Jeroen has contributed to R by developing a number of R web interfaces for popular CRAN packages, including ggplot2 for graphical exploratory analysis, lme4 for online random effects modeling, stockplot for stock predictions and irttool.com, an R web application for online IRT analysis. His web interfaces are being used in many applications, from charts tracking gender development through puberty to demographic studies.

Looking towards the future, Jeroen plans to continue developing in and contributing to the R environment as he pursues his Ph.D. from UCLA. “R is an open platform that everybody can use and contribute to,” he says. “The lifecycle of going from a good idea to production is extremely short, and greatly stimulates innovation. Once you publish your code in CRAN, others can immediately start using it or further develop it. Using R is unlike any other statistical software package: one automatically becomes part of an enthusiastic community that encourages you to share ideas, contribute code, and keep learning every day. It is science at its best”

The R-Files: Martin Morgan

"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.

 

Name: Martin Morgan Profession: Senior Staff Scientist at Fred Hutchinson Cancer Research Center Nationality: Canadian Years Using R: 7 Known for: Director of the Bioconductor project

Martin Morgan is a Senior Staff Scientist at the Fred Hutchinson Cancer Research Center (FHCRC) in Seattle. He is perhaps best known for running the Bioconductor project, which has emerged as the tool of choice for scientists conducting analyses of high-throughput genomic data.

His use of R stems from his early days working at FHCRC, where one of his colleagues was R project co-founder Robert Gentleman. Gentleman also founded the Bioconductor project. While he had worked with a fair share of statistical analyses as he pursued his Ph.D. in Evolutionary Genetics at the University of Chicago, he was especially impressed by R’s flexible, powerful nature.

Morgan describes his first real interaction with the wider R community, which came via the R and Bioconductor mailing lists. As he puts it, “I was impressed with both the level of sophistication and engagement of the users on the mailing list. I remember realizing that the people responding to questions were the very authors of the packages in question.”

As he became more familiar with R, Morgan took on an increasingly active role in Bioconductor, eventually taking the lead on the project. Morgan and his team of researchers and analysts at FHCRC have worked to develop a number of R packages for genomic research, including the popular Rsamtools, IRanges, GenomicRanges and GenomicFeatures packages for importing and using next-generation sequence data. Morgan was quick to credit Michael Lawrence, Herve Pages, Patrick Aboyoun, and Marc Carlson in particular for their efforts in developing these packages.

Morgan cites significant insights from the first R project he embarked on. “I’d initially written a C program that was literally thousands of lines of code. After a while, I decided to try programming in R; since the facilities for data input and optimization were already there, I was able to do it in just six lines of R code.”

When asked about particular areas in which he would like to see R evolve, he responded, “Because R is a programming language, there’s a risk that people lose sight of its biggest strengths — namely, its nearly limitless capacity for statistical analysis.” He paused, before continuing, “Some conservative elements of the R project help reproducible research, but encouraging more interoperability between packages would improve R’s core functionality.”

The R-Files: Dirk Eddelbuettel

"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.

 

Name: Dirk Eddelbuettel Background: Ph.D. (EHESS, France), Quantitative Analyst Nationality: Canadian Years Using R: About 15 Known for: RQuantlib, Rcpp/RInside, R packaging for Debian/Ubuntu, Task           Views for Finance and HPC, CRANberries

Dirk Eddelbuettel is an active member of the R community who has been contributing packages to CRAN for nearly a decade. An early adopter of Linux, he was first introduced to R in 1996. "The world was smaller then--much of the open source community were reading the same mailing lists and news groups at the time," he said. "I had known about S for several years, and was starting to use S+ in my first job out of graduate school. So when I learned about R,  there was a natural curiosity about this implementation and what it could do as a next step for data analysis."

By day, Eddelbuettel is a financial econometrician with a Chicago-based trading firm. He uses R heavily in his work and is quick to praise its data analytics capabilities. "R is built for programming with data; there's no other tool like it. As a quant, I use R for almost all analytical work. I go back and forth to C++ for high-performance work--which at times can be integrated with R--but it doesn’t compare to R’s capacity for data analysis.”

Eddelbuettel is also an early contributor to the Debian project, where he got in contact with Doug Bates. It was Bates, who was already a long-time S user and R Core contributor himself, who helped Eddelbuettel with some early steps in R. He became increasingly interested in R, started to co-maintain it in Debian with Bates, and began to use it for more and more projects. Since he already had a background with S, he was not as affected by the learning curve that can come with R and was soon actively developing  packages.

To date, Eddelbuettel has authored or co-authored well over a dozen packages in CRAN, including RQuantLib (an R interface to the QuantLib libraries), digest, RDieHarder,  and the popular Rcpp (C++ classes for extending R with C/C++ functions, which is now co-authored with Romain Francois counting on frequent contribution by Bates and John Chambers) and RInside (for embedding R in C++ applications, now also co-authored with Romain Francois). He has also been a contributor to other R-Forge projects including phylobase, inline and blotter. He is editor of the CRAN Task Views for Finance as well as High-Performance Computing, and serves as an editor at the Journal of Statistical Software.

When asked about what draws him to R, Eddelbuettel echoes a common sentiment. "Behind R stands an exceptional community that’s comprised of many of the world’s brightest, savviest statistical minds. I get so much out of R and I'm happy to be able to contribute a little bit back, be in the form of new packages or the Debian packaging." He adds, "While I do use R during my day job, it's also a hobby of mine. I build packages that I think solve interesting problems and hopefully benefit the community, and I know that there are many others out there with the same mindset, doing the same thing."

Looking to the future, Eddelbuettel acknowledges that it's difficult to predict new developments for R, given that it's driven by a diverse community with a multitude of interests and pursuits. "It's hard to speculate where R will go in the next few years--it's not like there's any official governing body. If you want to see something new in R, do it. That's the inherent beauty of R: it's driven by both the R Core group and the the community at large --especially the researchers and developers that continue to innovate and introduce new things to R."

The R-Files: Saptarshi Guha

"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.

 

Name: Saptarshi Guha Background: Ph.D. in Statistics, Purdue University Nationality: India Years Using R: 6 Known for: Developing RHIPE package for R + Hadoop integration

At just 31 years old, Saptarshi Guha has emerged as a cutting-edge contributor to the R community. Saptarshi earned a Ph.D. in Statistics from Purdue University in June 2010 (his advisor was William S. Cleveland, one of the pioneers of modern data visualization) and his current research focus is the analysis of large data sets. He is working to develop innovative approaches to the visualization and computing of statistical analyses. He has also worked on modeling network traffic for security and developing algorithms to detect human presence in SSH connections.

Saptarshi is best known for developing the popular RHIPE package that integrates the R statistical environment with the Hadoop framework. RHIPE allows R users to compute on terabyte-sized data sets a cluster using the MapReduce framework, thus offering the best of both worlds to users seeking to leverage the strength of R and Hadoop. People with very large data sets stored in the Hadoop Distributed File System can now easily process the data on hundreds or even thousands of nodes in parallel, using only the R language (no need to learn Java). They can even apply the statistical algorithms in R to boot.

While Saptarshi has studied the intersection of computer science and statistics for well over a decade — in the U.S. as well as his native India — it was not until he began his doctoral research at Purdue that he learned R. Given his statistical background, he quickly took to the language. "R is one of the most expressive languages I’ve ever encountered, and it’s perfectly geared towards comprehensive data analysis," he says.

In addition to RHIPE, Saptarshi has worked on packages that serialize R objects for operability with other languages, including Python, Java and C. He also wrote a package that saves R objects in a flexible data format so that individual objects can be lazy loaded. He has used R to perform statistical analysis on a wide array of topics, from network security modeling to generating reports and graphs for monthly expenditures and weight loss programs.

“The real beauty with R is that it’s constantly evolving,” Saptarshi says. “Is it perfect? No. But it’s being constantly refined by some of the most brilliant statistical minds today.” To that end, Saptarshi is working on several packages that will bring features of distributed computing to users working within the R environment. He’s also working on a package that integrates R with HBase and give users a fast query distributed data store that applies MapReduce computations across the data using RHIPE.

On October 12, Saptarshi will be delivering a talk at Hadoop World, where he’ll demonstrate the use of RHIPE to analyze 190 Gb of VoIP network data. (This project was joint work with Jin Xia and William S. Cleveland.) When you make a call over Skype, Google Voice, or even a landline routed over the internet, the call quality is of primary importance. One of the major factors influencing call quality is timing: when you speak, your voice is sampled every 20 milliseconds, but on the receiver’s end it might not arrive quite so regularly – perhaps 5ms too early or too late.

This “jitter” in packet arrival times degrades the audio quality; to investigate this, Saptarshi used R code to identify which packets corresponded to a single call. He used R’s robust regression algorithm to remove the effect of the gateway. In this way, he was able to process the data in a matter of minutes stored across a cluster of eightcomputers to assess the overall call quality metrics of the system.

If you’re interested in attending and are in the Greater New York area, you can register here. Cloudera also published an interview with Saptarshi last month.

The R-Files: Hadley Wickham

"The R-Files" is an occasional series from Revolution Analytics, where we profile prominent members of the R Community.

 

Name: Hadley Wickham Profession: Assistant Professor of Statistics, Rice University Nationality: New Zealand Years Using R: 10 Known for: Developing popular R packages including ggplot2, plyr, reshape; creator of crantastic.org; author of ggplot2: Elegant Graphics for Data Analysis

An Assistant Professor of Statistics at Rice University, Hadley Wickham has been using R for over 10 years. In that time, he has emerged as one of the most prominent members of the R community. Wickham’s research focuses primarily on data analysis and developing innovative tools that facilitate the understanding of complex statistical models through visualization. He is known for developing some of today’s most popular R packages, including ggplot2, plyr and reshape.

Wickham’s R packages are used by a wide range of high-profile institutions for complex statistical analyses, including the U.S. Air Force Research Laboratories, Mozilla Labs and the Vanderbilt University Center for Human Genetics Research. ggplot2 was also used earlier this year by NYU doctoral student Drew Conway to visualize the Wikileaks data, offering far greater insight into troop movements and conflict hot zones in Afghanistan than was previously available to the public. 

He first encountered R in an undergraduate statistics course at the University of Auckland—the birthplace of R. Admittedly, he found the language a bit overwhelming at first. Of his early experiences with the language, he says, "I remember when I first used R, I found it to be the most enormously frustrating—yet magical—computer language I had ever encountered."

For Wickham, the frustration came from R's organic growth, which—he’s quick to add—is also its greatest strength. Thanks to its open source roots and community of users and contributors, R has grown without a roadmap or goal. While you can do just about anything with R, Wickham notes that it can be an intimidating language for new users to learn, and does not have the most user-friendly interface. 

As a doctoral student at Iowa State University, Wickham became increasingly familiar with—and fond of—R. Working under the tutelage of Di Cook and Heike Hoffman, he began to master the language’s wide suite of functional capabilities and eventually started to develop R packages to contribute back to the community. To date, Wickham has developed 20 packages on top of R, with an emphasis on visualization and manipulation of data. 

In his own classes, Wickham uses R to teach a new generation of statisticians. Among some of his favorite subjects for teach with R are tracking the recent American sub-prime mortgage crisis and using data released by the Social Security Administration to study the evolution of baby names. "R is the most powerful statistical computing language on the planet," Wickham says. "Students today have such an advantage over previous generations; R has nearly limitless statistical capability and allows them to perform a far greater range of analyses than was previously possible."

While critics argue that R is not as "pretty" a programming language as Java and C++, Wickham sees that as one of its greatest strengths. "One of the things I like about R is that it is so pragmatic," he says. "It doesn’t try to hold to theories that might be lovely and elegant, but don’t work in practice. It’s the result of a brilliant community coming together to make it work, and it’s constantly evolving."

Wickham cites the community behind R, which is over 2-million users strong, and their willingness to support one another as the primary driver for its great success. "When you ask a question, you’re getting free help from the people who wrote R. Not only are they great programmers, they’re internationally recognized statisticians who have contributed and developed leading theories of modern statistics."