Revolutions

New versions of plyr, ggplot2 released

Hadley Wickham has announced that new versions of his popular grammar-of-graphics charting package ggplot2 and his general-purpose data reshaping tool plyr for R are now available. 

plyr boasts several new features, most notably a new join function which should simplify what can sometimes be a difficult process in R: merging two data sets. A simplified SQL-like terminology should make it much easier to specify how the datasets should be combined. But the most significant changes to plyr may be behind the covers -- this version offers a number of performance improvements that should make the process of restructuring large data frames much easier.

The ggplot2 package will also benefit from these performance improvements, and also includes a number of bug fixes to improve upon the previous version.

Hadley Wickham: ggplot2 and plyr

June 2010 edition of R Journal available

The latest edition of the R Journal is now available. This issue includes in-depth articles on the packages IsoGene, glmmBUGS, cshapes, tmvtnorm, neuralnet, glmperm and exactci/exact2x2, plus an example of reproducible research in R. There's also a review of the book A Beginner's Guide to R. The complete edition is available for download in PDF format at the link below.

The R Journal: Volume 2/1, June 2010

Connecting Revolution R to MySQL on Windows

My colleague Saar Golde was having some troubles connecting Revolution R to MySQL on Windows (64-bit). Turned out the problem was the lack of an environment variable. He documented the instructions for fixing the problem on Windows 7, below. Thanks, Saar!

The Problem:

A client is about to send me a couple of large MySQL tables, so I needed to install a MySQL server and connect it to R so I can do some analysis on it.

The Process:

1. Install a MySQL server.

For some reason I could not get MySQL 5.1 to talk to R using the RMySQL package. Not sure what was wrong – maybe it’s the lack of an available MySQL 5.1 server for 64-bit windows. So I opted to use MySQL 5.0 (the exact version is 5.0.91-community-nt MySQL Community Edition).

2. Install the RMySQL package

Downloaded from Revolution’s repository, but CRAN should also work fine.

3. Load RMySQL

This is the tricky step:

> require(MySQL)

does not work right off the bat. You get an error like this one:

Error in utils::readRegistry("SOFTWARE\\MySQL AB", hive = "HLM", maxdepth = 2) :Registry key 'SOFTWARE\MySQL AB' not found

Error : .onLoad failed in 'loadNamespace' for 'RMySQL'

4. Manually add MYSQL_HOME variable to the list of system variables

This is for Windows 7 –  it should not be very different for other versions. Right click on ‘my computer’, choose ‘properties’, click on ‘advanced system settings’, under the ‘advanced’ tab click on ‘Environment Variables…’. Create a new system variable (not a user variable!) named ‘MYSQL_HOME’ and enter the MySQL directory address there. In my case it is ‘C:/Program Files/MySQL/MySQL Server 5.0’ (notice the slash instead of the standard Microsoft backslash).

5. Restart R

If there is an alternative way for R to recognize the change in the system variable, that may be preferable. In any case, restarting works. require(RMySQL) works now.

6. Go have a beer. Repeat if necessary.

Package Update Roundup: Feb 2010

This is a list of new or updated packages that were released for R in February, as announced on the r-packages mailing list. To include other updates on this list, please email David Smith. For a complete list of all updates on CRAN, see the CRANberries archive for February 2010. Follow package name links for ratings and other information on crantastic.org.

ggplot2, the graphics system based on the Grammar of Graphics, has been updated with a new transform and several bug fixes.

GWRM, a new package for fitting Generalized Waring Regression Models, has been released.

interval, a package that calculates weighted logrank tests and nonparametric maximum likelihood estimates for interval censored data, has been updated to improve performance.

mboost, a package for model-based boosting, has been updated with many new features, including new families available for ordinal, expectile and censored regression. 

np, a package implementing nonparametric kernel smoothing methods for mixed data types, has been updated with new nonparametric entropy-based testing methods.

RFLPtools, a new package providing analysis functions for DNA fragment molecular weights and nucleotide sequence similarities, has been released.

rms, the Regression Modeling Strategies package, has been updated to add several new capabilities.

solaR, a new package with calculation methods of solar radiation and performance of photovoltaic systems, has been released.

survey, a package for the analysis of complex survey samples, has been updated to allow database-backed designs can now use replicate weights and to add some multivariate statistics.

tsne, a new package implementing the T-distributed Stochastic Neighbor Embedding algorithm, has been released.

Package Update Roundup: Dec 2009 - Jan 2010

A special double edition of the Package Update Roundup this month! This is a list of new or updated packages that were released for R in December and January, as announced on the r-packages mailing list. To include other updates on this list, please email David Smith. For a complete list of all updates on CRAN, see the CRANberries archive for December 2009 and January 2010. Follow package name links for ratings and other information on crantastic.org.

dcemriS4, a package for medical image analysis, has been released to update package dcemri.

Deducer, a cross-platform data analysis GUI, has been updated.

doMPI, a backend for the foreach package for parallel computing, has been updated. 

exact2x2, a package for exact conditional tests and confidence Intervals for 2x2 tables, was updated to allow optionally give an exact McNemar's test together with odds ratio estimates and confidence intervals.

exactci, a new package that calculates exact tests and confidence intervals for binomial and Poisson tests, has been released.

ff and bit, two packages that help manage large datasets in R by mapping them to disk instead of storing them in memory, have been updated.

fortunes, a package delivering snippets of amusing R wisdom, has been updated (and now includes more than 250 fortunes!)

GGally, a package to draw plot matrices (like pairs does, but using ggplot2), was released.

ggplot2, the comprehensive high-level graphics system for R, has been updated adding several new features.

itertools, a new package of tools for manipulating iterators, has been released. 

mecdf, a package for multivariate ECDF based models, has been released.

micEcon, micEconAids, and miscTools, three packages for microeconomic analysis and microeconomic modelling have been released (these packages were previously combined into a single micEcon package)

mspath, a new package to fit multi-state path-dependent models in discrete time, has been released.

parcor, a new package of tools to estimate the matrix of partial correlations based on different regularized regression methods, has been released.

pgfSweave, a package that improves the process of including graphs in Sweave documents, has been released.

randomSurvivalForest, a package for ensemble survival analysis based on a random forest of trees, has been updated with new model controls.

randtoolbox, a package for random number generation and testing, has been updated with new features.

RobASt, a family of packages for robust asymptotic statistics, has been updated.

Rcpp, the package interfacing R with C++, has been updated to make integration between R and C++ even easier.

sp, a package that  provides class definitions for spatial data, and utilities for spatial data handling and manipulation, has been updated to improve performance.

sqldf, a package for accessing data in data frames using SQL, has been updated.

New features in ggplot2 version 0.85

Hadley Wickham recently updated the ggplot2 package for R to version 0.8.5. In addition to bugfixes and performance improvements, this version introduces some handy new features. Among them: the ability to display mathematical equations as text, automatic legends for color scales,  and user-configurable axis labels and legend titles. The Learning R blog runs through the new features with worked examples.

Learning R: New Features in ggplot2 version 0.8.5

Equivalents to SAS, SPSS modules in R

Bob Muenchen, the author of R for SAS and SPSS Users, has created a handy table listing the R packages that implement the statistical methods included in add-on modules for SAS and SPSS. For example, if you're looking for the equivalent structural equation modeling functions you might be using in PROC CALIS from SAS/STAT or the SPSS "Amos" software, the table points you to the OpenMX and sem packages for R where you can find similar functionality.

By the way, Bob has greatly expanded the materials he provides to support his book at a new website, r4stats.com. You'll also find there some teaser information about his forthcoming book, R for Stata Users. 

r4stats: R-SAS-SPSS Add-on Module Comparison

R's exponential package growth, ctd.

Reader SK has collected the most recent data on R's package growth, through the latest 2.10 release. The three most recent releases fall slightly below the exponential growth line, which isn't altogether surprising (that's a lot of growth to sustain!). Another interesting thing to look at would be the combined rate of new packages submitted to CRAN and packages updated on CRAN: as R packages begin to cover the domain space of possible applications I'd expect updates to be where much of the package activity lies.

For anyone interested, the raw data (as calculated by SK) is available after the jump.

The number of R packages is growing exponentially

The second issue of the R Journal is out now, and in addition to a bevy of contributed articles and some news from the R Core Group on the new help system introduced in R 2.10, there's an invited section called, intriguingly, "The Future of R". In that section John Fox provides an exhaustively researched and insightful review of the history of the R project, its social structure as an open-source project, and an analysis of why the project succeeded and its prospects for the future. A recommended read. One tiny nugget that caught my eye though, is quantitative evidence for a claim that I've made anecdotally for a long time: R is growing exponentially. The proof is in the packages: the chart below shows the number of packages on CRAN for every release since 2001. The Y axis is the number of packages (on a log scale); the X axis is time (annotated by the version number). 

 

Exponential growth. It's tailed off very slightly for the last two releases shown (and 2.10 isn't included, having just been released), but it's a clear indicator of the growth of the R community that creates and submits the wealth of packages on CRAN. (Update Jan 7: An updated chart is available.)

The R Journal: Aspects of the Social Organization and Trajectory of the R Project

10 Must-Have R Packages for Social Scientists

Political scientist Drew Conway (he of the awesome live-tweeting social analysis demo) lists his top 10 must-have packages for social scientists using R. They are:

1. Zelig, for a uniform regression methodology
2. ggplot2, for graphics
3. Statnet and igraph, for social network analysis
4. plyr, for data manipulation
5. Amelia II, for missing data imputation
6. nlme, for nonlinear mixed-effects models
7. SNOW and Rmpi for parallel computing (what, no love for foreach?)
8. xtable/apsrtable for print-ready tables
9. plm, for analysis of panel data, and
10. sqldfm, for using SQL to process R data

Check out the full article for Drew's discussion of each package and why it makes the top 10 for social scientists. 

Zero Intelligence Agents: Must-Have R Packages for Social Scientists