by Joseph Rickert

Last week, I mentioned a few of the useR tutorials that I had the opportunity to attend. Here are the links to the slides and code for all but one of the tutorials:

Regression Modeling Strategies and the rms Package - Frank Harrell

Using Git and GitHub with R, RStudio, and R Markdown - Jennifer Bryan

Effective Shiny Programming - Joe Cheng

Missing Value Imputation with R -Julie Josse

Extracting data from the web APIs and beyond - Ram, Grolemund & Chamberlain

Ninja Moves with data.table - Learn by Doing in a Cookbook Style Workshop - Matt Dowle

Never Tell Me the Odds! Machine Learning with Class Imbalances - Max Kuhn

MoRe than woRds, Text and Context: Language Analytics in Finance with R - Das & Mokashi

Time-to-Event Modeling as the Foundation of Multi-Channel Revenue Attribution - Tess Calvez

Handling and Analyzing Spatial, Spatiotemporal and Movement Data - Edzer Pebesma

Machine Learning Algorithmic Deep Dive - Erin LeDell

Introduction to SparkR Part1, Part 2 - Venkataraman & Falaki

Using R with Jupyter Notebooks for Reproducible Research de Vries & Harris

Understanding and Creating Interactive Graphics Part 1, Part 2- Hocking & Ekstrom

Genome-Wide Association Analysis and Post-Analytic Interrogation Part 1, Part 2 - Foulkes

An Introduction to Bayesian Inference using R Interfaces to Stan - Ben Goodrich

Small Area Estimation with R - Virgilio Gómez Rubio

Dynamic Documents with R Markdown - Yihui Xie

Granted that since the tutorials were not videotaped they mostly fall into the category of a "you had to be there" experience. However, many of the presenters put a significant effort into preparing their talks and collectively they comprise a rich resource that is worth a good look. Here are just of couple examples of what is to be found.

The first comes from Julie Josse's Missing Data tutorial where a version of the ozone data set with missing values is used to illustrate a basic principle of exploratory data analysis: visualize your data and look for missing values. If there are missing values try to determine if there are any patterns in their location.

maxO3 T9 T12 T15 Ne9 Ne12 Ne15 Vx9 Vx12 Vx15 maxO3v 20010601 87 15.6 18.5 NA 4 4 8 0.6946 -1.7101 -0.6946 84 20010602 82 NA NA NA 5 5 7 -4.3301 -4.0000 -3.0000 87 20010603 92 15.3 17.6 19.5 2 NA NA 2.9544 NA 0.5209 82 20010604 114 16.2 19.7 NA 1 1 0 NA 0.3473 -0.1736 92 20010605 94 NA 20.5 20.4 NA NA NA -0.5000 -2.9544 -4.3301 114 20010606 80 17.7 19.8 18.3 6 NA 7 -5.6382 -5.0000 -6.0000 94

These first two plot from made with the aggr() function in the VIM package shows proportion of missing values for each variable and relationship of missing values among all of the variables.

The next plot shows a scatter plot of two variables along boxplots along the margins that show the distributions of missing values for each variable. (Here blue represents data that are present and red the missing values.) The code to do this and many more advanced analyses is included on the tutorial page.

It looks like missing values are pretty much spread among the data.

Frank Harrell's tutorial provides a modern look at regression analysis from a statisticians point of view. The following plot comes from the section of his tutorial on Modeling and Testing Complex Interactions. If you haven't paid much attention to the the theory behind interpreting linear models in a while you may find this interesting.

Finally, I had one of those "Aha" moments right at beginning of Ben Goodrich's presentation on Bayesian modeling. MCMC methods work by simulating draws from a Markov chain whose limiting distribution converges to the distribution of interest. This technique works best when the simulated draws are able to explore the entire space of the target distribution. In the following the figure, the target is the bivariate normal distribution on the far right. Neither the Metropolis nor Gibbs Sampling algorithms come close to sampling from the entire target distribution space, but the Hamiltonian Monte Carlo "NUTS" algorithm in the STAN package displays very good coverage.

For reasons I described last week I believe that this year's useR tutorial speakers have raised the bar on both content and presentation. I am going to do my best to work through these before attending next year's conference in Brussels.

Intro to SparkR part 0: http://databricks.com/ce

Intro to SparkR part 1: https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/4445213449192764/386285766047251/3865211757615045/latest.html - use Import Notebook in upper right

Intro to SparkR part 2: https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/4445213449192764/386285766047279/3865211757615045/latest.html - use Import Notebook in upper right

Posted by: Harris McGehee | July 07, 2016 at 10:22

So no videos?

Posted by: myschizobuddy | July 08, 2016 at 03:40

Here is a better link for the SparkR tutorial.

https://databricks.com/blog/2016/07/07/sparkr-tutorial-at-user-2016.html

Posted by: Harris McGehee | July 11, 2016 at 15:06