Revolutions

Bay Area R User Group at Strata and PAW

by Joseph Rickert

I always think of Strata Hadoop World and Predictive Analytics World as initiating the Spring conference season here in the San Francisco Bay Area. The rainy season is usually over by the end of March and it is a perfect time to visit. If you are traveling to either of these conferences from out of town and you are an R aficionado please arrange your plans to include one one or both of the Bay Area useR Group, BARUG, meetings that will be held in conjunction with these events.

For the past few years, the folks at O'Reilly and the organizers of PAW have very generously arranged for us to hold meetings on-site at the conference locations. This year, even though the two BARUG meeting will be held within a week of each other, we have manged to assemble a panel of speakers that I expect will rival the line up at the conferences themselves.

Our San Jose, Strata meeting will be held on March 29th. David Smith will begin the evening by describing how R is being integrated into the warp and woof of Microsoft's offerings. Hadley Wickham will present his ideas about managing many models with the help of the tidyr, purrr and broom R packages and H20's Erin LeDell will follow up on Hadley's theme and talk about model management with model stacking or "super learning" as it is sometimes called.  This recent video of Hadley speaking at the WOMBAT conference in February covers some of the themes that he will be presenting. 

 

We have assembled an even more diverse and eclectic line up of speakers for our April 4th Predictive Analytics World meeting in San Francisco. Keith Everett will present the GLMNet model behind his Oscar Predictions model. Then Megan Price, a long-time member and supporter of BARUG, will present the statistical techniques and R packages that the Human Rights Data Analysis Group, HRDAG, uses to compensate for the missing data problems that plague attempts to estimate conflict related casualties in Syria. This will be Megan's first BARUG talk since becoming Executive Director at HRDAG.

After Megan, Blair Hull, legendary trader, founder and managing partner of Ketchum Trading, will talk about market timing, big data and machine learning. Blair is all about algorithms and Ketchum Trading was a founding member of the R Consortium. 

Max Kuhn, co-author of Applied Predictive Modeling and frequent BARUG speaker, will close out the evening with a talk on rule based regression models. 

If you are planning on attending the meetings please register as soon as you can . Space will be limited.

For details and registration for the the San Jose, Strata meeting look here, and for the San Francisco PAW meeting go here. Also note that both Strata and PAW are offering conference discounts for BARUG members. The discount details are on the meeting sites linked to above.

Finally, if you haven seen the "Predict This" the dance video by PAW's Eric Siegel you can find it here.

Bay Area useR Group Lightning Talks

by Joseph Rickert

Earlier this month the Bay Area useR Group (BARUG) held it annual lightning talk meeting. This is by far our most popular meeting format: eight, 15 minute talks (12 minutes speaking and 3 minutes Q & A while the next speaker is setting up) packed into a two hour time slot. The intensity seems to really energize the speakers and engaged the audience. 

Bradley Shanrock-Solberg kicked off the event with delightful example of an R Monte Carlo simulation based on his wildpoker package that you can find on CRAN. I have never seen a more prepared lightning talk presenter: high energy, a royal flush presentation and a four color printed hand out just in case you have trouble keeping up with him for the 12 minutes. In a series of well conceived plots Bradley showed how, for a number of different poker variations, the best hand changes as the game progresses. The number of players who start the game, the number who stay until the showdown, wildcards and many more contingent events dynamically change the value of your hand. Bradley is definitely the guy for your next trip to Vegas.

 

If you are a poker player, you will definitely want to check out his package and supplementary material: his paper on Winning More at Dealers Choice Poker and his example of why rules of thumb fail.

William Sundstrom, professor of Economics at Santa Clara University, gave an entertaining and thought provoking presentation on teaching R to undergraduate Econometrics students. One interesting observation that generated some discussion was that even though today's students are "digital natives" having grown up using intelligent devices of all kinds, many of them are nevertheless "digital naïfs". The following slide, a reprint of an email from one of Professor Sundstrom's students, captures some of their frustration. 

 

David Ouyang MD, a Stanford resident, a guy who sometime puts in 73+ hour work weeks, presented some explorations of the Epic electronic medical records data set he is analyzing in his spare time. The following plot shows the distribution of physician's interactions with the Epic system over the course of a day.

Dennis Noren, a long time BARUG member and contributor showed some results from a recommendation App he is building on top of The Movie database. The following slide from his presentation shows a Shiny dashboard he build to drive a parallel plot. The interactivity really makes the plot useful.

If you are thinking about betting on the Academy Awards you might want to consider Keith Everett's predictions based on his GLMNet Model.

Nelson Auner talked about Modern KPI Tracking in R. My favorite slide describes the behavior of Execs who don't work for companies that sell BI Software:

And that wasn't all of it! We also had a update on data.table from Matt Dowle himself, and introduction to exploring large data sets with Apache Spark from Hossein Falaki.

Many thanks to Earl Hubbell and the folks at Thermo Fisher Scientific who hosted the meeting.

More R User Group Sites

by Joseph Rickert

Last month I wrote about how several R user groups were making use of GitHub and listed some sites that I thought had interesting material. A few readers were kind enough to point out sites that I had missed; so I would just like to give a couple of "shout outs" here. First of all, I should acknowledge LondonR as a leader in R user group web properties. Their website is very nicely done and valuable. The links to the presentations made to LondonR that they have collected comprise quite a resource to the community.

 

The following slide which comes from the first presentation on the list, the workshop on Network Analysis in R, is an example of what you can find there.

The LondonR Github repository can be found here.

The repository for the Berlin R User Group also shows great promise.  BerlinRUG is beginning to build out a clean, functional webpage and populate it with some very nice material. Honorable mention should go to R-ladies of San Francisco who are searching other GitHub sites for useful material like this data science tutorial from Jonathan Bower, SevillaR, and the Las Vegas R Users group. 

 

R User Groups in Poland

by Przemyslaw Biecek

The first meeting of R users in Poland took place in Wroclaw in 2008. It was a one-day conference with 27 participants and 6 talks.

Today, we have three large groups of R users in major Polish cities (according to meetup.com there is 640 users in SER - Warsaw, 235 in eRka - Cracow and 64 in PAZUR - Poznań). And there are also some data science groups that are full of R users anyway (450 users in Wroclaw and 115 in Łódź).

We have tried different forms of meetings. Some of them were working; some of them were not. Below I will summarize what we have learnt from them.

Today, the most common formula for our meetings is: two short talks (around 30 minutes) with 30 min break for pizza and networking. The main advantage of this formula is that it is very easy to organize. Just find a large room, order pizza or other snacks and find speakers. You can meet even without speakers, but with them the meeting is much nicer.

In Warsaw we have meetings every month. They usually start at 6pm. Currently, for meetings, we are using a class room at Warsaw Technical University. Previously we were meeting in pubs, coffee shops, other places, but finally it turns out that we need more space, good projector, good access to public transport, and we need place that rather quite silent (in coffee shop you sometimes compete with noise from coffee maker).

Among attenders there are students, some stuff (just few people) but majority of participants are graduates that are now working in data-oriented businesses. So 6pm is a good starting time since we can meet after work. We finish around 8pm – good time for after-party.

We also have organised hackatons (two in Warsaw, three in Cracow - watch the promo),

 

 

which were pretty cool, but such meetings require much more preparation and organization. Some of them resulted in interesting outcomes, like this diagram of flow between parties in Poland in previous cadence of Sejm. And there is fun in working with completely new people. After few hours it turns out that you can easily collaborate and you have learned a lot.

Talks are very diversified. Some of them are in Polish some in English, some are technical some are applied and some related to methodology. The 30 minutes per talk turns out to be enough to get people interested and ignite discussions.

Once we have tried a cinema–like meeting. Together (around 40 people) we were watching videos about deep networks, eating pizzas and sharing experience related to what we have heard. It wasn’t bad, but after all it is much better to have a live speaker.

After all I think that it is good to try different forms. It is interesting to see what is working and what is not.

The key ingredients are of course speakers. So I just would like to send thanks to our recent roster.

Karol Kubicki - Magic with pivotTable; Paweł Cejrowski – The package elastic; Zawistowski Paweł - Machine learning in production; Kornel Kiełczewski - From idea to model, What Could Possibly Go Wrong?; Markus Gesmann - Experience vs. Data; Emilia Kalarus - Non life insurance in R; Adam Wróbel - Stochastic mortality modelling; Piotr Migdal - Jupiter; Adolfo Alvarez - 5 lessons I have learned at Analyx; Wit Jakuczun – The protector – how to protect R code; Roger Bivand - Spatial data and R; Bartosz Meglicki - Machine learning and robotics; Wit Jakuczun - R in production; Paweł Chudzian - SVM in action; Piotr Dittwald - R and genomic rearrangements; Filip Grotkowski - R, robots and intelligent warehouses; Piotr Przybyła - Text mining of Sejm RP; Bogumił Kamiński - Integration of R and localsolver; Tomasz Żółtek - R and psychometrics; Paweł Teisseyre - Multilabel classification; Olga Mierzwa - price elasticity and the systemfit package; Teresa Ponikowska - R5 classes; Marek Gagolewski - the stringi package; Cezary Dendek - Shiny in buissness; Krzysztof Misztal - The package CEC (Cross Entropy Clustering); Zygmunt Zawadzki – R optimization; Piotr Ocalewicz - R and KNIME (video); Kamil Stanuch - Justin Bieber vs. The Beatles - text mining; Maciej Eder - The package Stylo (part 1); Ruprech von Waldenfels - Comparing the functional domain of linguistic elements across languages; Maciej Eder – The package Stylo (part 2); Jarosław Iwaniec - About hackathon. eRka and Onet.pl; Maciej Beręsewicz - RcppArmadillo; Alicja Szabelska - ampliQueso; Jakub Nowosad - machine learning with caret; Paweł Kleka - Item response theory in R, Mikołaj Bogucki and Krzysztof Jędrzejewski - Performance analysis of assessment content in an English language learning product; Łukasz Smaga - Functional data, Tomasz Górecki - Profiling in R; Other presentation from 13 PAZUR meetings

And we would like to thank our sponsors for they kind support (sometimes we get food and sometimes a good speaker): Revolution Analytics (now Microsoft) has supported us since 2009. Now, they are supporting all three polish R user groups!!!; datahero.tech; SmarterPoland.pl; WLOG Solutions;Codilime.com; Nationale-Nederlanden; Triple A - Risk Finance; INIME; Hub:raum; KoalaMetrics; Analyx.

The community is large and has big dreams. In 2014, we had three day Polish R conference called PAZUR. This year (October 12-14) we are going to organize European R users meeting [eRum] in Poznań. So be prepared to meet us at eRum!

R User Groups on GitHub

by Joseph Rickert

Quite a few times over the past few years I have highlighted presentations posted by R user groups on their websites and recommended these sites as a source for interesting material, but I have never thought to see what the user groups were doing on GitHub. As you might expect, many people who make presentations at R user group meetings make their code available on GitHub. However as best as I can tell, only a few R user groups are maintaining GitHub sites under the user group name.

The Indy UseR Group is one that seems to be making very good use of their GitHub Site. Here is the link to a very nice tutorial from Shankar Vaidyaraman on using the rvest package to do some web scraping with R. The following code which scrapes the first page from Springer's Use R! series to produce a short list of books comes form Shankar's simple example. 

# load libraries
library(rvest)
library(dplyr)
library(stringr)
 
# link to Use R! titles at Springer site
useRlink = "http://www.springer.com/?SGWID=0-102-24-0-0&series=Use+R&sortOrder=relevance&searchType=ADVANCED_CDA&searchScope=editions&queryText=Use+R"
 
# Read the page
userPg = useRlink %>% read_html()
 
## Get info of books displayed on the page
booktitles = userPg %>% html_nodes(".productGraphic img") %>% html_attr("alt")
bookyr = userPg %>% html_nodes(xpath = "//span[contains(@class,'renditionDescription')]") %>% html_text()
bookauth = userPg %>% html_nodes("span[class = 'displayBlock']") %>% html_text()
bookprice = userPg %>% html_nodes(xpath = "//div[@class = 'bookListPriceContainer']//span[2]") %>% html_text()
pgdf = data.frame(title = booktitles, pubyr = bookyr, auth = bookauth, price = bookprice)
pgdf

 

 This plot,which shows a list of books ranked by number of downloads, comes from Shankar's extended recommender example.

The Ann Arbor R User Group meetup site has done an exceptional job of creating an aesthetically pleasing and informative web property on their GitHub site.

I am particularly impressed with the way they have integrated news, content and commentary into their "News" section. Scroll down the page and have look at the care taken to describe and document the presentations made to the group. I found the introduction and slides for Bob Carpenter's RStan presentation very well done.

Other RUGs active on GitHub include:

• Cambridge R User Group: meetup site and GitHub site
• Inland Northwest R User Group: GitHub only
• Twin Cities R User Group: meetup site and GitHub site
• UVa R Users Group: meetup site and GitHub site

If your R user group is on GitHub and I have not included you in my short list please let me know about it. I think RUG GitHub sites have the potential for creating a rich code sharing experience among user groups. If you would like some help getting started with GItHub have a look at tutorials on the Murdoch University R User Group webpage.

 

7th Meeting of Spanish R Users. 5-6 November 2015. Salamanca (Spain)

By Virgilio Gómez Rubio, Spanish R Users Organizing Committee

As every autumn since 2009, Spanish R users gathered at their annual meeting. It is organised by Spanish R users group ‘Comunidad R-Hispano’and took place in 5-6 November in the historic city of Salamanca. The 7th Meeting of Spanish R Users attracted more than 100 R entusiasts and provided a mix of tutorials and contributed talks within the quarters of the University of Salamanca.

First of all, from Comunidad R-Hispano we would like to thanks our sponsors Revolution Analytics, Telefónica, Instituto de Ingeniería del Conocimiento, Open Sistemas and Datatons for their financial support. Publishers Springer, Wiley and CRC/Taylor & Francis also supported this meeting by providing flyers, discounts and samples of recent books.

Tutorials presented at the conference covered topics such as R on Spark, the analysis of data from telephone surveys, analysis of survival data,analysis of data obtained from randomised surveys, analysis of network data with igraph and how to use R for the analysis of questionnaire data. All these tutorials provided a hands-on approach to the subject and the materials are available from the conference website.

Contributed talks focused on four main areas: Applications, Interfaces/Data Mining, Statistical Methodology and Biostatistics. Altogether, there were 22 oral presentations on these topics.

Among the Applications, Marcos Fernández Arias showed how to use R to gather information available on-line to pay a fair price for a new car.Teresa González Arteaga also shared some of her teaching experiences with R in a Degree in Statistics. In the Interfaces/Data Mining section, Christian González Martel and co-authors explored the use of R to use Wikipedia searches of top Spanish companies for investment in the Spanish stock market.

Regarding the contributed session on Statistical Methodology, José Luis Vicente Villardón talked about his experience migrating his code for multivariate analysis using biplots from Matlab to R. Finally, in the Biostatistics section, Carlos Prieto and co-authors presented some interactive plots developed with R and other tools to visualize genomic data.

The prize to the best presentation by a young presenter was awarded to Karel López Quintero for his work on a Price Sensitivity Meter (PSM) with R.

Comunidad R-Hispano is already preparing the 8th Meeting that will take place in November 2016, in the University of Castilla-La Mancha in Albacete and it will be locally organised by the same team that took care of useR! 2013.

Looking forward to 2016

by Joseph Rickert

The following map of all of the R user groups listed in Microsoft's Local R User Group Directory is good way to visualize the R world as we rocket into 2016. As a member of the useR!2016 planning committee, foremost in my mind right now is that in just a few months people will be coming to Stanford from all points plotted and almost everywhere in between.

I suppose I have a strong bias here, but this conference promises to be an outstanding event where Silicon Valley welcomes R. The invited speakers for useR! 2016 are  Richard Becker, Donald Knuth, Deborah Nolan, Simon Urbanek, Hadley Wickham and Daniela Witten and the invited tutorials already scheduled to be on the program are:

But all of this is only half the story. The rest depends on you. There are still several open slots for tutorials, and there is plenty of time to think about submitting an abstract for a contributed talk. Please, use one of the templates to submit your tutorial proposal and make sure it is received at [email protected] before the submissions deadline: midnight January 10, 2016 PST. The clock for contributed talks starts on January 5th and runs till March 25th.

Moreover, due to a special alignment of the stars in 2016 the Bioconductor Conference will also be held at Stanford from June 24th through the 26th making it possible to attend both conferences! The details should be up on the Bioconductor site soon. Bay Area hotels are notoriously expensive and there is limited capacity close to the campus, so if you are going to either useR! 2016 or BioC 2016 make you housing arrangements soon.

And - don't let all of this cosmic California talk distract you from planning for R/Finance 2016 which will be held May 20th and 21st in Chicago. If you have any interest at all in Finance (or Chicago), R/Finance is as good as it gets. The deadline for submissions is January 29th. You can find the details here.

And - don't forget that January 10th is also the deadline for submitting a proposal to the R-Consortium for the next round of funding. (You can find the details for submitting a proposal here.) The Infrastructure Steering Committee (ISC) has already received proposals for improving distributed computing in R, developing a package and API for retrieving package download statistics and building a framework for collecting R usage information at the function level. Don't miss this opportunity to get your ideas for work that could benefit the greater R Community in front of the ISC.

Finally, if you are thinking about starting an R user group in 2016 consider applying for support from Microsoft's Data Science User Group Sponsorship Program to get RUG going. You can find the application form here.

R User Group Activity 2015

by Joseph Rickert

2015 has been a good year for R user groups, both in terms of activity and the number of new groups founded. The plot below which runs 12/30/2012 through the week beginning with Monday 11/23/2015 shows that the number of weekly meeting continues to drift up to the right. You can see the seasonal pattern of fewer meetings in the late summer and winter holiday season and increased activity in Autumn. We track weekly meetings in our Community Calendar.

The table below shows 18 new groups founded in 2015. (If I have missed your group please let me know.) 

 

Altogether, we have 174 R user groups listed in the Local R User Group Directory.

After running for more than 5 years the Revolution Analytics R user group support program closed down on September 30, 2015. However, without missing a beat, we immediately picked up with the new Microsoft Data Science User Group Program. Reflecting Microsoft's broad support for the open source community, this program provides support for data science user groups of all stripes. (even Python :) The new program works in a manner that is similar to the Revolution Analytics program. There are three levels of support Vector, Matrix and Array with the amount of funding and care package of goodies determined by a group's size and the frequency with which it meets. If you are a data science group organizer and you think a little cash and some swag can help your group grow please do apply to the program here. 

Finally, a shout out to RBelgium that postponed its meeting scheduled for this week, but will pick up again on 12/16 and to RLyon meeting on 11/27. 

R User Groups Highlight R Creativity

by Joseph Rickert

I have been a big fan of R user groups since I attended my first meeting. There is just something about the vibe of being around people excited about what they are doing that feels good. From a speaker's perspective, presenting at an R user Group meeting must be the rough equivalent of doing "stand-up" at a club where you know mostly everyone and you are pretty sure people are going to like your material. So while user groups don't necessarily ignite R creativity (people don't do their best work just to present at an R User group meeting), they do help to shine the spotlight on some really good stuff.

I attend all of the Bay Area useR group meetings, and quite a few other R related events throughout the year, but I only get to experience a small fraction of what is going on in the R world. In the spirit of sharing the "wish I was there" feeling, here are a few recent user group presentations from around the globe that look like they were informative, entertaining and motivating.

Tommy O'Dell gave "Welcome to dply" talk to the Western Australia R Group (WARG) on September 10th. This is a very good presentation until near the very end when it becomes an absolutely great presentation!! Apparently, motivated by a desire to use dplyr with R 2.12, an older R version of R not supported by dplyr, Tommy deconstructed the dplyr "magic" to write his own package, rdplyr. This is a wonderful example of how curiosity and open source can open up many possibilities. The following slide comes from the section where Tommy explains some of the problems he encountered and how he worked through them.

On the 16th of September, Kevin Little gave a talk to MadR about how he recovered after "hitting the wall" in failed first attempt to interface to the SurveyMonkey API using the Rmonkey package. Kevin's description of how he worked through the process which included wading into some JSON scripting is a motivational case study. Kevin wrote a blog post that provides background for the project and has made his slides available here.

Also in September Jim Porzak, a long-time contributor to the San Francisco Bay Area R community, described a detailed customer segmentation analysis in a presentation to BARUG. The following slide examines the stability of the clusters.

Finally, there is a small treasure trove of relatively recent work at the BaselR presentations page. These include a presentation from Aimee Gott on the Mango Solutions development environment and one from Anne Kuemmel on using simulations to calculate confidence intervals in pharma applications. Also have a look at Daniel Sabanes Bove's presentation on using R to produce Microsoft PowerPoint presentations, and some thoughtful advice from Reinhold Koch on how to go about creating a lively R community within your company.

_______________________________________________________________________________________

_______________________________________________________________________________________

 Let us all adopt this mindset!!

The R Consortium Gears Up For Business

by Joseph Rickert

This week, the Infrastructure Steering Committee (ISC) of the R Consortium unanimously elected Hadley Wickham as its chair thereby also giving Hadley a seat on the R Consortium board of directors. Congratulations Hadley!!

This is a major step forward towards putting the R Consortium in business. Not only is the ISC the group that will decide on what projects the R Consortium will undertake, but it will also be responsible for actually getting the work done. (Look here for the charter of the ISC. )

The whole process of funding, soliciting, selecting and executing projects will work something like this: The board of directors under the leadership of its chair, Richard Pugh of Mango Solutions, will establish a budget for projects. The ISC will solicit proposals for new projects both from R Consortium member companies and from the R Community at large.  With approval from the board, the ISC will decide which projects to fund. From there on, the ISC will assemble resources and manage the work. That’s the plan. The devil, of course, is in the details. There is much work to be done to put all of the necessary infrastructure in place, but Hadley’s election makes it possible for the ISC to begin bootstrapping the process.

So, while there is currently no formal proposal process in place, and the ISC and the R Consortium are not ready to begin the process of soliciting proposals from the public, it is not too early for the R Community to begin thinking about what work needs to be done. Now, is the time to begin thinking on a grand scale; well, at least on a scale that might be a bit more ambitious than creating a single R package.

What type of project might make the cut? I don’t want to set up any constraints here, or limit possibilities. But, just to pick one application area, it seems to me that that there were more than a few ideas kicked around in the HP Workshop on Distributed Computing in R held earlier this year that could be formulated into exciting and important projects. How about a unified interface for distributed computing in R?

If you have an idea for a project that you think would benefit the general R community but is more complicated than writing a simple package please start thinking about how you would write up your ideas, elaborating on the benefits to the R Community, technical feasibility, required resources etc. And, stayed tuned to R Consortium Announcements for information on when the proposal process will begin.

I’ll finish here by congratulating Hadley one more time, and state that I am very pleased to have the opportunity to work with him and the other members of the committee. I expect that with Hadley’s technical leadership, the guidance of the board of directors, and the participation of committed R users that the R Consortium will become an effective advocate and source of support for the R Community.

You can write to the ISC at:  [email protected]

Some facts about the R Consortium
Founded: June 19, 2015
Status: The R Consortium is organized as a Linux Foundation Collaborative Project
Member organizations: Alteryx, Google, Hewlett Packard, Ketchum Trading, Mango Solutions, Microsoft, Oracle, The R foundation, RStudio and Tibco
Board of Directors: David Smith (Microsoft), Hadley Wickham (RStudio), John Chambers (R Foundation), J.J. Allaire (RStudio), Louis Bajuk-Yorgan (Tibco) and Chair, Richard Pugh (Mango Solutions)
ISC Members: Hadley Wickham (RStudio), Joseph Rickert (Microsoft), Luke Tierney (R Foundation) and Stephen Kaluzny (Tibco)