Revolutions

RStudio Server on Azure

RStudio Server Pro is now available on the Azure Marketplace, the company announced on the RStudio Blog earlier this month. This means you can launch RStudio Server Pro on an virtual machine with the memory, disk, and CPU configuration of your choice, and pay by the minute for the VM instance plus a the RStudio software charge. Then, you can use a browser to access the remote RStudio Server (the interface is nigh-indistinguishable from the desktop version), with access to the commercial features of RStudio including support for multiple R version and concurrent R sessions, load-balancing and high availability instances, and enhanced security.

If you don't want or need those capabilities, you can use the open-source version of RStudio Server as well. The easiest way is simply to launch an instance of the Azure Data Science Virtual Machine, which comes pre-installed with the open-source RStudio Server. It also comes pre-installed with dozens of data science tools you can use with R and RStudio, including Jupyter Notebooks, Python (check out the reticulate package), Keras and Tensorflow, and much more. All of the tools are free and/or open-source, so you only pay the base virtual machine charge (which depends on your region and the size and power of the instance you select). Once you've launched the Ubuntu DSVM, simply log in and use systemctl to start the RStudio Server service, and then use your browser to access RStudio remotely.

If you don't need a server with all the tools the DSVM provides, you can configure your own VM with just RStudio Server and anything else you need, too. This handy blog post by Scott James Bell takes you through the steps. This includes setting up an Azure account (if you don't have one yet, click here for $200 in Azure Credits to get started), creating a virtual machine instance, and installing R, RStudio, and Shiny Server (for hosting Shiny applications) on it. Follow the link below for the complete details, and enjoy using RStudio on Azure!

My Year in Data: Running a Shiny Server on Microsoft Azure

AI, Machine Learning and Data Science Roundup: January 2019

A monthly roundup of news about Artificial Intelligence, Machine Learning and Data Science. This is an eclectic collection of interesting blog posts, software announcements and data applications from Microsoft and elsewhere that I've noted over the past month or so.

Open Source AI, ML & Data Science News

Preview of Tensorflow 2.0 (the public preview is expected "early this year").

pcLasso, an R package implementing a new method for supervised learning described by co-author Rob Tibshirani as "principal components regression meets the lasso".

R 3.5.2 has been released.

Industry News

A retrospective of Google Research activities in AI in 2018.

Google Cloud Platform now supports R jobs (via SparkR) in Cloud DataProc.

GCP App Engine support for Python 3.7 now generally available.

Amazon SageMaker now comes preconfigured with support for SciKit-learn.

Microsoft News

Microsoft Professional Program for Data Analysis, a new online course and certification. Other MPP tracks include Data Science and Artificial Intelligence.

[video] A single API key can now be used to access Cognitive Services APIs for language, vision and search.

AzureR, a suite of packages for interfacing with storage, virtual machines, containers and other Azure services from the R language, is now available on CRAN.

Learning resources

E-book by TWIML AI host Sam Charrington: Kubernetes for Machine Learning, Deep Learning and AI (requires free sign-up).

E-book by Patrick Hall and Navdeep Gill: Introduction to Machine Learning Interpretability (requires free registration).

An in-depth introduction to convolutional neural networks, from Ars Technica: How computers got shockingly good at recognizing images.

An on-line course from Databricks, Deep Learning Fundamental Series, with a focus on Keras and TensorFlow.

Getting Started with TensorFlow Probability, from R, a blog post from RStudio.

What can Neural Networks Learn?, an approachable look at the inner workings of neural network classifiers from Brandon Rohrer.

A blog post describing the computational graph concepts behind TensorFlow, including an illustrative implementation of core TensorFlow operations in numpy.

Applications

Using computer vision to monitor shelf stocking policies, a three-part series on complex image classification: Part 1, Part 2, Part 3.

[video] Building a pet breed identification application via transfer learning with Azure ML Service and Python. (Github repo here.)

Find previous editions of the monthly AI roundup here.

Use foreach with HPC schedulers thanks to the future package

The future package is a powerful and elegant cross-platform framework for orchestrating asynchronous computations in R. It's ideal for working with computations that take a long time to complete; that would benefit from using distributed, parallel frameworks to make them complete faster; and that you'd rather not have locking up your interactive R session. You can get a good sense of the future package from its introductory vignette or from this eRum 2018 presentation by author by Henrik Bengtsson (video embedded below), but at its simplest it allows constructs in R like this:

a %<-% slow_calculation(1:50)
b %<-% slow_calculation(51:100)
a+b

The idea here is that slow_calculation is an R function that takes a lot of time, but with the special %<-% assignment operator the computation begins and the R interpreter is ready immediately. The first two lines of R code above take essentially zero time to execute. The futures package farms off those computations to another process or even a remote system (you specify which with a preceding plan call), and R will only halt when the result is needed, as in the third line above. This is beneficial in Bengtsson's own work, where he uses the future package to parallelize cancer research on DNA sequences on high-performance computing (HPC) clusters.

The future package supports a wide variety of computation frameworks including parallel local R sessions, remote R sessions, and cluster computing frameworks. (If you can't use any of these, it falls back to evaluating the expressions locally, in sequence.) The future package also works in concert other parallel programming systems already available in R. For example, it provides future_lapply as a futurized analog of lapply, which will use whatever computation plan you have defined to run the computations in parallel.

The future package also extends the foreach package thanks to the updated doFuture package. By using registerDoFuture as the foreach backend, your loops can use any computation plan provided by the future package to run the iterations in parallel. (The same applies to R packages that use foreach internally, notably the caret package.) This means you can now use foreach with any of the HPC schedulers supported by future, which includes TORQUE, Slurm, and OpenLava. So if you you share a Slurm HPC cluster with colleagues in your department, you can queue up a parallel simulation on the cluster using code like this:

library("doFuture")
registerDoFuture()
library("future.batchtools")
plan(batchjobs_slurm)

mu <- 1.0
sigma <- 2.0
x <- foreach(i = 1:3, .export = c("mu", "sigma")) %dopar% {
  rnorm(i, mean = mu, sd = sigma)
}

The future package is available on CRAN now, and works consistently on Windows, Mac and Linux systems. You can learn more in the video at the end of this post, or in the recent blog update linked below.

JottR: Maintenance Updates of Future Backends and doFuture

Because it's Friday: A timeline of the elements

A few chemical elements: copper, iron, sulphur, and a few others have been known since the dawn of time. This animated timeline, created by Dr Jamie Gallagher, shows the year of discovery (or in some cases, the creation) of the rest of them:

That's all from the blog for this week. Have a great weekend, and we'll see you next week!

Who is the greatest finisher in soccer?

It's relatively easy to find the player who has scored the most goals in the last 12 years (hello, Lionel Messi). But which professional football (soccer) player is the best finisher, i.e. which player is most likely to put a shot they take into the goal?

You can't simply use the conversion rate (the ratio of shots taken to goals scored), because some players play more shots a long way from the goal while others get more set-ups near the goal. To correct for that, the blog Barça Numeros used a Bayesian beta-binomial regression model to weight the conversion rates by distance, and then ranked each player for their goal scoring rate at 25 distances from the goal. (The analysis was performed in R using techniques described in David Robinson's book Introduction to Empirical Bayes: Examples from Baseball Statistics, which is available online.)

Here's a chart comparing the ranks of Messi, Zlatan Ibrahimovic, Ronaldo Cristiano, Paulo Dybala and Allesandro Del Piero, at each distance, showing Messi to be the best finisher of these players at all ranges:

For an overall ranking for each player, the blog used the median rank across the 25 shot distances — a ranking that places Lionel Messi as the greatest finisher of the last 12 years.

For more details behind the analysis (and many more charts), check out the complete blog post linked below.

Barça Numeros: Who are the best finishers in contemporary football? (via @barcanumbers)

AzureR packages now on CRAN

The suite of AzureR packages for interfacing with Azure services from R is now available on CRAN. If you missed the earlier announcements, this means you can now use the install.packages function in R to install these packages, rather than having to install from the Github repositories. Updated versions of these packages will also be posted to CRAN, so you can get the latest versions simply by running update.packages.

The packages are:

• AzureVM (CRAN): a package for working with virtual machines and virtual machine clusters. It allows you to easily deploy, manage and delete a VM from R. A number of templates are provided based on the Data Science Virtual Machine, or you can also supply your own template. Introductory vignette.
• AzureStor: a package for working with storage accounts. In addition to a Resource Manager interface for creating and deleting storage accounts, it also provides a client interface for working with the storage itself. You can upload and download files and blobs, list file shares and containers, list files within a share, and so on. It supports authenticated access to storage via either a key or a shared access signature (SAS). Introductory vignette.
• AzureContainers: a package for working with containers in Azure: specifically, it provides an interface to Azure Container Instances, Azure Container Registry and Azure Kubernetes Service. You can easily create an image, push it to an ACR repo, and then deploy it as a service to AKS. It also provides lightweight shells to docker, kubectl and helm (assuming these are installed). Vignettes on deploying with the plumber package and with Microsoft ML Server. 
• AzureRMR: the base package of the AzureR family for interfacing with the Azure Resource Manager and which can be extended to interface with other services. There's an introductory vignette, and another on extending AzureRMR.

Click on the first links in each bullet above for a blog post introducing each package. The introductory vignette listed with each package provides further details, and will be up-to-date for the latest version of the package.

Feedback is welcome on all of these packages, which can be found as part of the cloudyr project on Github.

Because it's Friday: Synthetic faces, styled to your specifications

If you need someone's face to use in an application or some marketing materials, you might search one of the stock photography vendors for people of a given gender, skin tone, hairstyle, etc. Or, you could just ask the Generative Adversarial Network described in this paper by NVIDIA researchers to generate a brand new face for you. See it in action in the video below:

That's all for this week. Have a great weekend, and we'll be back with more from the blog next week.

In case you missed it: December 2018 roundup

In case you missed them, here are some articles from December of particular interest to R users.

R 3.5.2 is now available.

Roundup of AI, Machine Learning and Data Science news from December 2018.

AzureStor, a new R package to interface with Azure Storage.

How to use the "plumber" package to create an R service as a container with the AzureContainers package.

How to give money to the R Foundation in support of the R project.

The Revolutions blog is 10 years old, and I reflect on some milestones in a talk given at SatRDays DC.

Reshama Shaikh compares gender diversity within the R and Python communities.

AzureVM, a new package for managing virtual machines in Azure from R.

And some general interest stories (not necessarily related to R):

• Santa Claus and compliance with GDPR
• A parody on the use of CGI in film
• The story behind Fleetwood Mac's Go Your Own Way

As always, thanks for the comments and please send any suggestions to me at [email protected]. Don't forget you can follow the blog using an RSS reader, via email using blogtrottr, or by following me on Twitter (I'm @revodavid). You can find roundups of previous months here. 

Notebooks from the Practical AI Workshop

Last month, I delivered the one-day workshop Practical AI for the Working Software Engineer at the Artificial Intelligence Live conference in Orlando. As the title suggests, the workshop was aimed at developers, bu I didn't assume any particular programming language background. In addition to the lecture slides, the workshop was delivered as a series of Jupyter notebooks. I ran them using Azure Notebooks (which meant the participants had nothing to install and very little to set up), but you can run them in any Jupyter environment you like, as long as it has access to R and Python. You can download the notebooks and slides from this Github repository (and feedback is welcome there, too). 

The workshop was divided into five sections, each with its associated Notebook:

1. The AI behind Seeing AI. Use the web interfaces to Cognitive Services to learn about the AI services behind the "Seeing AI" app
2. Computer Vision API with R. Use an R script to interact with the Computer Vision API and generate captions for random Wikimedia images.
3. Custom Vision with R. An R function to classify an image as a "Hot Dog" or "Not Hot Dog", using the Custom Vision service.
4. MNIST with scikit-learn. Use sckikit-learn to build a digit recognizer for the MNIST data using a regression model.
5. MNIST with Tensorflow. Use Tensorflow (from Python) to build a digit recognizer for the MNIST data using a convolutional neural network.

The workshop was a practical version of a talk I also gave at AI Live, "Getting Started with Deep Learning", and I've embedded those slides below. 

Azure Notebooks: Practical AI for the Working Software Engineer