Revolutions

Saving Snow Leopards with Artificial Intelligence

The snow leopard, the large cat native to the mountain ranges of Central and South Asia, is a highly endangered species. With an estimated estimated 3900-6500 individuals left in the wild, conservation efforts led by the Snow Leopard Trust are focused on preserving this iconic animal.

But the snow leopard is an elusive creature: given their range and emote habitat (including the highlands of the Himalayas), they are difficult to study. In order to gather data about the creatures, researchers have used camera traps to capture more than 1 million images. 

But not all of those images are of snow leopards. It's a time-consuming process to classify those images as being of snow leopards, their prey, some other animal or nothing at all. To make things even more difficult, snow leopards have excellent camouflage, and can be difficult to spot even by experienced observers. To cut down on the 300 person-hours per camera survey (containing up to 100,000 images), Microsoft's Azure Machine Learning teamed up with the Snow Leopard Trust to build an automated classification system.

The system is based on a convolutional neural network (CNN) which used is to classify the camera trap images. Rather than training a CNN from scratch, transfer learning was applied to the ResNet-50 model, which has already been trained on images of objects, animals and people. The output from the upper layers of ResNet-50 is then used to build a traditional logistic regression model on human-classified images stored in Spark, using the open-source MMLSpark library. (You can see a similar example of transfer learning with MMLSpark here.)

This classifier is then made available to the researchers as an on-line service in Azure — "Snow Leopard Classification as a Service", if you will. All the researchers need to do is upload a file of images from the latest camera trap survey, and the system automatically identifies those that include a snow leopard. You can see the system in action in this presentation by Joseph Sirosh at the Strata Data conference earlier this month.

To learn more about the project, check out the blog post linked below.

Cortana Intelligence and Machine Learning Blog: Saving Snow Leopards with Deep Learning and Computer Vision on Spark

Recap: Applications of R at EARL London 2017

The fourth EARL London conference took place last week, and once again it was an enjoyable and informative showcase of practical applications of R. Kudos to the team from Mango for hosting a great event featuring interesting talks and a friendly crowd.

As always, there were more talks on offer than I was able to attend (most of the event was in three parallel tracks), but here are a few highlights that I was able to catch:

Jenny Bryan from RStudio gave an outstanding keynote: "Workflows: You Should Have One" (view the slides here). The talk was a really useful collection of good practices for data analysis with R. Note I didn't say "best practices" there, and that's something that resonated with me from Jenny's talk: sometimes the best practices are the ones that are "good enough" and don't add unnecessary complexity or fragility. Software Carpentry's "Good Enough Practices for Scientific Computing" is a great read that I hadn't come across before this talk.

Rachel Kirkham from the National Audit Office uses R and Shiny to scrutinize public spending on behalf of taxpayers in the UK. (View the slides here.) One interesting beneficial aspect of R for this application is the ability to bring R to the data. (Update Sep 21: I've edited this to remove my error mischaracterizing Rachel's talk. Rachel clarified via email: "We are deploying Shiny apps on client systems to eliminate the need to transfer data and associated risks of data loss. We are fully compliant with the Data Protection Act and associated data handling policies.")

Joy McKenney from Northumbrian Water gave a truly fascinating talk on using R to monitor and predict flows in a sewer system. (View the slides here.) One surprising application from this talk was being able to model whether an overflowing sewer line is due to rainfall or because of a blockage in the system — a tool that can be used to detect "fatbergs" clogging up the system.

Pieter Vos from Philips Research showed how they deploy applications to doctors to evaluate things like cancer risk. (View the slides here.) The applications call out to R to make the underlying statistical calculations.

Joe Cheng from RStudio gave a remarkable talk on a major concept for the R language itself: "promises". (View the slides here.) With promises, you can ask for the results of a long-running computation, but get back to the R command line instantaneously. The results will be available to you, well, when they're ready. It sounds like magic, but it's already working (in beta) with the promises package and promises (pun intended) to bring more responsiveness to Shiny applications.

Ashley Turner from Transport for London gave a talk that I was very sad I couldn't see (it conflicted with my own session), but fortunately the slides are available. R has been used for over 2 years to model how Londoners get about via car, bus and Tube. It was used for an official report on London transportation, which included some fascinating data on the routes certain commuters prefer for getting from (say) Kings Cross to Waterloo. 

Matthew Upson from the Government Digital Service reported on the ongoing program to implement a reproducible data science workflow for creating official reports. (View the slides here.) He says that this process has reduced production time for some official reports by 75%, and you can learn more about the program here.

I was also informed by the organizers that my own talk on Reproducible Data Science with R (slides here) won the "award" for loudest presentation at the conference. I think the audio desk had the volume up a little too high for this video! 

As I said, there are many more excellent talks beyond those listed above. You can explore the other talks by clicking here and then clicking through each speaker portrait — slides, where available, are linked at the bottom of each abstract.

The next EARL conference will take place in Boston, November 1-3 and promises to include even more practical applications of R. I hope to see you there!

Hurricane Harvey's rains, visualized in R by USGS

On August 26 Hurricane Harvey became the largest hurricane to make landfall in the United States in over 20 years. (That record may yet be broken by Irma, now bearing down on the Florida peninsula.) Harvey's rains brought major flooding to Houston and other coastal areas in the Gulf of Mexico. You can see the rainfall generated by Harvey across Texas and Louisiana in this animation from the US Geological Survey of county-by-county precipitation as the storm makes its way across land.

Watch #Harvey move thru SE #Texas spiking rainfall rates in each county (blue colors) Interactive version: https://t.co/qFrnyq3Sbm pic.twitter.com/md0hiUs9Bb

— USGS Coastal Change (@USGSCoastChange) September 6, 2017

Interestingly, the heaviest rains appear to fall somewhat away from the eye of the storm, marked in orange on the map. The animation features Harvey's geographic track, along with a choropleth of hourly rainfall totals and a joyplot of river gage flow rates, and was created using the R language. You can find the data and R code on Github, which makes use of the USGS's own vizlab package which facilitates the rapid assembly of web-ready visualizations. 

You can find more information about the animation, including the web-based interactive version, at the link below.

USGS Vizlab: Hurricane Harvey's Water Footprint (via Laura DeCicco)

Buzzfeed trains an AI to find spy planes

Last year, Buzzfeed broke the story that US law enforcement agencies were using small aircraft to observe points of interest in US cities, thanks to analysis of public flight-records data. With the data journalism team no doubt realizing that the Flightradar24 data set hosted many more stories of public interest, the challenge lay in separating routine, day-to-day aircraft traffic from the more unusual, covert activities.  

So they trained an artificial intelligence model to identify unusual flight paths in the data. The model, implemented in the R programming language, applies a random forest algorithm to identify flight patterns similar to those of covert aircraft identified in their earlier "Spies in the Skies" story. When that model was applied to the almost 20,000 flights in the FlightRadar24 dataset, about 69 planes were flagged as possible surveillance aircraft. Several of those were false positives, but further journalistic inquiry into the provenance of the registrations led to several interesting stories.

Using this model, Buzzfeed news identified several surveillance aircraft in action during a four-month period in late 2015. These included a spy plane operated by US Marshals to hunt drug cartels in Mexico; aircraft covertly registered to US Customs and Border Protection patrolling the US-Mexico border; and a US Navy contractor operating planes circling several points over land in the San Francisco Bay Area — ostensibly for harbor porpoise research.

You can learn more about the stories Buzzfeed News uncovered in the flight data here, and for details on the implementation of the AI model in R, follow the link below.

Github (Buzzfeed): BuzzFeed News Trained A Computer To Search For Hidden Spy Planes. This Is What We Found.

Reproducibility: A cautionary tale from data journalism

Timo Grossenbacher, data journalist with Swiss Radio and TV in Zurich, had a bit of a surprise when he attempted to recreate the results of one of the R Markdown scripts published by SRF Data to accompany their data journalism story about vested interests of Swiss members of parliament. Upon re-running the analysis in R last week, Timo was surprised when the results differed from those published in August 2015. There was no change to the R scripts or data in the intervening two-year period, so what caused the results to be different?

Image credit: Timo Grossenbacher

The version of R Timo was using had been updated, but that wasn't the root cause of the problem. What had also changed was the version of the dplyr package used by the script: version 0.5.0 now, versus version 0.4.2 then. For some unknown reason, a change in the dplyr package in the intervening package caused some data rows (shown in red above) to be deleted during the data preparation process, and so the results changed.

Timo was able to recreate the original results by forcing the script to run with package versions as they existed back in August 2015. This is easy to do with the checkpoint package: just add a line like

library(checkpoint); checkpoint("2015-08-11")

to the top of your R script. We have been taking daily snapshots of every R package on CRAN since September 2014 to address exactly this situation, and the checkpoint package makes it super-easy to find and install all of the packages you need to make your script reproducible, without changing your main R installation or affecting any other projects you may have. (The checkpoint package is available on CRAN, and also included with all editions of Microsoft R.)

I've been including a call to checkpoint on the top of most of my R scripts for several years now, and it's saved me from failing scripts many times. Likewise, Timo has created a structure and process to support truly reproducible data analysis with R, and it advocates using the checkpoint to manage package versions. You can find a description of the process here: A (truly) reproducible R workflow, and find the template in Github. 

By the way, SRF Data — the data journalism arm of the national broadcaster in Switzerland — has published some outstanding stories over the past few years, and has even been nominated for Data Journalism Website of the Year. At the useR!2017 conference earlier this year, Timo presented several fascinating insights into the data journalism process at SRF Data, which you can see in his slides and talk (embedded below):

Timo Grossenbacher: This is what happens when you use different package versions, Larry! and A (truly) reproducible R workflow

Applications in energy, retail and shipping

The Solutions section of the Cortana Intelligence Gallery provides more than two dozen working examples of applying machine learning, data science and artificial intelligence to real-world problems. Each solution provides sample data, scripts for model training and evaluation, and reporting of predictions. You can deploy a complete stack in Azure to implement the solution with the click of a button, or follow instructions to deploy on your own hardware. The internals of each solution is fully documented and open source, so you can easily customize it to your needs.

Here's a brief overview of some solutions that have recently been posted to the Gallery. Click on the links in bold to be taken to the main solution page.

Customer Churn Prediction. This solution uses historical customer transaction data to identify new customers that are most likely to churn (switch to a competitor) in the near future. Azure Machine Learning is used to train a boosted decision tree model (with some Python scripts for data wrangling) on data stored in Azure SQL Data Warehouse; Azure Event Hub and Azure Stream Analytics are used to generate predictions on incoming data. Results are presented in interactive Power BI dashboards. Details in the Solution Guide on Github.

Demand Forecasting for Shipping and Distribution. This solution uses uses historical demand data (such as shipping or delivery records) to forecast future demand across customers, products and destinations. Forecasts are generated from data in Azure SQL Server using R functions within Azure ML and presented as a dashboard in Power BI. Developed in conjunction with New Zealand supply-chain company Kotahi. Details in the Solution Guide on Github.

Energy Supply Optimization. This solution determines the optimal mix of energy types (substation, battery cell, wind, solar, etc.) for efficient management of an energy grid. Azure Batch is used to manage a cluster of Data Science Virtual Machines, which run Pyomo (within Python) to solve complex numerical optimization problems. Results are presented in Power BI from data exported to Azure SQL Database. Details in the Solution Guide in Github.

Oil and Gas Tank Level Forecasting. This solution helps oil and gas storage facilities to anticipate problems that can lead to spills or emergency shutdowns. Data from supply sources combined with tank sensor data and outflow measurements are used to predict tank levels and flag anomalies. The zoo package for R is used to process the time series data and generate the forecasts, with are presented in a Power BI dashboard as shown below. Details in the Solution Guide on Github.

For more solutions in various industries, browse the Gallery at the link below.

Cortana Intelligence Gallery: Solutions

Data Science Accelerator for Credit Risk Prediction

by Fang Zhou, Data Scientist; Graham Williams, Director of Data Science, all at Microsoft

Credit Risk Scoring is a classic but increasingly important operation in banking as banks are becoming far more risk careful when lending for mortgages, credit card payments or other commercial purposes, in an industry known for fierce competition and the global financial crisis. With an accurate credit risk scoring model, a bank is able to predict the likelihood of default on a transaction. This will in turn help evaluate the potential risk posed by lending money to consumers and to mitigate losses due to bad debt, as well as determine who qualifies for a loan, at what interest rate, and what credit limits, and even determine which customers are likely to bring in the most revenue through a variety of products.

Many banks nowadays are driving innovation to enhance risk management. For example, a largest bank in one of the Asian countries by market capitalization is exploring opportunities to segment a million of active credit card customer population to improve risk scoring to then identify opportunities to offer increased limits. Using advanced analytics for credit risk scoring involves traditional scorecard building and modelling, and extends to machine learning and ensemble, but will also pursue an innovation on customer oriented aggregation of transactions, multi-dimensional customer segmentation and conceptual clustering to identify multiple segments across which to understand bank customers.

In the data-driven credit risk prediction model, normally two types of data are taken into consideration.

1. Transaction dataTransaction records cover transaction id, account id, transaction date, transaction amount, merchant industry, etc. This transaction-level data could be dynamically aggregated and then provide transaction statistics and financial behavior information at account level.
2. Demographic and bank account informationThis type of data show the characteristics of individual customer or account credit bureau, such as age, sex, income, and credit limit. They are static and never change or solely increment deterministically over time.

The following graphic shows the data schema and the workflow for credit card fraud risk prediction.

In recent years, R has been gaining in popularity over SAS among statisticians and data scientists in solving variety of industrial business problems, including Financial Services and Risk Management. A data science accelerator for credit risk prediction is now shared in the github repository. This accelerator consists of four R templates which walk through the process of model development, scale-up and speed-up, deployment, and application development.

CreditRiskPrediction: Data-driven credit risk prediction in R, covering techniques of exploratory analytics, data aggregation, merging and cleansing, feature engineering, but more importantly, model building and evaluation.

CreditRiskScale: Faster and scalable credit risk models with Microsoft R Server , using the state-of-the-art machine learning algorithms provided by the MicrosoftML.

CreditRiskScale (part 2): Train multiple ML models with hyper-parameter selection in parallel by using rxExec().

CreditRiskDeploy: Deploy a credit risk model as a web service with R Server Operationalization, leveraging the mrsdeploy package.

CreditRiskShinyApp: Credit risk application through REST API, with integration with the Shiny framework.

You can find the data science accelerator for credit risk prediction on GitHub. The R scripts are provided as R markdown files, which you can use to generate documents in formats including PDF, html, and Word. By no means is this the endpoint of the data science journey. The accelerator is under regular revision and improvement and we welcome feedback via pull requests and issues.

Github (Microsoft): Data Science Accelerator - Credit Risk Prediction 

More Companies using R

Here's a quick roundup of some case studies published recently on the Microsoft Customer Stories portal, with examples of companies running R in production environments using the Microsoft stack.

eToro, the social trading network, uses Microsoft R Server and Azure Machine Learning so that investors can automatically duplicate market strategies and actual investment decisions of successful traders in the eToro network to use in their own portfolios.

Acxiom, the marketing analytics company, is using SQL Server R Services to build more accurate models using larger data sets, which results in more precisely targeted marketing for customers.

Merck, the pharmaceutical company, is using Microsoft R Server for Hadoop to analyze the various factors that affect the cold chain such as origin, destination, shipment route, external weather, and logistics providers. With an improved temperature-controlled supply route, Merck can reduce wastage caused by temperature fluctuations.

PROS, the dynamic pricing software company, uses Microsoft Azure and Microsoft SQL Server 2016 with R Services to to keep up-to-the-minute pricing accurate, competitive, and profitable.

WorldSmart, the retail technology specialist, uses Azure Stream Analytics services and Microsoft R Server to drive several of its Machine Learning forecasts and analysis.

dv01, the financial reporting and analytics platform, uses Microsoft SQL Server 2016 running on Azure Virtual Machines and R to bring greater transparency to the consumer lending market.

Heartland Bank, in New Zealand, replaced SAS with Microsoft R Server and SQL Server to analyze risk, evaluate credit lines, and understand cash flows.

You can find more case studies at the Microsoft Customer Stories portal.

How R is used by the FDA for regulatory compliance

I was recently alerted (thanks Maëlle and Mikhail!) to an enlightening presentation from last years' useR! conference. (This year's useR! conference takes place next week in Belgium.) Paul H Schuette, Scientific Computing Coordinator at the FDA Center for Drug Evaluation and Research (CDER), talked about how R is used in the process of regulating and approving drugs at the FDA. 

In what has become a common theme of FDA presentations at R conferences, Schuette refutes the fallacy that SAS is the only software that can be used for FDA submissions, by sponsors such as pharmaceutical companies. On the contrary, he says "sponsors may propose to use R, and R has been used by some sponsors for certain types of analyses and simulations (post-market)."

The myth persists despite the FDA's Statistical Software Clarifying Statement declaring that any suitable software can be used. This is probably because some data-exchange regulations do require the use of the "XPT" (also known as SAS XPORT) file format, but that data format is an open standard and not restricted to SAS. XPT files can be read into R with the built-in read.xport function, and exported from R with the write.xport function in the SASxport package. (If you have legacy data in other SAS formats, here's a handy SAS macro to export XPT files.) The R Foundation also provides guidance on how R complies with other FDA regulations in the document R: Regulatory Compliance and Validation Issues A Guidance Document for the Use of R in Regulated Clinical Trial Environments. 

In addition to sponsors using R in submission, R is also used internally at the FDA. Statisticians there may use the statistical package of their choice, provided it's fit for the purpose. The software used includes SAS, R, Minitab and Stata. Schuette notes that R is used specifically for:

• Statistical review of data analysis in clinical trial submissions. The primary goal here is, "Can we, on our own, replicate the conclusions of the sponsor?"
• Methodology development, innovation and evaluation.
• Graphics (in some cases, the detailed information folded and included with prescription medications feature R graphics).
• Simulations.
• The openFDA initiative, including this LRT Signal Analysis for a Drug Shiny application.

Check out the entire presentation, embedded below.

Channel 9: Using R in a regulatory environment: FDA experiences.

 

Applications of R at EARL San Francisco 2017

The Mango team held their first instance of the EARL conference series in San Francisco last month, and it was a fantastic showcase of real-world applications of R. This was a smaller version of the EARL conferences in London and Boston, but with that came the opportunity to interact with R users from industry in a more intimate setting. Hopefully Mango will return to the venue again next year, and if so I'll definitely be back!

As always with EARL events, the program featured many interesting presentations of how R is used to implement a data-driven (or data-informed) policy at companies around the world. With a dual-track program I couldn't attend all of the talks, but here are some of the applications that caught my interest:

• Ricardo Bion (AirBnB): An keynote with an update on data science practice at AirBnB: training for everybody, the Knowledge Repository, trends in Python and R usage, and even a version of the AirBnB app implemented in Shiny!
• Hilary Parker (Stitchfix): A wonderful keynote on bringing principles of engineering (and in particular blameless post-mortems) to data science (slides)
• Prakhar Mehrotra (Uber): Using R to forecast demand and usage (sadly no slides, but here's R charting Uber traffic data)
• David Croushore (Pandora): Using R and Shiny to monitor and forecast demand by users for different music services (slides).
• David Bishop (Hitachi Solutions). Using R in the clean energy industry, with a nice demo of a Power BI dashboard to predict wind turbine anomalies, and even visualize the turbines with HoloLens (slides)
• Tyler Cole (Amgen). Using R packages and Microsoft R Server for clinical trial submissions to the FDA (slides)
• Luke Fostvedt (Pfizer). How R and R packages are used at various stages of the drug development process at Pfizer (slides)
• Gabriel Becker (Genentech). Processes and R packages used at Genentech to manage the various tradeoffs for reproducibility in a multi-collaborator environment (slides)
• Shad Thomas (Glass Box Research): Using R and Shiny for segmentation analysis in market research (slides)
• Aaron Hamming (ProAssurance): Using R to combat the opioid epidemic by identifying suspect prescribers (slides)
• Eduardo Ariño de la Rubia (Domino Data Lab): Using R from APIs including Rook, Rapache, Plumbr, OpenCPU and Domino (slides forthcoming)
• Madhura Raju (Microsoft). Using R to detect distributed denial-of-service attacks (slides)
• Slides from my own presentation, Predicting patient length-of-stay in hospitals, are available too.

There are many more interesting presentations to browse at the EARL San Francisco website. Just click on the presenter name to see the talk description and a link to slides (where available).

EARL Conference: San Francisco 2017