Revolutions

AI for Good: slides and notebooks from the ODSC workshop

Last week at the ODSC West conference, I was thrilled with the interest in my Using AI for Good workshop: it was wonderful to find a room full of data scientists eager to learn how data science and artificial intelligence can be used to help people and the planet. The workshop was focused around projects from the Microsoft AI for Good program. I've included some details about the projects below, and you can also check out the workshop slides and the accompanying  Jupyter Notebooks that demonstrate the underlying AI methods used in the projects. 

The projects were drawn from the AI for Earth, AI for Accessibility and AI for Humanitarian Action programs, respectively:

The iNaturalist app, which uses image recognition and species distribution models to help citizen scientists identify animal and plant species from photos, and submit their observations to the scientific community. The app is available for iOS and Android smartphones, and you can also try this web-based demo with a picture of an animal or plant. 

The Seeing AI app (available for iOS), which helps blind and vision-impaired people navigate the world around them. (This video, featuring creator Saqib Shaikh, demonstrates how the app works.) The app detects and describes people and transcribes signs and documents, and you can try out the underlying vision APIs using just a web browser. You can also use R to drive the vision APIs, as shown in this notebook demonstrating the "Describe Scene" feature in Seeing AI. 

The AirSim Project, which is used to simulate image data for vision system training using the 3-D Unreal Engine. One application is to train a drone to automatically perform search and rescue operation after a disaster, as described in this video. There, 3-D simulations were provided training data for the Custom Vision service, which you can try out yourself with R and this Jupyter Notebook. 

You can try out all of the aforementioned notebooks by cloning this Azure Notebooks library, or download them at the Github repository below and run them in your own Jupyter Notebooks instance.

Github (revodavid): AI for Good Workshop

T-mobile uses R for Customer Service AI

T-Mobile, the global telecommunication company, is using R in production to automatically classify text messages to customer service  and route them to an agent that can help. The AI@T-mobile team used the keras library in R to build a natural language processing engine with Tensorflow, and deployed it to production as a docker container. The MRAN Time Machine ensures the container gets fixed R package versions for reproducibility. 

While you may think first of Python for building production AI models, the keras library in R has already proven to be a first-class interface for building AI models. The T-mobile team was able to build this system in just four months and with a small budget because they were able to explore in R and immediately deploy in R:

"Despite being an incredibly popular language for exploratory analysis, data scientists are repeatedly told that R is not sufficient for machine learning – especially if those ML models are destined for production. Though much exploratory analysis and modeling is done in R, these models must be rebuilt in Python to meet DevOps and enterprise requirements. Our team doesn’t see the value in this double work. We explore in R and immediately deploy in R. Our APIs are neural network models using R and TensorFlow in docker containers that are small and maintainable enough to make our DevOps team happy!"

If you'd like to try this out yourself, the T-mobile team has published an open source version of their message-classification container on Github, and you can read the details in the blog post by team leads at the link below.

T-Mobile blog: Enterprise Web Services with Neural Networks Using R and TensorFlow

Fringe FM conversation on AI Ethics

A few weeks ago, I had a lively conversation with Matt Ward for the Fringe FM podcast, where we discussed artificial intelligence, its applications, and the ethical implications thereof. 

During the conversation I mentioned offhand an article I'd read recently which suggested that program to identify criminals using face recognition in CCTV suffered from a high rate of misidentifications. I couldn't remember the exact rate at the time, and in the quote shown below I said "50% or 90%". Turns out I was being too conservative: the actual rate was 98%. Just as with human-based systems, no AI system is perfect, and classifications are based on a confidence score exceeding some threshold. Set the threshold too low, as was undoubtedly the case here, and the result will be many false positives. It's always important for AI developers to consider the impact of false positives and false negatives, and to take particular care to consider the impact of those negative determinations, especially when they relate to people's lives.

During the podcast I also mentioned the Microsoft group headed by Kate Crawford which is doing some of the fundamental research into AI ethics, but failed to mention the name of the group: Fairness, Accountability, Transparency and Ethics in AI (FATE). You can find their publications in the Microsoft Research Catalog — the AI Now Report in particular is worth reading for its recommendations.

You can find the Fringe FM podcast in your favorite podcast app (mine is Pocket Casts), or listen to the Episode 46 directly at the link below.

Fringe FM: 46. The Ethics of AI in an Era of Big Data | David Smith

AI, Machine Learning and Data Science Roundup: October 2018

A monthly roundup of news about Artificial Intelligence, Machine Learning and Data Science. This is an eclectic collection of interesting blog posts, software announcements and data applications from Microsoft and elsewhere that I've noted over the past month or so.

Open Source AI, ML & Data Science News

PyTorch 1.0 is now in preview, and brings a streamlined workflow from model development to production. Relatedly, Sam Charrington compares the growing PyTorch ecosystem with that of Tensorflow.

Tensorflow 1.11 is released, with binaries for cuDNN 7.2 and TensorRT 4, and new functions for querying kernels.

fastai, a new open library for deep learning built on PyTorch, has been released by fast.ai. It provides a single, consistent API for many deep learning applications and data types.

The R packages keras and tensorflow now allow definition of models that take advantage of eager execution.

Anaconda can now install and manage Tensorflow as a conda package.

Industry News

PyXLL, a commercial Excel add-in for calling Python functions from Excel.

IBM releases AI Fairness 360, an open source toolkit for investigating and mitigating bias in machine learning models.

Google open sources a TensorFlow package for Active Question Answering, a reinforcement learning based method to train artificial agents to answer natural-language questions.

Google Cloud Platform adds PyTorch support to several services, including Tensorboard, Kubeflow and Deep Learning VM images.

RStudio Package Manager, a commercial product to help organizations manage R packages, has been released.

RStudio adds support for Stan, the Bayesian modeling language.

Databricks releases MLflow 0.7.0, featuring a new R client API that allows you to log parameters, code versions, metrics, and output files when running R code and visualize the results in MLflow.

Cloudera and Hortonworks are merging. Thomas Dinsmore assesses the impact on the machine learning ecosystem.

Dataiku 5 is released. The enterprise data platform adds support for containerized R and Python recipes, and integration with Tensorflow and Keras.

Microsoft News

Microsoft has opened a new AI research facility in Shanghai in conjunction with INESA, the China organization developing smart city solutions.

SQL Server 2019 will provide integration with "big data clusters": external SQL Server, Spark or HDFS containers managed in Kubernetes. For AI and ML workloads, you can run Spark or SQL Server's R, Python and Java extensions against the cluster data.

Azure Machine Learning Services now provides Python developer libraries for data preparation, experiment tracking and model management, training on GPU clusters, automated model search and hyperparameter optimization, deploying trained models as containers, and many other new capabilities.

Visual Studio Code Tools for AI has been updated to provide a convenient interface to Azure Machine Learning for users of the popular open-source editor.

Speech Service in Azure Cognitive Services is now generally available, and includes a new neural text-to-speech capability for humanlike synthesized spech.

Azure Databricks is now supported in more regions, offers GPU support for deep learning, and Databricks Delta is now available (in preview) for transactional data capabilities.

Microsoft Bot Framework SDK v4 is now available. The Ignite presentation "Creating Enterprise-Scale Intelligent Agents and Bots" provides an overview and several examples.

Cortana Skills Kit for Enterprise, a development platform based on Azure Bot Service and Language Understanding, is now in private preview.  

ONNX Runtime, a high-performance engine for executing trained models represented in the open ONNX format, is now in preview.

PyTorch is now supported in many Azure services, including Azure Machine Learning service, Data Science Virtual Machine, Azure Notebooks, and Visual Studio Code Tools for AI.

The Microsoft Infer.NET machine learning framework has been released as open source. An online book, Model-Based Machine Learning, describes its probabilistic approach with several in-depth examples.

Learning resources

Microsoft Learn, which provides free interactive training for the Azure platform, is now available. Modules include Deep Learning with PyTorch and Computer Vision with Tensorflow.

A tutorial on converting a PyTorch model to ONNX, the cross-platform model-sharing format.

Chromebook Data Science, a free course in Data Science and R from the Johns Hopkins Data Science lab that requires only a browser to take.

Applications

The Ethics Certification Program for Autonomous and Intelligent Systems, a new IEEE standard with the goal of advancing transparency and reducing algorithmic bias in AI systems.

The fast.ai research datasets collection, which includes MNIST, CIFAR 10 and Imagenet, is now available on AWS Open Data.

A comprehensive list of AI Ethics resources, published by fast.ai.

Snip Insights, a Microsoft AI Lab project for image analysis and text extraction from screenshots.

Azure Healthcare AI blueprint, a process to deploy a HIPAA and HITRUST compliant environment in Azure for managing and running healthcare AI experiments.

Find previous editions of the monthly AI roundup here

Computer Vision for Model Assessment

One of the differences between statistical data scientists and machine learning engineers is that while the latter group are concerned primarily with the predictive performance of a model, the former group are also concerned with the fit of the model. A model that misses important structures in the data — for example, seasonal trends, or a poor fit to specific subgroups — is likely to be lacking important variables or features in the source data. You can try different machine learning techniques or adjust hyperparameters to your heart's content, but you're unlikely to discover problems like this without evaluating the model fit.

One of the most powerful tools for assessing model fit is the residual plot: a scatterplot of the predicted values of the model versus the residuals (the difference between the predictions and the original data). If the model fits well, there should be no obvious relation between the two. For example, visual inspection shows us that the model below first fairly well for 19 of the 20 subgroups in the data, but there may be a missing variable that would explain the apparent residual trend for group 19.

Assessing residual plots like these is something of an art, which is probably why it isn't routine in machine learning circles. But what if we could use deep learning and computer vision to assess residual plots like these automatically? That's what Professor Di Cook from Monash University proposes, in the 2018 Belz Lecture for the Statistical Society of Australia, Human vs computer: when visualising data, who wins?  

(The slides for the presentation, created using R, are also available online.) The first part of the talk also includes a statistical interpretation of neural networks, and an intuitive explanation of deep learning for computer vision. The talk also references a related  paper, Visualizing Statistical Models : Removing the Blindfold (by Hadley Wickham, Dianne Cook and Jeike Hofman and published in the ASA Data Science Journal in 2015) which is well worth a read as well.

AI, Machine Learning and Data Science Announcements from Microsoft Ignite

Microsoft Ignite, Microsoft's annual developer conference, wrapped up last week and many of the big announcements focused on artificial intelligence and machine learning. The keynote presentation from Microsoft's Cloud AI lead Eric Boyd showcases the major developments, or you can check out his accompanying blog post for a written summary. 

In this post, I'll dive down into the details of the specific announcements and provide links to the associated recorded presentations from Ignite and other technical documents.

Tools for AI developers

Azure Machine Learning Service (Azure ML) adds several new capabilities. These include a new Python package that developers can use to prepare data for analysis, automate machine learning and deep learning model training, track model performance under various conditions, and automate the model selection process (see below). It supports a variety of execution environments, from local desktops to GPU clusters, and deploy trained models with containers or even high-performance FPGA chips.  (The desktop "Workbench" client, previously in preview, is no longer required and has been deprecated.) Watch the Ignite presentation AI with Azure Machine Learning services: Simplifying the data science process for an overview and demo.

Automated Machine Learning, a new Azure ML service in preview, automates the selection of machine learning algorithms (and associated parameters and hyperparameters) to optimize predictive performance. This blog post provides the technical details, and another blog post describes some practical applications. A Microsoft Research paper describes the probabilistic matrix factorization technique behind the optimization. This Ignite presentation provides further details and includes a demo.  

Visual Studio Code Tools for AI has been updated to provide a convenient interface to Azure ML for users of the popular open-source editor.

SQL Server 2019 is now in public preview, with many improvements shown in this Ignite keynote. Updates include Spark support, and added Linux support for in-database R and Python (watch this Ignite presentation for details).

Azure Databricks is now supported in more regions, offers GPU support for deep learning, and Databricks Delta is now available (in preview) for transactional data capabilities.

Customized AI

Microsoft Bot Framework SDK v4 is now available. The Ignite presentation "Creating Enterprise-Scale Intelligent Agents and Bots" provides an overview and several examples.

Cortana Skills Kit for Enterprise, a development platform based on Azure Bot Service and Language Understanding is now in private preview.  

Pre-trained AI

Speech Service in Azure Cognitive Services is now generally available, and includes a new neural text-to-speech capability for humanlike synthesized speech. This Ignite presentation provides an overview.

This Ignite presentation provides an overview of Cognitive Services with some demos with applications of the pre-trained AI capabilities for computer vision, speech, search and more.

Programs

AI for Humanitarian Action announced. With this new AI for Good program, MIcrosoft will dedicate $40m over five years to deploy AI technologies to assist with disaster recovery; the needs of children, refugees and displaced people; and human rights. A short video summarizes the initiative.

And that's just the highlights — for example, I haven't included the many AI capabilities added to products like Windows and Office. (However I can't resist mentioning the new Insert Data from Picture feature in Excel, which I'm sure I'll be making a lot of use of.) For more from Ignite, take a look at Buck Woody's index of presentations at Ignite, with highlights of videos focused on AI, machine learning, and data science.

AI, Machine Learning and Data Science Roundup: September 2018

A monthly roundup of news about Artificial Intelligence, Machine Learning and Data Science. This is an eclectic collection of interesting blog posts, software announcements and data applications from Microsoft and elsewhere that I've noted over the past month or so.

Open Source AI, ML & Data Science News

ONNX 1.3 released. The standard for representing predictive models adds a cross-platform API for executing graphs on optimized backends and hardware.

A new foundation supporting the development of scikit-learn, the Python machine learning library.

Google open sources "dopamine", a Tensorflow-based framework for reinforcement learning research.

Google adds a "What-If Tool" to Tensorboard, to interactively explore the robustness and algorithmic fairness of machine learning models.

Industry News

Google introduces Dataset Search, an index of public datasets from the environmental and social sciences, government, and journalism.

The 2018.3 update to the Alteryx analytics platform brings interactive graphics, and Spark and Jupyter Notebook integration.

AWS Deep Learning AMIs now provide Tensorflow 1.10 and PyTorch with CUDA 9.2 support.

Rigetti Computing launches the Quantum Cloud Service with a $1M prize for the first application demonstrating "quantum advantage".

Thomas Dinsmore's commentary on Forrester Wave rankings for "Multimodal Predictive Analytics and Machine Learning Platforms" and "Notebook-Based Predictive Analytics and Machine Learning Solutions".

Microsoft News

Microsoft acquires Lobe, a Bay Area startup that produced a drag-and-drop interface for building machine learning models.

NVIDIA GPU Cloud now provides ready-to-run containers with GPU-enabled deep learning frameworks for use on Azure.

Microsoft introduces Azure CycleCloud, a tool for creating, managing, operating, and optimizing burst, hybrid, and cloud-only HPC clusters. Schedulers including Slurm, Grid Engine and Condor are supported.

New features in Azure HDInsight: Spark 2.3.0 and Kafka 1.1 support; ML Services 9.3 integration, with updated R engine and new statistical and machine learning algorithms; Apache Phoenix, for SQL-like queries for data in HBase; Apache Zeppelin, web-based notebooks for querying Phoenix tables; and more.

Learning resources

A review of the algorithms behind AutoML systems for model selection and hyperparameter optimization, from the H2O blog.

Joel Grus's criticisms of Jupyter Notebooks as a platform for reproducible and production-ready computing. Yihui Xiu offers an alternative: RMarkdown.

A comprehensive introduction to mixed-effects models, and fitting them in Python.

A collection of videos, presentations and essays by Brandon Rohrer with approachable explanations of the inner workings of deep learning and machine learning algorithms.

A tutorial on using Azure Batch AI to parallelize forecasts of energy demand.

A meticulously-researched accounting of the resources (natural, technological, and human) that enable an Alexa voice query, presented as highly-detailed map.

A review of the book SQL Server 2017 Machine Learning Services with R.

Applications

Data scientists use text similarity analyses in R to try and identify the author of that anonymous NYT op-ed.

Sketch2Code, an application to translate hand drawings into HTML forms.

Measuring building footprints from satellite images, with semantic segmentation.

Near real-time fraud detection for mobile banking, with a classification model implemented in Azure Machine Learning.

Credit card fraud detection with an Autoencoder neural network, using the Azure Data Science VM.

Shell deploys machine learning and AI systems to avert equipment failures, autonomously direct drill-bits underground, and improve safety.

Find previous editions of the monthly AI roundup here

Videos from NYC R Conference

The videos from the NYC R conference have been published, and there are so many great talks there to explore. I highly recommend checking them out: you'll find a wealth of interesting R applications, informative deep dives on using R (and a few other applications as well), and some very entertaining deliveries. In this post, I wanted to highlight a couple of talks in particular. 

The talk by Jonathan Hersh (Chapman University), Applying Deep Learning to Satellite Images to Estimate Violence in Syria and Poverty in Mexico is both fascinating and a real technical achievement. In the first part of the talk, he uses convolutional neural networks to identify buildings, roads, and vegetation in satellite images, and use that to predict poverty rates across all of Mexico. In the second part, he uses a similar technique to identify violent hotspots in Syria, this time by identifying damaged buildings and roads from the satellite images. In both cases, analyzing satellite images is a lot less expensive (and more consistent) than collecting samples at the ground level. The neural networks were implemented in Theano running on a GPU-enabled Microsoft Azure server.

Moving from applications to platforms, my colleague Marck Vaisman gave a great talk on R for Big Data in the cloud. His talk offers some practical considerations related to working with large data sets in R, and details some cloud technologies for big data that you can connect with R, like Spark and sparklyr (which is easy to use in the cloud with Azure Databricks). 

My talk on R and Minecraft is there as well, if you want to check it out. That talk and all of the rest from the conference are available for viewing at the link below.

New York R Conference: 2018 

AI, Machine Learning and Data Science Roundup: August 2018

A monthly roundup of news about Artificial Intelligence, Machine Learning and Data Science. This is an eclectic collection of interesting blog posts, software announcements and data applications I've noted over the past month or so.

Open Source AI, ML & Data Science News

ONNX Model Zoo is now available, providing a library of pre-trained state-of-the-art models in deep learning in the ONNX format.

In the 2018 IEEE Spectrum Top Programming Language rankings, Python takes the top spot and R ranks #7.

Julia 1.0 has been released, marking the stabilization of the scientific computing language and promising forwards compatibility.

A retrospective on the R Project, now in its 25th year, from Significance Magazine.

Industry News

Google announces Cloud AutoML, a beta service to train vision, text categorization, or language translation models from provided data. The fast.ai blog evaluates the claim that AutoML works without the need for machine learning skills.

Google announces Edge TPU, a hardware chip and associated software to bring AI capabilities to edge devices.

Oracle open-sources GraphPipe, a network protocol designed to simplify and standardize transmission of machine learning data between remote processes.

AWS Deep Learning AMIs now include ONNX, making it easier to deploy exported deep learning models.

Amazon Rekognition is available in two new regions: Seoul and Mumbai. The video analysis service caused a stir when the ACLU reported it matching pictures of members of Congress to a database of criminal mugshots.

RStudio introduces Package Manager, a repository management server product to organize and centralize R packages.

Data from the O'Reilly Machine Learning Adoption Survey reveal that most machine learning models are built by in-house data science or product development teams, with only 3% adopting cloud-based ML service.

Anaconda Enterprise 5.2 adds capabilities for GPU-accelerated machine learning.

Microsoft News

TechNative's review of Microsoft's AI philosophy and technologies, and the roadmap ahead.

Microsoft R Open 3.5.1 is now available.

Power BI Desktop now offers Python integration (in preview).

AI for Accessibility, a $25M Microsoft program applying artificial intelligence to help people with disabilities.

Azure SQL Database now offers reserved capacity at a discounted rate, and reservation discounts for virtual machines now apply when using differing VM sizes within a group.

Learning resources

The Chartmaker Directory, an index of dozens of data visualization types with examples in more than 30 software tools.

An overview of the benefits and limitations of FPGUs compared to CPUs and GPUs for numeric computing.

A guide to deep learning applied to natural language processing, for those new to the field.

Containerized R and Python Web services with Docker and Microsoft ML Server.

Design, architecture, and implementation of an R-based recommendation system in Azure.

A guide to distributed deep learning in Spark, using Azure Batch and Azure HDInsight.

Google's Machine Learning Guides, with machine learning tips and a guide to text classification.

Field Guide to Machine Learning, a 6-part video series from Facebook Research.

What Data Scientists Really Do, according to a Harvard Business Review article.

Survey analytics company Crunch uses R to provide a data visualization service.

Finding optimal staff composition for a professional services company, with Azure Machine Learning.

Find previous editions of the monthly AI roundup here

A quick tour of AI services in Azure

If you're after a quick overview of some of the services available in Azure to build AI-enabled applications, you might want to check out the 6-minute video below. It provides a tour of three services:

• Azure Cognitive Services, pre-trained machine learning models to detect and characterize faces, text sentiment, and more 
• Azure Machine Learning, a development, deployment and management framework for machine learning models you train yourself, and
• Azure Bot Service, a framework for creating intelligent chat-based interfaces.

Want to learn more? You can find more videos and resources at the Azure Essentials page.