Revolutions

Let me tell you what you missed at BUILD

If you weren't able to attend last weeks' BUILD conference in Seattle, you can always catch up on the keynotes and the session talks online, or read this recap by Charlotte Yarkoni. Or, if you have 45 minutes on Wednesday next week you can join Tim Heuer and me as we recap some of the biggest announcements of the conference, and few things you might have missed as well.  We'll focus mainly on the AI and Machine Learning related announcements, including:

• Project Brainwave, and the performance you can get from hardware-accelerated, real-time AI
• New, platform-agnostic options for AI models, including ONNX and docker containers
• New Azure ML Python packages for forecasting, computer vision, and text analytics
• Projects from the Microsoft AI Lab and AI for Accessibility
• ... and a few extra surprises as well.

The webinar will be at 10AM Pacific time on Wednesday, May 23 when you'll also have the opportunity to ask us live questions. You can register for the webinar below, which will also give you access to the on-demand recording if you can't make the live sessions. We hope to see you there!

Azure Webinar Series: Top Azure Takeaways from Microsoft Build

 

Mind Bytes: Solving Societal Challenges with Artificial Intelligence

By Francesca Lazzeri (@frlazzeri), Data Scientist at Microsoft

Artificial​ ​intelligence​ ​(AI)​ solutions​ ​are playing a growing role in our everyday life, ​​and​ ​are​ ​being adopted​ ​broadly, in private and public domains. ​ ​While​ ​the​ ​notion​ ​of​ ​AI​ ​has​ ​been around​ ​for​ ​over​ ​sixty​ ​years, real-world​ ​AI scenarios and applications​ ​have​ ​only​ ​increased​ ​in​ ​the​ ​last​ ​decade​ ​due​ ​to​ ​three​ ​simultaneous events: ​ ​​improved​ ​computing​ ​power, capability​ ​to​ ​capture​ ​and​ ​store​ ​massive​ ​amounts​ ​of​ ​​​data, and faster​ ​algorithms.  

AI solutions help determine the commercials you see online, the movie you will watch with your family, the routes you may take to get to work. Beyond​ ​the​ most popular apps, ​these​ ​systems​ ​are​ ​also​ ​being​ ​implemented​ ​in​ ​critical areas​ such as​ health care, immigration policy, ​finance, ​and​ the ​workplace. The design​ ​and​ ​implementation​ ​of​ ​these AI ​tools​ ​presents​ ​deep societal​ ​challenges​ ​​that​ ​will​ ​shape​ ​our​ ​present​ ​and near​ ​future. ​ ​ ​

To identify and contribute to the current dialog around the emerging societal challenges that AI is bringing, we attended Mind Bytes at the University of Chicago. Mind Bytes is an annual research computing symposium and exposition designed to showcase to more than 200 attendees cutting-edge research and applications in the field of AI. Some of the most interesting demos and posters presented were:

• An Online GIS Platform to Support China’s National Park System establishment – Manyi Wang
• 17 Years in Chicago Neighborhoods: Mapping Crime Trajectories in 801 Census Tracts – Liang CAI
• Characterizing the Ultrastructural Determinants of Biophotonic Reflectivity in Cephalopod Skin: A Challenge for 3D Segmentation – Stephen Senft, Teodora Szasz, Hakizumwami B. Runesha, Roger T. Hanlon
• Exploring Spatial Distribution of Risk Factors for Teen Pregnancy – Emily Orenstein and Iris Mire

The Mind Bytes panel on Solving Societal Challenges with Artificial Intelligence represented an incredible occasion for us to interact with students and many other AI experts from the field, and understand how we can work together to ensure that AI is developed in a responsible manner, so that people will trust it and deploy it broadly, both to increase business and personal productivity and to help solve societal problems.

Specifically, the panel focused on three fundamental areas of discussion related to current and future AI aspects:

• What areas do you see AI most successfully applied?
• What is the major challenge that you think should be met before getting the full benefit of AI?
• What can researchers and students do now to build a system able to address those challenges?

The following sections aim at answering these questions in more detail and reflect on the latest academic and industry research. AI is already with us, and we are now faced with important choices on how it will be designed and applied. Most promisingly, the approaches observed at Mind Bytes demonstrate that there is growing interest in developing AI that is attuned to underlying issues of fairness and equality.

Speed up R with Parallel Programming in the Cloud

This past weekend I attended the R User Day at Data Day Texas in Austin. It was a great event, mainly because so many awesome people from the R community came to give some really interesting talks. Lucy D’Agostino McGowan has kindly provided a list of the talks and links to slides, and I thoroughly recommend checking it out: you're sure to find a talk (or two, or five or ten) that interests you.

My own talk was on Speeding up R with Parallel Programming in the Cloud, where I talked about using the doAzureParallel package to launch clusters for use with the foreach function, and using aztk to launch Spark clusters for use with the sparklyr package. I've embdeded the slides below: it's not quite the same without the demos (and sadly there was no video recording), but I've included screenshots in the slides to hopefully make it clear.

Live Free or Dichotomize: Wrangling Data Day Texas Slides

Registration and talk proposals open Monday for useR!2018

Registration will open on Monday (January 22) for useR! 2018, the official R user conference to be held in Brisbane, Australia July 10-13. If you haven't been to a useR! conference before, it's a fantastic opportunity to meet and mingle with other R users from around the world, see talks on R packages and applications, and attend tutorials for deep dives on R-related topics. This year's conference will also feature keynotes from Jenny Bryan, Steph De Silva, Heike Hofmann, Thomas Lin Pedersen, Roger Peng and Bill Venables. It's my favourite conference of the year, and I'm particularly looking forward to this one.

This video from last year's conference in Brussels (a sell-out with over 1,1000 attendees) will give you a sense of what a useR! conference is like:

The useR! conference brought to you by the R Foundation and is 100% community-led. That includes the content: the vast majority of talks come directly from R users. If you've written an R package, performed an interesting analysis with R, or simply have something to share of interest to the R community, consider proposing a talk by submitting an abstract. (Abstract submissions are open now.) Most talks are 20 minutes, but you can also propose a 5-minute lightning talk or a poster. If you're not sure what kind of talk you might want to give, check out the program from useR!2017 for inspiration. R-Ladies, which promotes gender diversity in the R community, can also provide guidance on abstracts. Note that all proposals must comply with the conference code of conduct.

Early-bird registrations close on March 15, and while general registration will be open until June my advice is to get in early, as this year's conference is likely to sell out once again. If you want to propose a talk, submissions are due by March 2 (but early submissions have a better chance of being accepted). Follow @user!2018_conf on Twitter for updates about the conference, and click the links below to register or submit an abstract. I look forward to seeing you in Brisbane!

Update Jan 19: Registrations will now open January 22

useR! 2018: Registration; Abstract submission

eRum 2018 to be held in Budapest, May 14-18

The main international R user conference, useR!, is typically hosted in Europe every other year. Back in 2016 when useR! was held in the USA, an alternative conference was held in Poland. Next year (when the useR! conference will be in Australia) the tradition continues, and the 2018 eRum (European R Users Meeting) will be held in Budapest, Hungary on May 14-16.

Around 400-500 attendees are expected, and the program committee is sure to put together an excellent collection of workshops and talks. If you'd like to contribute, workshop proposals are due on January 14, and abstracts for talks and posters are due on February 25. You can submit your proposals here.

Conference registration will open in early January, but you can follow @erum2018 on Twitter to be notified or simply check at the URL below for ticket information.

eRum 2018: About the Conference

R/Finance 2018, Chicago June 1-2

The tenth annual R/Finance conference will be held in Chicago, June 1-2 2018. This is a fantastic conference for anyone working with R in the finance industry, or doing research around R in finance in the academic sector. This community-led, single-track conference always features a program of interesting talks in a convival atmosphere.

If you'd like to contribute a 20-minute presentation or a 5-minute lightning talk, the call for papers is open though February 2, 2018. If you'd like to attend, registrations will open in March 2018.

Microsoft is proud to be a continuing sponsor of R/Finance 2018. You can a summary of lat year's conference here, and watch recordings of the talks on Channel 9. For more on the 2018 conference, follow the link below. 

R/Finance 2018: Overview

Highlights from the Connect(); conference

Connect();, the annual Microsoft developer conference, is wrapping up now in New York. The conference was the venue for a number of major announcements and talks. Here are some highlights related to data science, machine learning, and artificial intelligence:

• There have been several updates to Azure Machine Learning, including the ability to deploy models to the Azure IoT Edge using Docker containers for intelligent Internet-of-Things applications. 

• Microsoft has joined the MariaDB Foundation, and now offers fully-managed MariaDB databases in Azure. (MySQL and MariaDB creator Monty Widenius made the announcement from Finland.) You can access MariaDB databases on Azure as you would any other instance, including via the RMariaDB package for R.

• Databricks Spark clusters are now available in Azure as a fully-managed services. You can connect to these instances using standard Spark APIs, for example the sparklyr package for R. 

• New capabilities for Azure Cosmos DB: Apache Cassandra API; Gremlin API support for querying graph databases (preview), and enhanced MongoDB and Spark connectivity.

• Visual Studio Tools for AI is now available (in preview), making it easy to deploy models from deep learning frameworks (including CNTK, Tensorflow and Caffe) to Azure. This video from Connect() demonstrates how it works.

Lastly, I wanted to share this video presented at the conference from Stack Overflow. Keep an eye out for R community luminary David Robinson programming in R!

You can find more from the Connect conference, including on-demand replays of the talks and keynotes, at the link below.

Microsoft: Connect(); November 15-17, 2017

Recap: EARL Boston 2017

By Emmanuel Awa, Francesca Lazzeri and Jaya Mathew, data scientists at Microsoft

A few of us got to attend EARL conference in Boston last week which brought together a group of talented users of R from academia and industry. The conference highlighted various Enterprise Applications of R. Despite being a small conference, the quality of the talks were great and showcased various innovative ways in using some of the newer packages available for use in the R language. Some of the attendees were veteran R users while some were new comers to the R language, so there was a mix in the level of proficiency in using the R language.  

R currently has a vibrant community of users and there are over 11,000 open source packages. The conference also encouraged women to join their local chapter for R Ladies with the aim of increasing the participation of women at R conferences and increasing the number of women who contribute R packages to the open source community.

The team from Microsoft got to showcase some of our tools namely the Microsoft ML Server and our commitment to support the open language R. Some of the Microsoft earned sessions were:

1. Deep Learning with R – Francesca Lazzeri
2. Enriching your Customer profile at Scale using R Server – Jaya Mathew, Emmanuel Awa & Robert Alexander
3. Developing Deep Learning Applications with CNTK – Ali Zaidi

Microsoft was a sponsor at the event and had a booth at the conference where there was a live demo using the Cognitive Services APIs — namely the Face API — to detect age, gender, facial expression.

In addition, some of the other interesting talks were:

1. When and Why to Use Shiny for Commercial Applications – Tanya Cashorali
2. HR Analytics: Using Machine Learning to Predict Employee Turnover – Matt Dancho
3. Using R to Automate the Classification of E-commerce Products – Aidan Boland
4. Leveraging More Data using Data Fusion in R – Michael Conklin

All the slides from the conference will be available at the conference website shortly. For photos from the conference, visit EARL’s twitter page.

Two upcoming webinars

Two new Microsoft webinars are taking place over the next week that may be of interest:

AI Development in Azure using Data Science Virtual Machines

The Azure Data Science Virtual Machine (DSVM) provides a comprehensive development and production environment to Data Scientists and AI-savvy developers. DSVMs are specialized virtual machine images that have been curated, configured, tested and heavily used by Microsoft engineers and data scientists. DSVM is an integral part of the Microsoft AI Platform and is available for customers to use through the Microsoft Azure cloud. In this session, we will first introduce DSVM, familiarize attendees with the product, including our newest offering, namely Deep Learning Virtual Machines (DLVMs). That will be followed by technical deep-dives into samples of end-to-end AI development and deployment scenarios that involve deep learning. We will also cover scenarios involving cloud based scale-out and parallelization.

This webinar runs from 10-11AM Pacific Time on Thursday October 26, and is presented by Gopi Kumar, Principal Program Manager, Paul Shealy, Senior Software Engineer, and Barnam Bora, Program Manager, at Microsoft. Register here.

Document Collection Analysis

With the extremely large volumes of data, especially unstructured text data, that are being collected every day, a huge challenge facing customers is the need for tools and techniques to organize, search, and understand this vast quantity of text. This webinar demonstrates an efficient and automated end-to-end workflow around analyzing large document collections and serving your downstream NLP tasks. We’ll demonstrate how to summarize and analyze a large collection of documents, including techniques such as phrase learning, topic modeling, and topic model analysis using the Azure Machine Learning Workbench.

This webinar runs from 10-11AM Pacific Time on Tuesday October 31, and will be presented by Ke Huang, Data Scientist at Microsoft. Register here.

Recap: Applications of R at EARL London 2017

The fourth EARL London conference took place last week, and once again it was an enjoyable and informative showcase of practical applications of R. Kudos to the team from Mango for hosting a great event featuring interesting talks and a friendly crowd.

As always, there were more talks on offer than I was able to attend (most of the event was in three parallel tracks), but here are a few highlights that I was able to catch:

Jenny Bryan from RStudio gave an outstanding keynote: "Workflows: You Should Have One" (view the slides here). The talk was a really useful collection of good practices for data analysis with R. Note I didn't say "best practices" there, and that's something that resonated with me from Jenny's talk: sometimes the best practices are the ones that are "good enough" and don't add unnecessary complexity or fragility. Software Carpentry's "Good Enough Practices for Scientific Computing" is a great read that I hadn't come across before this talk.

Rachel Kirkham from the National Audit Office uses R and Shiny to scrutinize public spending on behalf of taxpayers in the UK. (View the slides here.) One interesting beneficial aspect of R for this application is the ability to bring R to the data. (Update Sep 21: I've edited this to remove my error mischaracterizing Rachel's talk. Rachel clarified via email: "We are deploying Shiny apps on client systems to eliminate the need to transfer data and associated risks of data loss. We are fully compliant with the Data Protection Act and associated data handling policies.")

Joy McKenney from Northumbrian Water gave a truly fascinating talk on using R to monitor and predict flows in a sewer system. (View the slides here.) One surprising application from this talk was being able to model whether an overflowing sewer line is due to rainfall or because of a blockage in the system — a tool that can be used to detect "fatbergs" clogging up the system.

Pieter Vos from Philips Research showed how they deploy applications to doctors to evaluate things like cancer risk. (View the slides here.) The applications call out to R to make the underlying statistical calculations.

Joe Cheng from RStudio gave a remarkable talk on a major concept for the R language itself: "promises". (View the slides here.) With promises, you can ask for the results of a long-running computation, but get back to the R command line instantaneously. The results will be available to you, well, when they're ready. It sounds like magic, but it's already working (in beta) with the promises package and promises (pun intended) to bring more responsiveness to Shiny applications.

Ashley Turner from Transport for London gave a talk that I was very sad I couldn't see (it conflicted with my own session), but fortunately the slides are available. R has been used for over 2 years to model how Londoners get about via car, bus and Tube. It was used for an official report on London transportation, which included some fascinating data on the routes certain commuters prefer for getting from (say) Kings Cross to Waterloo. 

Matthew Upson from the Government Digital Service reported on the ongoing program to implement a reproducible data science workflow for creating official reports. (View the slides here.) He says that this process has reduced production time for some official reports by 75%, and you can learn more about the program here.

I was also informed by the organizers that my own talk on Reproducible Data Science with R (slides here) won the "award" for loudest presentation at the conference. I think the audio desk had the volume up a little too high for this video! 

As I said, there are many more excellent talks beyond those listed above. You can explore the other talks by clicking here and then clicking through each speaker portrait — slides, where available, are linked at the bottom of each abstract.

The next EARL conference will take place in Boston, November 1-3 and promises to include even more practical applications of R. I hope to see you there!