Revolutions

Because it's Friday: Shake that globe

It's been a while since a major earthquake has overtaken the headlines, but as this animation shows (source data here), major earthquakes aren't actually all that rare: it's just (relatively) rare that they occur in heavily populated areas.

It's a really nice information design: the size of the circle represents the magnitude of each earthquake, and the color represents its depth. The way the circles appear and then slowly shrink away is a great way of visualizing the geographic impact of these inherently short-lived incidents. I also like the way it's represented on an actual globe, even though half of the data is obscured at any one time, as it gives a better sense of the geographic relationships. Nonetheless, NOAA also produced a whole-earth projection version of the same animation:

That's all from us for this week. We'll be back on Monday, reporting from the useR! conference in Brussels. Have a great weekend!

 

R 3.4.1 "Single Candle" released

The R core team announced today the release of R 3.4.1 (codename: Single Candle). This release fixes a few minor bugs reported after the release of R 3.4.0, including an issue sometimes encountered when attempting to install packages on Windows, and problems displaying functions including Unicode characters (like "日本語") in the Windows GUI. The other fixes are mostly relevant to packages developers and those building R from source, and you can see the full list in the announcement linked below.

At the time of writing, Windows builds are already available at the main CRAN cloud mirror, and the Debian builds are out, but Mac builds aren't there quite yet (unless you want to build from source). Binaries for all platforms should propagate across the mirror network over the next couple of days.

r-announce mailing list: R 3.4.1 is released

How R is used by the FDA for regulatory compliance

I was recently alerted (thanks Maëlle and Mikhail!) to an enlightening presentation from last years' useR! conference. (This year's useR! conference takes place next week in Belgium.) Paul H Schuette, Scientific Computing Coordinator at the FDA Center for Drug Evaluation and Research (CDER), talked about how R is used in the process of regulating and approving drugs at the FDA. 

In what has become a common theme of FDA presentations at R conferences, Schuette refutes the fallacy that SAS is the only software that can be used for FDA submissions, by sponsors such as pharmaceutical companies. On the contrary, he says "sponsors may propose to use R, and R has been used by some sponsors for certain types of analyses and simulations (post-market)."

The myth persists despite the FDA's Statistical Software Clarifying Statement declaring that any suitable software can be used. This is probably because some data-exchange regulations do require the use of the "XPT" (also known as SAS XPORT) file format, but that data format is an open standard and not restricted to SAS. XPT files can be read into R with the built-in read.xport function, and exported from R with the write.xport function in the SASxport package. (If you have legacy data in other SAS formats, here's a handy SAS macro to export XPT files.) The R Foundation also provides guidance on how R complies with other FDA regulations in the document R: Regulatory Compliance and Validation Issues A Guidance Document for the Use of R in Regulated Clinical Trial Environments. 

In addition to sponsors using R in submission, R is also used internally at the FDA. Statisticians there may use the statistical package of their choice, provided it's fit for the purpose. The software used includes SAS, R, Minitab and Stata. Schuette notes that R is used specifically for:

• Statistical review of data analysis in clinical trial submissions. The primary goal here is, "Can we, on our own, replicate the conclusions of the sponsor?"
• Methodology development, innovation and evaluation.
• Graphics (in some cases, the detailed information folded and included with prescription medications feature R graphics).
• Simulations.
• The openFDA initiative, including this LRT Signal Analysis for a Drug Shiny application.

Check out the entire presentation, embedded below.

Channel 9: Using R in a regulatory environment: FDA experiences.

 

Useful tricks when including images in Rmarkdown documents

Rmarkdown is an enormously useful system for combining text, output and graphics generated by R into a single document. Images, in particular, are a powerful means of communication in a report, whether they be data visualizations, diagrams, or pictures. To maximize the power of those images, Zev Ross has created a comprehensive list of tips and tricks for working with images in R markdown documents. Those tips include:

• How to change the width and height of images created by R, and of imported image files.
• How to change the resolution of your images. (Usually not an issue for HTML output, where the resolution is implied by the figure height/width, but relevant for Word and PDF documents and when consuming output on Retina displays).
• How to automatically reduce the size of PNG images generated by R.
• How to insert multiple images (say, all images in a directory) into your document at once.
• How to apply CSS styles (like borders and background colors) to individual images (or R code or other Rmarkdown chunks, for that matter).

While Zev's guide focuses on manipulating images as generated by R, this might also be a good time to revisit these tips on making the graphics themselves as attractive and useful as possible. Although written many years ago, most of these tips are still relevant, in particular:

• If you plan to print (or these days, view on high-resolution displays), use PDF output
• Use PNG for R graphics output, and use a high-resolution if you plan to print. Don't ever use JPG for R graphics output.
• Think about the aspect ratio of your graphics.
• Remove unused whitespace around your graphics (more of an issue for base R grahics; ggplot2 handles this pretty well already).

You may also find the Rmarkdown cheat sheet helpful if you're working with Rmarkdown documents.

ZevRoss Technical Blog: Tips and tricks for working with images and figures in R Markdown documents

Because it's Friday: Mario in the Park

I got my first chance to use HoloLens just a couple of weeks ago. It was pretty amazing to see a virtual wind turbine appear in the room with me, and to be able to walk around it and see how it was performing. But here's a much more fun application of HoloLens: a recreation of the 2-D Super Mario "World 1-1" in the very 3-D world of New York's Central Park:

Sadly the blocks and platforms are purely virtual, which constrains the real-world player to walking on the ground. But it does make for some amusing antics for the passers-by (although in true New Yorker style, most ignore the random jumping and pointing!).

That's all for us for this week. Have a great weekend: we'll be back on Monday!

The R community is one of R's best features

R is incredible software for statistics and data science. But while the bits and bytes of software are an essential component of its usefulness, software needs a community to be successful. And that's an area where R really shines, as Shannon Ellis explains in this lovely ROpenSci blog post. For software, a thriving community offers developers, expertise, collaborators, writers and documentation, testers, agitators (to keep the community and software on track!), and so much more. Shannon provides links where you can find all of this in the R community:

• #rstats hashtag — a responsive, welcoming, and inclusive community of R users to interact with on Twitter
• R-Ladies — a world-wide organization focused on promoting gender diversity within the R community, with more than 30 local chapters
• Local R meetup groups — a google search may show that there's one in your area! If not, maybe consider starting one! Face-to-face meet-ups for users of all levels are incredibly valuable
• Rweekly — an incredible weekly recap of all things R
• R-bloggers — an awesome resource to find posts from many different bloggers using R
• DataCarpentry and Software Carpentry — a resource of openly available lessons that promote and model reproducible research
• Stack Overflow — chances are your R question has already been answered here (with additional resources for people looking for jobs)

I'll add a couple of others as well:

• R Conferences — The annual useR! conference is the major community event of the year, but there are many smaller community-led events on various topics. 
• Github — there's a fantastic community of R developers on Github. There's no directory, but the list of trending R developers is a good place to start.
• The R Consortium — proposing or getting involved with an R Consortium project is a great way to get involved with the community

As I've said before, the R community is one of the greatest assets of R, and is an essential component of what makes R useful, easy, and fun to use. And you couldn't find a nicer and more welcoming group of people to be a part of.

To learn more about the R community, be sure to check out Shannon's blog post linked below.

ROpenSci Blog: Hey! You there! You are welcome here

Interactive R visuals in Power BI

Power BI has long had the capability to include custom R charts in dashboards and reports. But in sharp contrast to standard Power BI visuals, these R charts were static. While R charts would update when the report data was refreshed or filtered, it wasn't possible to interact with an R chart on the screen (to display tool-tips, for example). But in the latest update to Power BI, you can create create R custom visuals that embed interactive R charts, like this:

The above chart was created with the plotly package, but you can also use htmlwidgets or any other R package that creates interactive graphics. The only restriction is that the output must be HTML, which can then be embedded into the Power BI dashboard or report. You can also publish reports including these interactive charts to the online Power BI service to share with others. (In this case though, you're restricted to those R packages supported in Power BI online.)

Power BI now provides four custom interactive R charts, available as add-ins:

• Time-series forecasting using the forecast package, with interactive range selection, data selection and tooltips (code)
• Time-series forecasting using the arima package, with data selection and tooltips (code)
• K-means clustering, with data selection and tooltips (code)
• Spline smoothing charts, with data selection and tooltips (code)

You can also create your own custom R visuals. The documentation explains how to create custom R visuals from HTML output, and you can also use the code on Github for the provided visuals linked above as a guide. For more on the new custom visuals, take a look at the blog post linked below.

Microsoft Power BI Blog: Interactive R custom visuals support is here!

Updated Data Science Virtual Machine for Windows: GPU-enabled with Docker support

The Windows edition of the Data Science Virtual Machine (DSVM), the all-in-one virtual machine image with a wide-collection of open-source and Microsoft data science tools, has been updated to the Windows Server 2016 platform. This update brings built-in support for Docker containers and GPU-based deep learning. 

GPU-based Deep Learning. While prior editions of the DSVM could access GPU-based capabilities by installing additional components, everything is now configured and ready at launch. The DSVM now includes GPU-enabled builds of popular deep learning frameworks including CNTK, Tensorflow, and MXNET. It also includes Microsoft R Server 9.1, and several machine-learning functions in the MicrosoftML package can also take advantage of GPUs. Note that you will need to use an N-series Azure instance to benefit from GPU acceleration, but all of the tools in the DSVM will also work on regular CPU-based instances as well.

Docker Containers. Windows Server 2016 includes Windows Containers, and the DSVM also includes the Docker Engine. This means you can easily run Windows containers from Docker Hub or the Azure Container registry. This is limited to Windows-based containers right now, but you can use Linux-based containers on the Data Science Virtual Machine for Ubuntu Linux, which likewise supports GPU-enabled deep learning.

It's easy to launch a Windows 2016 DSVM on Azure, and the documentation will help you get started using it. (An Azure account is required, but free Azure credits are available to help get you started.) You can find more information about this update at the link below.

Cortana Intelligence and Machine Learning Blog: Introducing the new Data Science Virtual Machine on Windows Server 2016

R leads, Python gains in 2017 Burtch Works Survey

For the past four years, recruiting firm Burtch Works has conducted a simple survey of data scientists with just one question: "Which do you prefer to use — SAS, R or Python". The results for this year's survey of 1,046 respondents are in:

• R: 40% (2016: 42%)
• SAS: 34% (2016: 39%)
• Python: 26% (2016: 20%)

Compared to last year's results, Python has gained 6 percentage points, mainly at the expense of SAS. (2016 results do not add to 100% due to rounding.) The trend for the commercial tool compared to the open-source tools is apparent looking at the data from all four years:

(Note: Python has only been an option for respondents in the last two surveys.) Demographic breakdowns, included in the survey analysis linked below, show that SAS remains popular with more years of experience (more than 16 years), although becomes less popular with greater educational attainment. Burtch Works will also be holding a mini webinar on June 29 to discuss the results (details at the link below).

Burtch Works: 2017 SAS, R, or Python Flash Survey Results

Using sparklyr with Microsoft R Server

The sparklyr package (by RStudio) provides a high-level interface between R and Apache Spark. Among many other things, it allows you to filter and aggregate data in Spark using the dplyr syntax. In Microsoft R Server 9.1, you can now connect to a a Spark session using the sparklyr package as the interface, allowing you to combine the data-preparation capabilities of sparklyr and the data-analysis capabilities of Microsoft R Server in the same environment.

In a presentation by at the Spark Summit (embedded below, and you can find the slides here), Ali Zaidi shows how to connect to a Spark session from Microsoft R Server, and use the sparklyr package to extract a data set. He then shows how to build predictive models on this data (specifically, a deep Neural Network and a Boosted Trees classifier). He also shows how to build general ensemble models, cross-validate hyper-parameters in parallel, and even gives a preview of forthcoming streaming analysis capabilities.

Any easy way to try out these capabilities is with Azure HDInsight 3.6, which provides a managed Spark 2.1 instance with Microsoft R Server 9.1.

Spark Summit: Extending the R API for Spark with sparklyr and Microsoft R Server