Google uses R to calculate ROI on advertising campaigns

Google has just released a new package for R: CausalImpact. Amongst many other things, this package allows Google to resolve the classical conundrum: how can we asses the impact of an intervention (for example, the effect of an advertising campaign on website clicks) when we can't know what would have happened if we hadn't run the campaign? For a marketer, the worry is that the spike in clicks was partially or wholly the result of something unrelated (say, a general increase in web traffic) rather than your campaign.

The CausalImpact package uses Bayesian structural time-series models to resolve this question. All you need is a second time series to act a a "virtual" control, which is unaffected by your actions but which is still subject to the extraneous effects you're worried about. (For the marketing example, you might choose web clicks from a region where the campaign didn't run.) Then, you can model the extraneous effects and subtract them from your actual results, so see how your things would have played out had the intervention not occurred.

In the chart below (from the Google Open Source blog post) you can see the results of the campaign in black, with the campaign launch at the dotted line. The blue line shows the estimated results had the campaign not run, clearly showing that it was effective.

Google uses R and the CausalImpact package to measure the return-on-investment on advertising campaigns its customers run:

We've been testing and applying structural time-series models for some time at Google. For example, we've used them to better understand the effectiveness of advertising campaigns and work out their return on investment. We've also applied the models to settings where a randomised experiment was available, to check how similar our effect estimates would have been without an experimental control.

Now that Google has made the CausalImpact package available, you can do the same for any kind of intervention, as long as you have a time series of results before and after the intervation, and a "control" time series where the intervention had no effect. Read more about the package and the methodology behind it at the link below.

Google Open Source Blog: CausalImpact: A new open-source package for estimating causal effects in time series

Comments

[gwern]

Seems like a strange approach - why assume that Google Trends or another time series is a good control? You'd think they'd validate the approach on more than one experiment in the paper...

[Emaasit]

Great article.

[Skylar]

Good share.