Revolutions

The PMML Revolution: Predictive analytics at the speed of business

This guest post is by Alex Guazzelli, VP of Analytics at Zementis Inc. -- ed.

PMML, the Predictive Model Markup Language, is the de facto standard to represent predictive analytics and data mining models. With PMML, it is extremely easy to move a predictive solution from one system to another, since it avoids proprietary issues and incompatibilities.

Companies around the globe are benefiting from PMML to make instant use of their predictive solutions. With PMML, there is no need for custom coding: you can easily move your solution from the scientist’s desktop, where it was built, to the production environment, where it is operationally deployed. Companies also use PMML as the common language between service providers and external vendors. In this way, it defines a single and clear process for the exchange of predictive solutions. It becomes the bridge not only between data analysis, model building, and deployment systems, but also between all the people and teams involved in the analytical process. This is extremely important, since PMML is used to disseminate knowledge and best practices, and to ensure transparency.

All the top analytical tools, commercial and open-source, support PMML. And, the language itself has reached a great level of maturity and refinement. PMML 4.1, its latest version, makes it extremely easy for predictive solutions to be represented in an open and standard way. With PMML, you can represent a myriad of pre- and post-processing steps, besides the predictive modeling techniques per se. PMML 4.1 allows for multiple models (model composition, chaining, segmentation, and ensemble, which includes random forest models), to be represented by a single and concise language element. It also allows for model outputs to be transformed into business decisions. Therefore, a PMML file is able to represent the entire solution, from raw data to business decision, with one or multiple predictive models.

The availability of a standard such as PMML combined with scoring solutions in the cloud, for Hadoop, and in-database make it possible for predictive analytics to fulfill its promise and crack the big data code. Zementis, Inc. has been in the forefront of PMML-based scoring, first through its ADAPA Scoring Engine, which is available for on-site deployment or as a service on cloud (Amazon and IBM), and lately through its Universal PMML Plug-in which is offered for a range of databases and for Hadoop. Zementis has partnered with Revolution Analytics, so that predictive solutions built in R can benefit from the vast scoring infrastructure already in place. I am proud to be associated with Zementis and excited to be part of an ever-growing PMML community.

A PMML package for R that exports all kinds of predictive models is available directly from CRAN.

Traditionally, the PMML Package offered support for the following data mining algorithms:

• ksvm (kernlab): Support Vector Machines
• nnet: Neural Networks
• rpart: C&RT Decision Trees 
• lm & glm (stats): Linear and Binary Logistic Regression Models 
• arules: Association Rules
• kmeans and hclust: Clustering Models 

Recently, it has been expanded to support: 

• multinom (nnet): Multinomial Logistic Regression Models;
• glm (stats): Generalized Linear Models for classification and regression with a wide variety of link functions 
• randomForest: Random Forest Models for classification and regression (click HERE for examples);
• rsf (randomSurvivalForest): Random Survival Forest Models;

And, this expansion is still on-going as the R community implements support for other packages and techniques. For more on the PMML package, please take a look at the paper we published with Graham Williams from Togaware in “The R Journal”. For that just follow the link below:

PMML: An Open Standard for Sharing Models

There may be quite a few reasons for you to move your predictive solution from R to an independent deployment platform. Among them, you may want parallel execution on big data or real-time scoring for applications such as fraud detection or recommender systems. With PMML you can easily move your model to the cloud or inside the database for scoring. Or, even have it executed on Hadoop. It is really up to you! On top of that, PMML allows for side-by-side deployment of predictive assets from R as well as other commercial data mining tools, supporting a multi-vendor environment as well as platform independent deployment.

More and more companies and individuals are using the PMML standard for the obvious benefits it provides, putting their predictive solutions on the fast track. With PMML, the speed of predictive solutions can be on par with the speed of business.

Dr. Alex Guazzelli is the VP of Analytics at Zementis Inc. where he is responsible for developing core technology and predictive solutions under ADAPA, a PMML-based decisioning platform. With more than 20 years of experience in predictive analytics, Dr. Guazzelli holds a PhD in Computer Science from the University of Southern California and has co-authored the book PMML in Action: Unleashing the Power of Open Standards for Data Mining and Predictive Analytics, now in its second edition (paperback and kindle). You can follow him at @DrAlexGuazzelli.

Cheat sheet for prediction and classification models in R

Ricky Ho has created a reference a 6-page PDF reference card on Big Data Machine Learning, with examples implemented in the R language. (A free registration to DZone Refcardz is required to download the PDF.) The examples cover:

• Predictive modeling overview (how to set up test and training sets in R)
• Linear regression (using lm)
• Logistic regression (using glm)
• Regression with regularization (using the glmnet package)
• Neural networks (using nnet)
• Support vector machines (using tune.svm from the e1071 package)
• Naïve Bayes models (using naiveBayes from the e1071 package)
• K-nearest-neighbors classification (using the knn function from the class package)
• Decision trees (using rpart)
• Ensembles of trees (using the randomForest package)
• Gradient boosting (using the gbm package)

The examples use the traditional built-in R data sets (such as the iris data, used to create the neural network above), so there's unfortunately not much of a "big data" aspect to the reference card. But if you're just getting started with prediction and classification models in R, this cheat sheet is a useful guide.

DZone Refcardz:  Big Data Machine Learning Patterns for Predictive Analytics

Orbitz and the Macs: Signals, not segmentation

By now you've probably heard about the fact that Orbitz users accessing the site via Macs are seeing more expensive hotel options when they search. But it seems worth clearing up a couple of fallacies.

First, it's not as if the same hotel room is being offered at a higher prices to Mac users. (So no, using Windows to access Orbitz won't get you a better deal.) Orbitz offers the same search results to Windows and Mac users, but Mac users may see more expensive hotels higher earlier in the search results.

And the reason why this is happening isn't segmentation, as some have claimed.  From what I remember of the Orbitz presentation at the R user conference a couple of years ago, all sorts of signals go into the R model to try and predict which hotels a given Orbitz user might be interested in: WebTrends data, search history, purchase history, planned travel dates ... and the user's browser and operating system. It would seem that the browser (perhaps Safari / iOS) and/or the operating system is a strong signal that the user is likely to purchase a higher-priced hotel room. The Orbitz system (as I understand it) is designed to rank search results showing the results the specific user is most likely to be interested in at the top of the list. If Mac users, on average, spend more on hotel rooms than other Orbitz visitors, then it's not surprising that more expensive results would appear at the top of the list. 

But it's not as if Mac users being singled out for special treatment. Segmentation is a marketing operation (and a conscious decision), whereas the use of signals like these is part of a predictive analytics operation (driven by data, not people). It's important to understand the distinction.

You can read more about how Orbitz uses R and Hadoop to drive its search at the link below.

Revolutions: How Orbitz uses Hadoop and R to optimize hotel search

R Tops Data Mining Software Poll

For the past 12 years, KDNuggets has conducted an annual poll asking "What analytics/data mining software you used in the past 12 months for a real project (not just evaluation)". In this year's poll, R was the top-ranked data mining solution, selected by 30.7% of poll respondents. Microsoft Excel was second, at 29.8%. Rapidminer, which took the #1 spot over R in 2011 and 2010, ranked third. And as Bob Muenchen notes, four of the top five ranked data mining solutions in this year's poll are open-source. R was also ranked in this poll as the most popular language for implementing data mining application, beating out SQL and Java.

See the link below for the complete list of tools ranked in the KDNuggets poll.

KDNuggets: Poll Results: Top Analytics,Data Mining, Big Data software used 

R is to SAS as Java is to COBOL

An interview with Revolution Analytics CEO Dave Rich was published this week by BeyeNetwork. During the interview, Dace was asked about how the statistical modeling platforms have changed over the decades:

People have been doing statistical modeling and predictive analytics for 50 years now, SAS and SPSS have been around since the early ‘70s. What’s different now -- what’s making this move toward other statistical and “big data” areas?

David Rich: Well, I think obviously SAS and SPSS have been around, as you pointed out, for decades. We call that sort of the first generation of analytics and insight-driven solutions. In my perspective, having been in the business for more than three decades, it reminds me a bit of what COBOL was back in the day relative to business software. I see R as the more modern language. In this analogy, R would represent Java or C++. What happened in the middle of the nineties when the shift occurred is very similar to where we are now with R. Open source is a worldwide collaboration innovation. It’s a way to tap into that channel for research, and I think the role that Revolution Analytics can play – very similar to what Red Hat did back in the Linux days – is to be the conduit between the community and enterprise deployment.

The conversation also touched on the future of big data analytics, impact of advanced analytics on business, and the benefits of R and Revolution R Enteprise to reduce costs and expand the scope of possibility with big data analytics. For the complete interview, follow the link below.

BeyeNetwork: Advanced Analytics, Big Data and the Power of R: A Q&A Spotlight with David Rich, CEO of Revolution Analytics

Simple tools for building a recommendation engine

By Joseph Rickert

Revolution’s resident economist, Saar Golde, is very fond of saying that “90% of what you might from a recommendation engine can be achieved with simple techniques”. To illustrate this point (without doing a lot of work), we downloaded the million row movie dataset from www.grouplens.org with the idea of just taking the first obvious exploratory step: finding the good movies. Three zipped up .dat files comprise this data set. The first file, ratings.dat, contains 1,000,209 records of UserID, MovieID, Rating, and Timestamp for 6,040 users rating 3,952 movies. Ratings are whole numbers on a 1 to 5 scale. The second file, users.dat, contains the UserID, Gender, Age, Occupation and Zip-code for each user. The third file, movies.dat, contains the MovieID, Title and Genre associated with each movie.

Although the movies dataset is not a large file, and can easily be read into my laptop’s memory (a Dell with 4, 1.86GHz cores and 8GB or RAM) it turns out that it doesn’t provide R with enough resources to run the simple regression model we had in mind. To get around this, we used the rxImport function to import the files into .XDF format used by the RevoScaleR package, transformed all of the variables except the times stamp into factors and merged them into one big file called “RUM”. The script “XDF import.R” shows how all of this was done. Everything in this script is straightforward except maybe the few lines of code beginning at line 101 where a little work needs to be done to make reconcile the levels of the MovieID factor variable between ratings and movies files. Working with factors can be a little in R, so it is nice to have the function rxFactors for setting factor levels in XDF files.

Once we got the RUM we could take the first naïve step of finding the best movie according to the user ratings. This can be done with the RevoScaleR function rxCube which produces a cross tabulation in long form. The following code which tabulates rating by titles produces the results in Table 1.

cube.3 <- as.data.frame(rxCube(Rating ~ fTitle,data="RUM"))

head(cube.3)

Table 1
	fTitle	Rating	Counts
1	Toy Story (1995)	4.147	2077
2	GoldenEye (1995)	3.541	888
3	City Hall (1996)	3.063	128
4	Curdled (1996)	3.050	20
5	Ed's Next Move (1996)	4.250	8
6	Extreme Measures (1996)	2.942	121

Rating contains the average rating and Counts gives the number of ratings that went into computing the average. After sorting cube.3 and including only movies with more than 50 ratings we come up with our top-six list in Table 2.

Table 2
	Title	Rating	Counts
1	Sanjuro (1962)	4.609	69
2	Seven Samurai (The Magnificent Seven) (Shichinin no samurai) (1954)	4.561	628
3	Shawshank Redemption, The (1994)	4.555	2227
4	Godfather, The (1972)	4.525	2223
5	Close Shave, A (1995)	4.521	657
6	Usual Suspects, The (1995)	4.517	1783

 (The code for this and the rest of our analysis is in the script “Best movie.R”.) This list certainly looked good to me: two films by Akira Kurosawa on top and the Godfather not far behind. These people know a good film when they see one. Well, maybe so, but maybe taking the ratings at face value from this particular movie buffs is not the best we can do. Better than just averaging ratings, with a simple regression we can look for the best movie while controlling for the users. The RevoScaleR functions to do this are:

form <- formula(Rating ~ UserID + fTitle)

mod.1 <- rxLinMod(form,data="RUM",cube=TRUE)

The regression looks inocuous enough. However, since UserID and fTitle are both factors we are computing a regression with 9,746 fixed effects! lm ran out of resource on my laptop trying to do this regression but rxLinMod completed the job in about 53 seconds. Sorting the model coefficients from high to low produces a new best movie list. Table 3 includes enough movies to show where the movies in our original list landed.

Table 3
	Title	Coef	Coef Std
1	Song of Freedom (1936)	3.298	0.9127
2	Mamma Roma (1962)	2.173	0.6445
3	Schlafes Bruder (Brother of Sleep) (1995)	1.717	0.9159
4	Hour of the Pig, The (1993)	1.637	0.6410
5	Smashing Time (1967)	1.636	0.6481
6	Ulysses (Ulisse) (1954)	1.583	0.9133
7	Gate of Heavenly Peace, The (1995)	1.577	0.5244
8	Apple, The (Sib) (1998)	1.554	0.3058
9	I Am Cuba (Soy Cuba/Ya Kuba) (1964)	1.532	0.4076
10	Lamerica (1994)	1.490	0.3243
11	One Little Indian (1973)	1.404	0.9066
12	Follow the Bitch (1998)	1.401	0.9178
13	Sanjuro (1962)	1.382	0.1213
14	Bells, The (1926)	1.323	0.6404
15	Lured (1947)	1.315	0.9090
16	Leather Jacket Love Story (1997)	1.311	0.6410
17	Chain of Fools (2000)	1.298	0.9127
18	Seven Samurai (The Magnificent Seven) (Shichinin no samurai) (1954)	1.281	0.0645
19	Shawshank Redemption, The (1994)	1.254	0.0568
20	Sunset Blvd. (a.k.a. Sunset Boulevard) (1950)	1.253	0.0678
21	For All Mankind (1989)	1.245	0.1822
22	Jar, The (Khomreh) (1992)	1.243	0.9045
23	Paths of Glory (1957)	1.232	0.0801
24	Close Shave, A (1995)	1.230	0.0641
25	Usual Suspects, The (1995)	1.229	0.0576
26	Godfather, The (1972)	1.229	0.0568

Pretty interesting, not only did the original top six move down, they also moved out of their original order. Moreover, having done the regression, we also have the standard deviations of the coefficients available. Now, we not only have a measure of how good a movie is, but we also have an estimate of the uncertainity of its quality, and we may want to use this to adjust our recommendations. For example, the 17^th movie on the list, Chain of Fools, has a coefficient of 1.298 with a standard deviation of 0.9127. This is a fairly large spread. Near the bottom of its range of uncertainty the coefficient of Chain of Fools would be smaller than the coefficients of the rest of the movies on the top 26 list. But, just below Chain of Fools in the 18^th slot, the coefficient of the The Seven Samurai, 1.281, has a much smaller standard deviation: 0.0645. It appears that this would be a better movie to recommend.

Although we have not shown it, our model also gives user fixed effects. So, we can see who the tougher raters are and, combined with other information about users, we might be able to figure out how to recognize a tough movie rater.

I think Saar made his point: good progress can be made with simple tools. Moreover, by thinking like a statistician (or an economist) and not merely letting the machine learning algorithms drive the process, it is possible to gain useful insight into how to make better recommendations.

Marketing optimization with LityxIQ

Marketing is one of the pioneering domains when it comes to applications of predictive analytics to Big Data. (For example, how Target used statistical modeling to predict demographic attribues of customers, like pregnancy, to target coupons.) To get such powerful insights into the hands of marketers, DC-area company LityxIQ provides a cloud-based solution: LityxIQ, "an integrated hosted analytics platform for marketing insights, predictive models, and marketing optimization".

LityxIQ allows marketers to automate the loading and managing multiple data sources, automatically build and manage predictive models, and optimize marketing budget and media decisions. The R language is used to implement those models behind the scenes, making use of its predictive algorithms and for its powerful variable selection capabilities. I asked Lityx CEO Paul Maiste why he chose R to implement these models instead of other statistical software packages. "We used R in developing the platform in order to take advantage of the flexibility of the language and wealth of supported algorithm", he said. "In addition, it allows us to fully integrate model building and scoring easily using its object-oriented architecture." Some of the R functions and packages used include glm (for generalized linear modeling), MASS, rpart (decision trees), nnet (neural netwoks), and rjson (for data exchange).

If you're interested in learning more about LityxIQ, there will be a free webinar this Thursday, presented by Lityx. I've included the webinar abstract and the registration link below.

Marketing Analytics, Modeling & Optimization with LityxIQ

Fortune 1000 organizations make big investments in predictive analytics to optimize their marketing budgets. Now, any size organization can develop the same marketing insights enjoying the same efficiency benefits with LityxIQ.

Learn how your organization can discover valuable insights about customers, predict their behavior and their future value, and optimize marketing budgets using a sophisticated and powerful marketing analytics platform designed for business users.

LityxIQ was developed by PhD statisticians and marketing analytic experts working with some of the largest companies and non-profits in the world. Its cloud-based delivery allows every organization to leverage their data through an affordable subscription model without internal IT investment.

Lityx: Marketing Analytics, Modeling & Optimization with LityxIQ

Big-data Naive Bayes and Classification Trees with R and Netezza

The IBM Netezza analytics appliances combine high-capacity storage for Big Data with a massively-parallel processing platform for high-performance computing. With the addition of Revolution R Enterprise for IBM Netezza, you can use the power of the R language to build predictive models on Big Data.

In the demonstration below, Revolution Analytics' Derek Norton analyzes loan approval data stored on the IBM appliance. You'll see the R code used to:

• Explore the raw data (with summary statistics and charts)
• Prepare the data for statistical analysis, and create training and test sets
• Create predictive models using classificiation trees and Naïve Bayes
• Predict using the models, and evaluate model performance using confusion matrices

 

Note that while R code is being run on Derek's laptop, the raw data is never moved from the appliance, and the analytic computations take place "in-database" within the appliance itself (where the Revolution R Enterprise engine is also running on each parallel core). 

This demo was included in the recent webinar, Turbo-Charge Your Analytics with IBM Netezza for which you can find slides and a replay at the link below.

Revolution Anlaytics Webinars: Turbo-Charge Your Analytics with IBM Netezza and Revolution R Enterprise: A Step-by-Step Approach for Acceleration and Innovation

Big Data Analytics to Revolutionize Services

Revolution Analytics' CEO Dave Rich was interviewed by Wikibon's David Vellante and SiliconAngle's John Furrier at the Strata 2012 conference last week. Given Dave's background at Accenture Analytics, the conversation naturally turned to impact of predictive analytics and R on business services. (See the video after the jump, below.)

Bret Latmore of SiliconANGLE provides highlights of the interview, including Dave's assessment of the opportinity for R in the enterprise:  

[Big Data revolutionizing business processes] is the reason Rich chose to become CEO of Revolution Analytics, which he calls "the Red Hat of R", which is the next-generation analytical programming language that has replaced SAS and SPSS in academia. Like Linux, R was developed by an open source community of mostly academics as an advanced analytics language.

Now, he says, the legacy language owners are milking their markets. IBM, for instance, hiked the price of SPSS licenses after buying it. As a result, R is growing in commercial use – for instance Oracle recently announced that it was standardizing on R. "We thought that was great. The same thing happened with Linux. Legacy customers were tired of the high license fees [for the commercial operating systems], so they put Red Hat in business. We're at that same inflection point."

See the video after the jump, and read more of the interview at the link below.

ServicesAngle: Big Data Analytics to Revolutionize Services, Says Industry CEO

The Uncanny Valley of Big Data

Three articles in recent weeks have touched on an important issue related to Big Data and predictive analytics: sometimes, the results can be downright creepy. It's kind of like the "Uncanny Valley" in computer animation: the reason why the human characters in Pixar animations are cartoon-like and not human-like is because trying to make animated humans photorealistic generally results in uncomfortable reactions from the viewer. The animations might look realistic, but something in our animal brain knows something isn't quite right, and it's just ... creepy.

The same thing can happen where the rubber meets the road of Big Data and predictive analytics: when offers or suggestions are made to individuals. You've probably had an experience similar to mine: after searching the web for a hotel deal in Vegas, suddenly every ad that appeared next to the blogs and websites I regularly read was for a Vegas-related deal. Creeeepy. (And also not particularly useful: after that trip I had no particular plans to return to Vegas anytime soon, yet the ads kept coming.)

A similar tale was related in the New York Times over the weekend. In the story "How Companies Learn Your Secrets" (reg. req.), statistician Andrew Pole (working for the retailer Target) described how he'd created a predictive model to identify from shopping habits when a shopper was likely to be pregnant. When the father of a young Target shopper saw the baby-related coupons sent to his daughter, he was outraged:

“My daughter got this in the mail!” [said the father]. “She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?”

It turns out the daughter actually was pregnant at the time, unbeknownst to her father. The creepy aspect here: why should a corporation be able to know (or rather, infer) such a personal fact, when close family members do not? Intriguingly, Target solved this problem by mixing future offers to identified pregnant shoppers with unrelated coupons, say, for lawnmowers or wineglasses. By deliberately making the predictions worse, the response was better: "as long as we don’t spook her, it works", said Pole. Personally, I wish web-advertisers would do the same thing. Not only do I not care about Vegas hotels anymore, the absence of other ads precludes the serendipity of discovering other products I might actually like, but which my activity history might not suggest. In a follow-up interview, article author Charles Duhigg suggested other areas where this technique might help alleviate the "creep factor".

In a similar vein, this week's Esquire profiles Tibco's CEO Vivek Ranadivé. Amongst several examples of the importance of collecting multiple streams of data to improve predictions from analytics, comes this anecdote about football fans visiting Oakland's Oracle Arena:

At the end of the third quarter, when the computer system showed that the concession stand near his seats had too many hot dogs, it could send him a buy-one-get-one-free offer — because it also knows that he sometimes buys hot dogs at games.

The right information to the right people at the right time in the right context. (Fans creeped out by this could opt out.)

This may be another example where moving the predictions outside the uncanny valley might prevent fans being creeped out.

Finally, another New York Times article from earlier this month, "The Age of Big Data" (reg. req.) looks into the lives and impacts of some of the "rock stars" of Big Data applications. While lauding many of the benefits of analytics on Big Data, it also strikes a cautionary tale at the end of the article:

Big Data has its perils, to be sure. With huge data sets and fine-grained measurement, statisticians and computer scientists note, there is increased risk of “false discoveries.” The trouble with seeking a meaningful needle in massive haystacks of data, says Trevor Hastie, a statistics professor at Stanford, is that “many bits of straw look like needles.”

This is a great point: treating analytics as a "black box process" — data in, predictions out — can lead to inapproprate predictions (more to the "zombie" side than the "angel" side of the Uncanny Valley). It takes the statistical expertise of a data scientist to ensure that such predictive analytics are creating sensible predictions ... and to help companies avoid the Uncanny Valley of Big Data.

(Read more articles from this blog on big data and predictive analytics.)