Most people know R as a statistics/analytics language for analysis of quantitative data, and don't think of it as a tool for processing raw text. But R actually has some quite powerful facilities for processing character data. And as Gaston Sanchez learned, text manipulation is an important part of a modern data scientist's repertoire:
Many years ago I decided to apply for a job in a company that developed data mining applications for big retailers. I was invited for an on-site visit and I went through the typical series of interviews with the members of the analytics team. Everything was going smoothly and I was enjoying all the conversations. Then it came turn to meet the computer scientist. After briefly describing his role in the team he started asking me a bunch of technical questions and tests. Although I was able to answer those questions related with statistics and multivariate analysis, I had a really hard time trying to answer a series of questions related with string manipulations.
I will remember my interview with that guy as one of the most embarrassing moments of my life. That day, the first thing I did when I went back home was to open my laptop, launch R, and start reproducing the tests I failed to solve. It didn't take me that much to get the right answers. Unfortunately, it was too late and the harm was already done. Needless to say I wasn't offered the job. That shocking experience showed me that I was not prepared for manipulating character strings. I felt so bad that I promised myself to learn the basics of strings manipulation and text processing. "Handling and Processing Strings in R" is one of the derived results of that old promise.
Gaston's Creative-Commons licensed 112-page e-book, Handling and Processing Strings in R, is an excellent and comprehensive review of R's string handling capabilities. It cover's R's basic string-handling capabilities (reading, converting, manipulating and formatting), and also devotes a chapter to the higher-level functions of Hadley Wickham's stringr package. The two chapters on regular expressions are a must-read for anyone who hasn't yet come to grips with the power of regexes for handing string-based data. There are a few practical examples at the end of the e-book (frequency counting, word clouds) but the book sticks mainly with the fundamentals, and doesn't stray into semantic analysis. Highly recommended for anyone working with strings or character data in R.
Gaston Sanchez: Handling and Processing Strings in R (via Sharon Machlis)
Some shameless advertising but this book on corpus linguistics with R deals with many different text processing capabilities: {http://www.amazon.com/dp/0415962706} and the companion website at {http://www.linguistics.ucsb.edu/faculty/stgries/research/qclwr/qclwr.html}.
Posted by: Stefan Th. Gries | March 22, 2014 at 15:24