Revolutions

R leads, Python gains in 2017 Burtch Works Survey

For the past four years, recruiting firm Burtch Works has conducted a simple survey of data scientists with just one question: "Which do you prefer to use — SAS, R or Python". The results for this year's survey of 1,046 respondents are in:

• R: 40% (2016: 42%)
• SAS: 34% (2016: 39%)
• Python: 26% (2016: 20%)

Compared to last year's results, Python has gained 6 percentage points, mainly at the expense of SAS. (2016 results do not add to 100% due to rounding.) The trend for the commercial tool compared to the open-source tools is apparent looking at the data from all four years:

(Note: Python has only been an option for respondents in the last two surveys.) Demographic breakdowns, included in the survey analysis linked below, show that SAS remains popular with more years of experience (more than 16 years), although becomes less popular with greater educational attainment. Burtch Works will also be holding a mini webinar on June 29 to discuss the results (details at the link below).

Burtch Works: 2017 SAS, R, or Python Flash Survey Results

Python and R top 2017 KDnuggets rankings

The results of KDnuggets' 18th annual poll of data science software usage are in, and for the first time in three years Python has edged out R as the most popular software. While R increased its share of usage from 45.7% in last year's poll to 52.1% this year, Python's usage among data scientists increased even more, from 36.6% of users in 2016 to 52.6% of users this year.

There were some interesting moves in the long tail, as well. Several tools entered the KDNuggets chart for the first time, including Keras (9.5% of users), PyCharm (9.0%) and Microsoft R Server (4.3%). And several returning tools saw big jumps in usage, including Microsoft Cognitive Toolkit (3.4% of users), Tensorflow (20.2%) and Power BI (10.2%). Microsoft SQL Server increased its share to 11.6% (up from 10.8%), whereas SAS (7.1%) and Matlab (7.4%) saw declines. Julia, somewhat surprisingly, remained flat at 1.1%.

For the complete results and analysis of the 2017 KDnuggets data science software poll, follow the link below.

KDnuggets: New Leader, Trends, and Surprises in Analytics, Data Science, Machine Learning Software Poll

Stack Overflow Trends

Developer Q&A site Stack Overflow recently introduced Stack Overflow Trends, a useful tool for tracking the growth and decline in the rate of questions asked on various topics (by their Stack Overflow tag). For example, you can see that activity around both R and Python has been increasing over the past 8 years:

As you'd expect from a general purpose language, the volume of Python questions is much greater than the volume of R questions. R is a data science language, so it's difficult to compare it with data science usages of Python. But inspired by David Robinson's blog post introducing Stack Overflow trends, here's a comparison of R with a couple of Python packages used primarily for data science:

For more examples of using Stack Overflow Trends, check out the blog post linked below.

Stack Overflow blog: Introducing Stack Overflow Trends

Statistician ranked as best job in 2017 by CareerCast

According to job hunting site CareerCast, the best job to have in 2017 is: Statistician. This is according to their 2017 Jobs Rated report, based on an evaluation of Bureau of Labor Statistics metrics including environment, income, employment and income growth, and stress factors.

In their rankings, Statistician is the role that took the top spot with a "work environment" score of 4 out of 199 (lower is better) , a stress factors score of 39, and a projected growth score of 3. The median salary is reported at USD$80,110 and projected to grow at 34% (per annum, I assume).

Also in the top ten: Data Scientist, in fifth place. This role scored similarly in Work Environment (12 out of 199) and stress factors (37), but had slightly lower prospects for growth (37).  Here, the median salary is reported at $111,267 and projected to grow at 15.75%.

The top 10 list is as follows:

1. Statistician
2. Medical Services Manager
3. Operations Research Analyst
4. Information Security Analyst
5. Data Scientist
6. University Professor
7. Mathematician
8. Software Engineer
9. Occupational Therapist
10. Speech Pathologist

You can see the full list of 200 jobs ranked by CareerCast at the link below.

CareerCast: Jobs Rated Report 2017: Ranking 200 Jobs (via Royal Statistical Society)

 

The Most Popular Languages for Data Scientists/Engineers

The results of the 2017 StackOverflow Survey of nearly 65,000 developers were published recently, and includes lots of interesting insights about their work, lives and preferences. The results include a cross-tabulation of the most popular languages amongst the "Data Scientist/Engineer" subset, and the results were ... well, surprising:

When thinking about data scientists, it certainly makes sense to see SQL, Python and R in this list. (I've only included the top 10 above.) But it's a real surprise to see JavaScript at the top of the list, and the presence of PHP is just unfathomable to me. I think it goes to show that the "Data Engineer" role is a very different type of job than "Data Scientist". Sadly, it's not clear what the relative proportion of Data Scientists to Data Engineers is in the survey, as it's not broken out elsewhere.

Nonetheless, there were several other interesting tidbits in the survey relevant to data scientists:

• Overall, Python is the fifth most popular language, used by 32% of respondents. R ranks #15, used by 4.4% of respondents.
• The top three databases are MySQL (44.3%), SQL Server (30.8%) and SQLite (21.2%).
• The most popular platforms are Windows (41%), Linux (33%) and Android (28%).
• AWS is used by 28% of respondents; Microsoft Azure by 11.4%.
• Python is the "Most Wanted" language (indicated by 21% of respondents as a language they'd like to be using).
• Oracle is the "Most Dreaded" database (63% of users would rather be using a different database).
• Visual Studio is the most popular development environment within all developer categories except "Sysadmin/DevOps" (in which case it's vim).
• In the US, R developers earn $100,000pa on average. For Python, it's $99,000. "Machine Learning specialist", "Developer with a statistics or mathematics background" and "data scientist" were the three highest-paying job categories in the US, all topping $100,000.
• Developers consider Elliot Alderson from Mr Robot to be the most realistic portrayal of a programmer in fiction.

You can find a complete analysis of the survey data, follow the link below.

StackOverflow: Developer Survey Results 2017

 

 

 

Data science languages score highly in RedMonk rankings

Redmonk have once again updated (a little later than usual) their bi-annual programming language report with their January 2017 rankings. If you haven't come across these rankings before, they are based on GitHub contributions and StackOverflow questions related to around 40 commonly-used programming languages. The raw data (as of January 2017) is shown below — as you might guess from the appearance of the chart, the analysis for the rankings is done in R.

Languages used by data scientists rank highly in this metric. Python is ranked #3 (up from #4 in the June 2016 rankings). R is ranked #14, down from its all-time high rank of #12 in June 2016. Redmonk's Stephen O'Grady attributes this dip to a change in the process of collecting the GitHub data and notes that R "remains popular in spite of a step back".

Other notable moves in the rankings are Swift, jumping from #26 to #16 in the rankings; Go (moving to #11 from #15); and Powershell (jumping to #19 from #36). For more the complete rankings and associated analysis, check out the RedMonk post linked below.

RedMonk (Tecosystems): The RedMonk Programming Language Rankings: January 2017

Job trends for R and Python

When we last looked at job trends from indeed.com, job listings for "R statistics" were on the rise but were still around half the volume of listings for "SAS statistics". Three-and-a-half years later, R has overtaken SAS in job listings for "statistics".

I added Python to the search this time; job listings for "Python statistics" have risen at a similar rate to those for R, but with a somewhat higher volume for R.

Since data science is popular job role these days, let's do the same search for "data scientist":

For "data scientist" jobs, R and Python track very closely, with Python just edging out R in the past few months. This is most likely because R and Python (but unlike SAS) appear together in many data scientist job listings.

You can explore other job titles at indeed.com. (And thanks to reader SK for the suggestion to revisit these searches!)

Three reasons to learn R today

If you're just getting started with data science, the Sharp Sight Labs blog argues that R is the best data science language to learn today.

The blog post gives several detailed reasons, but the main arguments are:

1. R is an extremely popular (arguably the most popular) data progamming language, and ranks highly in several popularity surveys.
2. Learning R is a great way of learning data science, with many R-based books and resources for probability, frequentist and Bayesian statistics, data visualization, machine learning and more.
3. Python is another excellent language for data science, but with R it's easier to learn the foundations.

Once you've learned the basics, Sharp Sight also argues that R is also a great data science to master, even though it's an old langauge compared to some of the newer alternatives. Every tool has a shelf life, but R isn't going anywhere and learning R gives you a foundation beyond the language itself.

If you want to get started with R, Sharp Sight labs offers a data science crash course. You might also want to check out the Introduction to R for Data Science course on EdX.

Sharp Sight Labs: Why R is the best data science language to learn today, and Why you should master R (even if it might eventually become obsolete)

2016 Data Science Salary Survey results

O'Reilly has released the results of the 2016 Data Science Salary Survey. This survey is based on data from over 900 respondents to a 64-question survey about data-related tasks, tools, and the salary they receive from doing/using them. The median salary reported in the survey was US$87,000; amongst data scientists in the US, the median salary was US$106,000.

Appropriately for a survey about data science, O'Reilly doesn't merely report aggregate statistics from the survey; they fit a linear regression model for a data, and extact coefficients from the model indicative of salary "bumps" (or downgrades) attributable to demographic factors. (The model apparently includes no interaction terms.) Factors that tended to increase salary included: working in cloud computing environments; working with Python; and being older. Factors that tended to decrease salary included: in the Education industry, working with Excel, and being female. (Since this is a regression model, that means female data scientists earned $7,800 per year less than their male counterparts for doing the same work.)

The survey also reports on use of tools, and the top 3 in each category were as follows (respondents could select multiple tools in each category):

• Operating Systems: Windows 74%; Linux 49%; Mac OS X 42%
• Databases: MySQL 37%; Microsoft SQL Server 33%; Oracle 23%
• Programming languages: SQL 70%; R 57%; Python 54%

Interestingly, the survey also reported on the tasks that data scientists perform: over 90% reported soem kind of coding in their day-to-day work. The tasks reported, in order of frequency or reporting and shown with corresponding salary ranges of the subset, are shown below:

For much more data and analysis from the 2016 survey, follow the link below to download a free copy from O'Reilly.

O'Reilly: 2016 Data Science Salary Survey

R moves up to 5th place in IEEE language rankings

IEEE Spectrum has just published its third annual ranking with its 2016 Top Programming Languages, and the R Language is once again near the top of the list, moving up one place to fifth position.

As I said last year (when R moved up to take sixth place), this is an extraordinary result for a domain-specific language. The other four languages in the top 5 (C, Java, Python amd C++) are all general-purpose languages, suitable for just about any programming task. R by contrast is a language specifically for data science, and its high ranking here reflects both the critical importance of data science as a discipline today, and of R as the language of choice for data scientists.

IEEE Spectrum ranks languages according to a large number of factors, including search rankings and trends, social media mentions, and job posting. (You can adjust the weighting of these factors to generate your own rankings using this interactive tool.) It also includes scholarly citations of the languages, a factur that influenced R's rise in this ranking:

Another language that has continued to move up the rankings since 2014 is R, now in fifth place. R has been lifted in our rankings by racking up more questions on Stack Overflow—about 46 percent more since 2014. But even more important to R’s rise is that it is increasingly mentioned in scholarly research papers. The Spectrumd efault ranking is heavily weighted toward data from IEEE Xplore, which indexes millions of scholarly articles, standards, and books in the IEEE database. In our 2015 ranking there were a mere 39 papers talking about the language, whereas this year we logged 244 papers.

In related news, R also increased its ranking in the recently-released RedMonk Language Rankings for June 2016, moving up one spot to take 12th place. Unlike IEEE Spectrum, RedMonk ranks language using just two criteria: activity of the language on GitHub and StackOverflow. Analyst Stephen O'Grady had this to say about R's performance in the RedMonk rankings:

Out of all the back half of the Top 20 languages, R has shown the most consistent upwards movement over time. From its position of 17 back in 2012, it has made steady gains over time, but had seemed to stall at 13 having stuck there for three consecutive quarters. This time around, however, R took over #12 from Perl which in turn dropped to #13. There’s still an enormous amount of Perl in circulation, but the fact that the more specialized R has unseated the language once considered the glue of the web says as much about Perl as it does about R.

Again, R's steady growth in this and numerous other surveys and rankings over time reflects the growing importance of data science applied using R.