The State of Data Science Fall 2019

Posted on Dec 9, 2019

Over the past decade, data science has emerged as an important and rapidly expanding field. Data Scientist has been the #1 job on Glassdoor’s “50 best jobs in America” for each of the last 4 years. Demand for Data Scientists has certainly increased over that span and is anticipated to grow into the future. While the field is growing quickly, the adoption of data science techniques is, in fact, moving surprisingly slowly. Even while businesses increasingly say they aspire to be  “data-driven,” they fail to take the necessary steps to achieve that goal.  

The NewVantage Partners’ 2019 Big Data and AI Executive Survey indicated that only 31% of companies say they are data-driven. Moreover, respondents indicated that the principal challenge to becoming data-driven was people (62.5%) and process (30%), not so much technology (7.5%). Part of this challenge in people and process is transforming the current culture. One vocal critic of the slow adoption of data science in business has been Peter Wang, the CEO of Anaconda:




According to GeekWire, Tableau had about 86,000 customer accounts, as of the end of 2018. In contrast, Anaconda, on its website lists that it has about 18 million users worldwide! This wide gap suggests there is a huge, as yet untapped business opportunity for data science in business, as Peter Wang alludes to and which the NewVantage Partners’ 2019 Big Data and AI Executive Survey confirms. Therefore, the adoption of data science techniques and tools in business has a long way to go.

What positions are companies currently hiring for and what skills are they looking for? 

After building a Python web scraper using the Selenium library to retrieve all the job postings on within 25 miles of New York, NY, here’s what I found after analyzing 7,319 unique postings for search terms: “Data Scientist,” “Data Engineer,” “Software Engineer,” “Python,” “Data Analyst," and “Machine Learning Engineer.”

These were the results:

  1. Analyst: 1,338
  2. Software Engineer: 1,331
  3. Data Scientist: 722
  4. Data Engineer: 460 
  5. Data Analyst: 427
  6. Machine Learning Engineer: 125 

The greatest majority of related job listings have "engineer" or "analyst" in the title; however, these job listings do not necessarily require data science specific skills. Data Scientist is actually third among the list of related jobs. 

Top Skills: 

  1. Machine Learning: 1,479
  2. Python: 1,403
  3. Analytics: 682
  4. R: 434 
  5. SQL: 431
  6. Statistics: 371

Although programming in R does make the list of required skills for data science roles, it appears as a supplementary skill alongside Python. Another important distinction is that while machine learning is one of the top skills being asked for, deep learning (96) and “artificial intelligence” (217) were not as common, since these are much more specialized skills. 

In addition to the technical skills above, perhaps even more important, are communication skills. Being able to summarize and articulate business insights and explain various data science methods and problems to colleagues and business executives is vital to being able to provide business value. This is the missing element of so many companies that are not yet data-driven.

As digital transformation of business continues and more businesses become increasingly data-driven, data science tools and techniques will become an indispensable part of every company. With so much room still left to grow, now is a great time to become a data scientist! 

Check out the web scraping code:

About Author

Mikolaj Wilk

Data Scientist, Developer, Scholar and Scoutmaster with a background in Physics, determined to unlock business value and solve problems with analytical and data-driven methods.
View all posts by Mikolaj Wilk >

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp