Data Science: The State of Data Science Fall 2019

Posted on Dec 9, 2019
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

Over the past decade, data science has emerged as an important and rapidly expanding field. Data Scientist has been the #1 job on Glassdoor’s “50 best jobs in America” for each of the last 4 years. Demand for Data Scientists has certainly increased over that span and is anticipated to grow into the future. While the field is growing quickly, the adoption of data science techniques is, in fact, moving surprisingly slowly. Even while businesses increasingly say they aspire to be  “data-driven,” they fail to take the necessary steps to achieve that goal.  

The NewVantage Partners’ 2019 Big Data and AI Executive Survey indicated that only 31% of companies say they are data-driven. Moreover, respondents indicated that the principal challenge to becoming data-driven was people (62.5%) and process (30%), not so much technology (7.5%). Part of this challenge in people and process is transforming the current culture. One vocal critic of the slow adoption of data science in business has been Peter Wang, the CEO of Anaconda:




According to GeekWire, Tableau had about 86,000 customer accounts, as of the end of 2018. In contrast, Anaconda, on its website lists that it has about 18 million users worldwide! This wide gap suggests there is a huge, as yet untapped business opportunity for data science in business, as Peter Wang alludes to and which the NewVantage Partners’ 2019 Big Data and AI Executive Survey confirms. Therefore, the adoption of data science techniques and tools in business has a long way to go.

What positions are companies currently hiring for and what skills are they looking for? 

After building a Python web scraper using the Selenium library to retrieve all the job postings on within 25 miles of New York, NY, here’s what I found after analyzing 7,319 unique postings for search terms: “Data Scientist,” “Data Engineer,” “Software Engineer,” “Python,” “Data Analyst," and “Machine Learning Engineer.”


These were the results:

  1. Analyst: 1,338
  2. Software Engineer: 1,331
  3. Data Scientist: 722
  4. Data Engineer: 460 
  5. Data Analyst: 427
  6. Machine Learning Engineer: 125 

The greatest majority of related job listings have "engineer" or "analyst" in the title; however, these job listings do not necessarily require data science specific skills. Data Scientist is actually third among the list of related jobs. 

Top Skills: 

  1. Machine Learning: 1,479
  2. Python: 1,403
  3. Analytics: 682
  4. R: 434 
  5. SQL: 431
  6. Statistics: 371


Although programming in R does make the list of required skills for data science roles, it appears as a supplementary skill alongside Python. Another important distinction is that while machine learning is one of the top skills being asked for, deep learning (96) and “artificial intelligence” (217) were not as common, since these are much more specialized skills. 

In addition to the technical skills above, perhaps even more important, are communication skills. Being able to summarize and articulate business insights and explain various data science methods and problems to colleagues and business executives is vital to being able to provide business value. This is the missing element of so many companies that are not yet data-driven.

As digital transformation of business continues and more businesses become increasingly data-driven, data science tools and techniques will become an indispensable part of every company. With so much room still left to grow, now is a great time to become a data scientist! 

Check out the web scraping code:

About Author

Mikolaj Wilk

Data Scientist, Developer, Scholar and Scoutmaster with a background in Physics, determined to unlock business value and solve problems with analytical and data-driven methods.
View all posts by Mikolaj Wilk >

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI