Data Science job market in the age of COVID-19

Avatar
Posted on Jun 29, 2020

According to California University of Pennsylvania “Data science was named the fastest-growing job in 2017 by LinkedIn, and in 2018 Glassdoor ranked data scientist as the best job in the United States.” In the age of COVID-19, we wonder how this trend is standing against the gruesome repercussions of COVID-19 on the economy. This project  aims at investigating the following questions.

  • What is the distribution of data science jobs per State? 
  • What industries are prominent in each State?
  • Which sector/industry holds the most Data Science jobs?
  • Is  the salary still  three figures?
  • How fast  are these positions being filled ?

Data Acquisition

Data used in the project was scrapped from Glassdoor using scrapy and selenium. A total of 864 jobs were scraped.  After cleaning and filtering, 634 unique jobs were used. Jobs scraped were the result of the query “data science” with location and all other fields left blank. The screenshot below shows a sample of the  cleaned data.

  • Company: Name of the company recruiting
  • City,State: Address of the position being filled
  •  Year_founded: Year the company was founded
  • Job_title: Title of the position 
  • Days_posted: Number of days the position has been on Glassdoor
  • Started_salary and Max_salary: Estimate of the minimum and maximum yearly salary by Glassdoor. 
  • Industry and Sector: The industry and sector of the company that is recruiting.

Exploratory Data Analysis

A first look into the data shows the distribution of jobs by states. As shown below, we can see that major tech hub States still hold quite a lot of positions in this industry.

A dive into the data slices it into sectors to understand which one is offering most positions.This is further grouped by State to show principal sector per State. This will give job seekers an inside look on how  their interest  match up with their preferred industry and location, so that they have a better idea of how to center their job search.

A look at salary estimate columns shows median, mean, minimum and maximum estimated lower and upper bound yearly salary per State. This valuable insight helps understand the pay rate of data scientists and what  new data scientists can expect. The two figures below show the minimum and maximum yearly salary of the top 20 States ordered by median income. It is worth noting that NH had a single position in the data set, so even though it is in the top tier it does not have enough data to show the spread of its income.

These figures show us the median of the lower bound salary for data science positions is above $100,000 per year. Therefore, we can safely suggest that even though some junior positions may not have a 6 figure salary, it is very likely that with experience they will achieve this pay scale in their career.

To close our investigation, we take a look at how quick positions were filling up by comparing the average posted duration of opened positions versus filled positions. Two days after the initial data were collected using scrappy, we ran the second script and recorded which positions were already filled or were not available anymore. The figure below shows that tech hub states like California, New York, Virginia,  filled their positions very quickly while other non-tech have trouble filling up positions. This is further illustrated on the average days a position stays open. We can see Nevada, Nebraska, etc. have the highest average posted days. We may  infer from this observations that non-tech oriented States have difficulties filling their positions. However these States could be an alternative for people who have trouble finding jobs in the most competing States -- their plan B when plan A doesn’t work out.. Even Though the salary might not be as high as in the high tier States, the lower cost of living in these States  may make up for that disparity.

Conclusion

In summary, we can say that data scientists still have a fair chance of finding a good paying job in the age of COVID-19. The job search  might not be easy, depending on which state and industry you are targeting. However, if one is open to expanding the search, it is worth considering the jobs in  locations  with less competition.

About Author

Avatar

Patrice Kontchou

Certified Data Scientist with a Masters in Software Engineering (concentration in Artificial Intelligence). Enthusiastic and self-motivated, backed by professional experience driving business insight from terabytes of data using visualization, statistical analysis and machine learning. Strong discipline and leadership...
View all posts by Patrice Kontchou >

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp