Data Science Job Market in the Age of COVID-19
The skills the author demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
According to California University of Pennsylvania “Data science was named the fastest-growing job in 2017 by LinkedIn, and in 2018 Glassdoor ranked data scientist as the best job in the United States.” In the age of COVID-19, we wonder how this trend is standing against the gruesome repercussions of COVID-19 on the economy. This project aims at investigating the following questions.
- What is the distribution of data science jobs per State?
- What industries are prominent in each State?
- Which sector/industry holds the most Data Science jobs?
- Is the salary still three figures?
- How fast are these positions being filled ?
Data Acquisition
Data used in the project was scrapped from Glassdoor using scrapy and selenium. A total of 864 jobs were scraped. After cleaning and filtering, 634 unique jobs were used. Jobs scraped were the result of the query “data science” with location and all other fields left blank. The screenshot below shows a sample of the cleaned data.
- Company: Name of the company recruiting
- City,State: Address of the position being filled
- Year_founded: Year the company was founded
- Job_title: Title of the position
- Days_posted: Number of days the position has been on Glassdoor
- Started_salary and Max_salary: Estimate of the minimum and maximum yearly salary by Glassdoor.
- Industry and Sector: The industry and sector of the company that is recruiting.
Exploratory Data Analysis
A first look into the data shows the distribution of jobs by states. As shown below, we can see that major tech hub States still hold quite a lot of positions in this industry.
A dive into the data slices it into sectors to understand which one is offering most positions.This is further grouped by State to show principal sector per State. This will give job seekers an inside look on how their interest match up with their preferred industry and location, so that they have a better idea of how to center their job search.
Salary
A look at salary estimate columns shows median, mean, minimum and maximum estimated lower and upper bound yearly salary per State. This valuable insight helps understand the pay rate of data scientists and what new data scientists can expect. The two figures below show the minimum and maximum yearly salary of the top 20 States ordered by median income. It is worth noting that NH had a single position in the data set, so even though it is in the top tier it does not have enough data to show the spread of its income.
These figures show us the median of the lower bound salary for data science positions is above $100,000 per year. Therefore, we can safely suggest that even though some junior positions may not have a 6 figure salary, it is very likely that with experience they will achieve this pay scale in their career.
Average Positions Closed or Posted
To close our investigation, we take a look at how quick positions were filling up by comparing the average posted duration of opened positions versus filled positions. Two days after the initial data were collected using scrappy, we ran the second script and recorded which positions were already filled or were not available anymore. The figure below shows that tech hub states like California, New York, Virginia, filled their positions very quickly while other non-tech have trouble filling up positions. This is further illustrated on the average days a position stays open.
We can see Nevada, Nebraska, etc. have the highest average posted days. We may infer from this observations that non-tech oriented States have difficulties filling their positions. However these States could be an alternative for people who have trouble finding jobs in the most competing States -- their plan B when plan A doesn’t work out.. Even Though the salary might not be as high as in the high tier States, the lower cost of living in these States may make up for that disparity.
Conclusion
In summary, we can say that data scientists still have a fair chance of finding a good paying job in the age of COVID-19. The job search might not be easy, depending on which state and industry you are targeting. However, if one is open to expanding the search, it is worth considering the jobs in locations with less competition.
About Author
