DATA STUDYING THE LABOR MARKET DURING A PANDEMIC
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
Introduction
During the Covid´s pandemic the world´s economy has suffered from a radical slowdown. This has been harshly felt in the labor market, affecting the opportunities of citizens to earn an income. This project aims not at providing an explanation of the general impact of the pandemic in the labor market, but instead to use data provide a snapshot to help identify possible niches of opportunity during one of the worst moments of the pandemic.
Thus, the aim is not to produce general claims about the labor market in the country, but to offer a picture of the available opportunities in the neighboring States of New York, New Jersey and Pennsylvania.
The Data snapshot
The data has been obtained scraping Glassdoor. The popular job portal allows gathering information about vacancies within a geographical range and filtering by other categories such as industry and salary. For this project, any restriction was introduced in terms of salary or industry and the geographical range has been chosen to cover the States of New York, New Jersey and Pennsylvania.
From each vacancy, data has been collected for: the particular job function, the company offering the vacancy, the industry in which it operates, the number of employees (expressed in a range), the State in which the company is located and the salary offered for the particular job. Right before performing the analysis, the database accounts for 563 observations and 7 variables.
Exploratory data visualization
The insights obtained from the exploratory data visualization are displayed below and offer an account of the labor market´ situation in the States of New York, New Jersey and Pennsylvania.
Firstly, we visualize the number of vacancies offered in each of the industries that conform our dataset.
Figure 1: Number of vacancies per industry
As we can see in Figure 1, IT is the industry offering a higher number of jobs. This is probably due to more demand for on-line services during the pandemic combined with the fact that telework is easier to implement in this industry. Health care is the second industry that offers a higher amount of vacancies. This is coherent with the greater need of health care professionals during the pandemic. As we can already begin to realize, the pandemic affects firms and workers in different industries differently.
Figure 2: Salaries per industry
When it comes to salaries, Figure 2 shows that Business Services is the industry that offers workers the highest remunerations. IT appears again as a good option in the second place, another signal that IT workers are doing better than most others during the pandemic. Health care is also an industry in which high salaries are offered. On the other hand, entertainment, non-profit and construction appear as the industries with the worst paying vacancies at the time. Especially in the case of the construction industry, this may be affected by the restrictions on the work opportunities, as most of the work cannot be done remotely.
Figure 3: Number of vacancies per State
As expected, figure 3 shows that New York is the state with a higher amount of offered vacancies. Pennsylvania´s low number is probably due to the geographical distance from New York, which makes jobs in New York and New Jersey to be more likely to be displayed by Glassdoor.
Figure 4: Average salaries per State
Figure 4 shows similar average salaries for New York and New Jersey, with Pennsylvania offering slightly lower ones. This may be a surprising result, as the prior might be that New York would offer higher salaries. In order to clarify what is driving this result, we compare average salaries across industries between the State of New York and New Jersey.
Figure 5: Average salaries across industries between the States of New York and New Jersey.
Figure 5 shows how the average salary per industry is actually higher in New York for every industry we have data on with only one exception. This is the Business Services industry, in which there is a big gap between both states. In the next Figure, we dig deeper into this industry to check what is behind this big difference.
Figure 6: Average salaries per firm in the business service industry
As we can see in Figure 6, there is one company in New Jersey, OptiBrand Rx, that is offering an average salary sharply above other companies in the industry. A close look at the data reveals that OptiBrand Rx is looking to hire for many executive positions, which imply higher wages. In addition, this company seems to be in a hurry to find suitable candidates, as it has published each of those vacancies several times, with slight salaries increases, thus augmenting the weight of the company in the State´s average. This appears to be a partial explanation for the difference in average salary between both States.
Figure 7: Proportion of vacancies per industry and State.
Figure 7 shows another possible explanation. It displays the proportion of vacancies offered by industry in each State. This allows to check whether the difference in average salaries may not only be due to differences across industries but also because a certain State offers a higher proportion of its vacancies in high paying industries.
As we can see in the graph, New Jersey offers a higher proportion of its vacancies in high paying industries like IT or insurance. Despite this, it is difficult to know for certain the net effect as it also offers a smaller proportion of vacancies in another high paying industry such as healthcare. Nevertheless, a look at the graph provides the insight that this could also help to explain why there are no differences in average salaries between New York and New Jersey although New York pays higher average salaries across almost every industry.
Figure 8: Salaries per number of employees in the firm.
In Figure 8 we turn to analyze how the number of employees in a firm is related to the salaries it offers.
Intuitively, one may expect firms with higher numbers of employees to offer higher salaries. However, we see that this is not the case. Actually, small firms (1 to 50 employees), seem to offer the highest ones while big companies offer lower wages. An explanation for this may be that firms hire a mix of employees with different sets of skills and wages. Following this reasoning, bigger firms may offer above average wages to certain employees but this may be offset by the wage offered to workers with a lower skill level. We try to confirm if this is the case in the next two figures.
Figure 9: Number of vacancies per salary level and number of employees in the firm.
In Figure 9, we break down the vacancies offered into two categories, low pay and high pay. In order to classify them, we calculate the average market salary and assign a vacancy as high pay if the offered salary is above the average market salary and low pay otherwise. Then, we compare the number of each of these types of vacancies across firms with different numbers of employees.
As we can see in the graph, smaller firms actually tend to offer a higher proportion of low paying vacancies that bigger companies do, with the exception of firms in the range of 1 to 50 employees. Now we can see how firms with more than 10.000 employees offer a higher proportion of high paying jobs. Why then do smaller firms offer higher salaries on average? In order to try to provide a more definite answer, we compare the average salaries of the low pay vacancies previously classified across firms with different numbers of employees.
Figure 10: Data on Average salaries of low pay vacancies per number of employees.
In Figure 10, the order in which the firms are displayed have been reversed to show the ones with the highest number of employees above. As we can see, companies with more employees offer lower average salaries in their low paying vacancies in comparison to smaller firms.
Conclusion
We provide evidence to support possible niches in the labor market that have been less affected during the pandemic. Specifically, the IT industry in a firm above 10.000 employees located in New York stands as one of the best options. The healthcare industry also provides good economic opportunities, although applicants should also weigh the health risk.
For high skilled workers, companies with a high number of employees stand as a better choice than smaller ones. However, this is not the case for job-seekers with less valuable skill sets. These are probably better off in a smaller firm. Finally, New York offers better economic opportunities than New Jersey or Pennsylvania. This may be due to the higher health risk associated with living and working in a huge metropolis in comparison to less densely populated areas in the neighboring States.
The relevant code and data for this project, including web scraping and analysis, can be found here.
About Author
