Prospect of data-related jobs across United States

Posted on Jul 25, 2016

I. Introduction

Contributed by Jiaxu Luo. He is currently in the NYC Data Science Academy 12 week full time Data Science Bootcamp program taking place between July 5th to September 23th, 2016.This post is based on his first class project - R visualization (due on the 2nd week of the program)


The Office of Foreign Labor Certification (OFLC) provides labor certification to US companies that want to hire someone from outside of the United States for a special tier or technical position. Typically, companies have to state how much they are planning on paying the employee when they file an application. Additionally,  the average amount an employee with similar skills and background typically gets paid for the same position is also required.

The OFLC has begun releasing program disclosure data on a quarterly basis on their OFLC Case Disclosure Data page. The dataset in the following data analysis is a subset of the 2011 to 2015 data publicly available on the website, which has a volume of nearly 16,000 rows and 28 variables. Each row of the dataset represents the general information of a foreign applicant.

Using this dataset we can get a glimpse of the prospect of data-related jobs across United States for foreign laborers.

 II. Analysis Plan

This analysis identifies how the demand and average salary of specific job-titles("data scientist","data analyst" and "business analyst") relate to the location (State, city) and company.


 Questions and hypothesis
(1) what is the distribution across United States?
hypothesis: The California, Great Chicago area and greater New York areas should have the greatest demand for data-centric employees.
(2) which states, cities, provide the highest average salaries for data-related jobs?
 hypothesis: California rank first and then New York State
(3) which companies provide the best salaries for data-related jobs?
hypothesis: Google, Facebook, LinkedIn, Airbnb


Variables of Interest
Case Number:Unique identifier assigned to each application
JOB_TITLE_SUBGROUP:Category of job title
WORK_STATE: State of intended employment
WORK_CITY: City of intended employment
EMPLOYER_NAME: Name of employer requesting certification
PAID_WAGE_PER_YEAR:  Wages employer is going to pay the employee


III. Visualization Analysis

Firstly I  selected the "JOB_TITLE_SUBGROUP" ("data scientist","data analyst" and "business analyst") of interest and I aggregated over the "case number" by "WORK_STATE " level and then counted the  total number of these data-related jobs. In the map below, the size of the bubble indicates the number of data related jobs by state.
The first graph primarily goes hand in hand with what I expected. California and New York State is leading way ahead in demand of skilled individuals with analytical skill. It makes sense because Silicon Valley and Wall Street alone are playing a important role in the demand. However, as the graph suggests, the job demand density on the eastern coast is much higher than the western coast. We can easily see that the circle representing NY, NJ and MA demand overlap with each other, indicating there are much more opportunities around these areas. Interestingly, the sheer demand of data-related talent in Texas surpasses my expectation. I suspect that there are some special local policies for many modern companies to set their foot in Texas.


While the visualization provides insight about where the demand for data specialists is, demand is not the the only factor determining where best to look for a job. Salary  also matters. I then calculated the mean“PAID_WAGE_PER_YEAR ” by state. In the map below, a color gradient is used to code the salary level.  The darker the blue, the more average salary a data geek could expect in that state.
It turns out WA, along with CA and NY consecutively were the three most generous state for  data-related jobs. It is a little bit surprising that WA salary is so high given that the demand is no match for the salary. One explanation is that Seattle is where many Internet companies are situated, even though the IT industry is no match for those in CA. For example, Amazon's headquarter and Microsoft's super large data center is right there.


With this general picture in mind, I explored further to see how the cities with the most data-related jobs welcomed states differ from each other in terms of demand and salary. Here a treemap is plotted to demonstrate the intertwining relationship between four variables "state","city", "average salary", "demand").The cities were first grouped by state and the size of the rectangle represents the demand and salary respectively in each city.
The treemap delivered a lot more useful details for us to give a bottom-up view of the prospect of data-related jobs in a given city. As I expected, in New York State, New York City is the most ideal place to work; NJ is in high demand for a lot of people to engage in data analysis. The result is not surprising because there are many pharmaceuticals companies there using data  to fuel the industry; In general, cities in TX have a similar structure as those in NJ; Notice three cities in WA, provide decent salaries for data-related jobs.This makes sense since WA ranks first in the average paid wages of data-related jobs;As for CA, this graph reveals compelling evidence that there are quite a few cities to work with average salaries ranging from 60,000$ up to 120,000$. No wonder California is the cradle of technology. Talents in technology are generally appreciated there.
Lastly, I was curious about what companies in the specified states provide the most attractive salaries. On the one hand, it  is helpful for job seekers in data field to identity companies that are data-centric, if not, at least data-aided. On the other hand, job seekers can also adjusted their expectation according to the level of their skills. In order to prevent bias caused by the scenario where some companies might hire people for just one or two senior positions, which would result in incredibly high average salary yet be of little use for the general job seeker, I applied a condition here that the names of the companies would only be seen provided that they hired more than 8 employees for this dataset.


 Among the top four are companies that one might imagine would pay high salaries for data related jobs, as these are successful companies that are data centric by their nature.  For example,  Netflix is famous for its recommendation algorithm, which once caught the attention of teams of machine learning experts across the world to optimize existing algorithm. Walmart has long been focusing on the value of data for digital marketing and operation optimization. And Twitter, after realizing the benefit of the API issuing and data sharing, is now creating its own data team. 


As a by-product of looking at company salaries, an interesting finding was the range of different industries and types of businesses represented here. As a by-product of looking at company salaries, an interesting finding was the range of different industries and types of businesses represented here, such as general retailer, banking, software and media.

IV. Conclusion

(1)CA,WA,TX,NJ and NY are the most  prosperous states in data industry. Among them, CA seems to be  the best place to work generally.
(2)In CA, you have many choices in which city to work with promising prospect in data industry. In NY, however, if you do not feel comfortable  living in New York City, it might not be a good choice to work there.
(3)If you are not ambitious enough to seek for a job with very high salary, NJ, with large demand and comparatively lower average salaries, might be a good choice.

About Author

Jiaxu Luo

Jiaxu Luo is a recent MS graduate from Columbia University. As a child born in a business family and a student with diverse academic background in Material Science and Chemical Engineering, he grasps the importance of asking questions...
View all posts by Jiaxu Luo >

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI