Exploratory Data Analysis of Real Estate in Manhattan
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
Overview of Project
Using Python's Scrapy package, the goal of this project was to conduct an exploratory data analysis of on residential real estate listings in Manhattan. This analysis aims to answer three questions regarding the borough's real-estate listings:
- What are some of the general trends in real-estate listings in Manhattan?
- What factors can affect the price of a Manhattan listing?
- How affordable and financially feasible is purchasing residential real-estate for the average New Yorker?
The data for this project was scrapped from Realtor.com and consists of 1663 individual listings containing information on the street address, price, number of bath & bedrooms, property type, and square footage.
General Dara Trends in Manhattan Listings
The bar plot above shows the total number of property types in the data set. Overall, the two predominant property types in Manhattan appear to be Co-ops and Condos. Townhouses, Single Family Homes, Multi Family Homes, and hybrid properties collectively account for less than 10% of the total number of listings.
The set of boxplots above depicts the listing prices for residencies across several property types. The prices were transformed on a logarithmic scale in order to better visualize the distributions. Townhouses have the largest overall listing price while condos and co-ops have the lowest overall prices.
Closer Examination of Co-Ops & Condos Data
Because Co-Ops and Condos comprised a large majority of the Manhattan real-estate listings, these two property types were further focused on for further analysis.
The following two histograms show the distribution of the total number of rooms in a listing between condos and co-ops. Overall the number of rooms clusters around 2-4 total rooms.
The boxplots above show the distribution of square footages among the Co-Op and condo property types. Overall the distributions between the two groups is largely similar with most listings ranging between 900 to 2000 square feet.
Evaluating Data on Factors Correlated with Listing Price
In addition to evaluating some of the trends regarding Manhattan listings, this analysis also examined which housing features showed strong correlations with the listing price. A Pearson correlation test was used to evaluate the correlation among each of the variables. The two variables with the highest correlations were square footage (r = 0.807) and the total number of rooms (r = 0.744).
The following graph is a plot of square footage against listing price modelled also with a trend line. The regression line shows a strong correlation near the start, but begins to show significant errors as the square footage increases
The following graph depicts the total number of rooms in a listing plotted against the listing price. A linear model was used to visualize the relationship between room number and price. From this graph, it appears that there is a positive correlation between the number of rooms and listing price.
Finances of Purchasing Manhattan Real-Estate
The pair of boxplots above show the distributions of monthly mortgages. The mortgages were calculated by finding the difference between the listing price and down payment dividing three hundred and sixty, which is the number of months in thirty years. The red plot shows the monthly mortgage after a twenty percent down payment, and the blue plot shows the monthly mortgage after a five percent down payment.
The boxplots above show the distribution of recommended yearly salaries based on the potential monthly mortgage of a listing. The recommended salary was determined by calculating a salary in which 30 percent of the annual income goes towards the monthly mortgage payments. The red plot shows the distribution of salaries if a listing had a twenty percent down payment, and the blue plot shows the distribution of salaries if a listing had a five percent down payment. The horizontal line below the twenty-fifth percentile of each plot is the median household income of Manhattan residents in 2018, which was about $60,000.
Conclusions & Future Steps
From the scrapped real-estate data, the following patterns observed in Manhattan residential real-estate:
- A large portion of listings consist mainly of Co-Op and Condos
- Some overlaps in the prices and size of Co-Op and Condo listings
- Very few large residential listings in Manhattan
- Square footage and the number of rooms were positively correlated with the price of a listing
- Most of the residential real-estate listings are potentially unaffordable to the average New York City resident
The data gathered for this analysis represents less than a third of the total number of Manhattan listings available on Realtor; however, due to time-constraints and technical issues, the analysis was conducted with just the data on hand, so the estimates do not offer a full & comprehensive overview on Manhattan residential real-estate. Furthermore, the data do not indicate how prices have shifted as a result of the COVID-19 pandemic, and the data also do not look towards prices in areas surrounding Manhattan like New Jersey, Upstate New York, and Queens. Therefore, a future iteration of the project would broaden the scope real-estate listings & make comparisons between historical pricing data.
About Author
