Pandemic Effects on the Ames Housing Market and Lifestyle
Introduction
Purpose
The purpose of this project is to assist real estate agents in their service to out-of-town clients who are planning to move to Ames from a more urban setting. To assess the factors that affect the sale price of each home, consideration will be given to the proximity of desired services (e.g., gyms, spas, organic groceries, hospitals, etc.) as well as other housing features. Additionally, the project will analyze the housing market change due to the pandemic (i.e., 2019 vs 2021 data) and determine the neighborhood preferences for this particular client group.
Data
The data was sourced from the Ames City Assessor website that archives recent home sales up to 2021. For the purposes of data cleaning, undeveloped lots were excluded from analysis, and only house sales from 2019 and 2021 were considered. Important dataset features include Sale Price, Year Built, Year Sold, Neighborhood, Property Address, and Other Dwelling Assessor Value (i.e., dwelling value = total value – land value).
Data Exploration
Sale Price
Figure 1: Ames neighborhood map by average Sale Price (color coded)
Figure 1 shows house sales distribution by neighborhood in Ames, Iowa. The annotations above show the average Sale Price within each selected neighborhood group along with the number of residences. Of note, condos generally had the lowest Sale Price, while the northern and western parts of the city exhibited the highest Sale Prices.
Services
Figure 2: Selected service and business locations in Ames, Iowa
Figure 2 depicts the selected services (e.g., Organic Grocery stores, Parks, High Schools, Gyms and Religious Institutions) based on possible interests of out-of-town home buyers. The individual businesses were manually chosen based on search engine user ratings (e.g., Google, Yelp) and popularity. Most businesses were located in the downtown area (i.e., Old Town).
Year Built
Figure 3: Ames neighborhood map by Year Built (color coded)
Among the features that could influence Sale Price, Year Built seemed to be significant. Figure 3 shows the progression of new house construction in Ames. The oldest houses sold were built in the 1880s in Old Town, while the newest houses were built in 2020 and appear in the northern and western suburbs of town. Comparing Figure 3 to Figure 1, it appears that the newer houses generally have a higher average Sale Price.
House Sales
Figure 4: Frequency of house sales per month (2019 vs. 2021)
Another point of interest is the house sales activity in 2019 and 2021. Figure 4 shows the distribution of house sales by month for both years. A chi squared test indicated that there was a significant difference (p-value < 0.05) in the activity across both years, even when accounting for house sales in 2021 ending in August. Year 2021 shows a more even distribution of sales throughout the year compared to 2019.
Feature Engineering
Lat Longs
Creating visualizations for this project entailed converting the Property Address field into lat long coordinates. This was done by calling the Geoapify API service. To validate the lat longs through a cross check with the original street name, we used Nominatim for a reverse lookup. Any discrepancies were then manually corrected to provide the most accurate locations. This process was performed for both houses and businesses.
Code Block 1: Geoapify API call function that takes in address input and returns lat long string
Drive Time
Code Block 2: OSMR API call function for drive times lookups that takes in as input the origin destination lat longs and returns the driving time to or from the destination
For every business and house combination, the driving time and driving distance was calculated using an OSMR API call as seen in Code Block 2. This allowed for the measurement of proximity to local businesses in terms that home buyers could understand. In addition, more advanced features were calculated, including the closest service to each house, the number of businesses within a certain driving time, and the average driving time to services.
Figure 5: Number of businesses within 180 vs. 480 seconds to each house
After the creation of various driving time fields, a select few were chosen based on their relative significance. For instance, in Figure 5, there is a large gap in the distribution of the number of businesses between a 3 minute (180 seconds) versus 8 minute (480 seconds) drive from each house. In other words, most businesses were located within 3 to 8 minutes of each residence.
Figure 6: Ames neighborhood map by minimum average driving time to Closest 5 Services (color coded)
In addition, the closest businesses to each home was determined, allowing for the calculation of the average driving time to the Closest, Closest 5, and All services. In the context of the Closest 5 services, neighborhoods in the far west of Ames, in Figure 6, had the longest average drive times, while those in the center and north of Ames had the shortest average drive times. Old Town had the absolute shortest drive time, as expected due to its centrality. Overall, however, the effect on Sale Price due to the proximity of the Closest 5 services is not entirely clear.
House Features
Determining the factors that affect Sale Price also required the conversion of house features into engineered fields. In terms of area, the Total Area field (Gross Living Area + Basement Area) and the Total Area with Garage field (Total Area + Garage Area) were calculated. Furthermore, boolean flags were created for the presence of significant amenities: Fireplace, Basement, and Garage.
Figure 7: Ames neighborhood map by average Total Area with Garage and Basement (color coded)
Figure 7 shows the distribution of Total Area with Garage. Neighborhoods in the northwest of Ames had the largest, while condo neighborhood subdivisions and Old Town had the smallest. Based on Figure 1, the Sale Price corresponds to a larger living area (e.g., North Ridge).
Covid Effects
Regression
Figure 8: Comparison of Sale Price to Other Dwelling Assessor Value pre-pandemic vs. post-pandemic
In order to compare the effects before and after Covid, it is necessary to analyze important features using descriptive models and testing. Figure 10 shows that the Other Dwelling Assessor Value field is the most important factor in determining the Sale Price.
In Figure 8, a regression is performed to demonstrate how the year affects the correlation between this feature and the Sale Price. Even though 2021 has a smaller number of records, there was a greater range of Sale Prices for that year. The plot also shows that the Sale Price for any given Assessor value was higher in 2021 than in 2019, meaning that the pandemic may have increased the demand for housing.
Hypothesis testing
Table 1: A/B testing results for categorical features across both 2019 & 2021
Table 2: Hypothesis testing results for numerical features across both 2019 & 2021
A/B testing was performed to compare categorical features across both years (Table 1). 2019 showed a preference for more moderately priced neighborhoods (Sawyer West) whereas 2021 seemed to split between more expensive and less expensive neighborhoods (North Ridge and Old Town). Overall, the houses bought in 2021 have more extreme features and Sale Prices.
Hypothesis testing was performed to compare numerical features across both years (Table 2). Sales in 2021 indicate that more houses were bought closer to services in general, as seen by the reduced average drive times (Closest 5 and All).
Closest Service Analysis
Figure 9: Closest service percentage comparison (2019 vs. 2021)
Finally, in Figure 9, a stark contrast between 2019 and 2021 is seen when comparing the frequencies of the Closest Service to each house. In 2019, gym and recreation services had a higher percentage of nearby house sales. However, in 2021, restaurant, religion, and shopping services had a higher percentage. There may have been a migration towards downtown during the pandemic since restaurant, religious, and shopping services were located in the center of town, as opposed to gym and recreation services which were located more in the suburbs.
Predictive Modeling
Feature Reduction
Due to the large number of features, it was important to reduce the feature space in order to alleviate multicollinearity issues. Sequential Feature Selection (SFS) using tree-based ensemble regression was utilized to reduce the number of features by half. Notable features that were determined to significantly influence the Sale Price included Total Area with Garage, Other Dwelling Assessor Value, Sale Condition, and Year Built as seen in Figure 10.
Figure 10: Top important features from final Gradient Boosting predictive model
Several regression models were tested, including Multiple Linear, Support Vector, Random Forest, and Gradient Boosting. Gradient Boosting, with GridSearchCV tuning its hyperparameters, resulted in the best model. The final optimized model reached a coefficient of determination (R2) of ~89%. As seen in Figure 10, the Other Assessor Value and Total Area with Garage have the highest importances over all other features (~50% and ~25% respectively).
Sensitivity Analysis
Table 3: Sale Price prediction based off the addition of selected features and consumer preferences
To utilize the predictive model, 3 records were chosen from the dataset representing a townhome, condo, and single-family detached home. All records were standardized by Year Sold, Sale Price, and Year Built. Specific features were changed or added to predict Sale Price for each unit type. In Table 3, the cheapest change for 2 out of 3 homes was hypothetically having shopping as the Closest Service, while the most expensive was adding a fireplace.
Notably, 2021 housing prices were significantly larger than 2019 for all house types. Condos had the largest percent change when adding amenities, whereas single family detached homes had the smallest. Overall, proximity to services, in Table 3, had less of an effect on Sale Price than either Year Sold or the addition of amenities.
Conclusion
In addressing the clients concerns, most houses in Ames were within 3 to 8 minutes of all services (gyms being the closest). The house amenities that had the greatest impact on Sale Price were fireplaces and central air: fireplaces distinguish expensive homes from average homes, whereas central air distinguishes average homes from inexpensive homes.
In addressing real estate firm concerns, house Sale Prices post-pandemic were more extreme in 2021 than in 2019. Real estate agents should focus on selling homes with more extreme Sale Prices: less expensive or more expensive than average. Based on the predictive model, proximity to any given service has no real effect on Sale Price. Post-pandemic, the best neighborhoods to recommend would be North Ridge and Old Town, which are both convenient to all services. However, based on Total Area with Garage in Figure 7, we see that North Ridge is best for families and Old Town is best for single individuals.
Links & References
- Real Estate Dataset Source: Ames City Assessor Reports
- Housing Dataset Source: Ames Housing Data Kaggle
- Original Paper: Ames, Iowa: Alternative to Boston Housing by Dean De Cock
- Github & Final Presentation: Pandemic Effects Project
About Author
