Data Forecasting Zillow Rent Index
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
GitHub
In working with our partner company, our objective was to create a data model to forecast the Zillow Rent Index (ZRI) for multifamily homes in the United States. Multi-Family homes are buildings with 5 or more housing units. The motivation for this project is to provide an accurate model for what rent prices will be in 1 to 3 years. Clients are interested in this information to decide if they should raise/lower rent prices or offer move-in specials. Rent price forecasting of specific zip codes is also important to real estate developers and investors who have to determine if they will be able to recoup their investment.
Data Overview
We utilized five data sources to build our models. Using data from the American Community Service (ACS), 5-year aggregate zip code level data on demographics, income information, rental housing units, and education level. We also used IRS data on the zip code level, labor force data by county, mortgage delinquency data from consumer finance monthly by county, and the United States Geological Survey’s Natural amenity Index.
Preprocessing
Two zip codes were not represented in the data we gathered and were dropped. The zip codes that had missing rent for the month were removed. One more zip code, that of Riverhead, NY, was dropped because of outlier rent prices.
Data on Feature Engineering
When we were doing our exploratory data analysis, we saw the need to engineer several features. Population change, income per capita change, rental vacancy rate and supply change were all created by comparing the change from the previous year’s ACS data, to the current year ACS data. Education was created using the ACS data by creating a weighted average of type of degree. Season was created by looking at which season had the lowest median rent price, with a 1 assigned to the lowest median rent season and a 4 to the season with the highest median rent.
Data on Tree Models
Using this data, we started with tree base models. In trying to predict future rent, we wanted to look at feature importance. With and without using the previous year’s ZRI we can see how well just our public data features are performing. As seen in the table, there is not surprisingly, a huge increase in the R2.
When including ZRI as a predictor, it overwhelmingly dominates feature importance and does not allow us to look at differences between the rest of the features. We decided to look at the feature importance of Random Forest without ZRI as a predictor. We can see in the graph below that TotTaxes (which is the total amount of taxes paid, divided by the population) is the most important feature. That is closely followed by zip code and the USGS’s natural amenity rank, a ranking that the United States Geological Service created to rank areas based on their natural beauty. This graph guides our decision on where we would start with our linear models.
Our best performing tree model was an XGBoost model. The above chart shows the residuals of our model that are mostly randomly distributed. There is a horn-shaped area of errors around 2500 to 2700 predicted value range that we will look into later to try to determine why our model performed poorly there.
Linear Models
We decided to use linear regression in rent forecasting for its advantage of interpretability. As part of the preprocessing for the linear model, we took the log transformation of rent prices to obtain a normal distribution and dropped outliers.
Our first model, with 15 features, was overfitted so we ran a penalized regression model with Lasso. Lasso zeroed out the education, Hispanic pop, unemployment rate, population change, and median age, features that were dropped in the next model.
Our final model has a train accuracy score of 96% and a test score of 95%. We checked for multicollinearity by checking VIF, variance inflation factor, and we found no issues with multicollinearity between the features so we kept all 10 in the final model.
We looked at the correlation between the 10 features and ZRI. Interesting to note here, the 'vacant housing units for rent' feature was negatively correlated with rent prices. This is to be expected as when the supply of rental units is greater than demand, prices will drop.
Residual of Final Linear Model
When we checked the residuals of our final linear model we saw no real pattern in our residuals, except for a slight horn in the upper end.
We decided to explore this horn further by splitting our dataset between high and low populations. The population was cut at the median; anything above 36k was classified as high, and anything at or below it as low. Both the linear model and the XGBoost model performed slightly better with the low population data and the MSE error improved by $2 in both.
You can see this change much more clearly in the residual plot for each. In the high pop dataset, the horn still persists but in the low pop, the horn pattern completely disappears.
Further examining the population, we broke the dataset into 4 quartiles. The top quartile performs the worst in our XGBoost model but the performance between different population quartiles was not significant.
Because the model didn’t perform drastically differently in two population or 4 quartiles models or the quartiles population model we decided to explore the zip codes of the residuals closer.
A closer look at the residuals
We then investigated the worst residuals in our model to see what those observations had in common. This graph shows us the zip codes with the worst residuals. We graphed their ZRI value over time to look for a pattern. The group of lines with the higher ZRI are all from the Los Angeles metro area but the other zip codes are from upstate New York. They all show a steep decline in ZRI between 2016 and 2019. It is not surprising that our model performed poorly given that previous ZRI is the strongest predictor of future ZRI, and the model was trained on 2015 data to predict 2018 zri. So from here, we wanted to investigate what was happening in these places.
We wanted to see what in these missing zip codes could cause such a drastic decline, but nothing stood out. One possibility is the lack of rental units in these zip codes on Zillow. Searching these areas on zip codes show only a few listings and a few rental properties could possibly have thrown the ZRI off.
The above graphs show our model performance before and after dropping those zip codes that our model was bad at predicting. There is a slight improvement in the model but nothing really of note. The main takeaway is that if ZRI happened to have a large decrease, our model was unable to capture that effect.
Conclusion & Future Work
Not surprisingly, the previous ZRI was the best predictor of future rent. The problem our model had was that since ZRI almost always increased, a decrease in ZRI was hard to capture. The number of vacant housing units had a negative correlation with ZRI. Finding a better data source for this feature may be helpful in predicting decreasing rent prices. Nevertheless, all this information would have never been able to predict a pandemic and the effect it has had on the housing market as people look to leave rental properties in urban areas.
Future work could include implementing a time series model to see how that would perform, especially with our zip codes that our model performed poorly on. We would also like to explore creating a model that forecasts the Zillow Observed Rent Index (ZORI) which is a new measure of rent. Zillow has begun using ZORI to fix issues they have come across with ZRI. This could explain some of the large decreases in rent that we came across with ZRI and could contribute to a model with better predictive power.
About Authors
Isabel Alvarez de Lugo
View all posts by Isabel Alvarez de Lugo >