Predicting ZORI using COVID-19 and Census Data

Introduction

The COVID-19 pandemic drastically shifted consumer preferences in housing. This created a scenario that made housing prices difficult to model, with price variability changing constantly as the pandemic continues.

Objective

To observe these changes and how they will affect the future of the market, we created models that can accurately forecast and predict the Zillow Observed Rent Index (ZORI) by using historical ZORI, US Census, and Covid-19 data. Predictions can be used to see which areas are recovering from Covid-19 more quickly. This kind of information can be useful to:

  • Government Organizations: find areas in need of resources
  • Investors: find which neighborhoods to start businesses in and which to stay away from
  • Business Owners: gain insight into your own neighborhood to decide when to increase or decrease inventory

Datasets

Zillow Observed Rent Index

  • Monthly rent prices of the center 20% (40th-60th percentile) of the entire observed market, Smoothed & Seasonally adjusted. (Jan. 2014 - June 2021)
  • 1743 rows by 93 cols.

US Census Data

  • Yearly data collected and/or predicted by the US Census Bureau pulled from BigQuery (2014-2018)
  • Housing, Income and Demographic datasets pulled from the Census Website (2019)
  • 33,120 rows by 232 cols, 6 years of data

COVID-19 Data from Johns Hopkins

Data collected by the Johns Hopkins Center for Systems Science and Engineering (JHCSSE)

  • Daily counts of US COVID-19 cases and deaths by US county (Jan 22 2020 - July 17 2021)
  • 3342 rows by 555 cols

Data Cleaning and Feature Engineering

ZORI Data:

Missing Value Interpolation

  • Interpolated linearly from Jan. 2014 (start) to June 2021 (end)
  • Any ZORIs missing at the time period’s start or end were calculated using that year’s mean change in ZORI 

Outliers

  • Each ZIP code was checked for any ZORI that was >3 stddevs away from that year’s mean rent price
  • No outliers were found, and no ZIP codes were removed

Census Data:

Raw Data

  • 2014-2018 data comes from 5 BigQuery Census Datasets
  • 165,600 observations (33,120 ZIP codes x 5 years of data)
  • 232 Features

Missing Data

  • Removed 4 Features that had 5,000+ null values and little to no correlation with the target feature.

Combining Census Years: 2014-2018 and 2019

  • Could only pull 47 matching features between the datasets
  • 2019 census data is duplicated for 2020, will be replaced with actual data in 9/2021

Feature Selection

  • Reduced to 36 Features using a Forward Stepwise function

Ended with 13,920 observations (1740 zip codes*8 Years) and 36 Features

Covid-19 Data:

Cleaning

The JHCSSE provides a daily count of global COVID-19 cases and deaths from 01/22/2020 to present day

  • Removed all observations in non-US counties
  • Capped data to be from 01/22/2020 to 06/30/2021
  • Cumulative numbers of cases and deaths were the only metrics kept alongside location & time attributes

The count of cases and deaths on the last day of each month was used as that entire month’s case and death count. 

Feature Engineering

After cleaning & filtering the data, we engineered features for ‘New Cases’ and ‘New Deaths’, which were defined as the current month’s cases & deaths subtracted from the previous month’s. We then had to convert county-level COVID cases and death metrics to the corresponding ZIP codes within each county. To do that, we assumed that the COVID cases and deaths were proportional to each ZIP Code’s population within the county, using ZIP Code and county information from the Census data.

This ‘New Case Rate’ feature would be the exogenous variable we will use later in our SARIMAX model, which is a seasonal time-series model with exogenous inputs.

Combining Datasets:

To combine all three datasets, first we joined COVID-19 data with the Census data on the county & state, then COVID-19 case/deaths were split to the ZIP Code level based on the (population of ZIP code) / (total county population), and finally, this data was combined with the ZORI dataset. After matching all datasets together, 103 ZIP codes were not included in the SARIMA and SARIMAX models due to the lack of available corresponding Census/COVID data

Models and Methods

The following models were implemented to achieve our objective:

  1. Multiple Linear Regression (MLR) to predict the yearly and semiannual ZORI in 2021
  2. Vector AutoRegression (VAR) to forecast the ZORI from 2020 to mid-2021
  3. Seasonal Autoregressive Integrated Moving Average (SARIMA) to forecast ZORI in the first half of 202
  4. Seasonal AutoRegressive Integrated Moving Average with eXogenous regressors (SARIMAX), using COVID-19 data as the exogenous variables, to forecast ZORI in the first half of 2021

Model

Features Used

Target
MLR

ZORI (yearly, semiannually), Census 2014-20

ZORI (semiannually) from

01/2021 to 06/2021

 
VAR

ZORI (monthly) from 01/2014 to 12/2019

ZORI (monthly) from

01/2020 to 06/2021

SARIMA

ZORI (monthly) from 01/2014 to 12/2020

ZORI (monthly) from

01/2021 to 06/2021

 

SARIMAX

ZORI (monthly) from 01/2014 to 12/2020;  COVID-19 (monthly) from 1/2020-12/2020

ZORI (monthly) from

01/2021 to 06/2021

MLR Model

Using the available data, we created models to predict:

  • The current rent price
  • The rent price in 1 year while including the current rent price as a feature
  • The rent price in first 6 months of 2021 with the last 6 months of 2020 as the test set, using a biannual aggregation; training on the full data for the full time period, and data for only the pandemic time period

MLR Results

The first 2 models were used to gauge how well MLR could predict the ZORI. This give us a basis to compare the pandemic models to.

We, then, included the current ZORI and forecasted for the ZORI in one year. This gave us a much worse score, as expected. The model was trained on a time period without Covid-19 to predict on a period with Covid-19.

It would obviously result in a bad model.

Model Attributes (Yearly)Test Scores
Current Rent Price w/ PCA - 2019 (Non-Pandemic)0.7304
Rent Price in 1 Year w/ PCA - 2020 (Includes Pandemic)0.6602

For the next 2 models, the  data was redone to aggregate and interpolate semiannually. This was done to utilize the ZORI from 1/2021 - 6/2021, as the pandemic data was limited to the past 18 months.

When training on 2014-2020 data, we got a decent score, but it improved much more when only training on 2019-2020 data

Non-pandemic data seemed to skew the data away from the actual values.

Model Attributes (biannually)Test Scores
Rent Price in 6 months w/ PCA - 2020 (Pandemic)0.9593  (train on 1/2014-6/2020)
Rent Price in 6 months w/ PCA - 2020 (Pandemic)0.9966 (train on 1/2019-6/2020)

VAR Model

  • VAR Model was run on all 1743 ZORI ZIP codes as a ‘baseline’ time series forecast
  • VAR Model proved to not be a great predictor based on RMSE scores
  • However, it was a better model overall than ARIMA, ARMA, and VARMAX

VAR Results

Looking at the best RMSE’s and 5 worst RMSE’s  gave us a list of ZIP codes to highlight in later models-- can SARIMA and SARIMAX yield accurate predictions were VAR cannot?

SARIMA Parameter Tuning

Grid search was performed to determine the optimal order, seasonal order, and trend parameters for SARIMA models on a monthly mean of ZORI inputs. The optimal parameters yielded an AIC of 8.0:

  • Order (p, d, q) = (0, 0, 1)
  • Seasonal (P, D, Q, seasonality) = (0, 1, 0, 12)
  • Trend =  ‘t’ 
    • (linear parameter controlling the deterministic trend polynomial)

The same parameters were used when modeling with SARIMAX. 

SARIMA Results

We created seasonal 6 month forecasts for each of the 1640 available ZIP Codes, at first with no exogenous COVID features, for our first model. A summary table is below, with the Root Mean Square Errors the metric we were most interested in minimizing. 

From our initial VAR model we were able to get a list of five ZIP Codes that VAR performed well and was able to predict rent indices accurately, and five other ZIP Codes that had very high root mean squared errors, and we wanted to get a sense of how predictions looked from the SARIMA model. The SARIMA model results of those ZIP Codes are below.

SARIMAX Results

Next we created models for each ZIP Code with exogenous factor data from the COVID & Census data--we used 'Monthly New Cases' as  a percent to total ZIP Code population as the outside variable, with the target variable still being the rent index. We could then compare this model with the previous SARIMA model, because inputs and target variables were the same for the two seasonal models. Summaries of the full SARIMAX model are below.

Just as with the SARIMA model, we further looked into the SARIMAX forecasts for previous models' good and bad predictions--that table and graph are below (we are still mainly interested in minimizing the RMSE for our forecasts).

Results Summary

Model

Target

Results

MLR

ZORI (semiannually) in 2021

R^2: 0.9966 test on 7/2020 - 12/2020

VAR

ZORI (monthly) from

01/2020 to 06/2021 

RMSE Mean (all predictions, 18 months): $102.83

SARIMA*

ZORI (monthly) from

01/2021 to 06/2021

RMSE Mean (all predictions, 6 months): $66.66

SARIMAX*

ZORI (monthly) from

01/2021 to 06/2021

RMSE Mean (all predictions, 6 months): $69.56

Comparing Models

Overall, MLR seems promising for predicting the ZORI during the pandemic at the yearly and semiannual level, only when limited to time periods where Covid-19 was a factor. The trends in 2014 to 2018 influence the model too much when included, worsening the score.

SARIMA proved to be our best time-series model, yielding a lower overall RMSE for 55% of ZIP Codes compared to SARIMAX (45% with lower RMSE)

Both of these models had a lower RMSE than VAR, but that may be because they were both trained on 2020 data (VAR was not).

SARIMAX versus SARIMA

One major downside of SARIMAX is that it requires COVID-19 and Census exogenous actual inputs in order to forecast. 

Future Census and COVID data can be forecasted separately and then passed as the exogenous variable. However, the accuracy lost to forecasting on top of forecasts would likely increase the RMSE of SARIMAX models, as compared to SARIMA.

The mean of SARIMAX’s RMSE predictions was slightly higher than that of SARIMA, but it forecasted with a lower RMSE for almost half (45%) of the data. 

Further work needs to be done to determine qualitative aspects of the ZIP codes whose better predictor was SARIMAX versus SARIMA.

Comparing SARIMA and SARIMAX RMSE

SARIMAX yielded an improved (lower) RMSE for 741 ZIP codes (45%, out of 1640):

rmse_diff = x_rmse - s_rmse (Where x_rmse is SARIMAX RMSE and s_rmse is SARIMA RMSE)

Conclusion

SARIMA and SARIMAX

Models were created to accurately predict ZORI by ZIP code. Each modeled provided a different insight into where ZORI are increasing and decreasing the most.

SARIMA and SARIMAX models each performed better on about half of the dataset each. We would follow-up to determine what about those ZIP codes led to better predictions with one models vs the other.

Including COVID-19 as exogenous was not as useful as we had hoped. The pandemic influenced many markets; pinpointing the best data to use for exogenous is required. Furthermore, it is reasonable to infer that as populations moved out of highly-populated ZIP codes, they further tipped rent prices more favorably for the less-populated ZIP codes.

Multiple Linear Regression

We used our final model on features from the 1/2021 - 6/2021 to  predict the ZORI in the 7/2021-12/2021. We calculated the percent change between the 2 to create the chart you see on the right.

It shows the largest increases and decreases in rent. From this, you can find:

  • areas for investment
  • areas that need more government assistance

Future Work

SARIMAX showed an improved RMSE for many ZIP codes, leading us to believe that improving our exogenous variables we can improve our forecasts:

  • Find recent/accurate population data (monthly, by ZIP would be optimal)
    • Or use 2020 census data when it becomes available
  • Find accurate COVID-19 data by ZIP code
  • Find other datasets that can be used for exogenous variables
    • Dominant political party during the pandemic
    • Rates of COVID-19 recovery and/or vaccination
  • Conditionally forecast 2021, 2022 ZORI based on whether COVID-19 diminishes or follows current ZIP code/county trends

About Authors

Nixon Lim

I am a data science fellow at NYC Data Science Academy with a Bachelors in Mathematics and Psychology. I am looking for opportunities to improve efficiency and maximize resource utilization using data visualization and statistical analysis.
View all posts by Nixon Lim >

Aleksey Klimchenko

BS in Bioinformatics & Molecular Biology RPI '17
View all posts by Aleksey Klimchenko >

Alexander Pinkerton

Extensive background leading merchandise planning, allocations, and buying functions, excited about the intersection of Data Science and consumer retail to drive the future of smarter decision making. NYC Data Science Academy Student Cohort 025 WashU '12 Bachelors of...
View all posts by Alexander Pinkerton >

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp