Using Data From COVID-19 and Census to Predict ZORI
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
Introduction
The COVID-19 pandemic drastically shifted consumer preferences in housing. This created a scenario that made housing prices data difficult to model, with price variability changing constantly as the pandemic continues.
Objective
To observe these changes and how they will affect the future of the market, we created models that can accurately forecast and predict the Zillow Observed Rent Index (ZORI) by using historical ZORI, US Census, and Covid-19 data. Predictions can be used to see which areas are recovering from Covid-19 more quickly. This kind of information can be useful to:
- Government Organizations: find areas in need of resources
- Investors: find which neighborhoods to start businesses in and which to stay away from
- Business Owners: gain insight into your own neighborhood to decide when to increase or decrease inventory
Datasets
Zillow Observed Rent Index
- Monthly rent prices of the center 20% (40th-60th percentile) of the entire observed market, Smoothed & Seasonally adjusted. (Jan. 2014 - June 2021)
- 1743 rows by 93 cols.
US Census Data
- Yearly data collected and/or predicted by the US Census Bureau pulled from BigQuery (2014-2018)
- Housing, Income and Demographic datasets pulled from the Census Website (2019)
- 33,120 rows by 232 cols, 6 years of data
COVID-19 Data from Johns Hopkins
Data collected by the Johns Hopkins Center for Systems Science and Engineering (JHCSSE)
- Daily counts of US COVID-19 cases and deaths by US county (Jan 22 2020 - July 17 2021)
- 3342 rows by 555 cols
Data Cleaning and Feature Engineering
ZORI Data:
Missing Value Interpolation
- Interpolated linearly from Jan. 2014 (start) to June 2021 (end)
- Any ZORIs missing at the time period’s start or end were calculated using that year’s mean change in ZORI
Outliers
- Each ZIP code was checked for any ZORI that was >3 stddevs away from that year’s mean rent price
- No outliers were found, and no ZIP codes were removed
Census Data:
Raw Data
- 2014-2018 data comes from 5 BigQuery Census Datasets
- 165,600 observations (33,120 ZIP codes x 5 years of data)
- 232 Features
Missing Data
- Removed 4 Features that had 5,000+ null values and little to no correlation with the target feature.
Combining Census Years: 2014-2018 and 2019
- Could only pull 47 matching features between the datasets
- 2019 census data is duplicated for 2020, will be replaced with actual data in 9/2021
Feature Selection
- Reduced to 36 Features using a Forward Stepwise function
Ended with 13,920 observations (1740 zip codes*8 Years) and 36 Features
Covid-19 Data:
Cleaning
The JHCSSE provides a daily count of global COVID-19 cases and deaths from 01/22/2020 to present day
- Removed all observations in non-US counties
- Capped data to be from 01/22/2020 to 06/30/2021
- Cumulative numbers of cases and deaths were the only metrics kept alongside location & time attributes
The count of cases and deaths on the last day of each month was used as that entire month’s case and death count.
Feature Engineering
After cleaning & filtering the data, we engineered features for ‘New Cases’ and ‘New Deaths’, which were defined as the current month’s cases & deaths subtracted from the previous month’s. We then had to convert county-level COVID cases and death metrics to the corresponding ZIP codes within each county. To do that, we assumed that the COVID cases and deaths were proportional to each ZIP Code’s population within the county, using ZIP Code and county information from the Census data.
This ‘New Case Rate’ feature would be the exogenous variable we will use later in our SARIMAX model, which is a seasonal time-series model with exogenous inputs.
Combining Datasets:
To combine all three datasets, first we joined COVID-19 data with the Census data on the county & state, then COVID-19 case/deaths were split to the ZIP Code level based on the (population of ZIP code) / (total county population), and finally, this data was combined with the ZORI dataset. After matching all datasets together, 103 ZIP codes were not included in the SARIMA and SARIMAX models due to the lack of available corresponding Census/COVID data
Models and Methods
The following models were implemented to achieve our objective:
- Multiple Linear Regression (MLR) to predict the yearly and semiannual ZORI in 2021
- Vector AutoRegression (VAR) to forecast the ZORI from 2020 to mid-2021
- Seasonal Autoregressive Integrated Moving Average (SARIMA) to forecast ZORI in the first half of 202
- Seasonal AutoRegressive Integrated Moving Average with eXogenous regressors (SARIMAX), using COVID-19 data as the exogenous variables, to forecast ZORI in the first half of 2021
|
Model |
Features Used |
Target |
| MLR |
ZORI (yearly, semiannually), Census 2014-20 |
ZORI (semiannually) from 01/2021 to 06/2021 |
| VAR |
ZORI (monthly) from 01/2014 to 12/2019 |
ZORI (monthly) from 01/2020 to 06/2021 |
|
SARIMA |
ZORI (monthly) from 01/2014 to 12/2020 |
ZORI (monthly) from 01/2021 to 06/2021 |
|
SARIMAX |
ZORI (monthly) from 01/2014 to 12/2020; COVID-19 (monthly) from 1/2020-12/2020 |
ZORI (monthly) from 01/2021 to 06/2021 |
MLR Model
Using the available data, we created models to predict:
- The current rent price
- The rent price in 1 year while including the current rent price as a feature
- The rent price in first 6 months of 2021 with the last 6 months of 2020 as the test set, using a biannual aggregation; training on the full data for the full time period, and data for only the pandemic time period
MLR Results
The first 2 models were used to gauge how well MLR could predict the ZORI. This give us a basis to compare the pandemic models to.
We, then, included the current ZORI and forecasted for the ZORI in one year. This gave us a much worse score, as expected. The model was trained on a time period without Covid-19 to predict on a period with Covid-19.
It would obviously result in a bad model.
| Model Attributes (Yearly) | Test Scores |
| Current Rent Price w/ PCA - 2019 (Non-Pandemic) | 0.7304 |
| Rent Price in 1 Year w/ PCA - 2020 (Includes Pandemic) | 0.6602 |
For the next 2 models, the data was redone to aggregate and interpolate semiannually. This was done to utilize the ZORI from 1/2021 - 6/2021, as the pandemic data was limited to the past 18 months.
When training on 2014-2020 data, we got a decent score, but it improved much more when only training on 2019-2020 data
Non-pandemic data seemed to skew the data away from the actual values.
| Model Attributes (biannually) | Test Scores |
| Rent Price in 6 months w/ PCA - 2020 (Pandemic) | 0.9593 (train on 1/2014-6/2020) |
| Rent Price in 6 months w/ PCA - 2020 (Pandemic) | 0.9966 (train on 1/2019-6/2020) |
VAR Model
- VAR Model was run on all 1743 ZORI ZIP codes as a ‘baseline’ time series forecast
- VAR Model proved to not be a great predictor based on RMSE scores
- However, it was a better model overall than ARIMA, ARMA, and VARMAX
VAR Results
Looking at the best RMSE’s and 5 worst RMSE’s gave us a list of ZIP codes to highlight in later models-- can SARIMA and SARIMAX yield accurate predictions were VAR cannot?
SARIMA Parameter Tuning
Grid search was performed to determine the optimal order, seasonal order, and trend parameters for SARIMA models on a monthly mean of ZORI inputs. The optimal parameters yielded an AIC of 8.0:
- Order (p, d, q) = (0, 0, 1)
- Seasonal (P, D, Q, seasonality) = (0, 1, 0, 12)
- Trend = ‘t’
- (linear parameter controlling the deterministic trend polynomial)
The same parameters were used when modeling with SARIMAX.
SARIMA Results
We created seasonal 6 month forecasts for each of the 1640 available ZIP Codes, at first with no exogenous COVID features, for our first model. A summary table is below, with the Root Mean Square Errors the metric we were most interested in minimizing.
From our initial VAR model we were able to get a list of five ZIP Codes that VAR performed well and was able to predict rent indices accurately, and five other ZIP Codes that had very high root mean squared errors, and we wanted to get a sense of how predictions looked from the SARIMA model. The SARIMA model results of those ZIP Codes are below.
SARIMAX Results
Next we created models for each ZIP Code with exogenous factor data from the COVID & Census data--we used 'Monthly New Cases' as a percent to total ZIP Code population as the outside variable, with the target variable still being the rent index. We could then compare this model with the previous SARIMA model, because inputs and target variables were the same for the two seasonal models. Summaries of the full SARIMAX model are below.
Just as with the SARIMA model, we further looked into the SARIMAX forecasts for previous models' good and bad predictions--that table and graph are below (we are still mainly interested in minimizing the RMSE for our forecasts).
Results Summary
|
Model |
Target |
Results |
|
MLR |
ZORI (semiannually) in 2021 |
R^2: 0.9966 test on 7/2020 - 12/2020 |
|
VAR |
ZORI (monthly) from 01/2020 to 06/2021 |
RMSE Mean (all predictions, 18 months): $102.83 |
|
SARIMA* |
ZORI (monthly) from 01/2021 to 06/2021 |
RMSE Mean (all predictions, 6 months): $66.66 |
|
SARIMAX* |
ZORI (monthly) from 01/2021 to 06/2021 |
RMSE Mean (all predictions, 6 months): $69.56 |
Comparing Models
Overall, MLR seems promising for predicting the ZORI during the pandemic at the yearly and semiannual level, only when limited to time periods where Covid-19 was a factor. The trends in 2014 to 2018 influence the model too much when included, worsening the score.
SARIMA proved to be our best time-series model, yielding a lower overall RMSE for 55% of ZIP Codes compared to SARIMAX (45% with lower RMSE)
Both of these models had a lower RMSE than VAR, but that may be because they were both trained on 2020 data (VAR was not).
SARIMAX versus SARIMA
One major downside of SARIMAX is that it requires COVID-19 and Census exogenous actual inputs in order to forecast.
Future Census and COVID data can be forecasted separately and then passed as the exogenous variable. However, the accuracy lost to forecasting on top of forecasts would likely increase the RMSE of SARIMAX models, as compared to SARIMA.
The mean of SARIMAX’s RMSE predictions was slightly higher than that of SARIMA, but it forecasted with a lower RMSE for almost half (45%) of the data.
Further work needs to be done to determine qualitative aspects of the ZIP codes whose better predictor was SARIMAX versus SARIMA.
Comparing SARIMA and SARIMAX RMSE
SARIMAX yielded an improved (lower) RMSE for 741 ZIP codes (45%, out of 1640):
rmse_diff = x_rmse - s_rmse (Where x_rmse is SARIMAX RMSE and s_rmse is SARIMA RMSE)
Conclusion
SARIMA and SARIMAX
Models were created to accurately predict ZORI by ZIP code. Each modeled provided a different insight into where ZORI are increasing and decreasing the most.
SARIMA and SARIMAX models each performed better on about half of the dataset each. We would follow-up to determine what about those ZIP codes led to better predictions with one models vs the other.
Including COVID-19 as exogenous was not as useful as we had hoped. The pandemic influenced many markets; pinpointing the best data to use for exogenous is required. Furthermore, it is reasonable to infer that as populations moved out of highly-populated ZIP codes, they further tipped rent prices more favorably for the less-populated ZIP codes.
Multiple Linear Regression
We used our final model on features from the 1/2021 - 6/2021 to predict the ZORI in the 7/2021-12/2021. We calculated the percent change between the 2 to create the chart you see on the right.
It shows the largest increases and decreases in rent. From this, you can find:
- areas for investment
- areas that need more government assistance
Future Work
SARIMAX showed an improved RMSE for many ZIP codes, leading us to believe that improving our exogenous variables we can improve our forecasts:
- Find recent/accurate population data (monthly, by ZIP would be optimal)
- Or use 2020 census data when it becomes available
- Find accurate COVID-19 data by ZIP code
- Find other datasets that can be used for exogenous variables
- Dominant political party during the pandemic
- Rates of COVID-19 recovery and/or vaccination
- Conditionally forecast 2021, 2022 ZORI based on whether COVID-19 diminishes or follows current ZIP code/county trends
About Authors
Nixon Lim
View all posts by Nixon Lim >
Aleksey Klimchenko
View all posts by Aleksey Klimchenko >