Machine Learning Informed Home Improvements
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
Introduction
When it comes time to sell a home, everyone hopes to sell for as high as possible. Many features of a home contribute to its overall value, and it’s not always clear which carry the most weight. While certain factors are out of the control of owners, such as neighborhood, there are improvements and renovations that can potentially increase their home’s value. Things like basement refinishing, kitchen upgrade, etc. The question is: which features deliver the most return on investment in terms of resale value? Our project with machine learning sets out to answer this.
Through Kaggle.com, we acquired data on the home sale prices of homes. Located in Ames, Iowa, including over 80 features specific to each house. Through the use of machine learning algorithms paired with EDA and feature engineering, we were able to create two models. Identify features with the most impact on value and to predict the value of home improvements.
Through our analysis, we found that the most important features are the overall quality, number of fireplaces and full bathrooms, quality of the kitchen, and number of basement full bathrooms. Our final prediction model had an R-squared value of 0.805.
Machine Learning EDA
Upon first glance at the dataset, there was an overwhelming amount of NA’s within many of the categorical features. With a closer look at the documentation, we see that each NA describes the lack of a feature. Thus to improve interpretability of the data, we have imputed string values for those homes with NA under a feature.
Feature Selection for Machine Learning
With 36 categorical variables, we noticed that not all features within a category were significant enough to include in our model. We also realized that dummifying all features would introduce a large amount of sparsity and increase the total computing power needed. To avoid this, we analyzed the mean, medians, and distributions of each categorical feature through the use of box plots. Those with IQRs and medians occupying a similar space would be considered for grouping into one feature. However, box plots do not visualize the count of homes with a certain categorical. We categorized the home prices into bins to create bar charts to visualize this. While a certain feature might occupy a significantly different distribution space and median, the bar chart will show us if the count is large enough for the feature to stand alone or if it should be grouped with others.
With over 80 features of a home documented, there was bound to be multicollinearity among its features. To combat this issue we regressed each numerical category against those remaining. Regressions with a large R-squared indicate a heavy presence of collinearity, and that was used as the basis to drop or retain certain numerical values.
For our final step of feature selection, we ran a lasso regression on the remaining features. We used a grid search with a C value iterating from 1 to 1000. Lasso regression helped to assess which features were still multicollinear and which features held the most predictive value. As the C value on the regression increases, the coefficients of the less significant features converge to 0. We continued to examine our lasso regressions until we found a C value wherein only 4 upgradable features remained.
Modeling for Machine Learning
While there are many models to choose from, our main focus was the dollar amount interpretability of a model’s output. Certain models actually performed better in terms of R-squared however, their outputs could only be interpreted in terms of relative importance. As a result, we decided on using a multiple linear regression (MLR) model to predict home sale values. With a MLR model we were able to ask and answer the following question: “While holding all other variables constant, how much does a unit increase in a given feature increase the value of a home?”
For our categorical features that required dummy variables (kitchen quality and basement finish type), we dropped the low-end features. Thus, these low-end features represented the baseline for our model.
Conclusions
From our coefficients, we see that kitchen quality has the potential to add the most value to a home, followed by number of full bathrooms, number of basement full baths, and basement finishing type.
In order for the renovation to be worth it, the value added must exceed the renovation costs. We estimated these costs using average prices quoted specifically for Iowa. For quality variables, such as the kitchen and basement finish type, the cost was estimated based on how much renovation would be needed to move from low-end to mid quality and the mid to high quality.
From our graph you can see that an excellent kitchen offers the most return on investment followed by upgrading to a good kitchen. While the net estimated return for a full bath might be positive based on our results, the narrow margin it offers warrants more investigation for its impact on the specific home in question. Unfortunately, a renovated basement is not worth the costs due to the high cost of renovation.
Our group demonstrated that Machine Learning can be used for practical and non-academic purposes. We were able to accurately predict home sale prices using advanced modeling techniques (Random Forest, XG Boost, etc.), but eventually opted to use a multiple linear regression model that had moderate predictive value but high interpretive value. We hope that these results will benefit future homeowners. Moreover, we hope that machine learning and modeling may always be tied to purpose and business interpretability.
Contributors
Pengyun Liu - www.linkedin.com/in/pengyun-liu
Peter Liu - www.linkedin.com/in/peter-liu100
Michael Link - www.linkedin.com/in/data-science-link
Featured Image
About Authors
Michael Link
View all posts by Michael Link >
Pengyun Liu
View all posts by Pengyun Liu >