Machine Learning Informed Home Improvements

Avatar
Avatar
Avatar
, and
Posted on Jul 1, 2020

Introduction

When it comes time to sell a home, everyone hopes to sell for as high as possible. Many features of a home contribute to its overall value, and it’s not always clear which carry the most weight. While certain factors are out of the control of owners, such as neighborhood, there are improvements and renovations that can potentially increase their home’s value like basement refinishing, kitchen upgrade, etc. The question is: which features deliver the most return on investment in terms of resale value? Our project sets out to answer this.

Through Kaggle.com, we acquired data on the home sale prices of homes in Ames, Iowa, including over 80 features specific to each house. Through the use of machine learning algorithms paired with EDA and feature engineering, we were able to create two models: one to identify features with the most impact on value and one model to predict the value of home improvements based on the previous model’s selected features.

Through our analysis, we found that the most important features are the overall quality, number of fireplaces and full bathrooms, quality of the kitchen, and number of basement full bathrooms. Our final prediction model had an R-squared value of 0.805.

EDA

Upon first glance at the dataset, there was an overwhelming amount of NA’s within many of the categorical features. With a closer look at the documentation, we see that each NA describes the lack of a feature. Thus to improve interpretability of the data, we have imputed string values for those homes with NA under a feature.

Feature Selection

With 36 categorical variables, we noticed that not all features within a category were significant enough to include in our model. We also realized that dummifying all features would introduce a large amount of sparsity and increase the total computing power needed. To avoid this, we analyzed the mean, medians, and distributions of each categorical feature through the use of box plots. Those with IQRs and medians occupying a similar space would be considered for grouping into one feature. However, box plots do not visualize the count of homes with a certain categorical. We categorized the home prices into bins to create bar charts to visualize this. While a certain feature might occupy a significantly different distribution space and median, the bar chart will show us if the count is large enough for the feature to stand alone or if it should be grouped with others.

With over 80 features of a home documented, there was bound to be multicollinearity among its features. To combat this issue we regressed each numerical category against those remaining. Regressions with a large R-squared indicate a heavy presence of collinearity, and that was used as the basis to drop or retain certain numerical values.

For our final step of feature selection, we ran a lasso regression on the remaining features. We used a grid search with a C value iterating from 1 to 1000. Lasso regression helped to assess which features were still multicollinear and which features held the most predictive value. As the C value on the regression increases, the coefficients of the less significant features converge to 0. We continued to examine our lasso regressions until we found a C value wherein only 4 upgradable features remained.

Modeling

While there are many models to choose from, our main focus was the dollar amount interpretability of a model’s output. Certain models actually performed better in terms of R-squared however, their outputs could only be interpreted in terms of relative importance. As a result, we decided on using a multiple linear regression (MLR) model to predict home sale values. With a MLR model we were able to ask and answer the following question: “While holding all other variables constant, how much does a unit increase in a given feature increase the value of a home?”

For our categorical features that required dummy variables (kitchen quality and basement finish type), we dropped the low-end features. Thus, these low-end features represented the baseline for our model.

Conclusions

From our coefficients, we see that kitchen quality has the potential to add the most value to a home, followed by number of full bathrooms, number of basement full baths, and basement finishing type.

In order for the renovation to be worth it, the value added must exceed the renovation costs. We estimated these costs using average prices quoted specifically for Iowa. For quality variables, such as the kitchen and basement finish type, the cost was estimated based on how much renovation would be needed to move from low-end to mid quality and the mid to high quality.

From our graph you can see that an excellent kitchen offers the most return on investment followed by upgrading to a good kitchen. While the net estimated return for a full bath might be positive based on our results, the narrow margin it offers warrants more investigation for its impact on the specific home in question. Unfortunately, a renovated basement is not worth the costs due to the high cost of renovation.

Our group demonstrated that Machine Learning can be used for practical and non-academic purposes. We were able to accurately predict home sale prices using advanced modeling techniques (Random Forest, XG Boost, etc.), but eventually opted to use a multiple linear regression model that had moderate predictive value but high interpretive value. We hope that these results will benefit future homeowners.  Moreover, we hope that machine learning and modeling may always be tied to purpose and business interpretability. 

Contributors

Pengyun Liu - www.linkedin.com/in/pengyun-liu

Peter Liu - www.linkedin.com/in/peter-liu100

Michael Link - www.linkedin.com/in/data-science-link

Featured Image

House by Oatsy40

About Authors

Avatar

Michael Link

Michael’s background is in ecological and water resources engineering. He began his career as a water resources engineer in a Fortune 500 consulting firm. Having worked to industry-standard analytical techniques and software, he discovered that he had a...
View all posts by Michael Link >
Avatar

Peter Liu

Peter Liu has more than 14 years of experience in corporate credit risk management and held various financial analytics positions. He has an MBA and a BA in mathematics.
View all posts by Peter Liu >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp