Machine Learning Informed Home Improvements

, and
Posted on Jul 1, 2020
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

Introduction

When it comes time to sell a home, everyone hopes to sell for as high as possible. Many features of a home contribute to its overall value, and it’s not always clear which carry the most weight. While certain factors are out of the control of owners, such as neighborhood, there are improvements and renovations that can potentially increase their home’s value. Things like basement refinishing, kitchen upgrade, etc. The question is: which features deliver the most return on investment in terms of resale value? Our project with machine learning sets out to answer this.

Through Kaggle.com, we acquired data on the home sale prices of homes. Located in Ames, Iowa, including over 80 features specific to each house. Through the use of machine learning algorithms paired with EDA and feature engineering, we were able to create two models. Identify features with the most impact on value and to predict the value of home improvements.

Through our analysis, we found that the most important features are the overall quality, number of fireplaces and full bathrooms, quality of the kitchen, and number of basement full bathrooms. Our final prediction model had an R-squared value of 0.805.

Machine Learning EDA

Upon first glance at the dataset, there was an overwhelming amount of NA’s within many of the categorical features. With a closer look at the documentation, we see that each NA describes the lack of a feature. Thus to improve interpretability of the data, we have imputed string values for those homes with NA under a feature.

Feature Selection for Machine Learning

With 36 categorical variables, we noticed that not all features within a category were significant enough to include in our model. We also realized that dummifying all features would introduce a large amount of sparsity and increase the total computing power needed. To avoid this, we analyzed the mean, medians, and distributions of each categorical feature through the use of box plots. Those with IQRs and medians occupying a similar space would be considered for grouping into one feature. However, box plots do not visualize the count of homes with a certain categorical. We categorized the home prices into bins to create bar charts to visualize this. While a certain feature might occupy a significantly different distribution space and median, the bar chart will show us if the count is large enough for the feature to stand alone or if it should be grouped with others.

With over 80 features of a home documented, there was bound to be multicollinearity among its features. To combat this issue we regressed each numerical category against those remaining. Regressions with a large R-squared indicate a heavy presence of collinearity, and that was used as the basis to drop or retain certain numerical values.

For our final step of feature selection, we ran a lasso regression on the remaining features. We used a grid search with a C value iterating from 1 to 1000. Lasso regression helped to assess which features were still multicollinear and which features held the most predictive value. As the C value on the regression increases, the coefficients of the less significant features converge to 0. We continued to examine our lasso regressions until we found a C value wherein only 4 upgradable features remained.

Modeling for Machine Learning

While there are many models to choose from, our main focus was the dollar amount interpretability of a model’s output. Certain models actually performed better in terms of R-squared however, their outputs could only be interpreted in terms of relative importance. As a result, we decided on using a multiple linear regression (MLR) model to predict home sale values. With a MLR model we were able to ask and answer the following question: “While holding all other variables constant, how much does a unit increase in a given feature increase the value of a home?”

For our categorical features that required dummy variables (kitchen quality and basement finish type), we dropped the low-end features. Thus, these low-end features represented the baseline for our model.

Conclusions

From our coefficients, we see that kitchen quality has the potential to add the most value to a home, followed by number of full bathrooms, number of basement full baths, and basement finishing type.

machine learning

In order for the renovation to be worth it, the value added must exceed the renovation costs. We estimated these costs using average prices quoted specifically for Iowa. For quality variables, such as the kitchen and basement finish type, the cost was estimated based on how much renovation would be needed to move from low-end to mid quality and the mid to high quality.

From our graph you can see that an excellent kitchen offers the most return on investment followed by upgrading to a good kitchen. While the net estimated return for a full bath might be positive based on our results, the narrow margin it offers warrants more investigation for its impact on the specific home in question. Unfortunately, a renovated basement is not worth the costs due to the high cost of renovation.

Our group demonstrated that Machine Learning can be used for practical and non-academic purposes. We were able to accurately predict home sale prices using advanced modeling techniques (Random Forest, XG Boost, etc.), but eventually opted to use a multiple linear regression model that had moderate predictive value but high interpretive value. We hope that these results will benefit future homeowners.  Moreover, we hope that machine learning and modeling may always be tied to purpose and business interpretability. 

Contributors

Pengyun Liu - www.linkedin.com/in/pengyun-liu

Peter Liu - www.linkedin.com/in/peter-liu100

Michael Link - www.linkedin.com/in/data-science-link

Featured Image

House by Oatsy40

About Authors

Michael Link

Michael’s background is in ecological and water resources engineering. He began his career as a water resources engineer in a Fortune 500 consulting firm. Having worked to industry-standard analytical techniques and software, he discovered that he had a...
View all posts by Michael Link >

Peter Liu

Peter Liu has more than 14 years of experience in corporate credit risk management and held various financial analytics positions. He has an MBA and a BA in mathematics.
View all posts by Peter Liu >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI