Ames Iowa Housing Prices Model

The skills the author demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.


Our goal was to construct the most accurate possible model to predict housing prices in Ames, IA.

Background on Ames

While Ames embodies the typical college town in the Midwest, it also represents a unique combination of demographics and history. Ames has the distinction of being the only city with a population of more than 65,000 where 45% of residents are enrolled in college or graduate school. This has a distorting effect on income demographics where the median household income is $46,000 and the 28% poverty rate is nearly 3 times the national average.

The housing market prices are considerably different on the basis of median price per square foot on a neighborhood basis. The highest priced neighborhood is the Green Hill section with an average $198.65 per square foot, while the Southwest of ISU neighborhood rounds out the bottom at $89.89 per square foot.

The real estate market often utilizes price per square foot as a defining metric and “location, location, location” as a mantra. Intuitively, it would seem that basing the dependent variable on these characteristics would yield the most accurate results. In practice, the results were less accurate than using Total Living Area and Neighborhood as features and “sale price” as the dependent variable. 

As the boxplot below illustrates, the median price per square foot is around $125 per foot. Using price per square foot, variance of the target variable is reduced, leading to a reduction in R Squared.

Ames Iowa Housing Prices Model

Feature Analysis and Selection - Pt. 1

In order to prepare the dataset for analysis, we cleaned the data. Duplicate records were removed. For the qualitative, categorical variables, we imputed all “NA” values with “None.” For numerical, continuous features that had missing values, we substituted the value of zero with the exception of the year the garage was built. In that case, we left the value as N/A since we did not use the feature.

Once we investigated and cleaned the data, we began the process of determining what variables would be included in the final model. The first thing we looked into was how we wanted to view the dependent variable (sale price). We quickly realized that the best way to view was using the log of the sale price as opposed to the raw number. The reasoning for this is that the log of the sale price provided a more normal distribution of values.

Ames Iowa Housing Prices Model

Feature Analysis and Selection - Pt. 2

Next, we looked into variable creation for any variables that would be helpful to add to the data set. We wound up creating a handful of variables to help model the data.



Code Name


Total Living Area


GrLivArea + TotalBsmttSF - BsmtUnSF

Unused Lot Size


LotArea - 1stFlrSF

Has Pool


PoolArea > 0

Has Basement


BsmtQual != None



HalfBath + FullBath + BsmtHalfBath + BsmtFullBath



FullBath + BsmtFullBath

House Age


max(YearBuilt) - YearBuilt + 1

House Age Squared


HouseAge ^ 2

Up/Down Ratio


2ndFlrSF / 1stFlrSF


Variables such as Total Living Area and Unused Lot Size were created by aggregating existing variables in the dataset because we felt the aggregated values would paint a better picture than the individual values. Has Pool and Has Basement were created because there was value in whether or not the item existed, though the actual size wasn’t particularly relevant.

Feature Analysis and Selection - Pt. 3

Once the necessary variables were added to the data set, we looked into the correlation across metrics to try and identify possible multicollinearity before beginning to model.

Ames Iowa Housing Prices Model

We also looked into how the individual metrics correlated with sale price.

Ames Iowa Housing Prices Model

Ames Iowa Housing Prices Model

Once we completed this process, we selected our variables for the initial modeling

Ames Iowa Housing Prices Model

Modeling the Prices of Houses in Ames

After selecting the overall features, they were separated into continuous, numeric and discrete, categorical variables. The categorical variables were “dummified” into a series of columns in which each column stood for a different value of the category. For instance, each neighborhood was represented by a column. All except one column would have zeros as values; the one column that represented the value of the feature would have a one as the value. This approach is known as One Hot Encoding.


The next step was to divide the features and targets into a training set (80% of the data) and a test set (the remaining 20%). The training set was used to fit the model while the test set was put aside to ensure that we did not overfit. 


Once the data was prepared, the following models were tested:


  1. Linear Regression
  2. Random Forest
  3. XGBoost
  4. LightGBM
  5. CatBoost
  6. Support Vector Machine

Linear Regression

In order to decrease multicollinearity and arrive at the most pertinent variables, Lasso regression was applied to the data set after scaling the variables. A grid search was performed on a variety of alpha penalization rates until we achieved a maximized test score where alpha equalled 4.6 x 10-5

This resulted in the following coefficients:

Since the regression model predicted the log of the sale price, we can interpret the coefficients to mean that  an increase in one unit of a given variable increases the log of the sale price by that coefficient (as long as all other variables are kept constant). For example, an increase of an additional square foot of Total Living Area would increase the log price by 2/100th of a cent. 

Boosting Models for Ames House Price Data

Boosting models are high-performing tree-based models. These models combine decision tree algorithms with boosting methods that train weak learners sequentially rather than in parallel like a Random Forest. Each subsequent tree in the boosting model attempts to correct the error of its predecessors.


Three variants of the boosting algorithm were tried:

  1. CatBoost
  2. LightGBM
  3. XGBoost


Each took a significant period of time to cross-validate the hyperparameters, so training was slow in general. What these models lacked in training speed, they more than made up for it in accuracy.


Depending on the model, different hyperparameters were tuned accordingly. Common among all of these were learning rate and maximum depth. CatBoost also offered a hyperparameter for L2 leaf regression. From our cross-validation, we learned that our best CatBoost model came from having a maximum depth of 7, an L2 Leaf Regularization parameter of 1, and a learning rate of 0.03.

Cross-Model Comparison

The following table details the best version of each model. From the table, we see that the boosting models tend to overfit, but the testing scores still outperform the other models.


Training R2

Testing R2














Random Forest




Support Vector Machine




Linear Regression






Among the estimated models, the best performing model is the CatBoost model. It explains around 92% of the variance in the dependent variable log sales price (Root mean squared error: 0.1135). As mentioned previously the model was estimated on a train data set and finally applied on a test data set. It is therefore an out of sample estimation.

Based on the variable importance plot the most important feature is the overall quality followed by the total living area. These two features capture 30 percent and 27 percent of the explained variance.


Variable Importance Plot



Overall Quality (OverallQual)


Total Living Area (TotalLivingArea)


House Age Squared (HouseAgeSq)


Garage Area (GarageArea)


Neighborhood (Neighborhood)


Overall Condition (OverallCond)


Unused Lot Size (UnusedLotSize)


# of Toilets (Toilets)


Basement Quality (BsmtQual)


Sale Condition (SaleCondition)


Building Type (BidgType)


Lot Frontage (LotFrontage)


Upstairs/Downstairs Ratio (UpDownRatio)



A downside of the model is that we cannot directly interpret the effects of the explanatory variables on the log sales price. This is a common disadvantage of this type of blackbox model where a direct interpretation is not possible. We can, therefore, not make a claim about how much the price will increase when the overall quality is improved or when there is an additional bathroom. Despite this disadvantage, the model still provides a good guideline for what an investor should consider when buying a house. An investor should definitely look at the mentioned features such as the overall quality of a house or the age.  


A nice advantage of our analysis is that we can make a direct prediction of a house's price as long as we have the considered explanatory variables from our model. Let’s assume we have a house with Overall Quality of 6, a total living area 330 square feet, a house age of 10 years; our model predicts a sale price of $420,000 USD. This can be compared to the offered price of a house with the same features for example on a web page such as Zillow. The comparison of our estimation and the offered price gives an investor a clear guideline if he should invest. An investor should invest if the offered price is lower in comparison to our estimated price.


About Authors

Ethan Zien

Data Analyst with a background in Social Media Advertising and a strong interest in sports analytics
View all posts by Ethan Zien >

David Wasserman

Data science professional with a background in finance and credit risk. Has the rare combination of technical skills and business acumen.
View all posts by David Wasserman >

Liam McDermott

Data science fellow with background in institutional trading.
View all posts by Liam McDermott >

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI