Data Analysis on Running Shoes

Posted on Oct 19, 2019
The skills the author demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.


In the shoe market there are many categories; athletic, running, casual, boots, hiking, etc. One thing is certain, people love shoes. Additionally, I find people are most critical of shoes that require performance more than just style. For that reason I wanted to use data to find out, what makes a great men's running shoe?

Zappos has a plethora of running shoes and many different ratings systems. They offer written customer reviews, overall shoe rating, total number of customer reviews, favorite likes, true-to-size feeling, true-to-width feeling and arch support. With all of these different ratings I was curious to find out which were the top brands and models. What do customers have to say about the highest rated shoes? What makes a running shoe stand out from the rest? Is it style, traction, light weight, price, fitment, or just plain comfort? Before we get to the results, let’s take a look a my method for scraping.



For scraping Zappos I used the automation testing framework Selenium. I was able to scrape the entire men's running shoe inventory, totaling over 500 models. The data was cleaned then loaded into a python environment to gain useful insights. Figure 1(a) shoes a web page sample. The info gathered here is brand, model, price, overall rating, and favorite likes. Figure 1(b) shows the details page where the info for total number of reviews, true-to-size, true-to-feel, and arch support were collected. Additionally customer reviews were collected for the top 4 models was collected. 


Data Analysis on Running ShoesFigure 1(a): Web page sample. Figure 1(b): Model detail page sample



Data Results:Data Analysis on Running Shoes

Figure 2: Overall ratings density plot for the top 5 brands


Above in figure 2 is a density plot of the overall ratings for the top 5 brands. This was determined by the summing the overall ratings for each model of a given brand. However this did not factor in the number of reviews per model. That will be addressed shortly.

Immediately we see Asics stands out with a large peak in the 4/5 category however, falls short in the 5/5 star category.  Weighing in both 4 and 5 star categories all 5 brands are close, but we can see the winner is Nike.

I wanted to find out how these top 5 brands ranked with other brands in price, number of reviews, and likes. Below we can see the results on horizontal bar graphs, in figures 3(a), 3(b), and 3(c).



Data Analysis on Running Shoes

 Figure 3(a): Horizontal Bar chart of price


We see from figure 3(a) that all top 5 brands price point fall in the middle to lower range. We can conclude that the average runner cares about quality but only within a certain price range.



 Figure 3(b): Horizontal Bar chart of number of reviews


We see from figure 3(b) that all top 5 brands rank in the top for number of customer reviews. This makes sense as we imagine people have the most to say about something they really like. And this same logic goes for things that people really dislike too.


Figure 4(a): Scatter plot of likes vs price. Figure 4(b): Scatter plot of likes vs price


We notice the highest number of likes, in the 1000+ range, are the in the price range of  50$ to 125$. Figure 4(b) is a scatter plot of overall rating vs likes. We see this plot verifies our intuition that likes more are associated with higher ratings.

It is worth mentioning there was a small surprise. That the most liked shoe has a 4/5 star rating. Upon looking into this matter, I found that this was a new model shoe that fell short of its beloved predecessor.



Top 7 models:

Figure 5: Top 5 running shoe models


Figure 5  is a bar chart of the top 5 running shoe models and displays the 4 categories of ranking; 5/5 star rating, true-to-size, true-to-width, and arch support. These results are a weighted average of the number of customer reviews. We can see there are really only 2 top running models. Top ranked is Brooks Ghost 11, and Second is the New Balance M990V4.

Word Clouds 

I wanted to gain insight into what customers were saying about these two top ranked shoes along with the next two following, Salomon XA Pro 3D and the Hoka One One Bondi 6. I did this by generating word clouds for each model from all of the associated customer reviews. Also, I made sure to filter out general words, and additionally for each model I filtered out brand name and model. Figures 6(a,b,c,d) show the results respectively.


Figure 6: Top 4 models word clouds


From the word clouds we see a few words stand out. Comfortable, fit and great. This also agrees with what qualities we imagine a performance shoe should have. That is it should be comfortable and fit perfectly. However I was surprised to find that no words associated with style or aesthetic bubbled to the top. 



From these results it's easy to verify that our general intuition about what qualities we seek in a running shoe are true. We seek comfort and fitment most. Although, it was surprisingly to find that style doesn't play a bigger role it the top models.  

Further these results agree with our intuition in that higher rated shoes are associated with more likes and number of customer reviews.


Future Work:

There is a whole other side to explore, women's running shoes. I’m curious how these results would compare to men’s running shoes. Would they simply be the same or would other factors drive a specific shoe to the top?Additionally I’d like to explore the subcategories of running shoes; trail running and regular running shoes. Also, how does the color pallet weigh in on customer’s preferences? And lastly I'd like to analyze bad reviews. What customer's don't like  



Thanks for reading!



About Author

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI