Web Scraping Chewy.com

Avatar
Posted on Dec 12, 2019

Background

The pet industry is a growing market in the United States. According to 2019-2020 American Pet Products Association (APPA), pet ownership of U.S. households increased from 56% to 67% over the last 30 years. The increasing demand may be due to the growing need of companionship from pets. Today people treat their pets as beloved family members and are willing to spend money to enrich their lives 

That accounts for the increasing demand for. Quality t products in the pet retail industry.  According to APPA, there was about a $72.6 billion expenditure on pets in 2018, and 41% of the expenditure was for food.

Like many markets in the present day, e-commerce has found a role in the retail market. Chewy, an online retailer of pet food and supplies, maintained its position while competing with e-commerce giant Amazon.

Motivation

Since dogs are by far the most popular domestic pet, I decided  to analyze dog food products on Chewy’s website and explore theses two questions:

  1.       How have dog food products on Chewy.com been performing over the past few years based on customer reviews?
  2.       How can Chewy.com further improve from its current position?

 

Data

Scraping

Scrapy is a great tool within Python to extract information from websites when the data does not exist in an analyzable format.

My Scrapy code extracted 3547 dog food products. The dataset includes 297,948 rows and 9 columns of data. The 9 columns are brand name, date of review, number of reviews, percent of recommendation, sale price, product name, product review content, review rating score and review title.

 

 

 

Analysis:

 

In 2019, about 77 percent of dog food products were given 5-star reviews. Ten percent of dog food products were given 4-star reviews. The overall review distribution looks positive for the dog food products on Chewy’s website.

The following word cloud is generated from 297,948 reviews (see image below). The most frequent phrases on these reviews are “dog love” and “picky eater.” It shows that many dogs love the dog food products from Chewy, and many pet owners who left reviews on the website have dogs who are picky eaters. Chewy’s website has a great selection of dog food products that all dogs love.

In addition, “grain free” appears a lot on the reviews as well. It is evident that “grain free” is an important factor that customers value when purchasing dog food products.

 

Although review rating and review word cloud in 2019 looks promising, the average review score for dog food products actually decreased in the past 7 years.

The reviews might  improve if Chewy focuses on  brands with better reviews. The graph below shows the top 10 brands with most products on Chewy’s website. Buffalo brand has the most dog food products, and its product availability is almost twice that of the next highest brand.

After finding the top ten brands on Chewy’s website with regards to product availability, further analysis was done to understand the overall trend for the brand average review score. The graph below clearly shows that the brand average review score has been decreasing over the past four years. One interesting finding is that the review score for Purina Pro Plan does not fluctuate like the other brands. Chewy should promote brands such as Purina Pro Plan on their website because the review score of Purina Pro Plan increased in 2016 and in 2018 despite the overall decreasing trend of review ratings.

It is possible that one reason that Purina is popular among customers is due to its price advantage.

I performed analysis between brand price and average review score. My findings show there is little to no correlation.

However, Purina Pro Plan is different from other brands. The average review rating for Purina Pro Plan is similar to other brands, as indicated by the bar chart. The box plot shows that the median sale price for Purina Pro Plan products is the lowest among the 10 top products (Cesar brand should be considered as an outlier because it only provides dog food trays on the Chewy website, while other brands provide dry dog food products, which are usually more expensive and provide more food). In other words, Purina Pro Plan keeps their  price low despite the fact that they have good customer reviews.

 

             

 

 

Recommendation 1:

Chewy should introduce brands like Purina Pro Plan with cheaper price and good brand review score so its customers could be more satisfied.

 

Recommendation 2:

It can be beneficial if Chewy considered getting more products from brands with increasing review scores in the future so review scores can be improved. I found some brands with increasing review scores, such as HI-TOR and Liver Bits. Further analysis is needed for choosing what brands to add to the website.

 

 

In conclusion:

Chewy has good review ratings on dog food from customers in 2019. However, the average review score has decreased in past years. To improve customer reviews, Chewy can introduce brands with cheaper prices and good review scores on their website. It is possible that Chewy can manage their brands to give more business to brands with increasing customer review trends.

 

Future Work

  • Current analysis shows that “grain free” is an important factor when customers choose food products. It would be interesting to analyze the reviews to find other ingredients that customers value.
  • Further analyze brands with a lot of reviews and a trend of rising  scores to identify which brands it pays to keep available for sale.
  • Find the release date of the products to better understand how the brand average rating score changes with new products added to the website.
  • If sales information is available, study the relationship between sales and review score. It is possible that the review score can be helpful for forecasting sales.

 

If you are interested, you can look at the code on my GitHub:

https://github.com/hmo99/Web_Scraping_Chewy.com

 

Resources:

https://www.americanpetproducts.org/press_industrytrends.asp

https://www.retaildive.com/news/the-state-of-the-pet-retail-industry/559309/

About Author

Avatar

Huimin Ou

Huimin Ou is NYC Data Science Fellow with a Bachelor's Degree in Industrial Engineering from University of Wisconsin-Madison. Before coming to NYCDSA, Huimin is an industrial engineer with experience in business data analytics, process improvement and project management....
View all posts by Huimin Ou >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp