Scraping for Scraps| A Tool for Sensitive Diets

Posted on Jun 25, 2021

Background

In recent years, studies have shown that people are becoming more and more sensitive to a variety of common food allergens (peanuts, egg, etc.). On the other hand, ease of transportation and improvements in agriculture have made a wider variety of food available to todays consumers. This opens the way for those with restrictive diets to more easily meet their needs with a wider variety of ingredients available, the next obstacle to those with strict dietary needs, is often favorable recipes to follow. Anyone with internet has access to recipe databases, however navigating recipe websites can be a slog, often filled with blog posts, advertisements, and opinion pieces. Coming from experience, even with search bars and categories, it isn't always quick or easy to find exactly what you need on a recipe website. 

What are we asking?

When looking to pull data from a site via web-scraping, it is very important to have an idea of what kind of data you are looking for. In this particular case, I wanted to see how popular, or unpopular, certain diet restrictive recipes were, as well as how long they took to cook, and how many servings they produced. More specifically:

  • Are any specific diets over/under represented? 
    • In amount of reviews and amount of ratings
      • How reliable is the data?
  • Are any specific diets easier to prepare than others? 
    • Time spent preparing meals?
  • Are any specific recipe types less popular?
    • How highly rated are they?
  • Which categories are able to leverage more meals per recipe?
    • Serving sizes?

 

The data sample

As a proof of concept, the data selected for this project consisted of 25 randomly selected recipes, pulled from the "staff picks" section of five different subcategories in the "diet & healthy recipes" section of the website. five recipes were taken from each subcategory: Gluten Free, Mediterranean Diet, Low Carb Diet, Diabetic Diet & Keto Diet

About the Data

Gluten Free Diets are for those who are gluten sensitive, or for those who wish to remove the protein from their diet for potential health benefits. A family of protein found in rye, wheat and barley, (wheat being the most common), gluten can be easily described as the part of bread and dough that gives it its sticky, elastic qualities. Those with Celiac Disease are well known for being gluten sensitive. Plaguing around 1% of the population, Celiac Disease is an auto-immune disorder that can result in nutrient deficiencies and damage to the digestive tract, if left unattended.  It is important for those with gluten sensitivity to avoid the protein in their diet as much as possible, in order to reduce the potential short term and long term effects on their bodies. 

Low Carb Diets are for those who are looking for weight loss assistance. This type of diet focuses on limiting carbohydrate intake, mainly those found in grains, starches and fruits. A high protein/high fat focused dietary plan, this diet focuses on eliminating a lot of the empty carbohydrates one might consume, paving the way for your body to properly burn its excess fat reserves. 

Keto Diets focus on High fat and low carb intake. Used for weight loss and by athletes for toning and shaping, this diet forces your body into releasing ketones, a state known as ketosis. Ketones are a product of the breakdown of fats in the body, into fuel. By eliminating excess carbohydrates and sugars from your diet, your body will begin to quickly burn its excess fat reserves . This diet focuses on eliminating most processed grains like bread and pasta, and added, processed sugars that come from common sweets. 

Diabetic Diets are for those who poignantly suffer from diabetes or pre-diabetes. Having issues with blood sugar regulation due to insulin production issues or imbalances, people with diabetes need to manually and artificially introduce insulin into their systems, in order to release the sugar from their blood streams into the rest of their bodies.  This diet focuses on eating regularly (three times a day) in order to better utilize insulin doses, as well as stereotypically eating healthier, in order to reduce the stress of processing unhealthy food, on the body. This diet includes lots of healthy carbs like fruits, vegetables, whole grains and low fat dairy products. 

Mediterranean Diets began to rise in popularity internationally in the 1960's. Doctors, scientists and nutrition specialists began to see data that showed better cardiovascular health and less fatalities as a result of cardiovascular disease in people residing in the Mediterranean. Consisting of a combination of daily starchy vegetables, whole grains, a helping of fish, legumes and eggs, this diet promotes a healthy heart  and is recommended for those with cardiovascular issues. This diet particularly focuses in limiting red meat intake. 

 

The figures

What does this mean?

At a glance, there seems to be a trend that leans towards the Mediterranean Diet being both highly reviewed and highly rated. Low Carb Diet recipes seem to be quick to make but not very highly reviewed or rated. Diabetic recipes appear almost as popular as Mediterranean diet recipes, when it comes to both  number ratings and reviews. Keto diets overall appear to take longer to prepare, therefore if you are looking for quick meal preparation, any of the other four diets might be your best best. If you're looking for a diet with the most crowdsourced feedback, the Mediterranean Diet and Diabetic recipes appear to be the places to look. With data analysis, there does not seem to be any significant difference among the categories in terms of both how long they take to make or how popular they are. However, this is a proof of concept sample size, and more data would yield a much more accurate, reliable and conclusive result

What went right and what went wrong

This project was a rollercoaster for me personally, and I had a lot of trouble, and spent a lot of hours learning how to web-scrape as opposed to learning the Pandas data management package. In terms of learning these skill quite recently, I was successfully able to navigate a reactive environment with a scraper in Python. I was also able to pull hrefs/urls, target specified information and store data to variable to sort out junk data from valued data. In terms of roadblocks with this project, they are prevalent, and given more time I probably could have worked my way through them. There were issues with getting my scraper to run on its own (However, it seemed to work line by line with my observation). There were obvious issues with implementing learned knowledge about web-scraping, made evident during my solo attempts an mentor assisted attempts at coding the program.  There were some issues with committing stored data to a csv properly, although during the lectures these issues were not a problem. Additionally, my knowledge working with Pandas suffered as result on working on web-scraping, as learning how to scrape on the fly proved time consuming and mentally intensive. 

What is next, and where to improve...

With more time, I would like to streamline the scraping process I designed in reactive environment, and to to be able to reliably pull and store any targeted information based on user dietary preferences/restrictions. I would also like to further improve working knowledge of Pandas (very important). Being able to manipulate and organize data in Python is very important, and a lot of time was spent learning to scrape. This being said, I need to improve my working knowledge of Python commands. I struggle to move quickly when working in python, and being able to speed up problem solving abilities working with Python would reduce some of the time management issues I had with this project. 

 

 

References

www.mayoclinic.org

https://www.freepik.com/free-photo/pots-vegetables-harvest_1440232.htm#page=1&query=cooking&position=11

Perkin, Michael R., et al. "Randomized trial of introduction of allergenic foods in breast-fed infants." N Engl J Med 374 (2016): 1733-1743.

About Author

Related Articles

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp