Scraping Recipes Using Data Science

Posted on Jun 25, 2021


By this article, we use data science to explore the preparation, cooking time among 5 different cuisine types using scraped data I prepared.

In recent years, studies have shown that people are becoming more and more sensitive to a variety of common food allergens (peanuts, egg, etc.). On the other hand, ease of transportation and improvements in agriculture have made a wider variety of food available to todays consumers.
This opens the way for those with restrictive diets to more easily meet their needs with a wider variety of ingredients available, the next obstacle to those with strict dietary needs, is often favorable recipes to follow. Anyone with internet has access to recipe databases, however navigating recipe websites can be a slog, often filled with blog posts, advertisements, and opinion pieces.

Coming from experience, even with search bars and categories, it isn't always quick or easy to find exactly what you need on a recipe website. 

What are we asking?

When looking to pull data from a site via web-scraping, it is very important to have an idea of what kind of data you are looking for. In this particular case, I wanted to see how popular, or unpopular, certain diet restrictive recipes were, as well as how long they took to cook, and how many servings they produced. More specifically:

  • Are any specific diets over/under represented? 
    • In amount of reviews and amount of rating
      • How reliable is the data?
  • Which categories are able to leverage more meals per recipe?
    • Serving sizes?
  • Are any specific diets easier to prepare than others? 
    • Time spent preparing meals?
  • Are any specific recipe types less popular?
    • How highly rated are they?
  • Which categories are able to leverage more meals per recipe?
    • Serving sizes?


The data sample

As a proof of concept, the data selected for this project consisted of 25 randomly selected recipes, pulled from the "staff picks" section of five different subcategories in the "diet & healthy recipes" section of the website. five recipes were taken from each subcategory: Gluten Free, Mediterranean Diet, Low Carb Diet, Diabetic Diet & Keto Diet

About the Data


Gluten Free Diets are for those who are gluten sensitive, or for those who wish to remove the protein from their diet for potential health benefits. A family of protein found in rye, wheat and barley, (wheat being the most common), gluten can be easily described as the part of bread and dough that gives it its sticky, elastic qualities. Those with Celiac Disease are well known for being gluten sensitive. Plaguing around 1% of the population,

Celiac Disease is an auto-immune disorder that can result in nutrient deficiencies and damage to the digestive tract, if left unattended.  It is important for those with gluten sensitivity to avoid the protein in their diet as much as possible, in order to reduce the potential short term and long term effects on their bodies. 


Low Carb Diets are for those who are looking for weight loss assistance. This type of diet focuses on limiting carbohydrate intake, mainly those found in grains, starches and fruits. A high protein/high fat focused dietary plan, this diet focuses on eliminating a lot of the empty carbohydrates one might consume, paving the way for your body to properly burn its excess fat reserves. 


Keto Diets focus on High fat and low carb intake. Used for weight loss and by athletes for toning and shaping, this diet forces your body into releasing ketones, a state known as ketosis. Ketones are a product of the breakdown of fats in the body, into fuel. By eliminating excess carbohydrates and sugars from your diet, your body will begin to quickly burn its excess fat reserves . This diet focuses on eliminating most processed grains like bread and pasta, and added, processed sugars that come from common sweets. 


Diabetic Diets are for those who poignantly suffer from diabetes or pre-diabetes. Having issues with blood sugar regulation due to insulin production issues or imbalances, people with diabetes need to manually and artificially introduce insulin into their systems, in order to release the sugar from their blood streams into the rest of their bodies. 

This diet focuses on eating regularly (three times a day) in order to better utilize insulin doses, as well as stereotypically eating healthier, in order to reduce the stress of processing unhealthy food, on the body. This diet includes lots of healthy carbs like fruits, vegetables, whole grains and low fat dairy products. 


Mediterranean Diets began to rise in popularity internationally in the 1960's. Doctors, scientists and nutrition specialists began to see data that showed better cardiovascular health and less fatalities as a result of cardiovascular disease in people residing in the Mediterranean.

Consisting of a combination of daily starchy vegetables, whole grains, a helping of fish, legumes and eggs, this diet promotes a healthy heart  and is recommended for those with cardiovascular issues. This diet particularly focuses in limiting red meat intake. 


The figures

What does this mean?

At a glance, there seems to be a trend that leans towards the Mediterranean Diet being both highly reviewed and highly rated. Low Carb Diet recipes seem to be quick to make but not very highly reviewed or rated. Diabetic recipes appear almost as popular as Mediterranean diet recipes, when it comes to both number ratings and reviews.

Keto diets overall appear to take longer to prepare, therefore if you are looking for quick meal preparation, any of the other four diets might be your best. If you're looking for a diet with the most crowdsourced feedback, the Mediterranean Diet and Diabetic recipes appear to be the places to look.

With data analysis, there does not seem to be any significant difference among the categories in terms of both how long they take to make or how popular they are. However, this is a proof of concept sample size, and more data would yield a much more accurate, reliable and conclusive result

What went right and what went wrong

This project was a rollercoaster for me personally, and I had a lot of trouble, and spent a lot of hours learning how to web-scrape as opposed to learning the Pandas data management package. In terms of learning these skill quite recently, I was successfully able to navigate a reactive environment with a scraper in Python. I was also able to pull hrefs/urls, target specified information and store data to variable to sort out junk data from valued data. In terms of roadblocks with this project, they are prevalent, and given more time I probably could have worked my way through them.

There were issues with getting my scraper to run on its own (However, it seemed to work line by line with my observation). Also, there were obvious issues with implementing learned knowledge about web-scraping, made evident during my solo attempts and mentor assisted attempts at coding the program. 

There were some issues with committing stored data to a csv properly, although during the lectures these issues were not a problem. Additionally, my knowledge working with Pandas suffered as result on working on web-scraping, as learning how to scrape on the fly proved time consuming and mentally intensive. 

What is next, and where to improve...

With more time, I would like to streamline the scraping process I designed in reactive environment, and to to be able to reliably pull and store any targeted information based on user dietary preferences/restrictions. I would also like to further improve working knowledge of Pandas (very important). Being able to manipulate and organize data in Python is very important, and a lot of time was spent learning to scrape.

This being said, I need to improve my working knowledge of Python commands. I struggle to move quickly when working in python, and being able to speed up problem solving abilities working with Python would reduce some of the time management issues I had with this project. 


The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.



Perkin, Michael R., et al. "Randomized trial of introduction of allergenic foods in breast-fed infants." N Engl J Med 374 (2016): 1733-1743.

About Author

David Green

Certified in data science, confident working in R, Python, Git and SQL development. Skilled in applying machine learning techniques in data analysis of large datasets, alongside traditional statistical analysis triage
View all posts by David Green >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI