Scraping Recipes Using Data Science
Background
By this article, we use data science to explore the preparation, cooking time among 5 different cuisine types using scraped data I prepared.
In recent years, studies have shown that people are becoming more and more sensitive to a variety of common food allergens (peanuts, egg, etc.). On the other hand, ease of transportation and improvements in agriculture have made a wider variety of food available to todays consumers.
This opens the way for those with restrictive diets to more easily meet their needs with a wider variety of ingredients available, the next obstacle to those with strict dietary needs, is often favorable recipes to follow. Anyone with internet has access to recipe databases, however navigating recipe websites can be a slog, often filled with blog posts, advertisements, and opinion pieces.
Coming from experience, even with search bars and categories, it isn't always quick or easy to find exactly what you need on a recipe website.
What are we asking?
When looking to pull data from a site via web-scraping, it is very important to have an idea of what kind of data you are looking for. In this particular case, I wanted to see how popular, or unpopular, certain diet restrictive recipes were, as well as how long they took to cook, and how many servings they produced. More specifically:
- Are any specific diets over/under represented?
- In amount of reviews and amount of rating
- How reliable is the data?
- In amount of reviews and amount of rating
- Which categories are able to leverage more meals per recipe?
- Serving sizes?
- Are any specific diets easier to prepare than others?
- Time spent preparing meals?
- Are any specific recipe types less popular?
- How highly rated are they?
- Which categories are able to leverage more meals per recipe?
- Serving sizes?
The data sample
As a proof of concept, the data selected for this project consisted of 25 randomly selected recipes, pulled from the "staff picks" section of five different subcategories in the "diet & healthy recipes" section of the website. five recipes were taken from each subcategory: Gluten Free, Mediterranean Diet, Low Carb Diet, Diabetic Diet & Keto Diet.
About the Data
Gluten Free Diets are for those who are gluten sensitive, or for those who wish to remove the protein from their diet for potential health benefits. A family of protein found in rye, wheat and barley, (wheat being the most common), gluten can be easily described as the part of bread and dough that gives it its sticky, elastic qualities. Those with Celiac Disease are well known for being gluten sensitive. Plaguing around 1% of the population,
Celiac Disease is an auto-immune disorder that can result in nutrient deficiencies and damage to the digestive tract, if left unattended. It is important for those with gluten sensitivity to avoid the protein in their diet as much as possible, in order to reduce the potential short term and long term effects on their bodies.
Low Carb Diets are for those who are looking for weight loss assistance. This type of diet focuses on limiting carbohydrate intake, mainly those found in grains, starches and fruits. A high protein/high fat focused dietary plan, this diet focuses on eliminating a lot of the empty carbohydrates one might consume, paving the way for your body to properly burn its excess fat reserves.
Keto Diets focus on High fat and low carb intake. Used for weight loss and by athletes for toning and shaping, this diet forces your body into releasing ketones, a state known as ketosis. Ketones are a product of the breakdown of fats in the body, into fuel. By eliminating excess carbohydrates and sugars from your diet, your body will begin to quickly burn its excess fat reserves . This diet focuses on eliminating most processed grains like bread and pasta, and added, processed sugars that come from common sweets.
Diabetic Diets are for those who poignantly suffer from diabetes or pre-diabetes. Having issues with blood sugar regulation due to insulin production issues or imbalances, people with diabetes need to manually and artificially introduce insulin into their systems, in order to release the sugar from their blood streams into the rest of their bodies.
This diet focuses on eating regularly (three times a day) in order to better utilize insulin doses, as well as stereotypically eating healthier, in order to reduce the stress of processing unhealthy food, on the body. This diet includes lots of healthy carbs like fruits, vegetables, whole grains and low fat dairy products.
Mediterranean Diets began to rise in popularity internationally in the 1960's. Doctors, scientists and nutrition specialists began to see data that showed better cardiovascular health and less fatalities as a result of cardiovascular disease in people residing in the Mediterranean.
Consisting of a combination of daily starchy vegetables, whole grains, a helping of fish, legumes and eggs, this diet promotes a healthy heart and is recommended for those with cardiovascular issues. This diet particularly focuses in limiting red meat intake.
The figures
What does this mean?
At a glance, there seems to be a trend that leans towards the Mediterranean Diet being both highly reviewed and highly rated. Low Carb Diet recipes seem to be quick to make but not very highly reviewed or rated. Diabetic recipes appear almost as popular as Mediterranean diet recipes, when it comes to both number ratings and reviews.
Keto diets overall appear to take longer to prepare, therefore if you are looking for quick meal preparation, any of the other four diets might be your best. If you're looking for a diet with the most crowdsourced feedback, the Mediterranean Diet and Diabetic recipes appear to be the places to look.
With data analysis, there does not seem to be any significant difference among the categories in terms of both how long they take to make or how popular they are. However, this is a proof of concept sample size, and more data would yield a much more accurate, reliable and conclusive result
What went right and what went wrong
This project was a rollercoaster for me personally, and I had a lot of trouble, and spent a lot of hours learning how to web-scrape as opposed to learning the Pandas data management package. In terms of learning these skill quite recently, I was successfully able to navigate a reactive environment with a scraper in Python. I was also able to pull hrefs/urls, target specified information and store data to variable to sort out junk data from valued data. In terms of roadblocks with this project, they are prevalent, and given more time I probably could have worked my way through them.
There were issues with getting my scraper to run on its own (However, it seemed to work line by line with my observation). Also, there were obvious issues with implementing learned knowledge about web-scraping, made evident during my solo attempts and mentor assisted attempts at coding the program.
There were some issues with committing stored data to a csv properly, although during the lectures these issues were not a problem. Additionally, my knowledge working with Pandas suffered as result on working on web-scraping, as learning how to scrape on the fly proved time consuming and mentally intensive.
What is next, and where to improve...
With more time, I would like to streamline the scraping process I designed in reactive environment, and to to be able to reliably pull and store any targeted information based on user dietary preferences/restrictions. I would also like to further improve working knowledge of Pandas (very important). Being able to manipulate and organize data in Python is very important, and a lot of time was spent learning to scrape.
This being said, I need to improve my working knowledge of Python commands. I struggle to move quickly when working in python, and being able to speed up problem solving abilities working with Python would reduce some of the time management issues I had with this project.
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
References
https://www.freepik.com/free-photo/pots-vegetables-harvest_1440232.htm#page=1&query=cooking&position=11
Perkin, Michael R., et al. "Randomized trial of introduction of allergenic foods in breast-fed infants." N Engl J Med 374 (2016): 1733-1743.
About Author
