A dish to die for?

Posted on May 8, 2017

You are exhausted from an intense day at work. You put one foot after the other, consuming all the energy left in reaching your destination. You finally get there and order your special. The dish arrives. Remembering how much you love that food you gobble what's in front of you. Once it's in your belly, you open your eyes, look around and realize you are the only one in your favorite restaurant. "But it's rush hour," you think to yourself. At that exact point a couple walks towards the door of the restaurant. You see them look at a paper stuck to the inside of the glass door. Their faces express concern and they walk away. You walk towards the door and see a big C staring back at you. At that point you realize the health inspector must have come around and was not impressed with what he saw. Your stomach bellows a howl. Oh-oh!

When you go out to eat, your top priority is to have good, clean food. For some odd reason, we always assume that every chef is like Monica from the famous sitcom Friends. Highly talented chef, super clean. Such a chef would pass the Department of Health and Mental Hygiene (DOHMH) score with flying colors. Unfortunately, that’s not the case for chefs in New York City restaurants.

Recently the DOHMH published violations, grades and scores of 8,444 restaurants in Manhattan, New York between 2012 and 2017. Clearly there are some places that carry higher risks than others. Before we poke our noses into who did what, it’s probably better to know some basics.



Every year the Health Department inspects 24,000 restaurants in Manhattan for City and State food safety regulations. As a result, each restaurant is given a score. A score between 0 to 13 points earns an A, 14 to 27 receives a B and those with more than 28 get a C. The lower the score the better. The scores are summed up based on the type of violation. You can read more about the different type of violations here. A health inspector can inspect a restaurant at least once and would inspect again if the restaurant has violated some critical violations.


The data

The DOHMH dataset was downloaded from NYC data center. It contains 167,777 rows and 15 columns. The columns contains information about

  1. The name of the restaurant
  2. Location in Manhattan (BORO, STREET)
  3. Zip code
  4. Type of cuisine
  5. Inspection date
  6. Action taken
  7. Violation code
  8. Inspection type
  9. Violation description
  10. Critical flag
  11. Score
  12. Grade
  13. Grade date
  14. Record date


With such a data at our hands what can we possibly find out?

With such a huge data set we could get answers to the following questions:

  • Are all cuisines violating protocol or are their specific cuisines that violate the most?
  • Restaurants that violate at a severe scale, what happens to them?
  • What counts as a violation?
  • How many of the violations were critical?
  • How many restaurants were awarded a grade?
  • Which grade was awarded the most?
  • Are good grades awarded to non-critical violations only?



Which cuisines had the highest violation?

The following are the top 20 cuisines (out of 81 in the data set) that had the highest frequency of violations.


Fig.1 The barplot above shows the highest frequencies of health violation by cuisine type.


The graph in Fig. 1 shows that American cuisines had the highest number of violations out of 81 cuisines in Manhattan.


Were there any actions taken for severe violations?

fig2Fig 2: Comparison between the total number of violators and non-violator.

Apparently, yes! Fig.2 shows that there were a lot of restaurants that performed better such that they did not need to shut down however there were 3129 restaurants that needed to shut down because they had severe violations. Only a handful of restaurants did not need to shut down.

What were those violations?

The dataset included 93 types of violations. The top 10 are graphed below.

fig3Fig 3: Top 10 reasons for penalization.

In Fig. 3 we can see some of the violations that are very crucial to the inspector. A clear majority of restaurants did not have the necessary setup to make a proper serving counter. Some restaurants did not make sure that:

  • The area was free of vermin or mice.
  • The food preparation counter was clean
  • Food was not prepared next to possible contaminants.
  • Filth flies were absent.
  • Food was prepared or cooked according to regulations.

How many of the violations were critical?

fig4Fig 4: The amount of critical, not critical flags.

There was a total of 92,831 critical violations. This was irrespective of the grade that was awarded.


How many restaurants were awarded a grade? Which grade was awarded the most?

fig5Fig 5: Types of grade and how many were awarded.

Turns out A! According to the Fig. 5 we can see that overall a lot of restaurants performed better and got good grades. The grading system works! Fig.6 below show how many grades were awarded each year.


Screen Shot 2017-05-05 at 12.07.45 AM

Fig. 6 The bar chart shows the grades awarded each year.

When we compare Fig. 4, Fig. 5 and Fig.6 we can notice a strange incidence. If many restaurants were given an A grade then why are there so many critical flags?


The evidence suggests that despite a restaurant getting a better grade, they still might have severe violations. However, the important objective to note is that the total number of violations are far fewer in number.



A tale of two case studies

Let’s get an accurate estimate of how to judge a restaurant. For this case study let’s look at two restaurants, La Grenouille and Dishes.

La Grenouille is an A grade restaurant. But do they have any skeletons in their closet?

yrgpFig. 7 The graph shows the number of violations that La Grenouille violated each year. La Grenouille’s worst year was 2015 with a total of 5 violations (3 critical and 2 noncritical).

It turns out La Grenouille did have some critical violations, but the total number of critical violations were fewer in number. Inspite of the violations of 2015, La Grenouille has managed to keep its critical violation low and maintain an A grade.


Now let’s look at Dishes.


Fig. 8 The graph shows the critical and noncritical violations for Dishes. Overall Dishes had a lot of critical violations from 2015 to 2017.

The graph in Fig. 8 shows that Dishes had had a lot of critical violations, and it hasn’t changed much for the preset year as well. Therefore, the grade has changed, moving between A, B and C (Fig.9).

Dishes2Fig. 9 Dishes' best grades were in 2014 and 2017. In 2015, the restaurant had a bit of trouble scoring an A.


Comparing both Fig.7 and Fig.8, we can conclude that as long as critical and non critical violations are under 14 points, a restaurant will be awarded an A grade.



You don't have to cancel your dinner plans

Before we conclude that you can't trust ratings, you should look over the choices of restaurants with A grades that had only noncritical violations. There are still 4,103 restaurants to pick from. You can visit the shiny app to have a look at both grade A and B restaurants with non-critical violations.


Future work on app:

  1. Make the best restaurants sub tab more dynamic
  2. Optimize the app such that
    1. Searching for a restaurant gives the grades it has been awarded over the year.
    2. Gives detail from the original data set.
  3. Get geo coordinates for all restaurants to display on Manhattan’s map. The dataset does not have geo coordinates and as a result Google’s API was used to convert addresses to geo coordinates. The API has a limitation of making 2500 calls a day.

About Author


Tariq Khaleeq

Tariq Khaleeq has a background in Bioinformatics and completed his masters from Saarland University, Germany. In his master thesis, he worked on prediction of non coding genes in breast cancer. After his masters he co-founded a company where...
View all posts by Tariq Khaleeq >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp