Using Data to Analyze NYC Open Restaurants and the Rise of 311 Complaints

Posted on May 27, 2022

Background

Within a few weeks of COVID hitting New York, the city shut down indoor dining. This was a devastating impact for the restaurant industry resulting in over 1,000 NYC restaurants permanently closing in 2020.

In an effort to save the restaurant industry, NYC launched the Open Restaurants Program in June of 2020.  This Program allowed restaurants to take up sidewalk or street space in front of their restaurants for outdoor dining.  It permitted restaurants to put chairs, tables, and build shed-like structures outside, with seemingly few restrictions.

Since the start, the Open Restaurants Program has been extremely popular.  The bar chart below shows how many restaurants have applied for the program by borough. As you can see, Manhattan has the most with over 6,000 applications, with Staten Island having the fewest. In total, over 12,000 restaurants in the city have applied for the program.

Number of Restaurants That Have Applied for the Open Restaurants Program

There can be no doubt that the Open Restaurants Program has been a lifesaver for the restaurant industry.  One article I read said that the program saved over 100,000 restaurant industry jobs.

And while that is great, I think it is also important to consider the negative side effects that have risen with adding over 12,000 outdoor dining areas in the city.  For my analysis, I used two datasets from the NYC open data portal. The first dataset I used contains information on all of the Open Restaurant applications that have been submitted since the start of the program. The second dataset contains information from the city’s 311 complaint program from 2010.  If you do not know, NYC has a program called 311, where residents can file complaints, essentially for anything whatsoever.  I explored these complaints to see whether I could learn anything about how the increase in outdoor dining has affected resident quality of life.

Outdoor Dining Complaints

In the 311 complaints dataset, complaints are broken down into dozens of different categories.  One particularly relevant category for my purpose was the “Outdoor Dining Complaint” category.  The city created this category specifically to track complaints related to the Open Restaurants Program.  So I thought it was a good place to start.

311 Page for "Outdoor Dining Complaints"

So how many “Outdoor Dining Complaints” have there been since the beginning of the Open Restaurants Program?  A lot.  As you can see in Manhattan, there have been approximately 5,000 complaints alone.

Number of “Outdoor Dining Complaints” Reported

Comparing the volume of complaints to volume of applications per borough, I was somewhat surprised by the sheer number of complaints; there are almost as many complaints as there are restaurants.

Number of “Outdoor Dining Complaints” vs. Number of Open Restaurant Applications

So, there are a lot of “Outdoor Dining Complaints,” but what are those exactly?  The chart below shows that most of the “Outdoor Dining Complaints” fall into a few categories, such as “Sidewalk zone blocked” or “Site setup condition.”

While these  complaint types provides some useful data (for example, residents clearly feel affected enough by blocked sidewalks and poor site setup conditions to submit a large number of complaints) these descriptors alone do not paint the full picture for assessing impacts to resident’s quality of life.

So I decided to look into the wider 311 dataset to look at other related complaint categories.

Assessing Quality of Life (“QOL”) Complaints

The wider dataset consists of over 28 million records, dating back from 2010, with dozens of complaint categories with even more sub-categories within.

I first identified what I thought were relevant complaint types and then grouped them into three main categories as structured below:

  • Dirty Conditions
    • Improper disposal/Dumping
    • Dirty sidewalk, Trash, Spillage
    • Rodents/Insects
  • Illegal Parking
    • Blocked Driveway/Bike Lane/Hydrant
    • Double Parking
    • Parking Sign Violation
  • Outdoor Noise (commercial & street)
    • Loud Music/Party
    • Loud Talking

Together, I’ll refer to these three categories of complaints as “quality of life” complaints.

The graph below shows the number of relevant Quality of Life complaints going back to 2010 for Manhattan.  You can see that during pre-pandemic times, there was a general trend upwards for these complaints (with seasonal spikes for each Summer/Fall time periods).  But what is really interesting is what happens post-pandemic.  From this chart, you can see that there was a huge influx for these complaints in 2020.

Relevant QOL Complaints in Manhattan (2010-2022)

 

Taking a Closer Look

This chart shows the same data, but is zoomed in to show only the past five years so that we can better see what happened during the pandemic. 

I’ve marked the date that restaurants were ordered to close and residents were encouraged to stay at home.  I’ve also marked the beginning of the Open Restaurants Program.  As you can see, the beginning of the Open Restaurants Program coincided with a huge influx in complaints.

Comparing Trends

So we know there was a huge spike in complaints in Manhattan around the time the Open Restaurants Program began. But can we attribute that to the Open Restaurants Program, or could there have been some other cause?  I wanted to explore the data a little bit more on this point.

The graph below shows a comparison in complaints between Manhattan and Staten Island. Going back from earlier, you’ll recall that Staten Island had the smallest volume of Open Restaurant applications.  Since the volume of open restaurants are very low in Staten Island, we wouldn’t expect to see the same surge in complaints filed.

And the graphs above shows that to be the case. As you can see, starting in 2010 until present day, there is a general upward trend in the number of complaints in Staten Island, but there is no spike in complaints around the time the Open Restaurants Program began.  In fact, the complaints in Staten Island today are about the same or even less than they were pre-pandemic.

Breaking Out the View by Category

Taking a closer look at each complaint, I broke out the trends to see which had the highest impact. This first view is filtered for outdoor noise complaints in manhattan from 2010 to date. We can see there are two large noticeable spikes in the summer seasons post-COVID with the first spike coinciding with the opening of the open restaurant program.

The month of June 2020 had over 16k outdoor noise complaints in Manhattan alone! June 2020 was also the beginning of phase 1 opening for the city and people were coming out of a 3 month stay at home order; we can assume New Yorkers were probably very excited to dine outdoor. The following summer of 2021 was not as high as 2020 but still almost twice as much as it was pre-pandemic era.

Outdoor Noise Complaints (Manhattan 2010 - to date)

As for dirty conditions, we see a noticeable spike but it does not occur until 2021 Summer/Fall. There was actually a decrease in 2020.

Dirty Condition Complaints (Manhattan 2010 - to date)

As for illegal parking complaints, other than the significant dip during the beginning months of covid with lockdown; surprisingly, we don’t see a significant surge in parking space related complaints as we did for noise and dirty conditions. It seems to follow the general upward trend since 2010.

With these breakdowns, it is clear that noise was the predominant factor driving complaint volume amongst these categories.

Complaints & Open Restaurant Density by Zip Code

Lastly, I wanted to view this relationship directly with a linear model.

The graph below demonstrates the aggregated volume of outdoor dining restaurants and complaints per zip codes to see if we can view any visual evidence that observes a relationship between these two variables.

The outlier zip codes above are all located in the Inwood/Washington Heights area and upon some googling found a New York Post article that describes these neighborhoods as famous for having the most noise complaints ever since the 311 help line was created.

Asides from the 4 obvious outliers, there seems to be a positive linear relationship that shows there are greater complaints within the areas with greater open restaurant density.

Conclusions

  • Our analysis shows evidence of a relationship with rise of Open Restaurants having an impact on NYC resident quality of life
  • The largest spikes in complaints found in Noise followed by Dirty Condition
  • We do not see any significant impact on illegal parking complaints but perhaps this is to be expected as parking has higher repercussions resulting in fines and ticketing
  • With Open Restaurant program becoming a permanent fixture for the city, we expect there to be improvements with better guidelines and more restrictions compared to the start of the program. With these changes, we’d hope to see some improvements in the coming summers.

The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

About Author

Related Articles

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI