A Retrospective- Data Study on Rats in NYC

Posted on Oct 15, 2017
The skills the author demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

32 million dollars.

Take a while, let that data sink in.

Thirty-two.

Million.

Dollars.

Three months ago, mayor de Blasio presented a $32 million dollar plan to β€œIce” some rats.Β  He said that this city wants more rat corpses.1

 

 

Are rats that big a deal? Can the government even curb the problem if they are?

I looked at governmental rat-reduction initiatives over the last 7 years, analyzed the effect of the three of theseΒ  initiatives, and then took a look at what drives rat sightings.

The short of it is that:

  • There seems to be little to no impact on rat sightings from governmental intervention
  • The data the government likely uses for their success metrics may be in need of some better controls
  • The recent trend in rat sightings may have little to do with the actual rat population.

 

Three initiatives, mixed data results

 

  • In 2010 the city cut their budget for pest control by 1.5 million dollars.2 The the average time to β€œclose out” a rat sighting is the highest during this period; however, this has no observable impact on the trend of rat sightings.
  • In 2013, plans to mass sterilize female rats were announced.3 While this may have had some level of success (I don’t know how widely the tests were being conducted) in 2014 we see a sizable increase in the reports of rat sightings.Β  This trend continues all the way through the start of 2017.
  • In 2014, 9 new inspectors and 600 thousand dollars used to target major infestations in bronx and manhattan with the goal of returning the average closing time to below 10 days.4Β  While the average closing time does decrease, the total reported sightings increase substantially.

A Retrospective- Data Study on Rats in NYCA Retrospective- Data Study on Rats in NYCA Retrospective- Data Study on Rats in NYC

So why don’t I like the average closing time metric?

My problem with this metric is that it doesn’t look like it has any actual impact on the rat sightings (see above) and that if it uses the same data that I worked with, it likely isn’t automatically updated in the system, and thus is prone to input error.Β  While cleaning up the data, I found approximately 10% of the over 102k sightings were missing a closing date, and that approximately 16.6% had closing dates that preceded their created date.Β  While I did filter that data out and use median for my average to filter out any input error that would create outliers, i’m still not sure that i’d put much confidence in it.

What drives rat sightings?

I originally decided on a map format to display all the rat sightings by whatever selected time period so we could see if there’s any major shifts in rats due to relocation or migration- unfortunately we did not see much of a shift in that way- however that does corroborate one of the documents I came across later which suggests that rats rarely travel more than 600 feet from their birthplace.13

Effects of Weather All Years

Effects of Weather 2010-2012

Β .Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β  Β Β 

 

 

 

 

 

 

The seasonal portion of rat sightings seems to have a pretty strong correlation with temperature.5

Precipitation?6Β not so much.Β  in 2010 to 2012 we see that the data all falls basically along the same line. The line shifts a little in 2013. In 2014 that line shifts up. The line shifts again in 2015, 2016, and 2017. When we change the data into a time series then decompose it (or rather, break the time series down into its seasonal change, its trend, and the β€˜white noise’ remainder) we can see that the trend shifts substantially at the end of 2013/start of 2014

Reported rat sightings as observed, as season, as trend, and as remainder

What changes the data trend?

Honestly, I don’t know.Β  I looked into trash strikes, income shifts (thinking more disposable income could lead to more garbage), but nothing I could find could have led to such a surge in rat population.Β  However, I’m not measuring rat population- just reported rat sightings. Something may have shifted the way New Yorkers think of rats in late 2013/early 2014.

So what happened in 2014?

  • A study found that New York rats carry Salmonella, E. coli, Seoul hantavirus, Leptospira, etc. -at least 18 viruses that are known to cause disease in humans7
  • Jonathan Auerback used a statistical model to argue that the rat population in NYC was closer to 2 million than 8 million8
  • YouTube videos of rats in NYC went viral9-11
  • Rent strike demanding rat problem taken care of- their slogan? β€œNo rent for rats!”12

In conclusion

Whether or not the rat population is actually increasing, the increased reports of rats indicate that New Yorkers perceive this as a growing problem. While prior attempts to curb the rat problem have been largely unsuccessful, there is the chance that this time it might work. But probably not.14

 

 

References

  1. https://www.nytimes.com/2017/07/12/nyregion/new-york-city-rat-problem.html
  2. http://www.nydailynews.com/new-york/bronx/pest-control-workers-union-fears-city-overrun-rats-bloomberg-cuts-57-84-jobs-article-1.169365
  3. As Rats Escape Death, MTA Turns to Sterilization - The New York Times
  4. ”Rodents winning New York rat race, but humans fight back"
  5. Avg temperature data source- Average Monthly & Annual Temperatures at Central Park
  6. Avg precipitation data source - Monthly & Annual Precipitation at Central Park
  7. http://mbio.asm.org/content/5/5/e01933-14
  8. Does New York City really have as many rats as ... - Wiley Online Library
  9. https://youtu.be/qdFF5C3ZR_I?t=30s
  10. Β "Rats scurry through food at Dunkin' Donuts in Manhattan (VIDEO)"
  11. https://youtu.be/zAw05LGeTkg
  12. Upper West Siders rat-chet up protest - NY Daily News
  13. https://www.nytimes.com/2015/04/26/magazine/the-rat-paths-of-new-york.html
  14. Rats can produce half a BILLION descendants in three years - Daily Mail

  15. Source for reported rat sightings- https://www.kaggle.com/new-york-city/nyc-rat-sightings
  16. Link to my Shiny App: https://bdbrunson.shinyapps.io/myshineyapp/
  17. Link to my Github Code for the app:Β https://github.com/Bdbrunson/NYRats

About Author

Ben Brunson

Ben Brunson is a man whose curiosity has led him to work in many industries. He handled day to day operations and special projects for AuST Development, a medical devices development company. He Managed paid search campaigns for...
View all posts by Ben Brunson >

Related Articles

Leave a Comment

hoodrat October 20, 2017
Dat rat phitty is amazing, rats should be the official NY Mascot or be in the coat of arms, I mean Sprinter is quite an inspiring character he led 4 turtles to become heroes for a generation!

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI