Are We In The Golden Age of Television? Scraping TV Data From Metacritic.

Julia Goldstein
Posted on Aug 10, 2017


I don't discriminate when it comes to television - I love everything from bad reality shows to prestige dramas.  But over the past few years, I've felt overwhelmed by the sheer number of shows being heralded as "must-see TV". I know I should watch Atlanta and Mr. Robot, but who has the time? I'm not alone here - the Financial Times, in an article entitled "Watch it While it Lasts: Our Golden Age of Television," wrote,  "We used to complain that there was never anything good on TV. Now we complain about the opposite." In addition, new companies like Netflix are spending billions of dollars each year on original content, which may contribute to the feeling that there is constantly something new to watch. 

The Data

I scraped TV data from Metacritic, a website that aggregates reviews of media products. Most notably, it aggregates both qualitative and quantitative critic reviews into a single "Metascore". It also contains detailed information about each product, though the TV section was not launched until 2005. Metacritic does have data on TV shows from prior years, but their listings are incomplete. I scraped several layers of data, including:

  1. Overall TV Rankings: From Metacritic's main TV page, I scraped each show's overall rank, name, and the link to the individual show page.
  2. TV Show Details: I scraped details for each individual television show, as well as a link to another “Details and Credits” page.
  3. TV Show Details, Continued: I scraped additional details for each television show.

From this process, I obtained 13 variables for over 2,000 shows, including Critic Rating, User Rating, Rating Description, Network, Season and Series Premiere Dates, and Genre(s).

Research Questions

For my analysis, I wanted to investigate my hypotheses about the influx of quality, must-see TV in recent years. I attempted to answer the following questions:

  1. Are there really more “good” shows than ever before?
  2. Are subscription/online networks displacing traditional channels?
  3. Should we trust the critics – do critics and users usually agree?
  4. What should I actually watch next?


Critics Vs. User Ratings

First, I wanted to explore the relationship between Critic and User Ratings. Metacritic provides a Critic Rating for TV shows with at least seven critic reviews, as well as an average User Rating for shows with at least four user reviews. I graphed both and colored the points based on the Rating Description, which is a qualitative variable based on the quantitative Critic Rating.Based on the plot, the relationship between Critic Rating and User Rating seemed fairly linear, and sentiments seem to converge as the Critic Rating improves. For shows that critics rated "Average" or "Unfavorable", user opinions vary widely. Further analysis revealed a correlation of 53%, indicating a significant, moderate linear relationship between Critic and User Rating. I also tested my theory of consensus among higher rated shows by conducting several two-sample t-tests on Critic and User Ratings for each Rating Description group, but I found that the population means were different across all four groups. We can look at the distribution of Critic and User Ratings as well:

Critics are "harsher" than users - the median rating given by critics is 65, while the median rating submitted by users is 74. The distribution of User Ratings is also narrower and is concentrated towards the higher end of the scale.

Critic and User Ratings may be correlated, but I wanted to know whether they matter for a show's success. By graphing the total number of seasons for each show, I found that almost half of the shows in the data set lasted for only one season. I also checked to see if there was any correlation between Critic and User Ratings and a show's total seasons, but there was a fairly weak linear relationship in both cases. Most shows only last for one or two seasons, regardless of how reviled or beloved they are.

The Era of Peak TV?

Next, I wanted to know whether there really has been an increase in the quantity and quality of TV over the last few years. I graphed the number of shows on TV by year, as well as the number of highly rated shows (Critic Rating > 60).Looking specifically at the years between 2005 (when Metacritic launched its TV section) and 2016 (since 2017 data is incomplete), it is clear that the number of shows, as well as the number of "good" shows, has increased. However, looking at the average Critic and User Rating by year does not show such a clear trend.Critic and User Ratings have fluctuated in recent years, even as the total number of highly rated shows has increased. We can conclude that shows are not necessarily getting better - we may just have more shows (and therefore more "good" shows)!

Trends in Television Networks

The increase in show volume may be related to the many networks that have entered the TV ring over the last few years (e.g., Netflix, Hulu). We can see this increase by looking at the number of networks with at least one show on the air by year.

The number of TV networks essentially doubled between 2005 and 2015 - no wonder we feel overwhelmed by our options! Access to many of these networks may require additional cable packages or online subscriptions beyond the basic channels, so how do we decide where to spend our time and money? To explore this, I looked at the top 10 and bottom 10 networks based on average Critic Rating. Of all networks with at least 15 existing shows, PBS and HBO have the most critical acclaim. A&E and NBC have the least.

To incorporate User Ratings and show volume into my analysis, I created a bubble chart. In the image below, the size of the bubbles represents the number of shows each network has produced. Only networks with at least 15 existing shows are included.

This chart shows that HBO is essentially the "model" network - it has produced a large number of shows that are beloved by both users and critics. Other subscription networks, such as Netflix and Showtime, also stack up well against the competition. On the other hand, traditional basic cable networks are lagging behind in terms of creating "prestige" TV. This reflects the growing industry trend of cable package cancellation in favor of streaming services and other online subscriptions.

What Should I Watch?

Finally, I used the data I collected to cut through all of the "buzz" and determine what I should actually watch next. I averaged Critic and User Ratings across all shows with at least three seasons, and selected the top 10 Dramas and Comedies. Hopefully this will help any of you out there looking for a new show to binge watch!


Based on the data from Metacritic, it is difficult to say whether we are really experiencing a "golden age of television". There are more shows being produced than ever before (and therefore more good shows), though the average quality of shows may not be improving. The influx of new networks has likely contributed to this increase, though many of the new "prestige" shows seem to be coming from platforms like HBO and Netflix, not traditional cable. Unless cable networks start producing "must-see TV" to improve demand, they will probably continue to see a drop in subscription rates among their customers.

Link to my Github.

About Author

Julia Goldstein

Julia Goldstein

Julia has over five years of experience delivering business insight through data analysis and visualization. As an analytics and management consultant, she was responsible for managing projects, identifying solutions, and developing support among senior-level stakeholders. Moving forward, Julia...
View all posts by Julia Goldstein >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp