Data Analysis of Movie Critics vs Movie Fans

Posted on Feb 12, 2021
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

Objective

A data analysis on how the top 100 movies from Metacritic and IMDb compare to one another and what that reveals about what factors a fan evaluates when rating a movie versus a critic. 

Introduction

When you are looking for a movie to watch, I bet you do a little diligence. I know this because if you're reading this blog post you're interested in or involved in Data Science or the NYCDSA, and that means you use data to make everyday decisions.

Many consumers use Yelp to find a new favorite dining spot or prowl through reviews to find the highest quality product on Amazon, but you take it to the next level. You click on each Reviewer. You try to understand them better, what other things have they reviewed, do they generally review things positively or negatively? If they generally review things negatively, does that mean they have too high standards or, the opposite, poor taste and judgment? If things seem too positive, you may wonder if this individual is being paid to write these favorable product reviews.

What if, on average, the reviewer seems to deviate from the consensus rating/opinion on the subject, does that mean they're hipster? If you're also a hipster that may be a good thing, but I digress...

Methodology

IMDb

Both IMDb and Metacritic rely on their consortium of users and subject matter experts to evaluate movies and other entertainment to provide a reliable and limited-bias metric for that very consortium. Just as your intuition as a Data Scientist tells you to look beyond a raw mean of ratings/number, IMDb and Metacritic give different weights and significance to some user ratings versus others. Let's look at IMDb's rating methodology first. 

“We take all the individual ratings cast by IMDb registered users and use them to calculate a single rating. We don't use the arithmetic mean (i.e. the sum of all votes divided by the number of votes), although we do display the mean and average votes on the votes breakdown page; instead, the rating displayed on a title's page is a weighted average

IMDb publishes weighted vote averages rather than raw data averages. The simplest way to explain it is that although we accept and consider all votes received by users, not all votes have the same impact (or ‘weight’) on the final rating. 

When unusual voting activity is detected, a different weighting calculation may be applied in order to preserve the reliability of our system. To ensure our rating mechanism remains effective, we don't disclose the exact method used to generate the rating.” [imdb.com]

Metacritic

Now we visit Metacritic's methodology. 

"Metascore is a weighted average in that we assign more importance or weight, to some critics and publications than others, based on their quality and overall stature. In addition, for music and movies, we also normalize the resulting scores (akin to "grading on a curve" in college), which prevents scores from clumping together." [metacritic.com]

Now that we have established that the ratings we see on either website are more than a straightforward raw mean score, let's proceed to analyze the differences in the data between them. 

Important considerations of the data:

-The data examined is a set of the top 100 movies on IMDb and a set of the top 100 movies on Metacritic

-IMDb rating scale 0.0-10.0 with approximately 1-2 million user inputs

-Metacritic rating scale is 0-100 with approximately 10-30 movie journalist inputs

First, we look at the distribution of ratings for both:

IMDb Data

Data Analysis of Movie Critics vs Movie Fans 

Metacritic Data 

Data Analysis of Movie Critics vs Movie Fans

We notice immediately that there is rightward skew, which intuitively makes sense. Higher scores should be harder to obtain, especially if you were to visualize a normal distribution curve of the entire collection of movies. 

Data on Distribution of Release Dates

We move on to something more interesting, the distribution of the release dates of each set of top 100 movies. 

Data Analysis of Movie Critics vs Movie Fans

Analysis 

As we can see, Metacritic favors older films more than IMDb, with a smaller mean and median. We also observe two peaks in the Metacritic distribution indicating a strong preference for movies before 1980 and movies after 2000. Interestingly, though this time period between 1980-2000 has limited favorability by Metacritic, it is actually favored the most by IMDb with 60% of the top 100 movies occurring in that time period. 

Moving on, we wonder if there are any overlapping movies between the two lists. Initially, in the top 10 and top 25 sets, there is little. Only one movie, The Godfather (1972), is shared between them. However, as the sets expand to contain the top 50, top 75, and top 100 movies the overlap grows from 6, 9, and 16 movies respectively.

Top 50

Top 75

Top 100  

Finally, we examine directors with multiple films (2 or more) in the top 100 lists. 

Data on Directors

Leading the charge in the IMDb list, are Christopher Nolan and Stanley Kubrick with a staggering 7 films in the top 100. These top 11 Directors make up 40% of the IMDb list while the top 11 directors of the Metacritic list would make up only 16% of the list. 

We end this analysis by asking, what might explain the large differences in the two sets?

Obviously, critics and movie fans use different criteria to evaluate a movie. Critics will use more technical analysis of the directing, cinematography, editing, and storyline than your average movie fan. Movie fans may rely more on visceral measures like emotional responses, special/visual effects, and simply the gut feeling one has at the end of the movie as the credits roll. Instead of conjecturing the wide array of criteria that is used across groups, let's identify a theme that both groups agree on which is cultural impact and the nuances of perspective within that theme.

IMDb evaluates cultural impact in real-time, as users flood the database with scores following a release of a major blockbuster movie like Avengers, which rose to the top 5 all-time list at a certain point closely following its release.

Metacritic evaluates cultural impact in hindsight, evaluating movies in respect to their relevance in that time period as well as their place in cinematic history. Movies that capture the zeitgeist are often featured in Metacritic such as Moonlight, Lady Bird, and Grapes of Wrath. Moreover, movies that left a lasting impression on the industry by changing the way films were directed and produced are common. This is evident through the high quantity of movies highly reviewed before 1960 and as far back as 1930 by directors like Alfred Hitchcock and Charlie Chaplin. 

About Author

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI