Using web scraper and K-means to find the colors in movie posters
Contributed by Amy(Yujing) Ma. She is currently in the NYC Data Science Academy 12 week full time Data Science Bootcamp program taking place between January 11th to April 1st, 2016. This post is based on her third class project – Web scraping(due on the 6th week of the program).
January 30, 2016
Each movie has their own posters. Even in today’s always-online climate, the movie poster remains a powerful form of advertising. Every movie poster has its own color scheme, based on the movie’s type, content, and tone. Best movie posters should catch people’s eyes. So what kinds of colors are more likely to be used in different types of movies?
To play with some plots in this post, please go to the related post on Rpubs.
Scrapping Images with Python
To answer this question, we need analysis movie posters of different movies. First of all, we need to build a training dataset of movie posters. So, I used the Bing Image search engine for this.
By using python to scrape all the movie posters from the website, I finally got 112 photos for 4 types of movies (horror, comedy, animation and action movies).
And the part of images I got looks like this:
Transforming the Images
To extract the color of images, we need to represent them in an easier and numerical way: Converting one image to a matrix of RGB pixels. For example, for a 200x200 px image, we convert it to a series of 40,000 RGB pixels.To maintain the size of the dataset, I resized the images down to 200x200.
Extract Colors from Posters
Then I used k-means clustering to do image segmentation, separate the pixels into different clusters based on their colors. Here is an example of image segmentation and compression from Bishop’s book Pattern Recognition and Machine Learning.
I used python to do the k-means analysis, the code is shown as below:
I tried k=3, k=5, and k=10 since lots of the movie posters used black letters and frames, k=3 and k=5 didn’t capture the main colors of posters. I chose k=10 to do k-means to all these 112 images. Finally, I got 1,120 colors of 112 movie posters
Part of the result shown as blow:
This result shows one of the weaknesses of k-means clustering, it’s sensitive to initialization, which leads to unwanted solutions. In this poster, K-means couldn’t capture any gold or orange colors.
For each type of movies, I plot its movie posters’ colors into a 3D scatterplot based on RGB values. Each point represents a poster color. By comparing these 4 plots, I found most of the horror movies use darker and red colors in their own posters while the comedy and the animation movies would choose various colors to correspond to their themes.
Comedy vs. Animation
Action vs. Horror
Transforming the Colors
1120 colors contribute to many to discover the specific patterns, so we need to reduce the number of colors and make them into several categories. For this, we can transform the colors into Lab color space, and use Delta E equations and colormath package in python to calculate the visual difference between poster colors and some basic colors.
Then I categorized these colors into 17 colors by choosing the smallest distance value in each row.
This is a part of the dataset of comedy movie posters:
Comparison based on Movie Types
After transforming the data, I counted a number of each basic color within each movie type.
By plotting the data together, we would confirm our previous findings and also we can see black, gray and white are most frequent used colors in movie posters. This is because I have the too small sample size of basic colors, and most the movie posters have black titles and frames.