Building a Painting Style Classifier with Tensorflow

Posted on Apr 5, 2017

The objective of my project was to build a painting style classifier with Tensorflow (Artificial Neural Network), that would be able to distinguish 28 different painting styles:

  1. Abstract
  2. African
  3. Chinese
  4. Color Field
  5. Concept Art
  6. Contemporary
  7. Cubism
  8. Expressionism
  9. Fauvism
  10. Futurism
  11. Hyperrealism
  12. Impressionism
  13. Indian
  14. Islamic
  15. Japanese
  16. Korean
  17. Landscape
  18. Lyrical Abstraction
  19. Minimalism
  20. Modern Art
  21. Modernism
  22. Mural
  23. Photorealism
  24. Pop Art
  25. Pop Out
  26. Still Life
  27. Surrealism
  28. Veduta

I attempted to collect 50,000 sample images of each style or 1.4 million images total. This required use of a search engine API, namely Bing Cognitive Web Search API. Unfortunately the API could not handle the 50,000 sample workload, so ultimately I only ended up with ~1,000 images per class (1/50 of the goal). Additionally, since I leveraged a search engine for this task, the images came from a wide range of sources and thus it was essentially impossible to create truly homogeneous samples for style.

Luckily I was fortunate enough to have a 32 processor server with 208GB of RAM and 8 NVIDIA K80 Graphic Cards at my disposal. Unfortunately Tensorflow is still an evolving framework and is very difficult to optimize, and has restrictions based on the OS you are using (e.g., on Windows Tensorflow only works with Python 3.5.2).

Having around 2 weeks to attempt this massive undertaking I sought out existing models I could leverage for my problem. I narrowed down my base model to VGG Net, which was proved very effective for a similar problem, the ImageNet Challenge 2014. They ended up finding that a 16 and 19 weight level variations of the model proved most effective. The weights from this model implementation have been used to create a Neural Artistic Style Transfer program that can transform any photo you give it, to have the theme of your favorite painting. See a breakdown of the VGG Net Model in the table below:


I used the Tensorflow playground to approximate the time and complexity to train a model for what I wanted. I approximated with perfect samples, my theoretical ceiling would be 78.5% at 5,000 epochs (see image below). I determined there would not be enough time to run and test multiple models since I believe it would take over 24 hours to run a single training model.

Screenshot (38)

So I ended up settling for demonstrating the application of the VGG-19 ImageNet to transform a photograph of Columbia University to the style of an Abstract Painting. See (a)Original Photo (b)Abstract Painting (c)Photo in the style of the painting


IMG_20170121_210744 (b)




The above image (c) took 36 hours to produce, which demonstrates how time consuming it would be to build a model that not only essentially performs the above transformation for 1.4 million images, but also applies the results of it to an optimization problem that can use the common characteristics to estimate the most probable painting style of a new image fed into the model. This is a problem that fascinates me and that I will continue to work on.

About Author

Marc Fridson

In addition to my current participation in the Data Science Academy, I am a Course Designer/Facilitator for Columbia University's Applied Analytics Program and the CEO/Founder of Instant Analytics an analytical technology start-up. Prior to this I was the...
View all posts by Marc Fridson >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI