used Tesla Model S web scraping

Posted on Jun 2, 2019

Github Repository | Linkedin


Electrically powered vehicle have become more popular due to less cost in maintenance and gasoline, good for the environment, driving range is similar to the traditional combustion vehicle and fun to drive based on the data collected. Tesla is one of the most popular and desirable brands that consumers want to have. However, most people are not able to afford the high initial cost of a brand new Tesla. Another alternative is to find a used Tesla. 

In this project, a Python spider is implemented to scrap used car websites: and Later, an explanatory data analysis will be performed to analyze which Tesla Model S is best suitable for one's need.

Web scraping: list page provides much important information for machine learning such as original price, current price, trim of the car, drivetrain, etc. Python spider can easily scrap the front page information. However, the Vin number can only be obtained after I clicked into the vehicle page.

The same thing goes for There is some useful information that can only be obtained after I clicked into the car page. For example, days on market, has accidents or not, how many previous owners the car has?

"Hot Car Prediction"

In, a car will be indicated as "HOT CAR" if predicts that with 70% of chance the car will be sold in seven days after posting. 

At the end of this project, a simple machine learning will be performed to predict if a car is HOT CAR or not base on the information I scraped from both websites.

Data Cleaning: data set contains 2115 rows, after I initial scrap. After I drop duplicated rows, there are only 1202 rows left. Filter model S since we only want to perform analysis on model S, there is only 871 rows left. This indicates that there are 871 used model S on sale on

This plot indicates that Model S has the majority count in used Tesla market. contains 1099 rows without any duplicates.

Most used Tesla Model S is concentrated between 2013 to 2017.



Merging data set:

Merging data sets only on Tesla Model S based on Vin number helped us to make sure that we are performing analysis on the same car even though the data sets are from different websites. Merging data set with 871 rows on and 1099 rows on, 710 rows of unique data set is left.

Exploratory Data Analysis:

Factors that affect the price

I want to see if there is a difference in pricing on and has a slightly higher average price base on year.

Mileage vs Count:

Majority of Model S has mileage between 20,000 miles to 40,000 miles. Mileage vs Price:

As expected, this plot shows the trend that cars with lower mileage will be more expansive. 


This scatter plot combines price, mileage and year variable to demonstrate overall trend between these three factors. Cars with an older year and higher mileage tend to have a lower price of around $40,000. 

If you are shopping for a used Tesla Model S, this information will help you to see what you expect to get with your budget and prevent people from selling you a low mileage and newer year, cheap Model S but it turns out to be an accident car.

Price mileage and car's trim scatter plot shows the price for different trims of the car. Majority of the Model S are 75 D AWD with the price of around $50,000. 
Accident comparison:

The average price difference between a Model S with and without an accident is $5,815. 


Price, Color comparison:

Green color and brown color are cheaper on average.  The other color has the highest average price due to its specialty. Usually, the dealer charges extra money on a special colored car. Green and brown colors are much lower on average price which means these two colors are not popular.
Days on market vs price.

Not as expected, longer the days on the market does not necessary means the cars will be cheaper. 



Fleet car price comparison:

A fleet car means a car that's being used as a rental car or company car. There is not a major price difference between a fleet car and nonfleet car.


Previous owner number price comparison:

Cars with the more previous owner have a lower price. 

In conclusion, the major factors that affect Model S' price are mileage, year, has an accident or not, the number of previous owners, exterior color and car's trim level. In the previous box plot, there are many outliers. They are expected to be cars with expensive trim.

HOT CAR Analysis:

I want to know how consider a car as a "HOT CAR".  We can examine the density plot and HOT CAR ratio to all cars plot for each important variables. In the rest of the blog, I will abbreviate HOT CAR as HC and not HOT CAR as NHC.


The density for price:

HC has more widely spread distribution than NHC.

This plot shows the ratio of HC to all cars. Cars below $10,000 have a higher ratio than other price range. However, an interesting finding that in the range $70,000 to $90,000, the ratios are extremely high. 

Let's take a closer look.


Two plots above show the histogram for NHC (right) HC (left). High price HC have a much higher ratio count than NHC. Therefore a car with a higher price means this car is a popular car, does not matter what price it is, people who love it will still buy it.


Distribution of HC is more concentrated in 24,000 miles range than NHCs.

The HC ratio in high mileage columns is extremely high. This is due to its low price and few cars have such high mileage.


The density of HC and NHC has a very distinct difference. HC peaked in 2013 and 2018. The reason might be the 2013 car is cheap and the 2018 car is relatively new and cheaper than a brand new car.

The 2014 car has the lowest HC ratio. 


Brown color car is more likely to be an HC


Days on market:

I am able to obtain the days on market information from to verify if an HC will be sold in 7 days. This density plot clearly shows that most HC will be sold in a very short time. However, there are still some remained in the market for a long time. 

From description indicates that 70% of HC will be sold in 7 days, in contrast, there is only 50% of HC Model S is sold in 10 days. This 70% accuracy is probably for entire used cars market but specified for Tesla Model S.

Owners Count:

It is surprising to see that the 4th hand car is more likely to be an HC. This might be the low number of sample in the 4th hand car.


Most of the HC does not have an accident.


Fleet car is more likely to be an HC.


Used Tesla Model S Exploratory Data Analysis helps buyer to see in a big picture that what to expect when shopping a used Model S. Later, HOT CAR analysis can be used for logistic machine learning. In future work, the logistic model and tree model will be used to predict whether it is a HOT CAR or not.

About Author

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI