Cars.com/Cargurus.com used Tesla Model S web scraping
Introduction:
Electrically powered vehicle have become more popular due to less cost in maintenance and gasoline, good for the environment, driving range is similar to the traditional combustion vehicle and fun to drive based on the data collected. Tesla is one of the most popular and desirable brands that consumers want to have. However, most people are not able to afford the high initial cost of a brand new Tesla. Another alternative is to find a used Tesla.
In this project, a Python spider is implemented to scrap used car websites: Cars.com and Cargurus.com. Later, an explanatory data analysis will be performed to analyze which Tesla Model S is best suitable for one's need.
Web scraping:
Cars.com list page provides much important information for machine learning such as original price, current price, trim of the car, drivetrain, etc. Python spider can easily scrap the front page information. However, the Vin number can only be obtained after I clicked into the vehicle page.
The same thing goes for Cargurus.com. There is some useful information that can only be obtained after I clicked into the car page. For example, days on market, has accidents or not, how many previous owners the car has?
"Hot Car Prediction"
In cars.com, a car will be indicated as "HOT CAR" if cars.com predicts that with 70% of chance the car will be sold in seven days after posting.
At the end of this project, a simple machine learning will be performed to predict if a car is HOT CAR or not base on the information I scraped from both websites.
Data Cleaning:
Cars.com data set contains 2115 rows, after I initial scrap. After I drop duplicated rows, there are only 1202 rows left. Filter model S since we only want to perform analysis on model S, there is only 871 rows left. This indicates that there are 871 used model S on sale on cars.com. 
This plot indicates that Model S has the majority count in used Tesla market.
Cargurus.com contains 1099 rows without any duplicates.
Most used Tesla Model S is concentrated between 2013 to 2017.
Merging data set:
Merging data sets only on Tesla Model S based on Vin number helped us to make sure that we are performing analysis on the same car even though the data sets are from different websites. Merging data set with 871 rows on cars.com and 1099 rows on cargurus.com, 710 rows of unique data set is left.
Exploratory Data Analysis:
Factors that affect the price
I want to see if there is a difference in pricing on cars.com and cargurus.com.
Cars.com has a slightly higher average price base on year.
Mileage vs Count:
Majority of Model S has mileage between 20,000 miles to 40,000 miles. 
Mileage vs Price:
As expected, this plot shows the trend that cars with lower mileage will be more expansive.
This scatter plot combines price, mileage and year variable to demonstrate overall trend between these three factors. Cars with an older year and higher mileage tend to have a lower price of around $40,000.
If you are shopping for a used Tesla Model S, this information will help you to see what you expect to get with your budget and prevent people from selling you a low mileage and newer year, cheap Model S but it turns out to be an accident car.
Price mileage and car's trim scatter plot shows the price for different trims of the car. Majority of the Model S are 75 D AWD with the price of around $50,000. 
Accident comparison:
The average price difference between a Model S with and without an accident is $5,815.
Price, Color comparison:
Green color and brown color are cheaper on average. The other color has the highest average price due to its specialty. Usually, the dealer charges extra money on a special colored car. Green and brown colors are much lower on average price which means these two colors are not popular.
Days on market vs price.
Not as expected, longer the days on the market does not necessary means the cars will be cheaper.
Fleet car price comparison:
A fleet car means a car that's being used as a rental car or company car. There is not a major price difference between a fleet car and nonfleet car.
Previous owner number price comparison:
Cars with the more previous owner have a lower price.
In conclusion, the major factors that affect Model S' price are mileage, year, has an accident or not, the number of previous owners, exterior color and car's trim level. In the previous box plot, there are many outliers. They are expected to be cars with expensive trim.
HOT CAR Analysis:
I want to know how cars.com consider a car as a "HOT CAR". We can examine the density plot and HOT CAR ratio to all cars plot for each important variables. In the rest of the blog, I will abbreviate HOT CAR as HC and not HOT CAR as NHC.
Price:
The density for price:
HC has more widely spread distribution than NHC.
This plot shows the ratio of HC to all cars. Cars below $10,000 have a higher ratio than other price range. However, an interesting finding that in the range $70,000 to $90,000, the ratios are extremely high.
Let's take a closer look.
Two plots above show the histogram for NHC (right) HC (left). High price HC have a much higher ratio count than NHC. Therefore a car with a higher price means this car is a popular car, does not matter what price it is, people who love it will still buy it.
Mileage:
Distribution of HC is more concentrated in 24,000 miles range than NHCs.
The HC ratio in high mileage columns is extremely high. This is due to its low price and few cars have such high mileage.
Year:
The density of HC and NHC has a very distinct difference. HC peaked in 2013 and 2018. The reason might be the 2013 car is cheap and the 2018 car is relatively new and cheaper than a brand new car.
The 2014 car has the lowest HC ratio.
Color:
Brown color car is more likely to be an HC
Trim:
Days on market:
I am able to obtain the days on market information from cargurus.com to verify if an HC will be sold in 7 days. This density plot clearly shows that most HC will be sold in a very short time. However, there are still some remained in the market for a long time.
From cars.com description indicates that 70% of HC will be sold in 7 days, in contrast, there is only 50% of HC Model S is sold in 10 days. This 70% accuracy is probably for entire used cars market but specified for Tesla Model S.
Owners Count:
It is surprising to see that the 4th hand car is more likely to be an HC. This might be the low number of sample in the 4th hand car.
Accident:
Most of the HC does not have an accident.
Fleet:
Fleet car is more likely to be an HC.
Conclusion:
Used Tesla Model S Exploratory Data Analysis helps buyer to see in a big picture that what to expect when shopping a used Model S. Later, HOT CAR analysis can be used for logistic machine learning. In future work, the logistic model and tree model will be used to predict whether it is a HOT CAR or not.
About Author
