Data Predicting Customer Churn Using Python

Posted on Jan 10, 2022

The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

Data Predicting Customer Churn Using Python

Data Predicting Customer Churn Using Python

Data Science Introduction

Companies incur a big loss on customer churn, also, known as attrition; which is defined as the number of customers that exited and stop doing business with a company. In this project, the work flow is : Define the problem by understanding stakeholders requirements, preprocess data, visualization analysis, modeling  , conclusion and recommendations.  This project is  on Bank customer attrition.

Banks incur a big loss on customer churn; this is one of the causes of higher banks expenditure. The cost of acquiring new customers is higher than keeping those existing  and is associated with higher cost, because of onboarding in terms of time and resources. Thus, there is  loyalty benefits when you keep existing customers, since, they are less likely to exit, especially when good services and products  are offered.

Focus of project

My analysis focusses on bank customers behavior ; those that are more likely to exit the bank by closing their accounts. The goal of this project is to predict customer churn using machine learning techniques, identify potential high risk customers that will churn and analyze the model to maximize business value and solve the  business challenges which are:  main factors that cause a customer to churn, what is the chance of a customer to churn and what strategies can be implemented based on findings to reduce customer churn.

The dataset is downloaded from Kaggle (https://www.kaggle.com/mathchi/churn-for-bank-customers ). It is a Bank dataset. Using Python, I show visualizations and analyzed  four models  to predict customer attrition: K Nearest Neighbors, Logistic Regression, Random Forest and Gradient Boosting.

Data Visualization and Analysis                                                                                

Data Predicting Customer Churn Using Python

Data Predicting Customer Churn Using Python

Data Predicting Customer Churn Using Python

Data Predicting Customer Churn Using Python

 

The above Pie chart shows the distribution of the target variable (Exited); There are more retained customers than churn, 79.6% of customers stayed , while 20.4% churned. The bar chart shows customers by Geography; France has the most customers, followed by Spain with a small difference than Germany.  In terms of churn, Germany has the highest customers that churned, while Spain has the least .

               

The above Bar chat shows that, there are more male customers and female customers are more likely to churn than males. In terms of age, the histogram shows, most of the customers are between the ages 30 to 40 years, whilst the least customers are over 70+ years. The highest churn is between the ages of 40 to 50 years, and the least churn is at the extreme ages of  20 to 30 years and 70+ years. There are few customers that are 90+ years that churned.

         

The Bar chart shows, there are more customers holding two products, followed by those holding one product. The highest churn rate is by customers holding one product and the least is by customers holding four products. For customers churn based on balance, the histogram shows, there are more customers with zero balance and this category of customers also, has the highest churn rate.

                     

The bar chart shows that, there are more active customers and inactive customers are more likely to churn. From the histogram, more customers are within  2 to 9 months tenure range and the least customers have a 10 months tenure.  The highest churn is by customers that are less than two months tenure and the least churn are customers with  10 months tenure.

Dataset and preprocessing

Before fitting the models, I did data wrangling, preprocessing, exploration and visualization. The dataset used in this project is the Churn for Bank Customers  data from Kaggle https://www.kaggle.com/mathchi/churn-for-bank-customers . It comprises 10,000 observations and 14 variables. There are no missing values. The target variable is Exited, with 1 indicating churn and 0 non-churn. I dropped irrelevant variables(Row number, Customer Id and Surname),  standardized features to the same scale ,I did  encoding and one hot encoding for categorical variables to create dummy variables and also,  examined the correlation between the features; there is no high correlation between them.

Model Fitting

After preprocessing, I split the data into 80% training and 20% testing sets, next, I conduct feature scaling by normalizing variables within a range of zero and 1. I chose the following  model classifiers for this project: K Nearest  Neighbors, Logistic Regression, Random Forest and Gradient Boosting.  In modeling, I setup hyperparameter grid for tuning, tuned hyperparameters, fit model on training data, get information about best hyperparameters, use 5 folds cross validation , predict model on test data and obtain model performance metrics as discussed below.

Model1-K Nearest Neighbors

I fit the first model KNN and tuned the hyperparameters using grid searching to maximize model performance; used K=24 and did cross-validation of 5-folds to reduce model overfitting. This model gives a cross-validation training score of 79.4% and for the sample set, it is 80.7%. Since, the training and test scores are close enough,  this indicates, there is no overfitting. The Area Under the Receiver Operation Characteristics Curve (AUROC)  is 56%; this indicates that the model doesn't perform well.

Model2-Logistic Regression

With Logistic Regression, the predicted probability of the model was obtained, where 1 represents a customer churn and 0 non-churn. To get model accuracy, we look at  probability greater than 0.5 as customer churn and less than 0.5 is non-churn. The cross-validation used is 5-folds. Grid search was used to select the highest number of regularization parameter  to avoid overfitting. From the result output, training score is 79% and test score is 80%.

The Area under the Receiver Operating Characteristics Curve is 68%; this model slightly performs better than the KNN.

Model3-Random Forest

With this ensemble method, hyperparameter space is tuned with n estimators and max features. I implemented randomized searching to tune 20 hyperparameters and the best was selected with an AUROC score of 87%.  The score on the training is  86% and test is 88%. This model performs far better than the KNN and LR.

Model4-Gradient Boosting

I fit the Gradient Boosting and tuned the hyperparameters by setting up hyperparameter grid for tuning. The SGB of the training  score is 86% and the test score is 88%. This scores indicates no overfitting. The AUROC score is 88% , this is the best performing model. The feature importance below shows age and number of products are the strongest features in the prediction.

Confusion Matrix

    SGB Confusion Matrix

I use a confusion matrix on the test set to evaluate correct and incorrect predictions of customers. There are 1,544 true negatives and 215 true positives customers; the true positives and true negatives are the customers that are correctly classified. There are 169 False negatives customers that are wrongly classified, since, our model predicts that they will stay but actually they churned.

The impact is that, because the model predicts they will stay, no incentives were given to them. In this case, the Bank lost revenue because nothing is done to keep them. The false positives are 72, these are the customers that the model predicts will churn but they actually stayed. In this case, the Bank lost revenue because they offered them incentives. Predicted yes to Churn are 287 customers and no to Churn are 1,713 customers. The total error rate is 12%; this is low and indicates the model performs better.

Compare Models

 

To compare the models, we use AUROC to get an estimation of customers who churn with higher probabilities than non-churn. The top right corner on the curve shows a decision threshold of 0 and the bottom left shows a threshold of 1. The Gradient Boosting is the best-performing model that predicts customer churn with the highest ROC curve of 0.88 as shown above.

Hight risk Churn customers                              Low risk  no Churn customers

   

Using the Gradient Boosting Classifier, high risk customers to Churn are identified with probabilities greater than 0.9 and low risk customers are also identified as no Churn with probabilities. By identifying these customers, incentives can be offered to the high risk customers to motivate them to stay, thereby increasing revenue. On the other hand, with the low risk customers, no incentives will be offered but high quality products and good services can be offered to all customers to reduce the Churn rate.

Data Conclusion

Key findings in this project are: long tenure customers are less likely to churn, in active customers are more likely to churn, customers with zero balance are more likely to churn, female customers are more likely to churn, age and number of products are the strongest features to predict churn, customers with three or four products are less likely to churn , older customers above 60 years are less likely to Churn and those  young customers between 40 to 50 years are more likely to churn.

The Gradient Boosting is the model that predicts churn rate with the best ROC (0.88) given the optimal trade-off of True positive rate and False positive rate . From the confusion matrix, no  incentives was given to 169 false negative customers , and they ended up to Churn causing a loss in revenue.

Also,  incentives offered to 72 false positive customers , since model predict they will Churn but they actually stayed  reduced revenue. It is recommended that management should focus on high risk customers to Churn and offer them incentives to stay. They should also, ensure products and services offered should be of high quality, and there should be minimal false negative rate and true positive rate should be maximized.

You can find the code in project link below.

https://github.com/tedebalogun/Predicting-Customer-Churn-Using-Pytho

References:

  1. Davenport T.H ; Predictive Analytics: The power to predict who will click, buy, lie or die
  2. O'reilly J.V; Python Data Science Handbook

About Author

Robert Willoughby

I live in Columbus, Ohio; working as a Data Analyst
View all posts by Robert Willoughby >

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI