Predicting Customer Churn Using Python

Posted on Jan 10, 2022

Introduction

Companies incur a big loss on customer churn, also, known as attrition; which is defined as the number of customers that exited and stop doing business with a company. In this project, the work flow is : Define the problem by understanding stakeholders requirements, preprocess data, visualization analysis, modeling  , conclusion and recommendations.  This project is  on Bank customer attrition. Banks incur a big loss on customer churn; this is one of the causes of higher banks expenditure. The cost of acquiring new customers is higher than keeping those existing  and is associated with higher cost, because of onboarding in terms of time and resources. Thus, there is  loyalty benefits when you keep existing customers, since, they are less likely to exit, especially when good services and products  are offered.

My analysis focusses on bank customers behavior ; those that are more likely to exit the bank by closing their accounts. The goal of this project is to predict customer churn using machine learning techniques and analyze the model to maximize business value and solve the  business challenges which are:  main factors that cause a customer to churn, what is the chance of a customer to churn and what strategies can be implemented based on findings to reduce customer churn.

I use python and churn.csv dataset from Kaggle  to show visualizations and analyzed  four models  to predict customer attrition: K Nearest Neighbors, Logistic Regression, Random Forest and Gradient Boosting.

Visualization Analysis                                                                                

               

The above Pie chart shows the distribution of the target variable (Exited); There are more retained customers than churn, 79.6% of customers stayed , while 20.4% churned. The bar chart shows customers by Geography; France has the most customers, followed by Spain with a small difference than Germany.  In terms of churn, Germany has the highest customers that churned, while Spain has the least .

               

The above Bar chat shows that, there are more male customers and female customers are more likely to churn than males. In terms of age, the histogram shows, most of the customers are between the ages 30 to 40 years, whilst the least customers are over 70+ years. The highest churn is between the ages of 40 to 50 years, and the least churn is at the extreme ages of  20 to 30 years and 70+ years. There are few customers that are 90+ years that churned.

         

The Bar chart shows, there are more customers holding two products, followed by those holding one product. The highest churn rate is by customers holding one product and the least is by customers holding four products. For customers churn based on balance, the histogram shows, there are more customers with zero balance and this category of customers also, has the highest churn rate.

                     

The bar chart shows that, there are more active customers and inactive customers are more likely to churn. From the histogram, more customers are within  2 to 9 months tenure range and the least customers have a 10 months tenure.  The highest churn is by customers that are less than two months tenure and the least churn are customers with  10 months tenure.

Dataset and preprocessing

Before fitting the models, I did data wrangling, preprocessing, exploration and visualization. The dataset used in this project is the churn.csv data from Kaggle; it comprises 10,000 observations and 14 variables. There are no missing values. The target variable is Exited, with 1 indicating churn and 0 non-churn. I standardized features to the same scale , encode categorical variables, create dummy variables and examined the correlation between the features; there is no high correlation between them.

Model Fitting

After preprocessing, I split the data into 80% training and 20% testing sets. I chose the following  model classifiers for this project: K Nearest  Neighbors, Logistic Regression, Random Forest and Gradient Boosting.  In modeling, I setup hyperparameter grid for tuning, tuned hyperparameters, fit model on training data, get information about best hyperparameters, predict model on test data and obtain model performance metrics as discussed below.

Model1-K Nearest Neighbors

I fit the first model KNN and tuned the hyperparameters using grid searching to maximize model performance; used K=24 and did cross-validation of 5-folds to reduce model overfitting. This model gives a cross-validation training score of 79.4% and for the sample set, it is 80.7%. Since, the training and test scores are close enough,  this indicates, there is no overfitting. The Area Under the Receiver Operation Characteristics Curve (AUROC)  is 56%; this indicates that the model doesn't perform well.

Model2-Logistic Regression

With Logistic Regression, the predicted probability of the model was obtained, where 1 represents a customer churn and 0 non-churn. To get model accuracy, we look at  probability greater than 0.5 as customer churn and less than 0.5 is non-churn. The cross-validation used is 5-folds. Grid search was used to select the highest number of regularization parameter  to avoid overfitting. From the result output, training score is 79% and test score is 80%. The Area under the Receiver Operating Characteristics Curve is 68%; this model slightly performs better than the KNN.

Model3-Random Forest

With this ensemble method, hyperparameter space is tuned with n estimators and max features. I implemented randomized searching to tune 20 hyperparameters and the best was selected with an AUROC score of 87%.  The score on the training is  86% and test is 88%. This model performs far better than the KNN and LR.

Model4-Gradient Boosting

I fit the Gradient Boosting and tuned the hyperparameters by setting up hyperparameter grid for tuning. The SGB of the training  score is 86% and the test score is 88%. This scores indicates no overfitting. The AUROC score is 88% , this is the best performing model. The feature importance below shows age and number of products are the strongest features in the prediction.

Confusion Matrix

Predicted Error rate
No Yes
Actual No 1539 77 0.048
Yes 168 216 0.56
Total 1,707 293 0.12

I use  confusion matrix on the test set to evaluate correct and incorrect predictions of customers. There are 1,539 true positives and 216 true negatives; the true positives and true negatives are the customers that are correctly classified. There are 168 False negatives customers that are wrongly classified, since, our model predicts that they will stay but actually they churned.  The impact is that, because  the model predicts they will stay, no incentives was given to them. In this case, the Bank lost revenue because nothing is done to keep them. The false positives are 77, these are the customers that the model predicts will churn but they actually stayed. In this case, the Bank lost revenue because they offered them incentives. The total error rate is 12%,  this is low and indicates the model performs better.

Compare Models

                 

To measure the performance of the models, we use AUROC to get an estimation of probability of customers who churn with higher probabilities than non-churn. The top right corner on the curve shows a decision threshold of 0 and the bottom left shows a threshold of 1. The Gradient Boosting is the best performing model that predicts customer churn with the highest AUROC curve of 0.88 as shown above.

Conclusion

Key findings in this project are: long tenure customers are less likely to churn, inactive customers are more likely to churn, customers with zero balance are more likely to churn, female customers are more likely to churn, age and number of products are the strongest features to predict churn and customers with three or four products are less likely to churn and those customers between 40 to 50 years are more likely to churn. The Gradient Boosting classifier is the best model that predicts churn rate with 88% accuracy. From the confusion matrix, management shouldn't give incentives to 168 true negative customers and incentives offered to 77 false positive customers will reduce revenue. Management should also, ensure products and services offered should be of high quality, and there should be minimal false negative rate and true positive rate should be maximized.

You can find the code in project link below.

https://github.com/tedebalogun/Predicting-Customer-Churn-Using-Pytho

 

About Author

Robert Willoughby

I live in Columbus, Ohio; working as a Data Analyst
View all posts by Robert Willoughby >

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp