Predicting Customer Churn Using Python
The skills I demoed here are taught in NYC Data Science Academy's Data Science with Machine Learning bootcamp..
Data Predicting Customer Churn Using Python
Data Science Introduction
Customer churn, also known as attrition, is the bane of businesses that struggle to not just win but retain customers to maximize their lifetime value. Churn is defined as the number of customers that exited and stop doing business with a company. For this project, we studied bank customer attrition. The work flow is: Define the problem by understanding stakeholders requirements, preprocess and data engineering, visualization analysis, modeling , conclusion and recommendations.
Customer churn is one of the causes of higher banks expenditure and a major contributor to loss of revenue. The cost of acquiring new customers is higher than keeping existing ones due to the cost of time and resources used for onboarding. Fostering loyalty pay because it helps customers feel connected to the business, so they are less likely to churn.
Focus of project
My analysis focuses on bank customers behavior, those that are more likely to exit the bank by closing their accounts. The goal of this project is to predict customer churn using machine learning techniques, identify potential high risk customers that will churn and analyze the model to maximize business value and solve the business challenges which are: identifying the main factors that cause a customer to churn, the odds of churn for particular customers, and what strategies can be implemented based on findings to reducer churn in the future.
The dataset is downloaded from Kaggle (https://www.kaggle.com/mathchi/churn-for-bank-customers ). It is a Bank dataset. Using Python, I show visualizations and analyzed four models to predict customer attrition: K Nearest Neighbors, Logistic Regression, Random Forest and Gradient Boosting.
Data Visualization and Analysis
Data Predicting Customer Churn Using Python
Data Predicting Customer Churn Using Python
The above Pie chart shows the distribution of the target variable (Exited); There are more retained customers than churn, 79.6% of customers stayed , while 20.4% churned. The bar chart shows customers by Geography; France has the most customers, followed by Spain with a small difference than Germany. In terms of churn, Germany has the highest levels of customer, while Spain shows the lowest.
The above Bar chart shows that there are more male customers, and female customers are more likely to churn than males. In terms of age, the histogram shows, most of the customers are between the ages 30 to 40 years, and the lowest number of customers are over 70 years-old. The highest churn is between the ages of 40 to 50 years, and the least churn is at the extreme ages of 20 to 30 years and 70+ years. There are few customers that are 90+ years that churned.
The Bar chart shows, there are more customers holding two products, followed by those holding one product. The highest churn rate is by customers holding one product and the least is by customers holding four products. For customers churn based on balance, the histogram shows, there are more customers with zero balance; this category of customers also has the highest churn rate.
The bar chart shows that there are more active customers, and inactive customers are more likely to churn. From the histogram, more customers are within 2 to 9 months tenure range, and the least customers have a 10 months tenure. The highest churn is by customers that have less than two months tenure, and the least churn are customers with 10 months tenure.
Dataset and preprocessing
Before fitting the models, I did data wrangling, preprocessing, exploration and visualization. The dataset used in this project is the Churn for Bank Customers data from Kaggle https://www.kaggle.com/mathchi/churn-for-bank-customers. It comprises 10,000 observations and 14 variables. There are no missing values. The target variable is Exited, with 1 indicating churn and 0 non-churn. I dropped irrelevant variables(Row number, Customer Id and Surname), standardized features to the same scale. I did encoding and one hot encoding for categorical variables to create dummy variables and also examined the correlation between the features. There is no high correlation between them.
Model Fitting
After preprocessing, I split the data into 80% training and 20% testing sets, next, I conduct feature scaling by normalizing variables within a range of zero and 1. I chose the following model classifiers for this project: K Nearest Neighbors, Logistic Regression, Random Forest and Gradient Boosting. In modeling, I setup hyperparameter grid for tuning, tuned hyperparameters, fit model on training data, get information about best hyperparameters, use 5 folds cross validation , predict model on test data and obtain model performance metrics as discussed below.
Model1-K Nearest Neighbors
I fit the first model KNN and tuned the hyperparameters using grid searching to maximize model performance; used K=24 and did cross-validation of 5-folds to reduce model overfitting. This model gives a cross-validation training score of 79.4% and for the sample set, it is 80.7%. Since the training and test scores are close enough, this indicates there is no overfitting. The Area Under the Receiver Operation Characteristics Curve (AUROC) is 56%, which indicates that the model doesn't perform well.
Model2-Logistic Regression
With Logistic Regression, the predicted probability of the model was obtained, where 1 represents a customer churn and 0 non-churn. To get model accuracy, we look at probability greater than 0.5 as customer churn and less than 0.5 is non-churn. The cross-validation used is 5-folds. Grid search was used to select the highest number of regularization parameter to avoid overfitting. From the result output, training score is 79% and test score is 80%. The Area under the Receiver Operating Characteristics Curve is 68%, making this model's performance slightly better than that of the KNN.
Model3-Random Forest
With this ensemble method, hyperparameter space is tuned with n estimators and max features. I implemented randomized searching to tune 20 hyperparameters and the best was selected with an AUROC score of 87%. The score on the training is 86% and 88% for the test . This model performed far better than the KNN and LR.
Model4-Gradient Boosting
I fit the Gradient Boosting and tuned the hyperparameters by setting up hyperparameter grid for tuning. The SGB of the training score is 86% and the test score is 88%. This scores indicates no overfitting. The AUROC score is 88%, making this is the best performing model. The feature importance below shows age and number of products are the strongest features in the prediction.
Confusion Matrix
I use a confusion matrix on the test set to evaluate correct and incorrect predictions of customers. There are 1,544 true negatives and 215 true positives customers; the true positives and true negatives are the customers that are correctly classified. There are 169 False negatives customers that are wrongly classified, since our model predicts that they will stay, though they churned.
The impact is that, because the model predicts they will stay, no incentives were given to them. In this case, the Bank lost revenue because nothing is done to keep them. The false positives are 72, these are the customers that the model predicts will churn but they actually stayed. In this case, the Bank lost revenue because they offered them incentives. Predicted yes to Churn are 287 customers and no to Churn are 1,713 customers. The total error rate is 12%; this is low and indicates the model performs better.
Compare Models
To compare the models, we use AUROC to get an estimation of customers who churn with higher probabilities than non-churn. The top right corner on the curve shows a decision threshold of 0 and the bottom left shows a threshold of 1. The Gradient Boosting is the best-performing model that predicts customer churn with the highest ROC curve of 0.88 as shown above.
Hight risk Churn customers Low risk no Churn customers
Using the Gradient Boosting Classifier, high risk customers to Churn are identified with probabilities greater than 0.9 and low risk customers are also identified as no Churn with probabilities. By identifying these customers, incentives can be offered to the high risk customers to motivate them to stay, thereby increasing revenue. On the other hand, with the low risk customers, no incentives will be offered, but high quality products and good services can be offered to all customers to reduce the Churn rate.
Data Conclusion
Key findings in this project are: long tenure customers are less likely to churn, inactive customers are more likely to churn, customers with zero balance are more likely to churn, female customers are more likely to churn, age and number of products are the strongest features to predict churn, customers with three or four products are less likely to churn, older customers above 60 are less likely to churn and those young customers between 40 to 50 years are more likely to churn.
The Gradient Boosting is the model that predicts churn rate with the best ROC (0.88) given the optimal trade-off of True positive rate and False positive rate . From the confusion matrix, no incentives was given to 169 false negative customers , and they ended up churning, causing a loss in revenue.
Also, incentives offered to 72 false positive customers , since model predict they will churn but they actually stayed reduced revenue. It is recommended that management should focus on high risk customers to churn and offer them incentives to stay. They should also ensure products and services offered should be of high quality. There also should be minimal false negative rate and true positive rate should be maximized.
You can find the code in project link below.
https://github.com/tedebalogun/Predicting-Customer-Churn-Using-Pytho
References:
- Davenport T.H ; Predictive Analytics: The power to predict who will click, buy, lie or die
- O'reilly J.V; Python Data Science Handbook
About Author
