Predicting Customer Churn Using Python
Companies incur a big loss on customer churn, also, known as attrition; which is defined as the number of customers that exited and stop doing business with a company. In this project, the work flow is : Define the problem by understanding stakeholders requirements, preprocess data, visualization analysis, modeling , conclusion and recommendations. This project is on Bank customer attrition. Banks incur a big loss on customer churn; this is one of the causes of higher banks expenditure. The cost of acquiring new customers is higher than keeping those existing and is associated with higher cost, because of onboarding in terms of time and resources. Thus, there is loyalty benefits when you keep existing customers, since, they are less likely to exit, especially when good services and products are offered.
My analysis focusses on bank customers behavior ; those that are more likely to exit the bank by closing their accounts. The goal of this project is to predict customer churn using machine learning techniques and analyze the model to maximize business value and solve the business challenges which are: main factors that cause a customer to churn, what is the chance of a customer to churn and what strategies can be implemented based on findings to reduce customer churn.
I use python and churn.csv dataset from Kaggle to show visualizations and analyzed four models to predict customer attrition: K Nearest Neighbors, Logistic Regression, Random Forest and Gradient Boosting.
The above Pie chart shows the distribution of the target variable (Exited); There are more retained customers than churn, 79.6% of customers stayed , while 20.4% churned. The bar chart shows customers by Geography; France has the most customers, followed by Spain with a small difference than Germany. In terms of churn, Germany has the highest customers that churned, while Spain has the least .
The above Bar chat shows that, there are more male customers and female customers are more likely to churn than males. In terms of age, the histogram shows, most of the customers are between the ages 30 to 40 years, whilst the least customers are over 70+ years. The highest churn is between the ages of 40 to 50 years, and the least churn is at the extreme ages of 20 to 30 years and 70+ years. There are few customers that are 90+ years that churned.
The Bar chart shows, there are more customers holding two products, followed by those holding one product. The highest churn rate is by customers holding one product and the least is by customers holding four products. For customers churn based on balance, the histogram shows, there are more customers with zero balance and this category of customers also, has the highest churn rate.
The bar chart shows that, there are more active customers and inactive customers are more likely to churn. From the histogram, more customers are within 2 to 9 months tenure range and the least customers have a 10 months tenure. The highest churn is by customers that are less than two months tenure and the least churn are customers with 10 months tenure.
Dataset and preprocessing
Before fitting the models, I did data wrangling, preprocessing, exploration and visualization. The dataset used in this project is the churn.csv data from Kaggle; it comprises 10,000 observations and 14 variables. There are no missing values. The target variable is Exited, with 1 indicating churn and 0 non-churn. I standardized features to the same scale , encode categorical variables, create dummy variables and examined the correlation between the features; there is no high correlation between them.
After preprocessing, I split the data into 80% training and 20% testing sets. I chose the following model classifiers for this project: K Nearest Neighbors, Logistic Regression, Random Forest and Gradient Boosting. In modeling, I setup hyperparameter grid for tuning, tuned hyperparameters, fit model on training data, get information about best hyperparameters, predict model on test data and obtain model performance metrics as discussed below.
Model1-K Nearest Neighbors
I fit the first model KNN and tuned the hyperparameters using grid searching to maximize model performance; used K=24 and did cross-validation of 5-folds to reduce model overfitting. This model gives a cross-validation training score of 79.4% and for the sample set, it is 80.7%. Since, the training and test scores are close enough, this indicates, there is no overfitting. The Area Under the Receiver Operation Characteristics Curve (AUROC) is 56%; this indicates that the model doesn't perform well.
With Logistic Regression, the predicted probability of the model was obtained, where 1 represents a customer churn and 0 non-churn. To get model accuracy, we look at probability greater than 0.5 as customer churn and less than 0.5 is non-churn. The cross-validation used is 5-folds. Grid search was used to select the highest number of regularization parameter to avoid overfitting. From the result output, training score is 79% and test score is 80%. The Area under the Receiver Operating Characteristics Curve is 68%; this model slightly performs better than the KNN.
With this ensemble method, hyperparameter space is tuned with n estimators and max features. I implemented randomized searching to tune 20 hyperparameters and the best was selected with an AUROC score of 87%. The score on the training is 86% and test is 88%. This model performs far better than the KNN and LR.
I fit the Gradient Boosting and tuned the hyperparameters by setting up hyperparameter grid for tuning. The SGB of the training score is 86% and the test score is 88%. This scores indicates no overfitting. The AUROC score is 88% , this is the best performing model. The feature importance below shows age and number of products are the strongest features in the prediction.
I use confusion matrix on the test set to evaluate correct and incorrect predictions of customers. There are 1,539 true positives and 216 true negatives; the true positives and true negatives are the customers that are correctly classified. There are 168 False negatives customers that are wrongly classified, since, our model predicts that they will stay but actually they churned. The impact is that, because the model predicts they will stay, no incentives was given to them. In this case, the Bank lost revenue because nothing is done to keep them. The false positives are 77, these are the customers that the model predicts will churn but they actually stayed. In this case, the Bank lost revenue because they offered them incentives. The total error rate is 12%, this is low and indicates the model performs better.
To measure the performance of the models, we use AUROC to get an estimation of probability of customers who churn with higher probabilities than non-churn. The top right corner on the curve shows a decision threshold of 0 and the bottom left shows a threshold of 1. The Gradient Boosting is the best performing model that predicts customer churn with the highest AUROC curve of 0.88 as shown above.
Key findings in this project are: long tenure customers are less likely to churn, inactive customers are more likely to churn, customers with zero balance are more likely to churn, female customers are more likely to churn, age and number of products are the strongest features to predict churn and customers with three or four products are less likely to churn and those customers between 40 to 50 years are more likely to churn. The Gradient Boosting classifier is the best model that predicts churn rate with 88% accuracy. From the confusion matrix, management shouldn't give incentives to 168 true negative customers and incentives offered to 77 false positive customers will reduce revenue. Management should also, ensure products and services offered should be of high quality, and there should be minimal false negative rate and true positive rate should be maximized.
You can find the code in project link below.