Increasing ROI for Investors Using LendingClub as an Example
The skills the authors demonstrated here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
INTRODUCTION
There are many avenues to investing which can be research through data. Each type of investment vehicle comes with a different risk-return profile. For the most risk-averse, higher interest savings accounts or CDs guarantee safety. Investors who seek a greater return and are willing to take on the risk of loss would likely invest in the stock market or mutual funds that invest in stocks, real estate, or REITs.
Those who seek alternative avenues to boost their investment returns or to diversify their portfolios beyond those standard investments, there’s also the option of capitalizing on the return of peer-to-peer loan investments which can yield a decent return while mitigating unnecessary risk.
However, investing in loans may not be the easiest to navigate with so many aspects of the loan, and the borrower to consider that the investor must understand before making an educated judgment. This analysis utilizes past data from LendingClub, the most popular online peer-to-peer lending platform since 2006 as an example to study return on investments.
BACKGROUND
Each loan is assigned a grade based on the borrower's FICO score, income, house ownership, and various other data the applicant fills out. We can see how many loans have a particular grade in Graph 1.
It can be observed that as the quality of the loan deteriorates, the chance of default increases. The investors would, therefore, require a higher interest rate to compensate for the possible loss of their capital. From the above graphs, we can predict that each loan’s return on investment can greatly differ depending on its grade.
Sometimes even the purpose of the loans can have an impact on returns due to different risk levels. For example, LendingClub rates are usually lower than normal credit card interest rates, making it easier for the borrowers to pay off their debt. On the other hand, rates are most likely higher at LendingClub than a typical mortgage or small business loan from traditional banks, hence chances of default would increase.
The underlying reasons for these patterns could have different impacts on the loan outcomes, but a simple analysis of the data may not show a complete picture. If an investor is able to get their hands on a credit report, there will be many features for them to take into consideration. In the current dataset, there are more than 150 features, including the grades, purposes, terms. We aim to reduce the complexity of selecting more profitable loan investments for new (and even experienced) investors.
PURPOSE
Our aim is to increase the ROI for investors who decide to invest in LendingClub loans by using machine learning models. Thus we will be able to save investors time, money, and stress while helping them make better decisions for their future.
The dataset we are using contains information for over 2 million loans and the 151 features that affect the outcome of the loan. Based on these numbers alone, it is apparent that classifying these loans can be a tricky and time-consuming task for a less-experienced investor. There are loans that look like they would do well and provide a decent ROI but end up defaulting. There are also loans that are paid off too early and do not provide a decent ROI.
Consequently, it is no simple matter to predict which loans will pay off the most even based on unique features. Our models predict which loans will end up being paid off and are more likely to provide a decent return. In order to reduce work, we used classification and confusion matrices to give a sense of direction.
METHOD
DATASET
First, we split data evenly into train and predict sets using only completed loans (loans with a definitive outcome of having been fully paid off or defaulted). Training data was used for the entire process, but the prediction dataset was only used to confirm our results.
BALANCING
After splitting the dataset, we balanced each dataset since labels were not split evenly. We ran our models after converting some features to numeric type to ease the load on RAM usage. Then we extensively engineered some features (for example ratios of income to monthly payment or income to loan amount) and realized that there was not much improvement in the ROC-AUC scores, which we used to measure accuracy.
Next, we reduced the features to create a minimal dataset to see if there were any glaring differences in the ROC-AUC scores. The reduced dataset performed similarly in terms of accuracy to the larger datasets with extensive feature engineering, scoring only about 1% less than the original dataset and the dataset with extensive feature engineering. The train and test scores for the 3 datasets are shown in Table 1.
FINAL DATASET
We were able to simplify the final dataset to include only 8 features. Those features are as follows: annual income, FICO Score, duration of the loan, the loan amount, homeownership status, the interest rate of the loan, the utilization rate of all credit lines, and debt-to-income ratio.
MODEL COMPARISON
To compare the performance of different machine learning models and to find the best model, we tested and tuned 3 different machine learning models including random forest classification, gradient boosting classification, and XGBoost classification.
CONFUSION MATRIX
We were able to achieve a ROC-AUC score in the range of 65-71% for our train sets. After checking the ROC-AUC scores, we used confusion matrices to see how possible outcomes were split.
Table 2 shows the confusion matrix of an un-tuned Gradient Boosting Classifier for our prediction dataset. Our labels were 0 to indicate paid off and 1 to indicate default. A total of 674,049 loans were analyzed, which is the sum of all quadrants. The top left quadrant of our confusion matrix is our true positive (TP), where we see 361,451 loans (53.6%) both actually paid off and predicted to be paid off by our models.
The top right quadrant of 177,918 loans (26.4%) is our false positive (FP), which was predicted to default but was actually paid off. Our true negative (TN) quadrant on the bottom right shows 85,225 loans (12.6%) that were defaulted and that our model predicted to default as well. The bottom left quadrant shows the false negative (FN) of 49,455 loans (7.3%) that our model predicted being paid off but the loan ended up actually defaulting.
This results in 66% overall accuracy, 65% sensitivity, and 66% specificity, where the sensitivity of a classifier is defined as TP/(TP+FN) and specificity is defined as TN/(TN+FP).
SIMULATION
We simulated the behavior of an investor looking to invest in 4 loans at random. The simulated investor would have 6 portfolios ranging from $1 - 60,000 in increments of $9,999. We applied the 3 best ML models as well as naive random sampling where no models were used in selecting the loans. We ran 10,000 iterations in this simulation to get the weighted average ROI.
RESULTS
In Graph 4, we can see a clear increase of ROI with the investor's portfolio when using our models compared to the naive sampling using no model. An increase of 20 - 120% in profit between using our model vs naive sampling was observed. The higher the investment, the more critical the need for implementing the model to ensure ROI does not decrease.
The decreasing trend of the ROI for naive sampling makes sense because higher investment ranges would require on average larger loans to be picked. As we can see from the boxplot in Graph 5, larger loans are often associated with lower grades, ie. grade F and G, which is correlated with a higher rate of default. Without any models to aid in the loan selection process, naive sampling would pick those loans with lower grades at random, increasing the rate of default. Overall, this would equate to decreasing the return on investment.
FUTURE IMPLEMENTATIONS
Until now, we only figured out a way to ensure that investors would be able to see an increase in returns. An easy-to-use app could be created to allow for user input of the investment range, the number of loans, the loan grades they are comfortable with, and the purposes of the loans. We can also revisit feature engineering and focus more on the features that have a greater impact on model prediction to increase our models’ performance to provide even better results for investors.
About Authors
