Predicting Loans with Positive ROI

DongHwi Kim
Lily Kuo
Avatar
Avatar
, , and
Posted on Dec 26, 2019

Introduction

San Francisco-based LendingClub is a peer-to-peer lending institution founded on Facebook in 2007. It counts as a digital-first investment platform with its algorithms mainly focused on matching investors with private loans. The company receives income through 1% - 6% origination fees to the borrower and 1% service fees to the investor.

  • Borrower requests unsecured installment loans of 3 or 5 years between $1,000 and $40,000 by posting a qualifying loan request on the Site.
  • Client buys fractional loans as an unsecured lender after reviewing multiple criteria through a search function.
  • Multiple clients share individual notes with a $25 minimum buy-in and receive monthly principal and interest payments.

Brief History of LendingClub

2007: Founded on Facebook, LendingClub received $10.26 million in a Series A funding round in August 2007.

2007 - 2014: PRE-IPO GROWTH, In Nov 2012, LendingClub surpassed $1 billion in loans issued since inception and announced cash flow positive.

12/10/2014: IPO, Viewed as a pioneer in fintech industry, the company was the largest technology IPO of 2014 in the US, raising $1 billion.

2016: CEO SCANDAL, Founder Renaud Laplanche was ousted as CEO for issues with data integrity and contract approval monitoring and review processes.

Dec 2018: FRAUD CHARGES, LendingClub was ordered to pay a $4 million penalty by the SEC for misrepresenting the quality of some of its loans.

Lending Club Issues

Though viewed as the gold standard in fintech startups, LendingClub experienced problems in early 2016, with difficulties in attracting investors, a scandal over some of the firm's loans. The scandal has taken a major toll on the company’s NYSE stock listing, with shares trading at around 10% of the IPO price in December 2019.

Part 1: Loan Approval Criterion Reverse Engineering

Rejected Loans

  • Observations: 20M (2007 – 2018Q4)

  • 9 Features: Before Loan Issued

  • Basic Applicant Information

Accepted Loans

  • Observations: 2M (2007 – 2018Q4)

  • 151 Features: Before / After Loan Issued

  • Basic Applicant Information | Detailed Applicant Information

  • Loan Details | Payments | Loan Status | Settlement | Hardship 

Merge & Process

  1. Selected 7 Features: Loan Amount | Purpose | Dti | State | Employment Length | Risk Score | Application Month/Year

  2. Assumptions: Risk Score in Reject = Average FICO score in Accept | Application Month in Reject = Loan Issued Month in Accept

  3. Feature Engineering: Impute missing values (Dti/Employment Length/Risk score) | Dummify: Loan Purpose (classify to 8 categories ) / State

Challenges

  • Data size beyond coverage of supervised / unsupervised algorithms

  • Highly imbalanced classification for supervised techniques

  • Unmatched data standard between rejected and accepted loans

Solutions

  • Start with small samples and push the size up to 200,000

  • Down sample majority class / Try different models / Logistic balanced argument

  • Make simplifying assumptions

Unsupervised Learning - Clustering

Supervised Models 

Models with higher accuracy have more type I error than type II error.

Feature Importance

Credit Policy Analysis Based on reverse engineering

  • Risk Score: Minimum FICO score of 660

  • Employment Length: 60% of Accepted Loan Application has at least 5 years of employment length. Only 9% of Accepted Loan Application has less than 1 year of employment length, and it has average FICO score of 700.

  • DTI: More than 98% Accepted loans with Debt-to-income ratio of below 40%

  • Purpose: For accepted loan application, more than 50% is for Debt Consolidation; more than 20% is for Credit Card. Loans with Other Purpose have higher possibility of being rejected 

  • Loan Amount: Accepted loans have higher average amount than rejected loans 

  • Conclusion:

    • LendingClub’s credit policy is straight-forward.

    • Prediction models can achieve high accuracy with limited features.

Return of Investment

  • Modern Portfolio Theory was improvised for assembling portfolio of assets. It is a formalization and extension of diversification of investing, consider each investment contributes to a portfolio’s overall risk and return

  • MPT assumes that investors risk averse,  given two portfolios that offers. The same expected return, investors will prefer the less risky one.

  • In general, the expected return is calculated by:

  • Distribution of ROI Curve via Monte Carlo Simulation

  • 20% of probability investors will end up with a negative return

Predicting Fully Paid vs Charged-Off/Default Loans

In order to pick out loans with positive returns, the highest factor are picking out loans that first pays off the loan completely therefore machine learning models were used predict which loans would come to full payment versus charged off.

The models that were used are listed below with its accuracy and precision scores.

Random Forest Classifier

  • Accuracy: 81%
  • Precision: 82.2%
  • n_estim = 500
  • max_feature= .25
  • balanced weight

Kneighbor Classifier

  • Accuracy: 77.6%
  • Precision: 81.7%
  • n = 5

Neural Network

  • Accuracy: 65%
  • Precision: 80%
  • hidden layers(10, 10,5,3)
  • balanced weight

Logistic Regression

  • Accuracy: 79.9%
  • Precision: 80.8%
  • C= 100
  • L2 Regularization

Model Conclusion

  • Random Forest seemed to have the best overall performance

  • The general trend looks as though if precision increases, accuracy decreases and vice versa

  • There are no determination on which model performs the best as the model used would depend on the goal achieved depending on higher accuracy or precision

The Trade Offs

It is important to understand how expensive False Negatives are compared with False Positives. For lending club, False Negatives are much more expensive as the amount of loss that 1 False Negative could bring far outweighs its potential return.  Therefore higher precision models might be more favored in return for lower accuracy.

 

Some models were able to reach 90% precision but sacrificed accuracy as it would drop to 70%.

In order to build a more sustainable business model, it is better to generate less revenue for future growth and stability by increasing precision of the loans and increasing the potential ROI of the loans

Future Works

Work on NLP for loan description or social media if information is available to increase prediction accuracy

Run the models for each sub-grade to increase accuracy and precision for the loans

 

About Authors

DongHwi Kim

DongHwi Kim

James (Dong-Hwi) Kim is NYC Data Science Fellow with a Bachelor's Degree in Applied Mathematics and Statistics from Stony Brook University. Before coming to NYCDSA James was a CEO and founder for a startup where he found a...
View all posts by DongHwi Kim >
Lily Kuo

Lily Kuo

Liyi (Lily) is a Data Science Fellow at the New York City Data Science Academy with a Masters in Biomedical Engineering and Education. She is aspired to be a data scientist who executes data-driven strategies to increase efficiency,...
View all posts by Lily Kuo >
Avatar

Huimin Ou

Huimin Ou is NYC Data Science Fellow with a Bachelor's Degree in Industrial Engineering from University of Wisconsin-Madison. Before coming to NYCDSA, Huimin is an industrial engineer with experience in business data analytics, process improvement and project management....
View all posts by Huimin Ou >
Avatar

Hanxiao Zhang

Hanxiao(Mia) Zhang is NYC Data Science Fellow with a Master's Degree in Finance from Fordham University. Before enrolling in the NYCDSA, she worked in the finance and business sector for over 4 years with extensive client interactions on...
View all posts by Hanxiao Zhang >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp