Using Data to Detect Healthcare Fraud

and
Posted on Jul 15, 2021
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

Background

The National Health Care Anti-Fraud Association (NHCAA) estimates that financial losses due to healthcare fraud are in the tens of billions of dollars each year. Healthcare fraud translates to higher premiums, out-of-pocket expenses, and reduced benefits and coverage for consumers. It also leads to higher costs for employers providing benefits to employees. With this in mind, our efforts to use data science to reduce healthcare fraud could have a  significantly positive impact on the industry.

Objectives

For this project, our primary goal was to create a machine learning algorithm to classify healthcare providers as either fraudulent or non-fraudulent. Secondly, we sought to identify the most important features that could be indicators of fraud. With these features in mind, we could recommend adjustments to optimize a health insurance company's screening process. Finally, we translated our findings into a dollar amount to project business cost savings and quantify the impact we could make.

 

Data Pre-Processing & EDA

Our dataset spans a full year of healthcare records (2009) and contains four separate parts- Beneficiaries, Inpatients, Outpatients, and Providers. Around 10% of the 5,410 providers are tagged potentially fraudulent.

Using Data to Detect Healthcare Fraud

Our first step was to combine these four datasets so we could get a more complete picture while performing exploratory data analysis. Our main objective for this step was to uncover some initial insights on fraud vs. non-fraud trends.

As shown in the plot below, both types of patients are affected by fraud, but inpatients are susceptible to it at a higher rate. We can attempt to rationalize this by keeping in mind that inpatient hospital visits are typically longer and more expensive, and thus would result in a higher payout per claim to fraudulent providers.

Using Data to Detect Healthcare Fraud

We also see that fraudulent claims tend to be for older patients admitted for longer hospital stays. The plot below illustrates this trend with a higher concentration of blue dots near the top right.

Using Data to Detect Healthcare Fraud

Data on Feature Selection & Engineering

We took findings like these into consideration when approaching feature engineering. Additionally, since fraud was tagged by provider, we consolidated several claim-level attributes to match. For example, patient age became mean patient age per provider, and claim states were expressed as a state count.

Feature engineering was part of an iterative process with modeling. Overall, we cycled through five rounds of feature adjustments based on the top coefficients from models and additional EDA. Our final cleaned dataset contained 27 new features.

 

Modeling Data

We ran the following binary classification models:

  • Logistic Regression
  • Linear Discriminant Analysis
  • Gaussian Naive Bayes
  • Support Vector Machine
  • Random Forest
  • Gradient Boosting
  • XG Boost

We also tested out the following approaches to balancing our dataset:

  • Setting a "balanced weight" model parameter
  • Undersampling: Edited Nearest Neighbor & Random Undersampling
  • Oversampling: Synthetic Minority Oversampling Technique (SMOTE) & Random Oversampling

SMOTE gave us the best results after some experimentation with hyperparameter tuning.

Preliminary Data Results

Because there are infinite combinations of features, models, and hyperparameters available, the modeling process involved significant trial and error. We evaluated our preliminary models using F1, recall, and precision scores, as well as ROC curve.

These plots demonstrate that Random Forest, Gradient Boosting, and XG Boost performed the best across all scoring metrics. We then ran a grid search to further tune hyperparameters for these models.

 

Model Scoring Metrics

To evaluate our models, we assumed that false negatives are more costly than false positives since a false positive can be addressed with a relatively quick investigation, whereas a false negative remains undetected. Thus, we used F2 score to emphasize recall over precision. The following plot shows our final model scores.

In the end, XG Boost narrowly outperformed the other two models and scored close to 97% on all three metrics.

Ensembling

We made one final attempt to improve our outcome with model ensembling. To do this, we used a simple voting classifier and combined our top three models.

We also tuned parameters for our Logistic Regression in an effort to incorporate another model category, and tried incorporating it into our ensemble as well. Interestingly, both sets of ensembling results were worse than XG Boost on its own. Thus, we felt satisfied choosing XG Boost as our final model.

 

Model Insights

 Now that we had a relatively high-performing model, our next step was to evaluate feature importance for additional insights. To do this, we performed additional EDA based on the values shown in the following plot.

Many of our most important features relate to diagnostic and procedure codes. We determined that, unsurprisingly, fraudulent claims are mostly assigned to the most commonly occurring codes. We found that duplicate claims and claims with no assigned physician are often fraudulent as well.

Cost Analysis

With all these findings in mind, our final objective was to project tangible cost savings for our model. We calculated the following figures to aid in this prediction:

  • Average claim cost: $998
  • Average number of claims per provider: 103
  • Cost to investigate a claim: $58 (assuming it takes 2 hours to investigate a claim, and medical claims adjusters make $29/hour on average)
  • Savings per year = (TP x 998) - (FP + TP) x 58 - (58 x FN)

We then scaled our XG Boost Confusion Matrix to match the dataset sample size, including adjustments for oversampling, to arrive at a projected savings of $5.2 million USD for the year.

 

Final Takeaways

Our analysis strongly reinforces the notion that health insurance fraud is a very complicated, multi-faceted issue. Thankfully, we can make a significant impact by utilizing machine learning concepts and ultimately produce some impressive cost savings. We recommend the following to any health insurance company:

  • Consider establishing an extra checkpoint for when the most common diagnostic and procedure codes come up
  • Closely monitor any duplicate claims, as well as claims submitted with no physician, as these are common red flag indicators of fraud
  • Fraudulent inpatient claims are significantly more prevalent than outpatient. Thus, focus the majority of investigatory resources on inpatients.

 

Further Analysis

Finally, here are a few opportunities for additional analysis:

  • As there are infinite possibilities, we can further combine some of our models using more advanced stacking/ensembling techniques. We can also consider incorporating other combinations of models, or assigning different weights to each model.
  • With more robust data, we can try classifying our fraud into more detailed categories such as duplicate claims, "upcoding", and billing for services never rendered. Health insurance companies will likely have different approaches for addressing these types of fraud. Fraudulent providers may also demonstrate different patterns based on what type of fraud they are committing.
  • The Covid-19 pandemic has brought about a new and unique set of challenges for the healthcare industry. In fact, Covid-19 related healthcare fraud has become a category unto itself. It would be valuable to re-run analysis on more recent data to understand how fraud has evolved.

View our code on Github.

Cover image source

About Authors

Tania Ghosh

Tania Ghosh is PH.D in Genetics and Forensic Science. Having research experience of 5+ years in the field of molecular biology and Genetics. Currently shifting aim to the world of Data Science. Tania Ghosh could be reached at...
View all posts by Tania Ghosh >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI