Credit Card Fraud Detection

Avatar
Posted on Aug 4, 2017

Introduction

One of the major pain points for the credit card industry has been to accurately find potential fraudulent transactions and to process them to completion. Every credit card transaction that requires the involvement of a customer service representative costs the company money. Credit card is often a payment method of convenience for the user. The more the genuine transactions get delayed due to checks in place to prevent fraud, the greater is the chance to alienate the consumer. Similarly, credit card firms will need to built a larger customer service workforce to ensure timely processing of transactions.

About the data

The data that has been used as part of this project is from kaggle. Quoting from kaggle, “The datasets contains transactions made by credit cards in September 2013 by european cardholders. This dataset presents transactions that occurred in two days, where we have 492 frauds out of 284,807 transactions. The dataset is highly unbalanced, the positive class (frauds) account for 0.172% of all transactions.

It contains only numerical input variables which are the result of a PCA transformation. Unfortunately, due to confidentiality issues, we cannot provide the original features and more background information about the data. Features V1, V2, … V28 are the principal components obtained with PCA, the only features which have not been transformed with PCA are ‘Time’ and ‘Amount’. Feature ‘Time’ contains the seconds elapsed between each transaction and the first transaction in the dataset. The feature ‘Amount’ is the transaction Amount, this feature can be used for example-dependant cost-senstive learning. Feature ‘Class’ is the response variable and it takes value 1 in case of fraud and 0 otherwise.

Given the class imbalance ratio, we recommend measuring the accuracy using the Area Under the Precision-Recall Curve (AUPRC). Confusion matrix accuracy is not meaningful for unbalanced classification.

The dataset has been collected and analysed during a research collaboration of Worldline and the Machine Learning Group (http://mlg.ulb.ac.be) of ULB (Université Libre de Bruxelles) on big data mining and fraud detection. More details on current and past projects on related topics are available on http://mlg.ulb.ac.be/BruFence and http://mlg.ulb.ac.be/ARTML

Please cite: Andrea Dal Pozzolo, Olivier Caelen, Reid A. Johnson and Gianluca Bontempi. Calibrating Probability with Undersampling for Unbalanced Classification. In Symposium on Computational Intelligence and Data Mining (CIDM), IEEE, 2015 "

Loading the data

The Class Column contains 0 or 1 and these numerical levels are converter to a column with two levels. Genuine or Fraud

ccfraud <- read.csv('creditcard.csv')
ccfraud$Class <- as.factor(ccfraud$Class)
levels(ccfraud$Class) <- c("Genuine", "Fraud"
)

Inspecting the data, we can confirm that there are 492 fraudulent transactions in our data file.  
dim(ccfraud
)

## [1] 284807 31

ftable(ccfraud$Class)
Genuine  Fraud
##                
##   284315    492

The Mice package was used to inspect if we have any missing values. There were none.

 data(ccfraud, package = "VIM")
## Show incomplete cases
sleep[!complete.cases(ccfraud),]

library(mice)

#md.pattern(ccfraud)

There are no missing data in the dataset. ## Plots on each of the variables

cn <- colnames(ccfraud[,1:30])
print(cn

)## [1] "Time" "V1" "V2" "V3" "V4" "V5" "V6"
## [8] "V7" "V8" "V9" "V10" "V11" "V12" "V13"
## [15] "V14" "V15" "V16" "V17" "V18" "V19" "V20"
## [22] "V21" "V22" "V23" "V24" "V25" "V26" "V27"
## [29] "V28" "Amount"

vars = paste("V", 1:30, sep="")
par(mfrow=c(29,4))
library(ggplot2)
for( var in cn){
   p <- ggplot(ccfraud, aes_string(x =  var, 
               fill = ccfraud$Class, colour = ccfraud$Class, alpha = 0.2)) +
               geom_density() + xlab(paste(var)) + 
               ylab("Density") +
               ggtitle(paste(var, "Density funcion"))
   print(p)
}



## Correlation Plot This is not really important here as input data is already a PCA tranformation.

library(corrplot)
cp <- corrplot(cor(ccfraud[, 1:30])) 

Class Imbalance and approaches to solve it

The data set featured is one that has a highly unbalanced set of data with regards to the class type. 284315 genuine records vs 437 fraudulent ones. In case of unbalanced data, its better to look at precision and recall values to determine if the model is good. Also, from a business stand point, the more the false positives, the more the effort required by customer service to clear it and more the false negatives, more is the danger of losing money due to fraudulent transactions.

There are couple of approaches to solve class imbalance and train a model correctly. We will try various approaches and will see if it helps us to generate a good model.

  1. Class Weights
  2. Up Sampling
  3. Down Sampling
  4. SMOTE(Synthetic Minority Sampling Technique)

Splitting Data for our train/test

library(caret)
## Loading required package: lattice
set.seed(123)

index <- createDataPartition(ccfraud$Class, p = 0.7, list = FALSE)
train_data <- ccfraud[index, ]
test_data  <- ccfraud[-index, ]
ftable(train_data$Class)
##  Genuine  Fraud
##                
##   199021    345
ftable(test_data$Class)
##  Genuine Fraud
##               
##    85294   147
MiscFactors <- c()
pcafactors <-paste("V", 1:28, sep="")
formula = reformulate(termlabels = c(MiscFactors,pcafactors), response = 'Class')
print (formula)
## Class ~ V1 + V2 + V3 + V4 + V5 + V6 + V7 + V8 + V9 + V10 + V11 + 
##     V12 + V13 + V14 + V15 + V16 + V17 + V18 + V19 + V20 + V21 + 
##     V22 + V23 + V24 + V25 + V26 + V27 + V28
ControlParamteres <- trainControl(method = "cv",
                                  number = 10,
                                  savePredictions = TRUE,
                                  classProbs = TRUE,
                                  verboseIter = TRUE
)
str(train_data)
## 'data.frame':    199366 obs. of  31 variables:
##  $ Time  : num  0 0 1 1 2 2 4 7 7 10 ...
##  $ V1    : num  -1.36 1.192 -1.358 -0.966 -1.158 ...
##  $ V2    : num  -0.0728 0.2662 -1.3402 -0.1852 0.8777 ...
##  $ V3    : num  2.536 0.166 1.773 1.793 1.549 ...
##  $ V4    : num  1.378 0.448 0.38 -0.863 0.403 ...
##  $ V5    : num  -0.3383 0.06 -0.5032 -0.0103 -0.4072 ...
##  $ V6    : num  0.4624 -0.0824 1.8005 1.2472 0.0959 ...
##  $ V7    : num  0.2396 -0.0788 0.7915 0.2376 0.5929 ...
##  $ V8    : num  0.0987 0.0851 0.2477 0.3774 -0.2705 ...
##  $ V9    : num  0.364 -0.255 -1.515 -1.387 0.818 ...
##  $ V10   : num  0.0908 -0.167 0.2076 -0.055 0.7531 ...
##  $ V11   : num  -0.552 1.613 0.625 -0.226 -0.823 ...
##  $ V12   : num  -0.6178 1.0652 0.0661 0.1782 0.5382 ...
##  $ V13   : num  -0.991 0.489 0.717 0.508 1.346 ...
##  $ V14   : num  -0.311 -0.144 -0.166 -0.288 -1.12 ...
##  $ V15   : num  1.468 0.636 2.346 -0.631 0.175 ...
##  $ V16   : num  -0.47 0.464 -2.89 -1.06 -0.451 ...
##  $ V17   : num  0.208 -0.115 1.11 -0.684 -0.237 ...
##  $ V18   : num  0.0258 -0.1834 -0.1214 1.9658 -0.0382 ...
##  $ V19   : num  0.404 -0.146 -2.262 -1.233 0.803 ...
##  $ V20   : num  0.2514 -0.0691 0.525 -0.208 0.4085 ...
##  $ V21   : num  -0.01831 -0.22578 0.248 -0.1083 -0.00943 ...
##  $ V22   : num  0.27784 -0.63867 0.77168 0.00527 0.79828 ...
##  $ V23   : num  -0.11 0.101 0.909 -0.19 -0.137 ...
##  $ V24   : num  0.0669 -0.3398 -0.6893 -1.1756 0.1413 ...
##  $ V25   : num  0.129 0.167 -0.328 0.647 -0.206 ...
##  $ V26   : num  -0.189 0.126 -0.139 -0.222 0.502 ...
##  $ V27   : num  0.13356 -0.00898 -0.05535 0.06272 0.21942 ...
##  $ V28   : num  -0.0211 0.0147 -0.0598 0.0615 0.2152 ...
##  $ Amount: num  149.62 2.69 378.66 123.5 69.99 ...
##  $ Class : Factor w/ 2 levels "Genuine","Fraud": 1 1 1 1 1 1 1 1 1 1 ...
model.glm <- train(formula, data = train_data,method = "glm", family="binomial", trControl = ControlParamteres)
## + Fold01: parameter=none 
## - Fold01: parameter=none 
## + Fold02: parameter=none 
## - Fold02: parameter=none 
## + Fold03: parameter=none
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## - Fold03: parameter=none 
## + Fold04: parameter=none 
## - Fold04: parameter=none 
## + Fold05: parameter=none 
## - Fold05: parameter=none 
## + Fold06: parameter=none 
## - Fold06: parameter=none 
## + Fold07: parameter=none 
## - Fold07: parameter=none 
## + Fold08: parameter=none 
## - Fold08: parameter=none 
## + Fold09: parameter=none 
## - Fold09: parameter=none 
## + Fold10: parameter=none 
## - Fold10: parameter=none 
## Aggregating results
## Fitting final model on full training set
exp(coef(model.glm$finalModel))
##  (Intercept)           V1           V2           V3           V4 
## 0.0001958784 1.0465226361 0.9692444177 1.0153588236 1.9545622788 
##           V5           V6           V7           V8           V9 
## 1.0040275509 0.8729683757 0.9622179694 0.8190839236 0.8090811225 
##          V10          V11          V12          V13          V14 
## 0.4658158788 1.0298687876 0.9696172856 0.7447382118 0.5935635399 
##          V15          V16          V17          V18          V19 
## 0.9115946086 0.8742182886 1.0436825514 0.9773124043 1.0100473879 
##          V20          V21          V22          V23          V24 
## 0.7444748231 1.4329545639 1.5347401880 0.8957445857 0.9727523021 
##          V25          V26          V27          V28 
## 0.8206291918 1.1151834700 0.5184477350 0.7885349948

Making Predictions

pred <- predict(model.glm, newdata=test_data)
accuracy <- table(pred, test_data[,"Class"])
sum(diag(accuracy))/sum(accuracy)
## [1] 0.9990754
pred = predict(model.glm, newdata=test_data)
confusionMatrix(data=pred, test_data$Class)
## Confusion Matrix and Statistics
## 
##           Reference
## Prediction Genuine Fraud
##    Genuine   85272    57
##    Fraud        22    90
##                                           
##                Accuracy : 0.9991          
##                  95% CI : (0.9988, 0.9993)
##     No Information Rate : 0.9983          
##     P-Value [Acc > NIR] : 5.509e-10       
##                                           
##                   Kappa : 0.6945          
##  Mcnemar's Test P-Value : 0.0001306       
##                                           
##             Sensitivity : 0.9997          
##             Specificity : 0.6122          
##          Pos Pred Value : 0.9993          
##          Neg Pred Value : 0.8036          
##              Prevalence : 0.9983          
##          Detection Rate : 0.9980          
##    Detection Prevalence : 0.9987          
##       Balanced Accuracy : 0.8060          
##                                           
##        'Positive' Class : Genuine         
##

The gradient Boosting model.

MiscFactors <- c()
pcafactors <-paste("V", 1:28, sep="")
formula = reformulate(termlabels = c(MiscFactors,pcafactors), response = 'Class')
print (formula)

ControlParamteres <- trainControl(method = "cv",
number = 10,
savePredictions = TRUE,
classProbs = TRUE,
verboseIter = TRUE
)
str(train_data)

model.gbm <- train(formula, data = train_data,method = "gbm", metric ="ROC", trControl = ControlParamteres)

#exp(coef(model.gbm$finalModel))
summary(model.gbm)
print(model.gbm)

Stochastic Gradient Boosting

199366 samples
28 predictor
2 classes: 'Geniune', 'Fraud'

No pre-processing
Resampling: Cross-Validated (10 fold)
Summary of sample sizes: 179429, 179429, 179429, 179430, 179430, 179430, ...
Resampling results across tuning parameters:

interaction.depth n.trees Accuracy Kappa
1 50 0.9983147 0.1565414
1 100 0.9983147 0.1565414
1 150 0.9983147 0.1565414
2 50 0.9983748 0.1829052
2 100 0.9984049 0.2482535
2 150 0.9984701 0.2647271
3 50 0.9985855 0.3733709
3 100 0.8990520 0.3826216
3 150 0.8990369 0.4040990

Tuning parameter 'shrinkage' was held constant at a value of 0.1
Tuning
parameter 'n.minobsinnode' was held constant at a value of 10
Accuracy was used to select the optimal model using the largest value.
The final values used for the model were n.trees = 50, interaction.depth = 3, shrinkage = 0.1
and n.minobsinnode = 10.

 

Making Predictions using GBM

pred <- predict(model.gbm, newdata=test_data)
accuracy <- table(pred, test_data[,"Class"])
print(accuracy)
sum(diag(accuracy))/sum(accuracy)
confusionMatrix(data=pred, test_data$Class)

 

Confusion Matrix and Statistics

Reference
Prediction Geniune Fraud
Geniune 85284 128
Fraud 10 19

Accuracy : 0.9984
95% CI : (0.9981, 0.9986)
No Information Rate : 0.9983
P-Value [Acc > NIR] : 0.2436

Kappa : 0.2155
Mcnemar's Test P-Value : <2e-16

Sensitivity : 0.9999
Specificity : 0.1293
Pos Pred Value : 0.9985
Neg Pred Value : 0.6552
Prevalence : 0.9983
Detection Rate : 0.9982
Detection Prevalence : 0.9997
Balanced Accuracy : 0.5646

'Positive' Class : Geniune

Making Predictions using Random Forest

mtry <- sqrt(ncol(train_data))

tuneGrid=expand.grid(.mtry=mtry)

ControlParamteres <- trainControl(method = "cv",
                                  number = 10,
                                  savePredictions = TRUE,
                                  classProbs = TRUE,
                                  verboseIter = TRUE
)
model.rf <- train(formula, data = train_data,method = "rf", family="binomial", metric="Accuracy",  trControl = ControlParamteres, tuneGrid=tuneGrid)
pred <- predict(model.rf, newdata=test_data)
accuracy <- table(pred, test_data[,"Class"])
print(accuracy)
##          
## pred      Geniune Fraud
##   Geniune   85289    28
##   Fraud         5   119

Making Predictions using Naive Bias

x=train_data[, -c(1,31)] # Removing Time and Class
y=train_data$Class
names(x)
grid <- data.frame(fL=c(0,0.5,1.0), usekernel = TRUE, adjust=c(0,0.5,1.0))

model = train(x,y,'nb',trControl=trainControl(method='cv',number=10),tuneGrid=grid)
model

prediction <- predict(model$finalModel,x)

table(prediction$class,y)

 

naive_ccfraud <- NaiveBayes(ccfraud$Class ~ ., data = ccfraud)
plot(naive_ccfraud)

x=test_data[, -c(1,31)] # Removing Time and Class
y=test_data$Class
prediction <- predict(model$finalModel,x)

table(prediction$class,y)

## Naive Bayes 
## 
## 199366 samples
##     29 predictor
##      2 classes: '0', '1' 
## 
## No pre-processing
## Resampling: Cross-Validated (10 fold) 
## Summary of sample sizes: 179430, 179430, 179429, 179429, 179429, 179429, ... 
## Resampling results across tuning parameters:
## 
##   fL   adjust  Accuracy   Kappa    
##   0.0  0.0           NaN        NaN
##   0.5  0.5     0.9978131  0.5481458
##   1.0  1.0     0.9957515  0.3954902
## 
## Tuning parameter 'usekernel' was held constant at a value of TRUE
## Accuracy was used to select the optimal model using  the largest value.
## The final values used for the model were fL = 0.5, usekernel = TRUE
##  and adjust = 0.5.
##    y
##          0      1
##   0 198799     75
##   1    222    270
GLM GBM Random Forest
Geniune 85272 57 85284 128 85289 28
Fraud 22 90 10 19 5 119

 


## pred Geniune Fraud
## Geniune 85289 28
## Fraud 5 119

About Author

Avatar

Smitha Mathew

Technology Enthusiast, with attention to detail, having global exposure. She is a self-motivated problem solver with experience analyzing data and deriving meaningful statistical information. Her goal is to be able to make a positive difference in peoples lives...
View all posts by Smitha Mathew >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Day Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet linear regression Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp