Exploring Data on Consumer Complaints

Posted on Sep 27, 2018
The skills the author demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.


The provision of quality is an important task for any firm. As the availability and scope of data on the consumer experience expands, companies must confront an important issue: how can their data spot emerging issues in the quality of their products and/or services?

In this project, I explore data on complaints made by consumers in regard to the services provided by their financial institutions. When a consumer is unhappy with their bank, they are able to submit a formal complaint to the U.S. Consumer Financial Protection Bureau (CFPB). In addition to investigating issues raised in the complaints, the CFPB also publishes the complaints online.

This complaint dataset is extremely detailed and provides data on the type of product, specific issue raised (and often the actual complaint text), the name of the financial institution, the individualโ€™s zip code, how the complaint was resolved (if at all), among other variables. In short, the dataset provides a great opportunity to explore how to visualize a large-scale dataset on consumer experiences. Effective visualization of this data could also help identify emerging issues at specific institutions that are being raised by consumers.

Visualizing the data:ย 

While I provide a few visualizations below, the current Shiny project can be found here. As the project will evolve, the analysis below provides an initial summary of my work. Future updates will be reflected in the Shiny page link above.

Initially, I analyze complaints at an aggregate level. From 2012 to 2017, total complaints submitted annually has risen from 70,290 to 233,099. Figure 1 shows that the composition of complaints also evolved tremendously during this period: mortgage-related issues have declined substantially, whereas debt collection and credit reporting issues have moved to the forefront. As of 2017, for example, complaints associated with credit reporting (mortgage-related) issues accounted for 38.81 % (12.56%) of complaints as opposed to 2.33% (53.07%) as of 2012.

Exploring Data on Consumer Complaints


Next, I explore how this composition differs across states as well as to examine a measure of severity (e.g., the % of complaints resolved in a timely manner) for any given product type. The figures shown below are treemaps that break down the complaints at the aggregate-level in the top figure and for Massachusetts in the bottom figure. Although this only compares the national breakdown to one state, the Shiny app also one to easily explore visualizations with different states and year combinations.

Exploring Data on Consumer ComplaintsExploring Data on Consumer Complaints

The figures above allow one to conveniently compare the state breakdown to the national average: while debt collection complaints are often not resolved in a timely manner, this is less of an issue in Massachusetts as of 2015. In this case, 8.4% of debt collection complaints are not resolved in a timely manner nationally, only 4.9% are not resolved in a timely manner in Massachusetts.

While one should expect state-specific issues, Shinyโ€™s interactive nature allows one to easily navigate across state-year combinations to quickly spot emerging issues. Switching from 2015 to 2016, for example, one can spot emerging issues in Massachusetts. The figures below visually illustrate how debt collection complaints grew rapidly in 2016: 9.7% of debt collection complaints were resolved in a timely manner as of 2016 relative to 4.9% as of 2015.


The figures above facilitate contrasting differences across states and time, but they also illustrate the distinction between scale and severity. Visualizing this distinction is helpful as some may differ in whether they consider scale or severity as the more import issue in any given context. Scale may be a more natural way to focus on problem areas, the vast majority of complaints in this data are resolved in a timely manner. One may thus prefer to focus on addressing problems in specific product areas that are less common overall, but where problems are severe. These charts illustrate these differences.

Once one has selected a specific area of focus, e.g. product type, it can be helpful to identify where those specific problems are concentrated across specific institutions and geography. ย The heatmap below illustrates the pattern of complaint growth across institutions and geography. The x-axis corresponding to different states and the y-axis corresponding to the largest 19 institutions in the data that have complaints in all states. It shows the concentration of complaint growth within Navient Solutions occurs across most states. It also shows some specific areas of growth: American Express in Maine and Barclays in New Hampshire.


While the states in in this figure correspond only to those in the Northeast, users can select different levels of geographic (as well as institution) coverage in the Shiny app. Most importantly, however, it also allows one to quickly identify the specific locations where an institutionโ€™s complaint growth is the most pronounced, benchmark patterns relative to other institutions, and also to also spot regional issues.


The visualizations above illustrate some of the difference across issues and institutions within the CFPB complaint data. The most important feature of this analysis is, however, the interactive visualizations provided through the Shiny app and the potential patterns they can allow one to spot quickly.

About Author

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI