Sentiment: Optimize Conversion Rates with Analytics

Posted on Oct 20, 2019
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.

I decided to apply sentiment analytics to understand how to go about improving customer targeting optimizing conversions. In e-commerce platforms, importance of online reviews cannot be overstated. Most online shoppers look at the reviews even before reading the description of the product they intend to buy. Due to social media and its wide use, positive or negative experience of consumers gets exposed at a speed of light.

Prospective consumers of a product, when they hear about these experiences, may alter their shopping habits and in the long run, bad reputation and increasing competition may create unexpected downturns for a brand and therefore the businesses are at no option of staying deaf. One single tweet can create cascades of problems for a brand and the use of sentiment analysis can be handy and yet, it is cost efficient.

In an e-commerce business, conversion rate is one of the most important tools. It’s a key indicator of effective reach that shows how often a click leads to a purchase and varies across product categories.

sentiment emoji brain

Sentiment analysis is an important concept and one of the most effective tools of improving the conversion rate. Reading the sentiment of consumers, not only enables businesses to reach out to their target audience, but also enables them understand their needs and feelings. It provides a bird-eye-view to brands and let them observe and protect their prestige. Additionally, it automates a cumbersome process of going through millions of lines of text to better read and listen to the demands and concerns of consumers.

That, in turn, helps manage unpredictable damaging scenarios and ease the cost of doing so. Daily, weekly, and monthly reports of sentiment analysis can help a brand improve its image, set its pricing appropriately, and improving its relationship with consumers. It can also be turned into a tool of tracking sector-wide trends and demands, including competitors’ contents and strategies, to contribute to a competitive advantage.

I created an app for sentiment analysis works for unigram (one-word sequence) sentiment analysis and loads small files. For more detailed analysis such as bigram (two-word sequence), trigram (three-word sequence), feel free to contact me. Although the app provides review rate prediction results for Amazon review dataset, I will explain it in another blog post.

The vast majority of data that exists is unstructured text data. Natural language processing seeks to extract information from text data through a variety of statistical techniques that range from simple word counting algorithms to more complex artificial intelligence algorithms. To get a sense of the text across a large set of text data, it is sometimes useful to analyze the most commonly occurring words. We can generate the frequency of each word or phrase by going through each column and summing up all the entries.

While word frequencies are useful at summarizing large amounts of text, the methods provide little information on how to interpret the data. A nice way of visualizing the frequency of text is a wordcloud. A wordcloud plot has larger values of each word or phrase the more frequent a word appears and smaller values for more infrequent words.

Text data, in this case the Amazon reviews and metadata, consists of descriptions of over a million products, their reviews, titles, and summaries. First, a table is created to calculate the percentages of each word occurrence and emotion present within the text file. I calculated the following emotions: Anger, Anticipation, Disgust, Fear, Joy, Sadness, Surprise, and Trust. These are the eight basic universal emotions conveyed in all cultures.

This allows us to calculate the positive and negative sentiment present within the text, since these emotions can be group into being either positive or negative.

After identifying the direction of sentiment, trajectory plot (top part of the following graph) is deployed to display the overall emotion of pieces of the text at different successive linear locations and will show how the emotional content of the text has changed over time from beginning of the text to the end.

The horizontal axis refers to how the text have changed, and the vertical axis refers to the positiveness or negativeness, upward or downward respectively. The output is a bit noisy and so needs to be decluttered. Smoothing process lies under the trajectory graph, which highlights the sentiment of the text clearly and gives us a better understanding about the direction of sentiment.

The picture above is the results of over a million reviews of Amazon reviews dataset across all categories, though it can be divided into sub-categories.

Lastly, lexical plot scans the uploaded text file, searches the word in text and marks it. It is quite a useful tool in spotting in where a certain text appears and at what frequency.

After visualizing and identifying the direction of sentiment, I tried to build a model to predict whether a review is positive or negative for reviewers who did not clearly specify their rating. What rate would have they given? The simplest type of sentiment classifier is the Bag-of-Words method which simply counts the frequency of positive or negative words as explained in this article. There are a few dictionaries that are used in sentiment classifying such as EmoLex (NRC), Loughran, Bing, and AFINN. I tried Bing lexicon in this analysis. After loading the dictionary, since I have the individual tokens, it counts co-occurrences within reviews to get an estimate.

Then the estimation can be used to compare against the actual sentiment to see how well the model predicts. The result shows that overall accuracy is at about 50%.This rate tells me how often I was correct. By including the contextual understanding and neutral and mixed state, trying a few other lexicons and methods, and researching more literature, the accuracy can be improved.

Sentiment analysis enables businesses to listen to the voices of people. Online shoppers have many alternatives of every product and they often decide fast. Therefore, reading the needs and demands of consumers, to stay competitive and profitable, is essential. In addition, sentiment analysis can be turned into a tracking tool of competitors, social stance, and brand image. The digital era requires close follow-up of textual content and we put our emotions into our texts. Therefore, understanding the feelings of customers and potential customers is essential for today’s business success.

About Author

ted dogan
View all posts by ted dogan >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories

Our Recent Popular Posts

View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI