Help Your Business: Sentiment Analytics

Ted Dogan
Posted on Oct 20, 2019

I decided to apply sentiment analytics to understand how to go about improving customer targeting optimizing conversions. In e-commerce platforms, importance of online reviews cannot be overstated. Most online shoppers look at the reviews even before reading the description of the product they intend to buy. Due to social media and its wide use, positive or negative experience of consumers gets exposed at a speed of light. Prospective consumers of a product, when they hear about these experiences, may alter their shopping habits and in the long run, bad reputation and increasing competition may create unexpected downturns for a brand and therefore the businesses are at no option of staying deaf. One single tweet can create cascades of problems for a brand and the use of sentiment analysis can be handy and yet, it is cost efficient. In an e-commerce business, conversion rate is one of the most important tools. It’s a key indicator of effective reach that shows how often a click leads to a purchase and varies across product categories.

sentiment emoji brain

Sentiment analysis is an important concept and one of the most effective tools of improving the conversion rate. Reading the sentiment of consumers, not only enables businesses to reach out to their target audience, but also enables them understand their needs and feelings. It provides a bird-eye-view to brands and let them observe and protect their prestige. Additionally, it automates a cumbersome process of going through millions of lines of text to better read and listen to the demands and concerns of consumers. That, in turn, helps manage unpredictable damaging scenarios and ease the cost of doing so. Daily, weekly, and monthly reports of sentiment analysis can help a brand improve its image, set its pricing appropriately, and improving its relationship with consumers. It can also be turned into a tool of tracking sector-wide trends and demands, including competitors’ contents and strategies, to contribute to a competitive advantage.

I created an app for sentiment analysis works for unigram (one-word sequence) sentiment analysis and loads small files. For more detailed analysis such as bigram (two-word sequence), trigram (three-word sequence), feel free to contact me. Although the app provides review rate prediction results for Amazon review dataset, I will explain it in another blog post.

The vast majority of data that exists is unstructured text data. Natural language processing seeks to extract information from text data through a variety of statistical techniques that range from simple word counting algorithms to more complex artificial intelligence algorithms. To get a sense of the text across a large set of text data, it is sometimes useful to analyze the most commonly occurring words. We can generate the frequency of each word or phrase by going through each column and summing up all the entries. While word frequencies are useful at summarizing large amounts of text, the methods provide little information on how to interpret the data. A nice way of visualizing the frequency of text is a wordcloud. A wordcloud plot has larger values of each word or phrase the more frequent a word appears and smaller values for more infrequent words.

Text data, in this case the Amazon reviews and metadata, consists of descriptions of over a million products, their reviews, titles, and summaries. First, a table is created to calculate the percentages of each word occurrence and emotion present within the text file. I calculated the following emotions: Anger, Anticipation, Disgust, Fear, Joy, Sadness, Surprise, and Trust. These are the eight basic universal emotions conveyed in all cultures.

This allows us to calculate the positive and negative sentiment present within the text, since these emotions can be group into being either positive or negative.

After identifying the direction of sentiment, trajectory plot (top part of the following graph) is deployed to display the overall emotion of pieces of the text at different successive linear locations and will show how the emotional content of the text has changed over time from beginning of the text to the end.

The horizontal axis refers to how the text have changed, and the vertical axis refers to the positiveness or negativeness, upward or downward respectively. The output is a bit noisy and so needs to be decluttered. Smoothing process lies under the trajectory graph, which highlights the sentiment of the text clearly and gives us a better understanding about the direction of sentiment. The picture above is the results of over a million reviews of Amazon reviews dataset across all categories, though it can be divided into sub-categories.

Lastly, lexical plot scans the uploaded text file, searches the word in text and marks it. It is quite a useful tool in spotting in where a certain text appears and at what frequency.

After visualizing and identifying the direction of sentiment, I tried to build a model to predict whether a review is positive or negative for reviewers who did not clearly specify their rating. What rate would have they given? The simplest type of sentiment classifier is the Bag-of-Words method which simply counts the frequency of positive or negative words as explained in this article. There are a few dictionaries that are used in sentiment classifying such as EmoLex (NRC), Loughran, Bing, and AFINN. I tried Bing lexicon in this analysis. After loading the dictionary, since I have the individual tokens, it counts co-occurrences within reviews to get an estimate.

Then the estimation can be used to compare against the actual sentiment to see how well the model predicts. The result shows that overall accuracy is at about 50%.This rate tells me how often I was correct. By including the contextual understanding and neutral and mixed state, trying a few other lexicons and methods, and researching more literature, the accuracy can be improved.

Sentiment analysis enables businesses to listen to the voices of people. Online shoppers have many alternatives of every product and they often decide fast. Therefore, reading the needs and demands of consumers, to stay competitive and profitable, is essential. In addition, sentiment analysis can be turned into a tracking tool of competitors, social stance, and brand image. The digital era requires close follow-up of textual content and we put our emotions into our texts. Therefore, understanding the feelings of customers and potential customers is essential for today’s business success.

About Author

Ted Dogan

Ted Dogan

With the right approach and tools, it is possible to create a digital footprint of behavior. Actions leave traces of preferences, a raw form of demand. Converting them into a meaningful business story is an art that requires...
View all posts by Ted Dogan >

Related Articles

Leave a Comment

No comments found.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

2019 airbnb Alex Baransky alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus API Application artist aws beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep Bundles California Cancer Research capstone Career Career Day citibike clustering Coding Course Demo Course Report D3.js data Data Analyst data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization Deep Learning Demo Classes Demo Day Demo Lesson Discount dplyr employer networking feature engineering Finance Financial Data Science Flask gbm Get Hired ggplot2 googleVis Hadoop higgs boson Hiring hiring partner events Hiring Partners Industry Experts Instructor Blog Instructor Interview Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter lasso regression Lead Data Scienctist Lead Data Scientist leaflet Lectures linear regression Live Chat Live Online Bootcamp Logistic Regression machine learning Maps matplotlib Medical Research Meet the team meetup Networking neural network Neural networks New Courses nlp NYC NYC Data Science nyc data science academy NYC Open Data NYCDSA NYCDSA Alumni Online Online Bootcamp Online Lectures Online Training Open Data painter pandas Part-time Portfolio Development prediction Prework Programming PwC python python machine learning python scrapy python web scraping python webscraping Python Workshop R R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking Realtime Interaction recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn Selenium sentiment analysis Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau team TensorFlow Testimonial tf-idf Top Data Science Bootcamp twitter visualization web scraping Weekend Course What to expect word cloud word2vec XGBoost yelp