Sentiment: Optimize Conversion Rates with Analytics
The skills I demoed here can be learned through taking Data Science with Machine Learning bootcamp with NYC Data Science Academy.
I decided to apply sentiment analytics to understand how to go about improving customer targeting optimizing conversions. In e-commerce platforms, importance of online reviews cannot be overstated. Most online shoppers look at the reviews even before reading the description of the product they intend to buy. Due to social media and its wide use, positive or negative experience of consumers gets exposed at a speed of light.
Prospective consumers of a product, when they hear about these experiences, may alter their shopping habits and in the long run, bad reputation and increasing competition may create unexpected downturns for a brand and therefore the businesses are at no option of staying deaf. One single tweet can create cascades of problems for a brand and the use of sentiment analysis can be handy and yet, it is cost efficient.
In an e-commerce business, conversion rate is one of the most important tools. It’s a key indicator of effective reach that shows how often a click leads to a purchase and varies across product categories.
Sentiment analysis is an important concept and one of the most effective tools of improving the conversion rate. Reading the sentiment of consumers, not only enables businesses to reach out to their target audience, but also enables them understand their needs and feelings. It provides a bird-eye-view to brands and let them observe and protect their prestige. Additionally, it automates a cumbersome process of going through millions of lines of text to better read and listen to the demands and concerns of consumers.
That, in turn, helps manage unpredictable damaging scenarios and ease the cost of doing so. Daily, weekly, and monthly reports of sentiment analysis can help a brand improve its image, set its pricing appropriately, and improving its relationship with consumers. It can also be turned into a tool of tracking sector-wide trends and demands, including competitors’ contents and strategies, to contribute to a competitive advantage.
I created an app for sentiment analysis works for unigram (one-word sequence) sentiment analysis and loads small files. For more detailed analysis such as bigram (two-word sequence), trigram (three-word sequence), feel free to contact me. Although the app provides review rate prediction results for Amazon review dataset, I will explain it in another blog post.
The vast majority of data that exists is unstructured text data. Natural language processing seeks to extract information from text data through a variety of statistical techniques that range from simple word counting algorithms to more complex artificial intelligence algorithms. To get a sense of the text across a large set of text data, it is sometimes useful to analyze the most commonly occurring words. We can generate the frequency of each word or phrase by going through each column and summing up all the entries.
While word frequencies are useful at summarizing large amounts of text, the methods provide little information on how to interpret the data. A nice way of visualizing the frequency of text is a wordcloud. A wordcloud plot has larger values of each word or phrase the more frequent a word appears and smaller values for more infrequent words.
Text data, in this case the Amazon reviews and metadata, consists of descriptions of over a million products, their reviews, titles, and summaries. First, a table is created to calculate the percentages of each word occurrence and emotion present within the text file. I calculated the following emotions: Anger, Anticipation, Disgust, Fear, Joy, Sadness, Surprise, and Trust. These are the eight basic universal emotions conveyed in all cultures.
This allows us to calculate the positive and negative sentiment present within the text, since these emotions can be group into being either positive or negative.
After identifying the direction of sentiment, trajectory plot (top part of the following graph) is deployed to display the overall emotion of pieces of the text at different successive linear locations and will show how the emotional content of the text has changed over time from beginning of the text to the end.
The horizontal axis refers to how the text have changed, and the vertical axis refers to the positiveness or negativeness, upward or downward respectively. The output is a bit noisy and so needs to be decluttered. Smoothing process lies under the trajectory graph, which highlights the sentiment of the text clearly and gives us a better understanding about the direction of sentiment.
The picture above is the results of over a million reviews of Amazon reviews dataset across all categories, though it can be divided into sub-categories.
Lastly, lexical plot scans the uploaded text file, searches the word in text and marks it. It is quite a useful tool in spotting in where a certain text appears and at what frequency.
After visualizing and identifying the direction of sentiment, I tried to build a model to predict whether a review is positive or negative for reviewers who did not clearly specify their rating. What rate would have they given? The simplest type of sentiment classifier is the Bag-of-Words method which simply counts the frequency of positive or negative words as explained in this article. There are a few dictionaries that are used in sentiment classifying such as EmoLex (NRC), Loughran, Bing, and AFINN. I tried Bing lexicon in this analysis. After loading the dictionary, since I have the individual tokens, it counts co-occurrences within reviews to get an estimate.
Then the estimation can be used to compare against the actual sentiment to see how well the model predicts. The result shows that overall accuracy is at about 50%.This rate tells me how often I was correct. By including the contextual understanding and neutral and mixed state, trying a few other lexicons and methods, and researching more literature, the accuracy can be improved.
Sentiment analysis enables businesses to listen to the voices of people. Online shoppers have many alternatives of every product and they often decide fast. Therefore, reading the needs and demands of consumers, to stay competitive and profitable, is essential. In addition, sentiment analysis can be turned into a tracking tool of competitors, social stance, and brand image. The digital era requires close follow-up of textual content and we put our emotions into our texts. Therefore, understanding the feelings of customers and potential customers is essential for today’s business success.
About Author
