Topic Modeling and User Clustering on Internet Discussion Forums - A Case Study

Posted on Oct 30, 2017

Overview

Internet forums or message boards are online discussion sites. They're used for a variety of purposes, including general socialization, asking specific questions, or sharing experiences.

Forums can be an abundant source of unstructured text data, often generated by specific communities united by shared interests. And analysis can be applied to these online discussions to uncover hidden patterns or to organize, summarize, and understand these communities -- both the members themselves and what they talk about.

In internet parlance, the participants are referred to as 'users', the conversations are called 'threads', and the unit of interaction is called a 'post'.

In this project, we use topic modeling to infer a structure or a set of common topics behind a collection of threads. Additionally, we try to group users to see if we can determine whether users fall into identifiable archetypes.

Dataset

We used Python's Scrapy library to gather user-posts from www.veggieboards.com, an internet community for vegetarians and vegans. The final dataset consists of ~450,000 posts from Oct. 2001 to Oct. 2017. The posts are divided into ~22,000 threads and made by ~14,600 users. We stored the individual posts as Python dictionaries in a MongoDB database - an example can be seen below:

Methodology

We followed the workflow set forth by Patrick Harrison who did Modern NLP in Python, relying heavily on the Python Spacy and Gensim libraries for natural language processing.

  1. Text Normalization: transforming all text into a consistent format in preparation for processing
    1. cleaning - removing HTML tags and meta-data
    2. tokenization - breaking text up into terms or words
    3. stemming / lemmatization - combining together different inflectional or derived forms of words, for example combining 'ran', 'run', 'running' - three tenses of the same root word - into a single token.
  2. Phrase Modeling: combining commonly co-occurring tokens into phrases. Below is what our text looks like after the phrase modeling step. We only modeled up to four token phrases. Additionally, all pronouns are replaced with a generic -PRON- term. The phrases below are outlined in red. 
  3. Dictionary Creation: defining a vocabulary of valid tokens
    1. In this step we can remove stopwords or very common words (occurring in over 40% of our documents) that might not provide any explanatory information
    2. We also remove very rare words (occurring less than 10 times) out of practicality -- because the processing time for subsequent steps depends on dictionary size
  4. Latent Dirichlet Allocation:
    1. We use the Gensim library to perform LDA, first treating all posts in a thread as our unit of analysis (referred to as a document in natural language processing) and then a second time treating all posts by a user as our documents.
    2. LDA is a probabilistic model that tries to uncover latent "topics" from all our documents (also referred to as a corpus). Unlike clustering, which would assign each document to a single cluster, the assumption is that documents may belong to a number of different topics.

Results

We can inspect the generated LDA models using the pyLDAvis library. The tool reduces the dimensionality of the model from K, the number of topics, down to 2D space for visualization purposes. The conceit being that topics that are more similar to each other are closer together and topics that are less similar are farther apart.

We chose 40 topics for the thread-centric approach, and 25 topics for the user-centric approach, based on human judgment -- looking at the topics and seeing if they made sense.

https://paultondata.github.io/vis/LDAthreads.html

https://paultondata.github.io/vis/LDAusers.html

Discussion

The topics generated by the LDA process aren't always human interpretable, and they aren't always easy for humans to distinguish between. Some topics are vague, and many topics share the same words. The clarity of division depends on the underlying documents themselves. Perhaps LDA may not be wholly appropriate for sub-forums, which are already narrowed down to specific categories. Even still, we were able to extract some promising topics.

The modeling process is fairly labor intensive, because it's difficult to evaluate the models without human intervention, especially if the goal of the model is for human understanding.

For modeling on users, the results are much worse, perhaps because generally users participate in a variety of different topics, taking part in many different threads. A lot of the pre-processing that's helpful in topic modeling such as removing stopwords, stemming, and lemmatization may actually be counter-productive for user differentiation. Those processes remove stylistic information unique to users or user groups, thus a user-centric study may benefit from a different type of dictionary.

Future Exploration

Potential avenues for further investigation include using LDA as an intermediary step for authorship attribution, troll detection, and deceptive opinion spam.

References

Veggieboards:

http://www.veggieboards.com

Analyzing Internet Forums:

http://psycnet.apa.org/record/2012-24262-002

Modern NLP in Python:

https://github.com/skipgram/modern-nlp-in-python/tree/master/executable

Improving LDA Topic Models:

https://dl.acm.org/citation.cfm?id=2484166

Authorship Attribution with LDA:

http://aclweb.org/anthology/W11-0321

About Author

Paul Ton

Paul is a software engineer by training. He has great respect for the theoretical underpinnings of data science, while still appreciating the practical considerations required to handle real world problems. He's always happy to discuss all things data!
View all posts by Paul Ton >

Related Articles

Leave a Comment

cessiobby August 31, 2019
The cleaning company performs cleaning of rooms of different sizes as well as setups. The business's specialists provide cleansing with the aid of contemporary innovations, have unique equipment, as well as also have licensed detergents in their collection. In addition to the above benefits, wines supply: positive rates; cleaning in a short time; top quality outcomes; more than 100 favorable reviews. Cleansing workplaces will certainly aid keep your work environment in order for the most productive job. Any kind of company is very essential atmosphere in the team. Cleaning up services that can be purchased cheaply currently can help to prepare it and offer a comfy room for labor. If necessary, we leave cleaning up the cooking area 2-3 hrs after positioning the order. You obtain cleaning up immediately. We offer expert maid services for exclusive clients. Using European devices as well as accredited devices, we accomplish optimal outcomes and also give cleaning in a short time. We provide discounts for those that use the solution for the first time, as well as beneficial terms of collaboration for routine clients. Our friendly team provides you to obtain familiarized with beneficial regards to teamwork for company clients. We properly approach our tasks, clean using specialist cleaning items and also specialized tools. Our employees are trained, have medical publications as well as recognize with the subtleties of getting rid of complex as well as hard-to-remove dust from surfaces. We supply high-quality cleaning for big business as well as small firms of various directions, with a price cut of approximately 25%.

View Posts by Categories


Our Recent Popular Posts


View Posts by Tags

#python #trainwithnycdsa 2019 2020 Revenue 3-points agriculture air quality airbnb airline alcohol Alex Baransky algorithm alumni Alumni Interview Alumni Reviews Alumni Spotlight alumni story Alumnus ames dataset ames housing dataset apartment rent API Application artist aws bank loans beautiful soup Best Bootcamp Best Data Science 2019 Best Data Science Bootcamp Best Data Science Bootcamp 2020 Best Ranked Big Data Book Launch Book-Signing bootcamp Bootcamp Alumni Bootcamp Prep boston safety Bundles cake recipe California Cancer Research capstone car price Career Career Day citibike classic cars classpass clustering Coding Course Demo Course Report covid 19 credit credit card crime frequency crops D3.js data data analysis Data Analyst data analytics data for tripadvisor reviews data science Data Science Academy Data Science Bootcamp Data science jobs Data Science Reviews Data Scientist Data Scientist Jobs data visualization database Deep Learning Demo Day Discount disney dplyr drug data e-commerce economy employee employee burnout employer networking environment feature engineering Finance Financial Data Science fitness studio Flask flight delay gbm Get Hired ggplot2 googleVis H20 Hadoop hallmark holiday movie happiness healthcare frauds higgs boson Hiring hiring partner events Hiring Partners hotels housing housing data housing predictions housing price hy-vee Income Industry Experts Injuries Instructor Blog Instructor Interview insurance italki Job Job Placement Jobs Jon Krohn JP Morgan Chase Kaggle Kickstarter las vegas airport lasso regression Lead Data Scienctist Lead Data Scientist leaflet league linear regression Logistic Regression machine learning Maps market matplotlib Medical Research Meet the team meetup methal health miami beach movie music Napoli NBA netflix Networking neural network Neural networks New Courses NHL nlp NYC NYC Data Science nyc data science academy NYC Open Data nyc property NYCDSA NYCDSA Alumni Online Online Bootcamp Online Training Open Data painter pandas Part-time performance phoenix pollutants Portfolio Development precision measurement prediction Prework Programming public safety PwC python Python Data Analysis python machine learning python scrapy python web scraping python webscraping Python Workshop R R Data Analysis R language R Programming R Shiny r studio R Visualization R Workshop R-bloggers random forest Ranking recommendation recommendation system regression Remote remote data science bootcamp Scrapy scrapy visualization seaborn seafood type Selenium sentiment analysis sentiment classification Shiny Shiny Dashboard Spark Special Special Summer Sports statistics streaming Student Interview Student Showcase SVM Switchup Tableau teachers team team performance TensorFlow Testimonial tf-idf Top Data Science Bootcamp Top manufacturing companies Transfers tweets twitter videos visualization wallstreet wallstreetbets web scraping Weekend Course What to expect whiskey whiskeyadvocate wildfire word cloud word2vec XGBoost yelp youtube trending ZORI