The Rise and Fall of Internet Slang

Sudhanva Manjunath, Sushmey Nirmal, Kyle Okura

This project explores how internet slang evolves over time on Reddit, tracking the birth, growth, and decline of popular terms like "based", "cringe", "rizz", "gaslight", and "goat". Using data mining techniques, we analyze millions of Reddit comments to understand the lifecycle patterns of these buzzwords and identify what makes some terms go viral while others fade quickly.

Overview

The project processes Reddit comment data to track slang term usage across different subreddits and time periods. We apply various analytical techniques including topic modeling, temporal analysis, and sentiment analysis to understand how these terms spread and evolve within online communities.

Project Structure

• Data Processing: Scripts for filtering and processing Reddit comment data stored in parquet format
• Analysis Notebooks: Jupyter notebooks for buzzword analysis, community analysis, topic modeling, and temporal trends
• Outputs: Results including topic models, case studies, and visualizations organized by analysis phase
• Training: Scripts for training classifiers to identify slang usage in context

Key Features

• Temporal analysis of slang term frequency over time
• Topic modeling to understand context and usage patterns
• Community analysis across different subreddits
• Case studies on specific terms (gaslight, goat, nerfed)
• Sentiment and distribution analysis

Setup

Install the required dependencies:

pip install -r requirements.txt

The project uses DuckDB for efficient data processing, along with standard data science libraries like pandas, numpy, and scikit-learn. NLP processing is handled through spaCy and NLTK.

Data

The project works with Reddit comment data stored in partitioned parquet files organized by month. The data includes comment text, timestamps, subreddit information, and engagement metrics.

Usage

The analysis is primarily conducted through Jupyter notebooks. Start with the basic analysis scripts to explore the data, then dive into specific notebooks for detailed investigations into buzzword lifecycles, topic modeling, or temporal trends.