X Algorithm Reimplementation

This is a personal research project to reverse-engineer and reimplement the core recommendation algorithms used by X (formerly Twitter). The goal is to build a tunable, transparent, and interactive version of the ranking pipeline to understand how social media feeds are constructed.

Architecture

The project consists of three main components:

1. Backend (FastAPI): Handles API requests, user management, and the core ranking algorithm.
2. Frontend (Next.js): A React-based web interface to view the feed and tune algorithm parameters.
3. Simulation: A Python script using LLMs (Gemini/OpenAI) to generate synthetic users and tweets to populate the system.

Setup

Prerequisites

• Python 3.11+
• Node.js 18+
• OpenAI or Google Gemini API Key

Backend

1. Navigate to backend/:

   cd backend

2. Create a virtual environment and install dependencies:

   python3 -m venv venv
   source venv/bin/activate
   pip install -r requirements.txt

3. Run the server:

   fastapi dev main.py

Frontend

1. Navigate to frontend/:

   cd frontend

2. Install dependencies:

   npm install

3. Run the development server:

   npm run dev

Simulation

1. Create a .env file in the root directory and add your API key:

   OPENAI_API_KEY=your_key_here
   # Alternative provider configuration
   GEMINI_API_KEY=your_key_here

2. Run the backend script, you will need to have activated the virtual environment.

   source backend/venv/bin/activate
   python -m uvicorn backend.main:app --reload --host 0.0.0.0 --port 8000

3. Run the frontend script

   cd frontend
   npm run dev

4. Run the simulation script

   python3 simulation/run_sim.py