Social Media Monitoring System

A high-performance, event-driven social media monitoring system that tracks specific keywords across social media platforms and sends real-time alerts to Telegram.

Features

• Twitter Scraping: Uses twscrape to fetch real Twitter data without official API access
• Modular Data Ingestion: Abstract provider pattern for easy integration with different data sources
• Smart Filtering: Filter posts by follower count and keyword matching
• Deduplication: Prevents duplicate alerts using efficient in-memory cache
• Telegram Integration: Real-time alerts sent to your Telegram channel
• Production Ready: Dockerized with resource limits optimized for VPS environments
• Async Architecture: Built with asyncio for high performance and low resource usage

Architecture

┌─────────────────┐
│  Data Provider  │  (twscrape Twitter Scraper or Mock)
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Filter Pipeline │  (Followers, Keywords, Dedup)
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Alert Manager   │  (Telegram Bot)
└─────────────────┘

Project Structure

telegram-tracking-app/
├── main.py                 # Main orchestration script
├── data_provider.py        # Abstract provider + Twitter scraper + Mock
├── alert_manager.py        # Telegram alert dispatcher
├── config.py               # Configuration management (Pydantic)
├── requirements.txt        # Python dependencies
├── Dockerfile              # Multi-stage Docker build
├── docker-compose.yml      # Docker Compose configuration
├── .env.example            # Environment variable template
├── accounts.txt.example    # Twitter accounts template
└── README.md              # This file

Prerequisites

• Python 3.11+ (for local development)
• Docker & Docker Compose (for containerized deployment)
• Telegram Bot Token (from @BotFather)
• Telegram Channel (where alerts will be posted)
• Twitter Account(s) (for real data scraping - 2-5 dedicated accounts recommended)

Quick Start

1. Create a Telegram Bot

1. Message @BotFather on Telegram
2. Send /newbot and follow the prompts
3. Save your bot token (looks like 123456789:ABCdefGHIjklMNOpqrsTUVwxyz)

2. Set Up Telegram Channel

1. Create a new Telegram channel
2. Add your bot as an administrator to the channel
3. Get your channel ID:
   • For public channels: @channelname
   • For private channels: Use @userinfobot

3. Configure Environment Variables

# Copy the example environment file
cp .env.example .env

# Edit .env with your credentials
nano .env

Update these required fields:

TELEGRAM_BOT_TOKEN=your_bot_token_here
TELEGRAM_CHANNEL_ID=@your_channel_name

4. Run with Docker (Recommended)

# Build and start the container
docker-compose up -d

# View logs
docker-compose logs -f

# Stop the container
docker-compose down

5. Run Locally (Development)

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Run the application
python main.py

Configuration

All configuration is managed through environment variables in .env:

Variable	Description	Default
TELEGRAM_BOT_TOKEN	Your Telegram bot token	Required
TELEGRAM_CHANNEL_ID	Target channel ID	Required
KEYWORDS	Comma-separated keywords to monitor	$hylo,hylo,hyusd,...
MIN_FOLLOWER_COUNT	Minimum followers to process	100
DEDUP_CACHE_SIZE	Max post IDs to track	10000
POLL_INTERVAL_SECONDS	Seconds between polls	60
USE_MOCK_PROVIDER	Use mock (true) or Twitter scraper (false)	true
MOCK_POSTS_PER_POLL	Mock posts per cycle	5
TWITTER_ACCOUNTS_FILE	Twitter credentials file path	accounts.txt
TWITTER_MAX_TWEETS	Max tweets per poll	50
TWITTER_SEARCH_TYPE	Search type: latest or top	latest
TWITTER_LANGUAGE	Language filter	en

Monitored Keywords

Default keywords (case-insensitive):

• $hylo
• hylo
• hyusd
• shyusd
• xsol
• xbtc
• xeth

Customize by editing KEYWORDS in .env.

Alert Format

Alerts are sent to Telegram in this format:

📢 2026-01-23 14:30:45 HYLO ALERT
🔗 https://twitter.com/username/status/123456789
👤 @username (5,432 followers)
💬 Just bought some $hylo! 🚀

Using Twitter Data with twscrape

The system uses twscrape to fetch real Twitter data without requiring official API access. This approach is more accessible but requires Twitter account credentials.

Setting Up Twitter Accounts

1. Create accounts file:

   cp accounts.txt.example accounts.txt
   nano accounts.txt

2. Add Twitter account credentials (format: username:password:email:email_password):

   myuser1:mypass1:email1@example.com:emailpass1
   myuser2:mypass2:email2@example.com:emailpass2
   

   Important notes:

   • Use dedicated accounts, not your personal Twitter account
   • Multiple accounts improve reliability and rate limits (recommended: 2-5 accounts)
   • Aged accounts work better than brand new accounts
   • Accounts should have email verification enabled
   • Keep accounts.txt secure (already in .gitignore)

3. Enable Twitter scraper in .env:

   USE_MOCK_PROVIDER=false
   TWITTER_ACCOUNTS_FILE=accounts.txt
   TWITTER_MAX_TWEETS=50
   TWITTER_SEARCH_TYPE=latest
   TWITTER_LANGUAGE=en

4. Run the system:

   # With Docker
   docker-compose up -d
   
   # Or locally
   python main.py

How twscrape Works

• No official API required: Scrapes Twitter directly using account sessions
• Keyword search: Automatically searches for all configured keywords
• Automatic filtering: Only returns tweets matching your criteria
• Rate limit handling: Rotates between multiple accounts to avoid limits
• Deduplication: Tracks seen tweets to avoid duplicate alerts

Twitter Scraper Configuration

Variable	Description	Default
TWITTER_ACCOUNTS_FILE	Path to account credentials file	accounts.txt
TWITTER_MAX_TWEETS	Max tweets per poll	50
TWITTER_SEARCH_TYPE	latest or top tweets	latest
TWITTER_LANGUAGE	Language filter (e.g., en)	en

Troubleshooting Twitter Accounts

Accounts getting locked:

• Use aged accounts (1+ months old)
• Don't scrape too aggressively (increase POLL_INTERVAL_SECONDS)
• Add more accounts to distribute load
• Use residential IP addresses when possible

No tweets found:

• Check that keywords match actual Twitter content
• Try TWITTER_SEARCH_TYPE=top for popular tweets
• Increase TWITTER_MAX_TWEETS
• Verify accounts are logged in (check logs)

Rate limits:

• Add more accounts to the pool
• Increase POLL_INTERVAL_SECONDS
• Reduce TWITTER_MAX_TWEETS

Resource Usage

Optimized for low-resource VPS environments:

• CPU: ~0.25-0.5 cores
• Memory: 128-256 MB
• Disk: ~50 MB (container)
• Network: Minimal (polling-based)

Docker Commands

# Build image
docker-compose build

# Start in background
docker-compose up -d

# View logs
docker-compose logs -f telegram-monitor

# Restart container
docker-compose restart

# Stop and remove
docker-compose down

# View resource usage
docker stats social-media-monitor

Monitoring & Debugging

Check Container Status

docker-compose ps

View Real-Time Logs

docker-compose logs -f

Execute Commands in Container

docker-compose exec telegram-monitor sh

Check Application Statistics

The application prints statistics periodically in logs:

[Monitor] Statistics: {
    'total_posts_processed': 150,
    'posts_filtered_followers': 45,
    'posts_filtered_keywords': 89,
    'posts_filtered_duplicates': 3,
    'alerts_sent': 13
}

Production Deployment

VPS Deployment

1. Connect to your VPS:

   ssh user@your-vps-ip

2. Clone repository:

   git clone <your-repo-url>
   cd telegram-tracking-app

3. Configure environment:

   cp .env.example .env
   nano .env  # Add your credentials

4. Deploy with Docker:

   docker-compose up -d

5. Set up auto-restart (optional):

   # Docker Compose will auto-restart on failure
   # To start on boot, enable Docker service:
   sudo systemctl enable docker

Security Best Practices

• Never commit .env file to version control
• Use environment-specific .env files
• Rotate bot tokens periodically
• Monitor logs for suspicious activity
• Keep Docker images updated

Troubleshooting

Bot not sending messages

1. Verify bot token is correct
2. Ensure bot is admin in the channel
3. Check channel ID format (@channelname or -100...)

No posts being processed

1. Verify USE_MOCK_PROVIDER=true for testing
2. For Twitter: Check accounts are valid and logged in
3. Review keyword configuration
4. Check POLL_INTERVAL_SECONDS is appropriate

High memory usage

1. Reduce DEDUP_CACHE_SIZE
2. Decrease MOCK_POSTS_PER_POLL
3. Increase POLL_INTERVAL_SECONDS

Container keeps restarting

# Check logs for errors
docker-compose logs --tail=100 telegram-monitor

# Verify environment variables
docker-compose config

Development

Running Tests

# With mock provider (default)
USE_MOCK_PROVIDER=true python main.py

# With Twitter scraper
USE_MOCK_PROVIDER=false python main.py

# Test with specific keywords
KEYWORDS="test,debug" python main.py

Code Style

• Type hints on all functions
• Async/await for I/O operations
• Docstrings for classes and methods
• PEP 8 formatting

License

This project is provided as-is for monitoring social media content.

Support

For issues or questions:

1. Check logs: docker-compose logs -f
2. Review configuration in .env
3. Verify bot permissions in Telegram

Roadmap

• Twitter data scraping with twscrape
• Database persistence for historical data
• Web dashboard for statistics
• Multiple channel support
• Sentiment analysis
• Custom alert templates
• Advanced filtering rules (regex, user lists)
• Twitter Spaces monitoring
• Support for other platforms (Reddit, Discord)

---

Built with ❤️ using Python, asyncio, twscrape, and Telegram Bot API