ScoutFlow — Autonomous Market Intelligence Daemon

An autonomous 24/7 background pipeline designed to monitor remote job boards, developer forums, and hiring threads. It extracts verified founder contacts using regex heuristics, enforces zero-cost atomic state deduplication with SHA-256 signatures, and dispatches high-priority alerts directly to Telegram.

---

📐 System Topology

┌────────────────────────────────────────────────────────┐
│          Continuous Schedule (GitHub Actions / VPS)    │
└───────────────────────────┬────────────────────────────┘
                            │ (Triggers every 3 hours)
                            ▼
┌────────────────────────────────────────────────────────┐
│                   ScoutFlow Engine                     │
│                                                        │
│   ┌────────────────────┐      ┌────────────────────┐   │
│   │ Jobicy Remote API  │      │ Hacker News Parser │   │
│   └─────────┬──────────┘      └─────────┬──────────┘   │
│             │                           │              │
│             └─────────────┬─────────────┘              │
│                           ▼                            │
│           ┌───────────────────────────────┐            │
│           │ Regex Email & Keyword Filter  │            │
│           └───────────────┬───────────────┘            │
│                           ▼                            │
│           ┌───────────────────────────────┐            │
│           │  SHA-256 State Deduplicator   │            │
│           │     (Atomic Hash Store)       │            │
│           └───────────────┬───────────────┘            │
└───────────────────────────┼────────────────────────────┘
                            │ Only New Opportunities
                            ▼
┌────────────────────────────────────────────────────────┐
│               Telegram Bot Alert Dispatch              │
│            (Immediate Actionable Leads)                │
└────────────────────────────────────────────────────────┘

---

⚡ Key Features

• Automated Web Scraping: Crawls high-volume developer hiring threads and remote APIs (Hacker News, Jobicy, RemoteOK).
• Regex Contact Extraction: Identifies direct hiring manager and founder contact email addresses automatically from unstructured markdown/HTML blocks.
• Atomic State Deduplication: Maintains a lightweight SHA-256 hash ledger (data/seen_hashes.json) ensuring each unique opportunity is only alerted once.
• Zero Heavy Infrastructure: Designed to run seamlessly in serverless cloud runners (GitHub Actions cron) or as a lightweight systemd Linux daemon on Ubuntu VPS.
• Rich Telegram Dispatch: Sends formatted markdown alerts including direct mailto links, company names, tech tags, and original post URLs.

---

🛠️ Tech Stack

• Language: Python 3 (3.10+)
• HTML Parsing: BeautifulSoup 4
• HTTP Client: Requests
• Automation: GitHub Actions Cron (.github/workflows/daemon-cron.yml)
• Messaging: Telegram Bot REST API

---

🚀 Quick Setup & Usage

1. Clone & Install Dependencies

git clone https://github.com/gurumaan/scoutflow.git
cd scoutflow
pip install -r requirements.txt

2. Environment Configuration

Copy the sample environment file:

cp .env.example .env

Fill in your credentials:

TELEGRAM_BOT_TOKEN=123456789:ABCdefGhIJKlmNoPQRstuvWxyz
TELEGRAM_CHAT_ID=987654321
CHECK_INTERVAL_SECONDS=3600
DRY_RUN=false

3. Execution Modes

Run Once (Simulated Dry-Run):

python -m src.daemon --once --dry-run

Run Continuous Daemon Loop:

python -m src.daemon

---

👤 Author & Architecture

Architected and maintained by Gursharan Singh (@gurumaan).

• Email: gurudeveloper05@gmail.com
• Portfolio: https://github.com/gurumaan/portfolio