🤖 AI Sales Agent - Autonomous Lead Generation & Outreach System

Multi-agent system for automated B2B lead generation, qualification, and personalized outreach using local AI models (Ollama)

🚀 Overview

This system orchestrates multiple specialized agents to automate the entire sales pipeline:

1. Web Scraping Workers - Extract contact data from business directories (Zillow, Realtor.com, LinkedIn, etc.)
2. AI Scoring Brain - Qualify leads using Ollama AI (phi3:medium, llama3.1)
3. AI Email Brain - Generate personalized outreach emails and send via SMTP
4. Orchestrator - Coordinates all agents, manages queue, provides dashboard

Privacy-First: All AI runs locally via Ollama. No data sent to OpenAI/Claude.

✨ Key Features

• 🧠 Local AI Processing - Uses Ollama (no API costs, full privacy)
• 🕷️ Intelligent Scraping - Playwright-based with anti-bot measures
• 📊 Lead Scoring - AI-powered qualification (1-10 score)
• 📧 Personalized Emails - Context-aware outreach generated by AI
• 🔄 Autonomous Operation - Set it and forget it (runs on schedules)
• 📈 Real-time Dashboard - Monitor all agents and queue status
• 🗄️ Dual Database Support - Supabase (primary) or Airtable (legacy)
• 🎯 CAN-SPAM Compliant - Built-in unsubscribe handling

🏗️ Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Orchestrator (Node.js)                  │
│                   Coordinates Everything                    │
└────────────┬────────────────────┬─────────────────┬────────┘
             │                    │                 │
   ┌─────────▼──────┐  ┌─────────▼──────┐  ┌──────▼─────────┐
   │  Worker Agent  │  │  Brain 1       │  │  Brain 2       │
   │  (Scraper)     │  │  (Scorer)      │  │  (Emailer)     │
   │                │  │                │  │                │
   │ • Web scraping │  │ • AI scoring   │  │ • AI email gen │
   │ • Playwright   │  │ • Ollama LLM   │  │ • SMTP sending │
   │ • Anti-bot     │  │ • Qualify/DQ   │  │ • Airtable API │
   └────────────────┘  └────────────────┘  └────────────────┘
             │                    │                 │
             └────────────────────┴─────────────────┘
                              │
                    ┌─────────▼──────────┐
                    │  Database          │
                    │  (Supabase/        │
                    │   Airtable)        │
                    └────────────────────┘

📋 Prerequisites

Required Software

• Node.js >= 18.0.0 (Download)
• Python 3.8+ (Download)
• Ollama (Download) - For local AI models
• Git (for cloning)

AI Models (Ollama)

# Install AI models
ollama pull phi3:medium      # Lightweight, fast (14GB)
ollama pull llama3.1:8b      # More powerful (4.7GB)

Database

Choose ONE:

• Supabase (recommended) - Free tier, PostgreSQL-based
• Airtable (legacy) - Simpler but limited API

Email Provider

• SMTP server (Hostinger, Gmail, SendGrid, etc.)
• Valid sender email and password

🚀 Quick Start

1. Clone & Install

git clone <your-repo-url>
cd "AI Sales Agent"
npm install

2. Configure Environment

Copy .env.example to .env and fill in your credentials:

cp .env.example .env

Required Variables:

# Email
HOSTINGER_EMAIL_1=your-email@yourdomain.com
HOSTINGER_PASSWORD=your_password

# Database (choose one)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_ANON_KEY=your_anon_key
# OR
AIRTABLE_API_KEY=your_airtable_key
AIRTABLE_BASE_ID=your_base_id

# Agent URLs (if running on multiple machines)
RIPI_1_URL=http://localhost:8001
M715Q_BRAIN_URL=http://localhost:6002
M73_BRAIN_URL=http://localhost:6001

3. Setup Database

Option A: Supabase (Recommended)

# Windows
setup-supabase.bat

# Mac/Linux
bash setup-supabase.sh

See SUPABASE_QUICKSTART.md for detailed instructions.

Option B: Airtable

Create tables: Leads, Profiles, Outreach, Unsubscribe (Schema included in docs/DATA_VALIDATION.md)

4. Start the System

All-in-One (Single Machine)

# Windows
START.bat

# Mac/Linux  
bash START.sh

This launches:

• Orchestrator on :5000
• Dashboard on :4000
• Workers on :8001, :8002
• Brains on :6001, :6002

Distributed (Multiple Machines)

See docs/SYSTEM_MAP.md for multi-machine setup.

# On Machine 1 (Worker)
bash START-RPI.sh

# On Machine 2 (Brain 1)
START-M715Q.bat

# On Machine 3 (Brain 2)
START-M73.bat

# On Main PC (Orchestrator)
START.bat

5. Access Dashboard

Open http://localhost:4000 in your browser to:

• Monitor all agents
• View scraping queue
• Check email statistics
• See real-time logs

📚 Documentation

File	Description
QUICK_START.md	Step-by-step launch guide
SYSTEM_MAP.md	Architecture & data flow
URL_DISCOVERY.md	How web scraping works
DATA_VALIDATION.md	Database schemas
SUPABASE_QUICKSTART.md	Supabase setup guide

🎯 How It Works

1. URL Discovery

The orchestrator uses AI to generate search queries:

// AI generates queries like:
"Miami real estate agent contact information"
"Florida realtors with email"

// Scrapes DuckDuckGo results
// Finds: zillow.com/profiles/..., realtor.com/agent/...

2. Profile Scraping

Workers scrape agent profiles:

# Extract from Zillow, Realtor.com, etc.
{
  "name": "John Smith",
  "email": "john@smithrealty.com",
  "phone": "(305) 555-1234",
  "location": "Miami, FL",
  "website": "smithrealty.com"
}

3. AI Lead Scoring

Brain 1 scores each lead (1-10):

// AI analyzes:
- Years of experience (newer = better)
- Business type (independent = best)
- Location hazard risk (high = better fit)
- Tech adoption (website/social media)

// Returns:
{
  score: 8,
  reasoning: "3-year agent in Miami (flood+hurricane) - perfect fit",
  qualification: "qualified"
}

4. AI Email Generation

Brain 2 generates personalized email:

// Context-aware based on:
- Agent's location (Miami = mention hurricanes)
- Business type (independent = emphasize white-label)
- Website presence (tech-savvy = mention API)

// Sends via SMTP
// Updates database: "contacted"

⚙️ Configuration

Target Market Customization

Edit .env to change who you target:

TARGET_MARKET=small_independent_agents
VALUE_PROPOSITION=low-cost white-label hazard reports

# Or customize for your business:
TARGET_MARKET=mortgage_brokers
VALUE_PROPOSITION=instant property risk assessments

AI Model Selection

# Faster, lighter (phi3:medium - 14GB)
SOCIAL_BRAIN_MODEL=phi3:medium
SALES_BRAIN_MODEL=phi3:medium

# More accurate (llama3.1:8b - requires more RAM)
SOCIAL_BRAIN_MODEL=llama3.1:8b
SALES_BRAIN_MODEL=llama3.1:8b

Timing & Intervals

In autonomous-orchestrator.js:

// Discovery: Every 60 minutes
setInterval(discoverNewURLs, 60 * 60 * 1000);

// Scraping: Every 5 minutes
setInterval(assignScrapingJobs, 5 * 60 * 1000);

In brain-m715q.js (scoring):

// Score leads: Every 3 minutes
setInterval(processLeads, 3 * 60 * 1000);

In brain-m73.js (emailing):

// Send emails: Every 5 minutes
setInterval(processQualifiedLeads, 5 * 60 * 1000);

🔒 Privacy & Compliance

CAN-SPAM Compliance

• ✅ Unsubscribe link in every email
• ✅ Physical address included
• ✅ Accurate "From" information
• ✅ No email to unsubscribed addresses

Data Privacy

• ✅ All AI processing done locally (Ollama)
• ✅ No data sent to OpenAI/Anthropic
• ✅ Scraped data stored in your database only
• ✅ SMTP credentials encrypted in .env

Rate Limiting

• Web scraping: 5-10 second delays between requests
• Email sending: 1 email per minute (configurable)
• AI processing: No external API rate limits

🐛 Troubleshooting

"Ollama connection refused"

# Check if Ollama is running
ollama list

# If not, start it
ollama serve

# Test endpoint
curl http://localhost:11434/api/generate

"No routes found" / Scraping Fails

# Check worker agent logs
curl http://localhost:8001/logs

# Common issues:
- Playwright not installed: npm install
- Anti-bot detected: Increase delays in proxy-manager.js
- Bad proxy: Check proxy-manager.js proxy list

"Email not sending"

# Test SMTP credentials
node -e "const nodemailer = require('nodemailer'); 
const t = nodemailer.createTransport({
  host: 'smtp.hostinger.com', 
  port: 587, 
  auth: {user: 'your@email.com', pass: 'yourpass'}
}); 
t.verify().then(console.log).catch(console.error);"

# Common issues:
- Wrong SMTP port: Use 587 for STARTTLS, 465 for SSL
- 2FA enabled: Create app-specific password
- IP blocked: Check spam blacklists

Dashboard Not Loading

# Check if all services are running
curl http://localhost:5000/health  # Orchestrator
curl http://localhost:4000          # Dashboard
curl http://localhost:8001/health  # Worker
curl http://localhost:6002/health  # Brain 1
curl http://localhost:6001/health  # Brain 2

📊 Performance

Expected Throughput

• Scraping: 50-100 profiles/hour (with anti-bot delays)
• Scoring: 100+ leads/hour (limited by AI inference speed)
• Emailing: 10-20 emails/hour (SMTP rate limits)

Resource Usage

Component	CPU	RAM	Disk
Orchestrator	2-5%	100MB	Minimal
Worker (Playwright)	10-20%	500MB	Minimal
Brain (Ollama phi3)	30-60%	8-10GB	14GB model
Brain (Ollama llama3.1)	40-70%	6-8GB	5GB model

Scaling

• Single machine: All agents on one PC (16GB RAM minimum)
• Distributed: Separate machines for workers/brains (8GB each)
• Worker scaling: Add more Raspberry Pis for faster scraping
• Brain scaling: Add more PCs with Ollama for parallel processing

🛠️ Development

Project Structure

AI Sales Agent/
├── autonomous-orchestrator.js  # Main coordinator
├── brain-m715q.js              # Lead scoring agent
├── brain-m73.js                # Email sending agent
├── lead-processor.js           # Legacy processor
├── proxy-manager.js            # Proxy rotation
├── url-assignment.js           # Job distribution
├── url-discovery.js            # Search query generation
├── ripi_scraper.py            # Python scraper worker
├── supabase-db.js             # Supabase integration
├── dashboard/
│   ├── server.js              # Dashboard backend
│   └── public/                # Dashboard UI
├── scraper-agent/             # Python scraper package
└── docs/                      # Documentation

Adding New Scrapers

1. Add target site to ripi_scraper.py:

async def scrape_newsite(page, url):
    # Your scraping logic
    return {
        'name': extracted_name,
        'email': extracted_email,
        # ...
    }

2. Register in orchestrator:

const SUPPORTED_SITES = [
    'zillow.com',
    'realtor.com', 
    'newsite.com'  // Your new site
];

Customizing Email Templates

Edit brain-m73.js:

const emailPrompt = `Generate a personalized email for...

Your custom instructions here...

Agent Profile:
- Name: ${agentName}
- Location: ${location}
...`;

🤝 Contributing

Contributions welcome! Areas for improvement:

• Additional scraper sites (Redfin, LinkedIn Premium)
• More AI models (Mixtral, CodeLlama)
• Reply detection & auto-response
• A/B testing for email templates
• Advanced anti-bot evasion
• Multi-language support

📄 License

MIT License - Free to use, modify, and distribute.

See LICENSE file for details.

⚠️ Legal Disclaimer

This tool is for educational purposes and legitimate B2B outreach only.

Your Responsibilities:

• ✅ Follow robots.txt and website ToS
• ✅ Comply with CAN-SPAM, GDPR, CCPA
• ✅ Only email businesses (B2B), not consumers (B2C)
• ✅ Honor unsubscribe requests immediately
• ✅ Rate limit scraping to avoid server overload

Prohibited Uses:

• ❌ Scraping without respecting robots.txt
• ❌ Sending unsolicited consumer emails (spam)
• ❌ Harvesting personal data without consent
• ❌ DDoS or overwhelming target servers

We are not responsible for misuse. Use ethically and legally.

📞 Support

For issues or questions:

1. Check docs/ folder for detailed guides
2. Review troubleshooting section above
3. Open a GitHub issue with:
   • Error logs from /logs endpoints
   • Environment details (OS, Node version)
   • Steps to reproduce

🙏 Credits

Built by Robert @ RisqMap for automated B2B lead generation.

Technologies:

• Ollama - Local LLM inference
• Playwright - Web scraping
• Supabase - PostgreSQL database
• Node.js - Backend runtime
• Express - Web framework

---

Star ⭐ this repo if you find it useful!