Lead-Gen Automation Pipeline

Automating contact discovery and phone enrichment for local business leads.

---

Overview

This project automates business contact research for a client in commercial insurance sales. It replaces hours of manual searching with a two-stage enrichment pipeline powered by Google APIs and GPT.

The system takes a list of local businesses (e.g. bars, gyms, salons) and automatically fills in missing manager names, titles, and phone numbers — producing a clean, outreach-ready dataset.

---

Why it Matters

• Saved significant manual labour — what previously took hours per list now runs in minutes.
• High data quality — verified directly from Google APIs and GPT reasoning over real search snippets.
• Repeatable & auditable — each stage writes incrementally to Excel and skips existing values.
• Direct business impact — reduced lead-research costs for the client by thousands in labour time.

---

Pipeline Summary

Stage	Script	Description
1. Contact Discovery	src/main_pipeline.py	Uses Google Custom Search API to find web snippets per business, then GPT-4 extracts First Name, Last Name, Title.
2. Phone Enrichment	src/enrich_phone_numbers.py	Uses Google Places API (places:searchText) to retrieve each business’s verified phone number.

Each stage reads/writes to the same Excel sheet and can be re-run safely (idempotent).

---

Example Before & After

Before

Company Name	City	First Name	Last Name	Title	Phone 3 (Company)
Rosso Coffee Roasters	Calgary
Fit Republic Gym	Calgary
Hudsons Pub	Edmonton

After

Company Name	City	First Name	Last Name	Title	Phone 3 (Company)
Rosso Coffee Roasters	Calgary	John	Doe	Manager	+1 403-555-1034
Fit Republic Gym	Calgary	Sarah	Miller	Owner	+1 587-555-4412
Hudsons Pub	Edmonton	Tyler	Grant	General Manager	+1 780-555-8793

(Sample data shown for illustration.)

---

Architecture & Data Flow

1. Input: Excel sheet of location-based businesses (bars, gyms, etc.).

2. Contact Discovery:

   • Build query → “{Company} manager {City} Alberta”
   • Call Google Custom Search API → retrieve text snippets.
   • Pass snippets to GPT-4 → extract name/title into structured CSV format.

3. Phone Enrichment:

   • For each business, call Google Places API → retrieve verified phone number.

4. Output: Enriched Excel file ready for outreach or CRM import.

---

Tech Stack (and Why)

Component	Purpose
Python + Pandas	Data manipulation, Excel I/O
Google Custom Search API	Retrieve business-specific web snippets
OpenAI GPT-4	Extract structured contact info from snippets
Google Places API	Retrieve verified business phone numbers
python-dotenv	Manage secrets via .env file (never hard-coded)

---

Setup & Run

1. Install

pip install -r requirements.txt

2. Configure .env

GOOGLE_API_KEY=your_google_api_key
CSE_ID=your_google_cse_id
OPENAI_API_KEY=your_openai_api_key

(The phone enrichment script also uses GOOGLE_API_KEY.)

3. Verify config.py

INPUT_FILE = "data/input.xlsx"
OUTPUT_FILE = "data/output.xlsx"

4. Run Modules

Step 1 – Contact Discovery

python src/main_pipeline.py

Step 2 – Phone Enrichment

python src/enrich_phone_numbers.py

Both scripts will update the same Excel file incrementally.

---

Reliability & Design Notes

• Incremental writes: saves after each successful row.
• Rate-limit safety: built-in sleeps for Google API pacing.
• Idempotent: skips already-filled rows; safe to re-run.
• Structured output: CSV-style fields for easy CRM import.
• Secrets isolation: .env keeps keys out of source control.

---

Key Learnings

• API-first automation is reliable — structured endpoints beat scraping for production use.
• Idempotency matters — automation is most useful when safe to re-run anytime.
• Hybrid reasoning works best — pairing deterministic APIs (Google) with generative reasoning (GPT) produced high accuracy with minimal supervision.
• Business alignment — automation that directly cuts repetitive research time delivers real operational ROI.

---

Future Extensions

• Add email extraction using official APIs or domain-based lookup.
• Push results directly into a CRM (HubSpot / Pipedrive).
• Schedule daily/weekly runs via n8n, Zapier, or a lightweight cron job.

---

Quick Reference

# Contact discovery
python src/main_pipeline.py

# Phone enrichment
python src/enrich_phone_numbers.py