Lead Scraper

A two-stage B2B lead scraper built on Botasaurus, the all-in-one anti-detection web-scraping framework.

1. Discover — searches Google Maps for businesses matching your queries and pulls each listing's name, category, address, phone, website, rating and review count (real, undetected Chrome).
2. Enrich — visits each business website and its contact/about pages to harvest emails, phone numbers, and social profiles (LinkedIn, Facebook, Instagram, Twitter/X) over fast HTTP requests.

Results are de-duplicated and written to both JSON and a tidy, spreadsheet-ready CSV.

Why Botasaurus

Google Maps actively blocks naive scrapers. Botasaurus drives a humane, fingerprint-resistant Chrome that renders Maps like a normal user, while its @request mode gives the enrichment stage a browser-grade TLS fingerprint without the cost of a full browser. You get reliable discovery and cheap, parallel enrichment in one framework.

Install

python -m pip install -r requirements.txt

Requires Python 3.8+ and Google Chrome installed (Botasaurus manages the driver). The first run downloads a few framework dependencies automatically.

Usage

Edit your target searches in config.py:

QUERIES = [
    "digital marketing agencies in Austin, TX",
    "coffee roasters in Portland, OR",
]

Then run:

python main.py

Or pass queries straight on the command line:

python main.py "dentists in Miami, FL" "dentists in Tampa, FL"
python main.py "vets in Denver, CO" --max 30        # more results per query
python main.py "law firms in London" --no-enrich    # discovery only, faster

Flag	Default	Description
--max	20	Max businesses per query
--scrolls	10	Max result-feed scrolls per query
--no-enrich	off	Skip website email/phone/social mining
--name	leads	Output file basename

Output

Files land in ./output/ as leads.json and leads.csv. Each lead looks like:

{
  "name": "Brightside Digital Marketing",
  "category": "Marketing agency",
  "address": "600 Congress Ave Suite 1400, Austin, TX 78701",
  "phone": "+1 512-555-0143",
  "website": "https://brightsidedigital.example",
  "rating": "4.8",
  "reviews": "127",
  "emails": ["hello@brightsidedigital.example"],
  "phones_from_site": ["(512) 555-0143"],
  "socials": {
    "linkedin": "https://www.linkedin.com/company/brightside-digital",
    "facebook": "https://facebook.com/brightsidedigital"
  }
}

See examples/sample_output.json for a full sample.

Project layout

lead-scraper/
├── main.py            # CLI + pipeline orchestrator (discover -> enrich -> save)
├── config.py          # your search queries and run settings
├── src/
│   ├── gmaps.py       # Stage 1: Google Maps discovery (@browser)
│   ├── enrich.py      # Stage 2: website email/phone/social enrichment (@request)
│   └── utils.py       # extraction regexes, URL helpers, de-duplication
└── examples/          # sample output

How it works

• src/gmaps.py opens the Maps search feed, scrolls until it stops growing (or hits --scrolls), collects each listing's place URL, then opens each place and reads the structured detail panel. Selectors have fallbacks; if Google ships a markup change, the extractors in _extract_place are the place to adjust.
• src/enrich.py fetches the homepage plus common contact paths (/contact, /about, …), prioritising mailto:/tel: anchors and falling back to text regexes. It stays on the lead's own domain and stops early once it has solid contact data. Runs 5 sites in parallel with response caching.
• src/utils.py holds the email/phone/social extraction and the domain-based de-duplication that merges the two stages.

Notes & responsible use

• Google Maps' internal markup changes from time to time. If discovery returns empty, update the selectors in src/gmaps.py.
• Caching is on for enrichment — delete cache/ to force a fresh crawl.
• Scrape responsibly: respect each site's Terms of Service and robots.txt, keep volumes reasonable, and comply with applicable privacy/marketing laws (GDPR, CAN-SPAM, etc.) when contacting leads.

License

MIT