Skip to content

feat: add SerpBase (Google SERP API) as a fourth search engine - #14

Open
gefsikatsinelou wants to merge 1 commit into
Socialpranker:mainfrom
gefsikatsinelou:feat/add-serpbase-engine
Open

gefsikatsinelou wants to merge 1 commit into
Socialpranker:mainfrom
gefsikatsinelou:feat/add-serpbase-engine

Conversation

@gefsikatsinelou

Copy link
Copy Markdown

Summary

Adds SerpBase (serpbase engine) as a fourth direct search-engine API in scripts/search_query.py, alongside brave/tavily/exa. SerpBase returns real Google organic results over a REST API — useful for SEO/competitive-intel sub-questions where the current three engines (own-index/semantic) don't reflect actual Google ranking.

Background

The skill already documents that a second search axis is required when an engine key is present (references/capability_discovery.md, references/source_dispatch.md), and it already keeps a reference card for SerpAPI (references/api_sources/search/serpapi.md) — but SerpAPI is explicitly marked "не используется — платный, без free tier" in the env-audit table and stays out of KNOWN_KEYS (see the comment in runner/capabilities.py and DEAD_SEARCH_KEYS in tests/test_docs_swarm.py). SerpBase covers that same "real Google" niche with a free tier (100 free searches, no credit card) and pay-as-you-go pricing, so it passes the exact gate SerpAPI fails: it's a search engine the skill actually calls, not one it advertises and can't use.

Changes

  • scripts/search_query.py — new serpbase engine: _request_serpbase (GET https://api.serpbase.dev/google/search, X-API-Key header, q/num params), _normalize_serpbase (organic[] → title/link/snippet), registered in ENV_VARS, BUILDERS, NORMALIZERS; docstring updated. Key: SERPBASE_API_KEY.
  • runner/capabilities.py — SERPBASE_API_KEY added to KNOWN_KEYS (18 keys), so Phase 3.5 env audit surfaces it as an available second search engine.
  • references/api_sources/search/serpbase.md (new) — reference card mirroring the serpapi.md/exa.md format: contract, auth, when-to-use, example Phase-4 queries, fallback order.
  • references/capability_discovery.md, references/source_dispatch.md, scripts/README.md — engine list and env-key rows extended to include serpbase.
  • Tests — tests/test_search_query.py::test_serpbase_request_shape (request shape + normalization, mirrors the brave/tavily/exa tests); SERPBASE_API_KEY added to LIVE_SEARCH_KEYS in tests/test_docs_swarm.py so the wiring test covers it; key-count assertion updated in tests/test_capabilities.py.

Design decisions

Decision Rationale
X-API-Key header auth SerpBase API contract (matches its current docs); SERPBASE_API_KEY env convention matches *_API_KEY pattern of the other engines
organic[] → shared {rank,title,url,snippet} shape Same normalizer contract as brave/tavily/exa — zero changes needed downstream in normalize() / run_query()
Added to KNOWN_KEYS (unlike SerpAPI) SerpAPI is excluded because it has no free tier; SerpBase has 100 free searches — the audit only advertises engines users can actually enable
Docs updated in the same PR The skill decides search axes from capability_discovery.md/source_dispatch.md; code without docs would be a dead engine the skill never calls

Testing

  • python3 -m pytest tests/test_search_query.py tests/test_capabilities.py tests/test_docs_swarm.py — 28 passed
  • New smoke test: with SERPBASE_API_KEY set, run_query("serpbase", ...) issues GET https://api.serpbase.dev/google/search with X-API-Key and normalizes organic[]; empty organic → [] (no crash); missing key → exit 2 with the env-var name, same as the other engines.
  • ruff check on changed files: no new findings (repo-wide pre-existing findings unchanged).

- scripts/search_query.py: register serpbase engine (GET
  https://api.serpbase.dev/google/search, X-API-Key header) and normalize
  organic[] hits into the shared {rank,title,url,snippet} contract
- runner/capabilities.py: audit SERPBASE_API_KEY alongside brave/tavily/exa
- docs: capability_discovery.md, source_dispatch.md, scripts/README.md and a
  new references/api_sources/search/serpbase.md card (mirrors serpapi.md)
- tests: request-shape test + live-search-key wiring test

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant