Skip to content

Block credential/exploit scanner paths at middleware#26

Open
stockyard-dev wants to merge 1 commit intomainfrom
block-scanner-paths
Open

Block credential/exploit scanner paths at middleware#26
stockyard-dev wants to merge 1 commit intomainfrom
block-scanner-paths

Conversation

@stockyard-dev
Copy link
Copy Markdown
Owner

Diagnosis

24h Cloudflare analytics on stockyard.dev (2026-04-21) showed:

  • 820 total requests, ~168 from non-datacenter ASNs (the rest were Tencent, Amazon, Alibaba, Google Cloud, Microsoft, OVH, Hetzner, etc.)
  • 227 404s, ~75% from a single coordinated credential scanner probing ~60 variants of .env, phpinfo.php, wp-admin/install.php, wp-login.php, /debug.php, /php.php from rotating IPs
  • None of these paths exist on the marketing site, so they will always 404 — but they hit the origin and burn a request through the full handler chain (recovery → gzip → admin auth → auto-config → proxy auth → RBAC → self-service auth → mux lookup → NotFound)

Fix

Adds scannerBlockMiddleware (internal/engine/scanner_block.go) wrapped as the outermost handler so it runs first on every request. Matches the path against 6 substrings (.env, phpinfo, wp-admin, wp-login, /debug.php, /php.php) and short-circuits with 403 before any other middleware or handler runs.

Cost per blocked request: a handful of strings.Contains calls. Cost per real request: same calls, all returning false.

Why not Cloudflare WAF

WAF custom rules at the zone level are available on Pro, but the API token I had was read-only and the dashboard navigation surfaced "Account-level WAF" (Enterprise add-on) instead of the per-zone Custom rules path. In-app middleware is portable, tracked in git, Cloudflare-independent, and visible to future-you. WAF can still be added later as defense-in-depth without conflict.

Test coverage

scanner_block_test.go runs 28 cases:

13 scanner paths blocked (403): /.env, /.env.bak, /.env.production, /.env.test, /backend/.env, /storage/.env, /phpinfo.php, /old_phpinfo.php, /hosting/phpinfo.php, /wp-admin/install.php, /wp-login.php, /debug.php, /php.php

15 real site paths pass through (200): /, /pricing/, /desktop/, /api/installer/macos, /api/installer/linux, /api/installer/windows, /api/recommend, /api/toolkit-count, /robots.txt, /sitemap.xml, /favicon.ico, /site-assets/..., /for/solo-developers/, /tools/, /playground, /v1/chat/completions

Expected impact

  • Eliminates ~170–200 daily 404s from analytics noise
  • Frees ~75% of origin Railway requests currently spent on scanner traffic
  • Real site paths unaffected (verified by test)
  • Cache hit rate dashboard becomes more meaningful (current 30/734 ratio is dragged down by uncacheable scanner 404s)

Out of scope (next steps if desired)

  1. Block the Singapore Tencent pricing-page scraper (8 different Chrome UA versions hammering /pricing/ from TENCENT-NET-AP-CN) — needs a separate ASN-based rule, ideally at Cloudflare
  2. Investigate /best-self-hosted-developer-tools/ 404 (5 hits, only non-scanner 404 in top 25) — possibly a missing SEO landing page someone is linking to
  3. Cache-everything page rule for /site-assets/* to lift cache hit rate

Adds outermost middleware that 403s requests for known scanner paths
(.env, phpinfo, wp-admin, wp-login, /debug.php, /php.php) before any
handler, template, or DB work runs.

Cloudflare WAF would do this at the edge, but custom rules require a
WAF-Edit token scope we don't currently have provisioned, and the
in-app approach is portable, tracked in git, and Cloudflare-independent.

Source: 24h Cloudflare analytics on stockyard.dev (2026-04-21) showed
~227 404s/day, ~75% from a single coordinated scanner probing ~60
.env/phpinfo/wp-admin variants. None of these paths exist on the site,
so blocking is safe.

Test covers all 13 observed scanner patterns + 15 real site paths
(/, /pricing/, /desktop/, /api/installer/{macos,linux,windows},
/api/recommend, /api/toolkit-count, /robots.txt, /sitemap.xml, etc.)
to confirm zero false positives.
@cloudflare-workers-and-pages
Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs
stockyard 91e8d2e Apr 22 2026, 02:28 AM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant