Skip to content

[Bug] Flaky auto-fetch #48

Description

@mszczepanczyk

Hi, the auto-fetch feature seems very flaky in docker. I've managed to fix a couple of issues (you can check the last commits in my fork: main...mszczepanczyk:domain-locker:main) but in the end I've given up. For some domains it works, for some it doesn't. Setting DNS_DUMPSTER_TOKEN doesn't help, haven't tried shodan.

Here's what happens when it does work:

Nov 20 11:12:24 homelab docker-domain-locker-start[2940573]: 🔵 [domain-info] Resolving domain info for: google.com
Nov 20 11:12:25 homelab docker-domain-locker-start[2940573]: 🟢 [whois] Got WHOIS data via whois-json for google.com
Nov 20 11:12:54 homelab docker-domain-locker-start[2940573]: 🟢 [domain-info] Successfully resolved: google.com

Here's what happens when it doesn't:

Nov 20 11:13:19 homelab docker-domain-locker-start[2940573]: 🔵 [domain-info] Resolving domain info for: mariusz.sh
Nov 20 11:13:19 homelab docker-domain-locker-start[2940573]: 🟢 [whois] Got WHOIS data via whois-json for mariusz.sh
Nov 20 11:13:29 homelab docker-domain-locker-start[2940573]: 🟡 [domain-info] SSL cert fetch failed: getaddrinfo ENOTFOUND mariusz.sh
Nov 20 11:13:48 homelab docker-domain-locker-start[2940573]: 🟢 [domain-info] Successfully resolved: mariusz.sh
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]: SyntaxError: Unexpected token '<', "<html>
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:  <h"... is not valid JSON
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at JSON.parse (<anonymous>)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at parseJSONFromBytes (node:internal/deps/undici/undici:5329:19)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at successSteps (node:internal/deps/undici/undici:5300:27)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at fullyReadBody (node:internal/deps/undici/undici:1447:9)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at async specConsumeBody (node:internal/deps/undici/undici:5309:7)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at async fetchSubdomains (file:///app/dist/analog/server/chunks/routes/domain-subs.mjs:41:18)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:     at async Object.handler (file:///app/dist/analog/server/chunks/routes/domain-subs.mjs:150:27)
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]: 🔴 [domain-subs] Error fetching from https://api.shodan.io/dns/domain/mariusz.sh: SyntaxError: Unexpected token '<', "<html>
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]:  <h"... is not valid JSON
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]: 🔴 [domain-subs] DNSDumpster token is not configured
Nov 20 11:13:49 homelab docker-domain-locker-start[2940573]: 🔴 [domain-subs] Error fetching subdomains for mariusz.sh: Error: Both primary and fallback services failed

Activity

  1. changed the title [-]Flaky auto-fetch[/-] [+][Bug] Flaky auto-fetch[/+] on Nov 20, 2025
  2. alexindigo commented on Dec 6, 2025

    @alexindigo

    Same here.
    Would be good to have config with ability to customize endpoints, and/or provide API keys to prevent rate limiting.

  3. liss-bot commented on Sep 22, 2026

    @liss-bot
    Collaborator

    Hey @mszczepanczyk, this has now been implemented in #128, and will be released shortly in v0.2.10 😇
    We're sorry this one took so long 😔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions