Skip to content

Recover a rotated ZDF key by reading the one its player uses - #125

Merged
ChrisonSimtian merged 1 commit into
developfrom
feat/zdf-key-discovery
Aug 23, 2026
Merged

Recover a rotated ZDF key by reading the one its player uses#125
ChrisonSimtian merged 1 commit into
developfrom
feat/zdf-key-discovery

Conversation

@ChrisonSimtian

Copy link
Copy Markdown
Contributor

Closes #112.

The spike first, since the issue said the answer might be "don't"

It isn't. The fear was that the key lives in a hash-named JS chunk, making the scrape rot faster than the key rotates. Measured against the live site instead:

GET https://www.zdf.de/                     → 5.6 MB, contains the key once
  \"apiAuthToken\":\"aa3noh4…Quah7el\"      ← exactly the key we ship
  \"appToken\":{\"apiToken\":\"aa3noh4…\",\"expiresAt\":\"2026-08-23T19:12:56+02:00\"}

extract → GET api.zdf.de/search/documents   → 200

Server-rendered data at a fixed URL, under a property name that is unambiguous — the page carries a second, different token under apiToken, and only apiAuthToken is the one the API wants. Sub-pages don't carry it, so the homepage it is, 5.6 MB and all.

What it does

On a 401/403 the client reads the current key, verifies it with a live call, and only then adopts it and retries. That verification is the whole safety argument: a scrape of an undocumented page fails by returning something plausible, and sending a garbage bearer would turn "your key is stale" into an unexplained failure inside the recovery path. A failed discovery lands exactly where today's behaviour lands — loud error, degraded health.

Other guards:

  • Rate-limited to one attempt per 15 minutes, single-flight. The source page is 5.6 MB; a fleet whose key just rotated would otherwise pull it once per failed request.
  • Finding the key we already send is not a recovery. It means the rejection is about something else, and "recovering" to the same value would loop.
  • The recovered key is memory-only, and health stays Degraded while it is in use, saying so. It is a recovery, not a second place to configure the key — a restart deliberately goes back to what you configured.

One reversal from the issue

I wrote "probably opt-in, default off, until it has been seen to work through at least one real rotation". It ships default on, because the reasoning behind that caution didn't survive the spike: the source is stable rather than brittle, and nothing is adopted without a live 200. Opt-in would mostly mean the feature is off in exactly the deployments that need it — unattended ones. Zdf__KeyDiscovery__Enabled=false turns it off.

Tests

Eight unit tests over a stub serving both the player page and the API: reads and verifies, refuses a candidate the API also rejects, survives a redesigned page, ignores a no-op "recovery", rate-limits, stays inert when disabled, recovers a crawl that used to throw, and reports the right health.

Plus a live test that reads the key off the real page and proves it authenticates — so a ZDF redesign fails in ./build.sh TestLive rather than during someone's outage.

./build.sh Test green: 500 passing.

The spike this issue asked for found the key in a far better place than feared:
ZDF's homepage publishes it as apiAuthToken in its own server-rendered data, at a
stable URL — not inside a hash-named JS chunk. Extracting it and calling the API
with it returns 200.

So a rejection now triggers a read of the current key, a verification call, and a
retry. Nothing is adopted before it has answered 200, which is what keeps a failed
scrape equal to today's behaviour rather than worse than it.

The recovered key is in-memory only and health stays degraded while it is in use:
it is a recovery, not a second place to configure the key.

Closes #112

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ChrisonSimtian ChrisonSimtian added the enhancement New feature or request label Aug 23, 2026
@ChrisonSimtian
ChrisonSimtian merged commit 31d7fe8 into develop Aug 23, 2026
1 check passed
@ChrisonSimtian
ChrisonSimtian deleted the feat/zdf-key-discovery branch August 23, 2026 01:17
@ChrisonSimtian ChrisonSimtian mentioned this pull request Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Discover the ZDF Api-Auth key from the web player when the configured one is rejected

1 participant