Skip to content

Wire the webhook signature-failure spike alert #2

Description

@OliverD25

The alert exists, the threshold exists, and nothing ever raises it.

  • AlertCode.WEBHOOK_SIGNATURE_FAILURE_SPIKE is defined at src/crypto_processing_api/alerts/notifier.py:61.
  • webhook_signature_failure_threshold (default 10) is defined at src/crypto_processing_api/config.py:145.
  • src/crypto_processing_api/api/webhooks.py raises 401 invalid signature and logs webhook.bad_signature, but never counts anything.

So a burst of forged BTCPay webhooks — somebody probing, or a real
misconfiguration after a secret rotation — is invisible unless an operator
happens to be reading logs.

What to build

Count 401s from POST /webhooks/btcpay over a rolling window and call
notify(...) with WEBHOOK_SIGNATURE_FAILURE_SPIKE when the count crosses
webhook_signature_failure_threshold.

Things to decide, and the constraints

  • Where the counter lives. In-process is simplest and resets on restart,
    which is acceptable for a spike detector; a database counter survives
    restarts but adds a write to an unauthenticated path, which is itself a
    small denial-of-service surface. Say which you chose and why in the pull
    request.
  • Alert once per window, not once per request. An alert that fires 400
    times is an alert that gets muted. Look at how notify is used in
    workers/reconciliation.py for the existing rhythm.
  • Nothing about the request body may be logged or stored. The endpoint is
    unauthenticated, so anything derived from the payload is attacker-controlled
    text. The existing log line deliberately records only the client host and the
    byte count.

Scope

No money code. Nothing under ledger/ or services/ should need to change.

Where to start

src/crypto_processing_api/api/webhooks.py, src/crypto_processing_api/alerts/notifier.py.
Tests: tests/integration/test_webhook_ingress.py for the endpoint,
tests/integration/test_outbound_and_ops.py for how alerts are captured in
tests (RecordingTransport).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: opsOperator surface: alerts, health, reconciliation, runbooksgood first issueGood for newcomers

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions