Skip to content

Webhook and event-replay endpoints have no contract tests: breaking changes go undetected #534

Description

@Xhristin3

Problem

The contract suite pins only the streams CRUD and auth endpoints. tests/contracts/src/ has auth.contract.ts (register, login) and streams.contract.ts (create, list, get, get-not-found, update) — and nothing else. The API surfaces added since the suite was built are unprotected: POST /webhooks and GET /webhooks/:id/deliveries (api/src/webhooks/webhooks.controller.ts), GET /streams/:id/events (event replay, issue #396), GET /streams/:id/analytics, GET /tags, POST/DELETE /streams/:id/tags, and the notifications endpoints (api/src/notifications/notifications.controller.ts).

The consequence is exactly what the contract suite exists to prevent (per tests/contracts/README.md and the contract.ts docstring: "the two sides can never independently drift"). A response-shape change to GET /streams/:id/events — e.g. the id/streamId stringification that already bit Stream and User — ships with green provider tests (api/src/contract-provider.spec.ts matches *.spec.ts) and green SDK consumer tests (xstreamroll-sdk/__tests__/contract.consumer.test.ts) because neither side asserts the shape. The SDK's WebhookDelivery type (xstreamroll-sdk/src/types.ts) is hand-maintained against GET /webhooks/:id/deliveries with zero machine-checked coverage; a field rename or a number-vs-string id change would not be caught.

Root cause

// tests/contracts/src/index.ts (exports) — only auth + streams are registered
export const contracts = [...authContracts, ...streamsContracts]

Why this is architecturally hard

  1. The contracts need real fixture state. streams.contract.ts already solves this with PLACEHOLDER.EXISTING_STREAM_ID substituted by the provider suite (api/src/contract-provider.spec.ts resolves placeholders at test time). The webhook contracts need the same treatment: create a stream, register a webhook, then exercise deliveries — the provider suite must sequence dependent contracts (register → list deliveries), which the current flat Contract[] array does not express natively. Decide how dependent chains are represented without rewriting the harness.
  2. The consumer side (xstreamroll-sdk/__tests__/contract.consumer.test.ts) mocks responses with nock; every new contract needs a matching mock shape on the SDK side, which means the SDK's types and the contract schemas must be authored together or they drift immediately.
  3. Some endpoints change state (webhook registration creates a delivery-capable subscription; notifications endpoints are user-scoped), so the provider suite needs per-contract setup/teardown hooks or the contracts must be order-independent. The existing suite avoids this by testing read-only shapes; webhooks break that assumption.

Acceptance criteria

Coverage

  • Contracts exist and pass (provider + consumer) for: POST /webhooks, GET /webhooks/:id/deliveries, GET /streams/:id/events, GET /streams/:id/analytics, and GET /notifications.
  • The contract schemas assert the exact field types the SDK types declare for WebhookSubscription and WebhookDelivery (xstreamroll-sdk/src/types.ts) — including the id/streamId string-vs-number choice — so a server-side type change fails CI.

Harness

  • Dependent contracts (create stream → register webhook → list deliveries) run in order without duplicating setup logic in the provider suite.
  • Consumer mocks in xstreamroll-sdk/__tests__/contract.consumer.test.ts mirror the provider responses; both suites pass with cd api && npm test and cd xstreamroll-sdk && npm test.

Tests

  • npm run build --workspace=tests/contracts succeeds and CI's contract jobs (api provider, sdk consumer) are green.
  • A deliberate breaking change to one covered endpoint (verified manually during development, not committed) fails the provider suite.

Out of scope

Expanding coverage to every endpoint (e.g. admin, audit, health) — this issue covers the webhook, event-replay, analytics, and notification surfaces.

Getting started

Real files in scope: tests/contracts/src/contract.ts (the Contract shape and PLACEHOLDER), tests/contracts/src/schemas.ts (zod schemas), tests/contracts/src/streams.contract.ts (pattern to copy), api/src/contract-provider.spec.ts, xstreamroll-sdk/__tests__/contract.consumer.test.ts, xstreamroll-sdk/src/types.ts (WebhookSubscription, WebhookDelivery).

Verify with:

npm run build --workspace=tests/contracts
cd api && npm test
cd ../xstreamroll-sdk && npm test

Good first files to read: tests/contracts/src/streams.contract.ts, api/src/contract-provider.spec.ts, xstreamroll-sdk/__tests__/contract.consumer.test.ts.

Metadata

Metadata

Labels

GrantFox OSSIssue tracked in GrantFox OSSMaybe RewardedIssue may be eligible for a GrantFox rewardThird CampaignCampaign: Third CampaignapiREST API design and endpointsdxsdkRelated to xstreamroll-sdk/testingTests and test coverage

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions