Implement v1 API: versioned, dataset-scoped, standards-compliant backend

[tomanizer]

Overview

The QueryService API needs to be rebuilt to v1 standards before any external clients are added. This is a greenfield redesign — no backwards compatibility is required since the API is not yet live.

The full design is specified in docs/api-v1-proposal.md.

Design goals

• All routes versioned under /api/v1/
• Dataset identity in the URL path, not the request body
• Named fields in all responses (no positional index maps)
• Full aggregation parity with the Huey frontend (all 23 functions)
• Dataset discovery endpoints
• Standards-compliant HTTP semantics (correct status codes, HEAD support, ETag, sparse responses)
• One error envelope, machine-readable error codes

Implementation order

Issues are listed in recommended implementation order. Each issue depends on those above it in the same group.

Foundation (implement first — everything depends on this)

• [v1 API] 1 — Versioning infrastructure: register all routers under /api/v1/ #404 — Versioning infrastructure: register all routers under /api/v1/, service root, versioned OpenAPI
• [v1 API] 2 — Response headers: X-API-Version, X-RateLimit-*, client metadata via request headers #405 — Response headers: X-API-Version, X-RateLimit-*, client metadata via request headers

Discovery

• [v1 API] 3 — Dataset discovery: GET /api/v1/datasets, /{id}, /{id}/schema #406 — Dataset discovery: GET /api/v1/datasets, GET /api/v1/datasets/{id}, GET /api/v1/datasets/{id}/schema

Core query API (implement together — tightly coupled)

• [v1 API] 4 — Query endpoints: versioned paths, dataset in URL, updated request model #407 — Query endpoints: versioned paths, dataset in URL, updated request model
• [v1 API] 5 — Cells response: named rows/columns/cells arrays, sparse cells, window totals #408 — Cells response: named rows/columns/cells arrays, sparse cells, window totals
• [v1 API] 6 — Members endpoint: per-value count, meta on all query responses, tuples named items #409 — Members endpoint: per-value count, meta on all query responses, tuples named items

Exports

• [v1 API] 7 — Export API redesign: dataset-scoped submit, list, DELETE, HEAD on file, 202, new formats #410 — Export API redesign: dataset-scoped submit, list, DELETE, HEAD on file, 202, new formats

Query capabilities (independently parallelisable after #407)

• [v1 API] 8 — Filter operator expansion: gt, gte, lt, lte, is_null, not_null #411 — Filter operator expansion: gt, gte, lt, lte, is_null, not_null
• [v1 API] 9 — Cells axis sorting: sort rows and columns by measure or field #412 — Cells axis sorting: sort rows and columns by measure or field
• [v1 API] 10 — Aggregation function expansion: full parity with Huey frontend (23 functions) #413 — Aggregation function expansion: full 23-function parity with Huey frontend
• [v1 API] 11 — Derivations: named field transforms on dimension axes #414 — Derivations: named field transforms on dimension axes (depends on [v1 API] 10 — Aggregation function expansion: full parity with Huey frontend (23 functions) #413)
• [v1 API] 12 — Parameterized aggregations: quantile_cont, quantile_disc, iqr #415 — Parameterized aggregations: quantile_cont, quantile_disc, iqr (depends on [v1 API] 10 — Aggregation function expansion: full parity with Huey frontend (23 functions) #413)
• [v1 API] 13 — Bivariate aggregations: correlation with field2 #416 — Bivariate aggregations: correlation with field2 (depends on [v1 API] 10 — Aggregation function expansion: full parity with Huey frontend (23 functions) #413)

Advanced analytics

• [v1 API] 14 — Outlier detection endpoint: POST /api/v1/datasets/{id}/query/outliers #417 — Outlier detection endpoint: POST /api/v1/datasets/{id}/query/outliers

Target route map

GET  /health/liveness
GET  /health/readiness
GET  /health/startup

GET  /api/v1
GET  /api/v1/datasets
GET  /api/v1/datasets/{id}
GET  /api/v1/datasets/{id}/schema

POST /api/v1/datasets/{id}/query/tuples
POST /api/v1/datasets/{id}/query/cells
POST /api/v1/datasets/{id}/query/members
POST /api/v1/datasets/{id}/query/outliers

POST   /api/v1/datasets/{id}/exports
GET    /api/v1/exports
GET    /api/v1/exports/{export_id}
DELETE /api/v1/exports/{export_id}
GET    /api/v1/exports/{export_id}/file  (HEAD also)

GET  /api/v1/openapi.json
GET  /api/v1/docs
GET  /api/v1/redoc

[tomanizer]

Progress update: opened PR for #405 and implemented the response-header/client-metadata slice on branch backend-v1-headers.

PR: will attach automatically in the issue timeline once created.

[tomanizer]

Progress update: #405 is complete and ready to merge; moving on to #406 (dataset discovery) next.

[tomanizer]

Progress update: opened PR for #406 (dataset discovery) on branch backend-v1-datasets.

This adds /api/v1/datasets, /api/v1/datasets/{dataset_id}, and the dataset-scoped schema endpoint with ETag support.

[tomanizer]

Progress update: #421 was updated after review with discovery-link cleanup, conditional GET fixes, path-safe dataset discovery routes, and a lighter list-path profiling strategy. Waiting on refreshed CI before merge.

[tomanizer]

Progress update: #406 is complete and merged; moving on to #407 (query endpoint request/response redesign) next.

[tomanizer]

Progress update: opened PR for #407 on branch backend-v1-query-endpoints.

This moves the query endpoints to the new top-level v1 request shapes, removes dataset_id from query bodies, makes date_range optional, and renames the members endpoint to /query/members.

[tomanizer]

Progress update: #407 is complete and merged. Starting #408 (cells response redesign) next on a fresh branch from dev.

[tomanizer]

Progress update: opened PR for #408 on branch backend-v1-cells-response.

This replaces the positional cells payload with named rows, columns, sparse cells, axis window totals, and response meta, while keeping the remote frontend path working.

[tomanizer]

Progress update: #408 is complete and merged. Moving on to #409 next to finish the query-response sprint.

[tomanizer]

Backend sprint update:

• [v1 API] 6 — Members endpoint: per-value count, meta on all query responses, tuples named items #409 is now implemented on branch backend-v1-members-response and is being opened for review.
• Scope: tuples now return named items + meta, members now return { field, items[{value,count}], meta }, and the frontend remote consumers/tests/docs were updated in lockstep.
• Validation completed locally:
  • ./.venv-server/bin/ruff check server/
  • ./.venv-server/bin/pytest server/tests -q
  • ./.venv-server/bin/pytest tests/test_backend_benchmark_runner.py -q
  • npm run test:unit
  • npm run lint:js
  • npm run test:ui -- --grep "remote datasource pivot with measure renders without typeId error" (Chromium passed; Firefox/WebKit unavailable locally because Playwright browsers are not installed)

Proceeding with the PR/review cycle now.

[tomanizer]

Backend sprint update:

• #425 merged and #409 is now closed.
• Tuples now return named items with shared meta; members now return { field, items[{value,count}], meta }.
• Follow-up fix included in the same PR: dimension-cache prewarming now uses the members cache key/payload shape, with regression coverage in server/tests/test_prewarm.py.

Proceeding to #411 next to expand filter operators and unblock #417.

[tomanizer]

Backend sprint update:

• #411 is implemented on branch backend-v1-filter-operators and is being opened for review.
• Scope: lowercase/case-insensitive filter operators, new comparison/null operators, filter-specific 422 FILTER_INVALID responses, SQL builder support, and remote adapter alignment.
• Validation completed locally:
  • ./.venv-server/bin/ruff check server/
  • ./.venv-server/bin/pytest server/tests -q
  • ./.venv-server/bin/pytest tests/test_backend_benchmark_runner.py -q
  • npm run test:unit
  • npm run lint:js

Proceeding with the PR/review cycle now.