Skip to content

api-reference.md

Latest commit

 

History

History
601 lines (456 loc) · 13.8 KB

api-reference.md

File metadata and controls

601 lines (456 loc) · 13.8 KB
Raw

QueryService API Reference

Base URL examples:

  • Local: http://localhost:8000
  • Containerized (mapped): http://<host>:8000

Interactive OpenAPI:

  • /api/v1/docs
  • /api/v1/redoc
  • /api/v1/openapi.json

Conventions

Authentication

  • Protected endpoints require X-API-Key only when QUERYSERVICE_AUTH_ENABLED=true.
  • Health endpoints do not require API key.

Request metadata headers

  • X-Request-ID is accepted and echoed in response headers.
  • X-Client-Version is accepted on requests and recorded in server logs.
  • Error envelopes usually include request_id when available.

Standard response headers

  • X-API-Version: 1 is returned on API responses.
  • X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset are returned when rate limiting is enabled for the endpoint.
  • Retry-After is returned on 429 responses.

Shared request shapes

{
  "date_range": {"type": "single", "date": "2026-03-01"},
  "... endpoint-specific fields ..."
}

date_range schema:

  • Single day: {"type": "single", "date": "YYYY-MM-DD"}
  • Inclusive range: {"type": "range", "start": "YYYY-MM-DD", "end": "YYYY-MM-DD"}

date_range behavior by dataset:

  • If dataset source defines a time filter column, date_range is applied to that column.
  • If dataset source has no time filter, date_range is accepted but ignored.

Common error envelope

{
  "code": "DATASET_NOT_FOUND",
  "message": "Dataset not found: trades_v1",
  "request_id": "trace-123",
  "details": {"dataset_id": "trades_v1"}
}

Endpoints

GET /health/liveness

Process liveness probe.

Authentication: not required.

Response 200:

{"status": "ok"}

GET /health/readiness

Readiness probe (checks DuckDB health).

Authentication: not required.

Responses:

  • 200:
{"status": "ok"}
  • 503:
{"status": "unavailable"}

GET /api/v1/datasets

Returns registered datasets with summary metadata and discovery links.

Authentication: conditional API key.

Query parameters:

  • limit (1..200, default 50)
  • cursor (opaque pagination cursor)

Success 200 example:

{
  "items": [
    {
      "id": "trades_v1",
      "display_name": "Equity Trades",
      "description": "Daily equity trade records",
      "field_count": 3,
      "row_count": 8,
      "time_dimension": {
        "field": "date",
        "min": "2026-03-01",
        "max": "2026-03-02",
        "max_range_days": 365
      },
      "links": {
        "self": "/api/v1/datasets/trades_v1",
        "schema": "/api/v1/datasets/trades_v1/schema",
        "tuples": "/api/v1/datasets/trades_v1/query/tuples",
        "cells": "/api/v1/datasets/trades_v1/query/cells",
        "members": "/api/v1/datasets/trades_v1/query/members"
      }
    }
  ],
  "cursor": null,
  "total_count": 1
}

GET /api/v1/datasets/{dataset_id}

Returns full discovery metadata for one dataset.

Authentication: conditional API key.

Path parameters:

  • dataset_id (required, string)

Response headers:

  • ETag
  • Cache-Control: private, max-age=60

Success 200 example:

{
  "id": "trades_v1",
  "display_name": "Equity Trades",
  "description": "Daily equity trade records",
  "source_kind": "sample_table",
  "row_count": 8,
  "version": "v1-1234abcd",
  "time_dimension": {
    "field": "date",
    "min": "2026-03-01",
    "max": "2026-03-02",
    "max_range_days": 365
  },
  "fields": [
    { "name": "date", "type": "date", "role": "dimension", "nullable": null, "distinct_count": 2 },
    { "name": "symbol", "type": "string", "role": "dimension", "nullable": null, "distinct_count": 5 },
    { "name": "volume", "type": "int64", "role": "measure", "nullable": null, "distinct_count": 8 }
  ],
  "links": {
    "self": "/api/v1/datasets/trades_v1",
    "schema": "/api/v1/datasets/trades_v1/schema",
    "tuples": "/api/v1/datasets/trades_v1/query/tuples",
    "cells": "/api/v1/datasets/trades_v1/query/cells",
    "members": "/api/v1/datasets/trades_v1/query/members"
  }
}

Conditional GET:

  • Send If-None-Match with the current ETag to receive 304 Not Modified.

GET /api/v1/datasets/{dataset_id}/schema

Returns schema metadata for a configured dataset.

Authentication: conditional API key.

Path parameters:

  • dataset_id (required, string)

Example:

curl 'http://localhost:8000/api/v1/datasets/trades_v1/schema'

Success 200 example:

{
  "dataset_id": "trades_v1",
  "version": "v1-1234abcd",
  "fields": [
    {"name": "date", "type": "date", "role": "dimension"},
    {"name": "symbol", "type": "string", "role": "dimension"},
    {"name": "volume", "type": "int64", "role": "measure"}
  ]
}

Error statuses:

  • 404 DATASET_NOT_FOUND
  • 401 auth failure when auth enabled
  • 304 when If-None-Match matches the current ETag

POST /api/v1/datasets/{dataset_id}/query/tuples

Returns distinct tuple values for selected fields.

Authentication: conditional API key.

Request body fields:

  • date_range (optional)
  • fields (array of objects):
    • field (string, required)
    • sort (ASC or DESC, optional)
    • derivation (optional)
    • include_totals (optional)
  • filters (optional):
    • field (string)
    • operator (include, exclude, like, between, gt, gte, lt, lte, is_null, not_null; case-insensitive)
    • values:
      • Required for: include, exclude, like, between, gt, gte, lt, lte
      • Must be omitted or an empty array for: is_null, not_null
      • Cardinality by operator:
        • gt, gte, lt, lte, like: exactly 1 value
        • between: exactly 2 values
        • include, exclude: between 1 and 1000 values
  • paging (optional):
    • limit (1..10000, default from config)
    • offset (>=0)

Request example:

{
  "date_range": {"type": "single", "date": "2026-03-01"},
  "fields": [{"field": "symbol", "sort": "ASC"}],
  "filters": [{"field": "symbol", "operator": "INCLUDE", "values": ["AAPL", "GOOG"]}],
  "paging": {"limit": 10, "offset": 0}
}

Success 200 example:

{
  "total_count": 2,
  "items": [{"symbol": "AAPL"}, {"symbol": "GOOG"}],
  "paging": {"limit": 10, "offset": 0, "returned": 2},
  "meta": {
    "execution_ms": 4.1,
    "cache_status": "miss",
    "request_id": "srv-tuples123"
  }
}

Error statuses:

  • 404 DATASET_NOT_FOUND
  • 409 DATASET_UNAVAILABLE
  • 422 VALIDATION_ERROR
  • 401 auth failure when auth enabled
  • 429 if rate limiting enabled and exceeded

POST /api/v1/datasets/{dataset_id}/query/cells

Returns aggregated cells grouped by row/column axes.

Authentication: conditional API key.

Request body fields:

  • date_range (optional)
  • axes.rows (array of { "field": string })
  • axes.columns (array of { "field": string })
  • axes.measures (array):
    • field (string)
    • aggregation
      • Supported: sum, avg, min, max, count, distinct_count, median, mode, stdev, variance, geomean, entropy, kurtosis, skewness, mad, and, or, count_if_true, count_if_false, list, unique_list, first, last
      • Explicitly rejected: histogram (AGGREGATION_NOT_SUPPORTED)
    • alias (string, optional)
    • sort_by (string, required only for first and last)
  • window.rows and window.columns (optional):
    • offset (>=0)
    • limit (>=1)
  • filters (optional)
  • filters[*].operator supports the same 10 case-insensitive operators as tuples/cells

Request example:

{
  "date_range": {"type": "single", "date": "2026-03-01"},
  "window": {
    "rows": {"offset": 0, "limit": 10}
  },
  "axes": {
    "rows": [{"field": "symbol"}],
    "columns": [],
    "measures": [{"field": "volume", "aggregation": "sum", "alias": "sum_volume"}]
  }
}

Success 200 example:

{
  "rows": [
    { "symbol": "AAPL" },
    { "symbol": "GOOG" }
  ],
  "columns": [
    {}
  ],
  "cells": [
    { "row": 0, "col": 0, "sum_volume": 1500 },
    { "row": 1, "col": 0, "sum_volume": 2200 }
  ],
  "window": {
    "rows": { "offset": 0, "limit": 10, "total": 2 },
    "columns": { "offset": 0, "limit": 1, "total": 1 }
  },
  "meta": {
    "execution_ms": 12.4,
    "cache_status": "miss",
    "request_id": "srv-abc123"
  }
}

Error statuses:

  • 400 CELLS_WINDOW_TOO_LARGE
  • 404 DATASET_NOT_FOUND
  • 409 DATASET_UNAVAILABLE
  • 422 VALIDATION_ERROR
  • 422 SORT_BY_REQUIRED
  • 422 AGGREGATION_NOT_SUPPORTED
  • 401 auth failure when auth enabled
  • 429 if rate limiting enabled and exceeded

POST /api/v1/datasets/{dataset_id}/query/members

Returns distinct values for one field, typically used for filter UIs.

Authentication: conditional API key.

Request body fields:

  • date_range (optional)
  • field (string)
  • search (string, optional; * is translated to SQL % wildcard)
  • filters (optional)
  • paging (optional)

Request example:

{
  "date_range": {"type": "range", "start": "2026-03-01", "end": "2026-03-02"},
  "field": "symbol",
  "search": "A*",
  "paging": {"limit": 10, "offset": 0}
}

Success 200 example:

{
  "field": "symbol",
  "total_count": 2,
  "items": [{"value": "AAPL", "count": 2}, {"value": "AMZN", "count": 1}],
  "paging": {"limit": 10, "offset": 0, "returned": 2},
  "meta": {
    "execution_ms": 3.8,
    "cache_status": "hit",
    "request_id": "srv-members123"
  }
}

Error statuses:

  • 404 DATASET_NOT_FOUND
  • 409 DATASET_UNAVAILABLE
  • 422 VALIDATION_ERROR
  • 401 auth failure when auth enabled
  • 429 if rate limiting enabled and exceeded

POST /api/v1/datasets/{dataset_id}/exports

Submits async export job and returns job id immediately.

Authentication: conditional API key.

Path parameters:

  • dataset_id (string, required)

Request body fields:

  • date_range (required)
  • query.axes (optional; rows/columns/measures)
  • query.filters (optional)
  • query.max_rows (1..100000, default 10000)
  • query.format (parquet, csv, csv_with_bom, ndjson, sqlite, or duckdb; default parquet)

Request example:

{
  "date_range": {"type": "single", "date": "2026-03-01"},
  "query": {
    "axes": {
      "rows": [{"field": "symbol"}],
      "measures": [{"field": "volume", "aggregation": "sum", "alias": "total_volume"}]
    },
    "max_rows": 1000,
    "format": "parquet"
  }
}

Success 202 example:

{
  "export_id": "exp-1234abcd",
  "dataset_id": "trades_v1",
  "status": "pending",
  "links": {
    "self": "/api/v1/exports/exp-1234abcd",
    "file": "/api/v1/exports/exp-1234abcd/file"
  }
}

Error statuses:

  • 404 DATASET_NOT_FOUND
  • 409 DATASET_UNAVAILABLE
  • 429 TOO_MANY_EXPORTS
  • 422 VALIDATION_ERROR
  • 401 auth failure when auth enabled
  • 429 if rate limiting enabled and exceeded

GET /api/v1/exports/{export_id}

Returns export job status.

Authentication: conditional API key.

Path parameters:

  • export_id (string)

Success 200 example:

{
  "export_id": "exp-1234abcd",
  "dataset_id": "trades_v1",
  "status": "complete",
  "format": "parquet",
  "created_at": "2026-03-10T01:00:00Z",
  "expires_at": "2026-03-10T02:00:00Z",
  "download_url": "/api/v1/exports/exp-1234abcd/file",
  "size_bytes": 2183640,
  "completed_at": "2026-03-10T01:00:12Z",
  "links": {
    "self": "/api/v1/exports/exp-1234abcd",
    "file": "/api/v1/exports/exp-1234abcd/file"
  }
}

Other statuses include pending, processing, failed, expired, and cancelled.

Error statuses:

  • 404 EXPORT_NOT_FOUND
  • 401 auth failure when auth enabled

GET /api/v1/exports

Lists export jobs newest-first.

Authentication: conditional API key.

Query parameters:

  • limit (default 20, max 100)
  • cursor (opaque pagination cursor)
  • status (optional job-state filter)

Success 200 example:

{
  "items": [
    {
      "export_id": "exp-1234abcd",
      "dataset_id": "trades_v1",
      "status": "complete",
      "format": "parquet",
      "row_count": 15420,
      "size_bytes": 2183640,
      "created_at": "2026-03-10T01:00:00Z",
      "expires_at": "2026-03-10T02:00:00Z",
      "links": {
        "self": "/api/v1/exports/exp-1234abcd",
        "file": "/api/v1/exports/exp-1234abcd/file"
      }
    }
  ],
  "cursor": null
}

GET /api/v1/exports/{export_id}/file

Downloads completed export artifact.

Authentication: conditional API key.

Path parameters:

  • export_id (string)

Success 200:

  • Includes Content-Length, ETag, and Last-Modified response headers
  • For parquet export: binary file, Content-Disposition: attachment; filename="<id>.parquet"
  • For csv / csv_with_bom export: text/csv, Content-Disposition: attachment; filename="<id>.csv"
  • For ndjson export: application/x-ndjson, Content-Disposition: attachment; filename="<id>.ndjson"

Error statuses:

  • 404 EXPORT_NOT_FOUND
  • 409 EXPORT_NOT_READY
  • 404 EXPORT_FILE_NOT_FOUND
  • 401 auth failure when auth enabled

HEAD /api/v1/exports/{export_id}/file

Returns the same file metadata headers as GET /api/v1/exports/{export_id}/file without a response body.

DELETE /api/v1/exports/{export_id}

Cancels active export jobs or deletes terminal jobs and their files.

Success 204 with no response body.

Error statuses:

  • 404 EXPORT_NOT_FOUND
  • 401 auth failure when auth enabled

Pagination, Filtering, and Sorting Summary

  • Pagination:
    • query.paging.limit and query.paging.offset on tuples/picklist
    • Defaults: tuples_default_limit, picklist_default_limit
  • Filtering operators:
    • INCLUDE, EXCLUDE, LIKE, BETWEEN
  • Sorting:
    • query.fields[].sort on tuples (ASC/DESC)
  • Cells windows:
    • query.rows/query.columns with start_index and count
    • Limited by max_axis_cardinality and max_cells_per_response

Rate Limits and Throttling

  • Disabled by default (QUERYSERVICE_RATE_LIMIT_ENABLED=false)
  • Query endpoints use QUERYSERVICE_RATE_LIMIT_QUERY
  • Export submission uses QUERYSERVICE_RATE_LIMIT_EXPORT
  • Exceeded limits return 429; clients should honor Retry-After