Skip to content

Encode wire-protocol UUIDs as raw bytes #285

Description

@conradbzura

Description

Carry every UUID-valued identifier in the wire protocol as its raw 16 bytes rather than as a text encoding, roughly halving each id's on-frame footprint, while keeping all such ids human-readable wherever they surface for a person — log lines, tracebacks and error messages, reprs, and diagnostic output.

The wire schema today serializes every id as text:

  • Task.id / TaskEnvelope.id — the task id (str(uuid), 36 chars)
  • Task.caller / TaskEnvelope.caller — the calling task's id for nested dispatches (str(uuid), 36 chars)
  • Task.proxy_id — the routine proxy id (str(uuid), 36 chars)
  • WorkerMetadata.uid — the worker instance id (str(uuid), 36 chars)
  • ChainManifest.id — the logical execution chain id (.hex, 32 chars)
  • ContextVar.spent_tokens / unspent_tokens — the per-var wool.Token ids (.hex, 32 chars)

Each is a 16-byte UUID inflated to 32 or 36 bytes of text, and the message types carrying them ride the hot path: ChainManifest and its token ledgers on every Request and Response frame, Task on every dispatch, WorkerMetadata on discovery. Restructure the id representation so these identifiers cross as their raw 16-byte form. The readability the text encodings currently provide for free must be preserved by rendering each id back to a legible hex string at every point it is shown to a human; the mechanism is left to implementation-time research and is not prescribed here.

Motivation

Identifiers make up the fixed, non-payload overhead of a frame, and several ride the wire on every dispatch — the chain id and its token ledgers on every Request and Response, the task and caller ids on every task frame. A UUID needs 16 bytes; the current text encodings spend 32 (hex) or 36 (canonical dashed) for the same entropy — a 2×–2.25× overhead paid per id, per frame.

The encodings are also inconsistent: chain and token ids serialize as 32-char hex, while task, caller, proxy, and worker ids serialize as the 36-char dashed form. Moving every id to a single raw-byte encoding both shrinks and unifies them.

The text encodings were chosen partly because they are greppable and copy-pasteable in logs — a task id, a chain id, or a spent token id can be matched across frames and across process logs by eye. Trading that for raw bytes reclaims the footprint but forfeits the free readability, so the identifiers must stay legible for humans by other means. Precedent already exists in the tree: the shared-memory WorkerReference transport encodes the worker UUID as its raw 16 bytes and still presents a readable short-hash string, showing the two goals are compatible.

This complements #226, which bounded how many token ids ride a frame; this bounds how many bytes each id — token or otherwise — costs.

Expected Outcome

  • Every UUID-valued identifier in the wire protocol crosses as its raw 16-byte form, roughly halving each id's on-frame footprint and unifying today's mixed hex/dashed encodings. Task dispatch, chain propagation, token single-use enforcement, and worker discovery are unchanged in observable behavior.
  • Every such id stays a readable, greppable hex string wherever it surfaces to a human — log lines, tracebacks, error messages, reprs, and diagnostic output. No raw-byte or base64 blobs appear anywhere a person reads an id. The guarantee holds regardless of the in-process representation chosen for the id.
  • The wire-format change is coordinated as a compatibility break suited to the v1.0.0 milestone: peers must agree on the new encoding, and the transition does not silently mis-decode ids exchanged between disagreeing versions.
  • Tests cover the raw-byte round-trip for each affected message type and pin the human-readable rendering of ids in reprs, logs, and error messages, and the suite stays green.

Metadata

Metadata

Assignees

No one assigned

    Labels

    breakingNon-backwards-compatible changeoptimizationPerformance optimization

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions