Summary
khive has the event plane already: stream.append / stream.read / stream.batch are the durable
log, and brain.event_counts proves the read side is buildable — it returns windowed counts grouped
by kind, actor and verb. What is missing is a telemetry pack: the same shape over an arbitrary
stream rather than over brain's own plane, plus the one piece that is genuinely absent, a declared
assignment of event kinds to carriers.
This is deliberately a small ask. Most of it is already served; the argument is for naming the
service so consumers stop each building the aggregation half in process.
What a consumer has today, as one worked example
One runtime's telemetry package is about 1,100 lines across six modules. Its parts, and where each
would land:
| part |
what it does |
khive today |
| typed envelope |
validates an event's payload against a per-kind schema before it is emitted |
nothing |
| channel table |
assigns each kind a carrier, a failure posture and a cursor kind |
nothing |
| durable append |
event-first append to the log, ahead of any subscriber |
stream.append |
| ephemeral ring |
bounded broadcast, never stored; a reconnect from before the window sees a gap |
nothing |
| counters |
per-run and per-job rollups so a reader answers "what happened" without replaying |
brain.event_counts, but only over brain's plane |
| cursors |
resume a reader from where it stopped |
stream.read |
Two of those six are already khive. One (counters) exists for exactly one plane. Three do not exist.
The part that is a design decision, not a port
The channel table is configuration, not code. Each event kind is assigned three things:
- carrier —
durable (goes to the log, readers get everything since their cursor) or ephemeral
(goes to a bounded ring, a reader that fell behind sees a gap).
- failure posture —
stop (a refused append raises out of the emitting task) or gap (a dropped
event surfaces as a gap at reconnect, never as an error).
- cursor kind —
log or ring, which follows from the carrier but is worth stating because it
is what a reader's resume logic keys on.
The reason this belongs in a table rather than at the emit site is that the choice is an operator's,
not an emitter's: whether a streaming delta is worth durable storage depends on the deployment, and
an emitter that decides it has hard-coded one deployment's answer. A worked table looks like:
[telemetry]
stream = "telemetry"
default_carrier = "ephemeral" # fail-cheap: an unclassified kind is not silently made durable
[[telemetry.channels]]
kinds = ["run.started", "run.completed", "run.failed", "turn.completed", "turn.failed",
"tool.executed", "governance.denied", "governance.approved"]
carrier = "durable"
failure_posture = "stop"
[[telemetry.channels]]
kinds = ["turn.delta", "*.heartbeat"]
carrier = "ephemeral"
failure_posture = "gap"
[telemetry.ring]
capacity = 4096
One thing to decide deliberately: whether an unclassified kind defaults to durable or ephemeral,
or is refused. Defaulting to durable makes an unnoticed new kind expensive; defaulting to ephemeral
makes it silently unrecorded; refusing makes adding a kind a config change. The example above picks
ephemeral, and the choice should be khive's, stated in the verb's help, not inherited from whichever
default was easiest.
Proposed verbs
telemetry.emit(kind, payload, run_id?, actor?) -> {carrier, seq?, dropped?, receipt_id}
telemetry.read(stream, since?, limit?, kinds?) -> {events, next_cursor, gap?}
telemetry.counts(stream, window, group_by?, kinds?) -> {rows: [{key, count, ...}], window, total}
telemetry.channels() -> the effective table, as resolved from config
telemetry.emit returning the carrier it chose is the point: an emitter should be able to see
that its event went to the ring rather than the log, without reading the config itself.
dropped is a field rather than an inference, for the same reason cached is on a web fetch.
gap on the read side is the contract that makes an ephemeral carrier honest: a reader that fell
out of the window is told so and is not handed a shorter list that reads complete.
telemetry.channels() because a table that cannot be read back is a table nobody can debug.
How many backends
Three, and only one of them is new:
- The durable log — already khive streams. No new backend,
stream.append under the covers.
- The aggregation —
telemetry.counts. Can be computed at read time over the stream, the way
brain.event_counts already does, which means no new storage. An incremental rollup table is an
optimization to reach for when a measurement says read-time is too slow, not before.
- The ephemeral ring — the one genuinely new piece, and the one worth questioning. A ring is
naturally per-process; a cross-process ring is a pub/sub with different operational properties.
If khive would rather not own a broadcast bus, saying so is a fine answer, and the consequence is
that carrier = "ephemeral" means "the daemon drops it" rather than "the daemon holds a window of
it" — which is still a useful contract, just a weaker one. This is the fork worth ruling on first,
because the rest of the design does not depend on which way it goes.
Ask
A telemetry pack with the four verbs above, the channel table as config, and an explicit ruling on
whether khive owns an ephemeral broadcast window or only a drop. The durable half already works
through stream.append today; what a consumer cannot get from khive is the classification and the
rollup, and those are the two that every consumer otherwise rebuilds.
Summary
khive has the event plane already:
stream.append/stream.read/stream.batchare the durablelog, and
brain.event_countsproves the read side is buildable — it returns windowed counts groupedby kind, actor and verb. What is missing is a telemetry pack: the same shape over an arbitrary
stream rather than over brain's own plane, plus the one piece that is genuinely absent, a declared
assignment of event kinds to carriers.
This is deliberately a small ask. Most of it is already served; the argument is for naming the
service so consumers stop each building the aggregation half in process.
What a consumer has today, as one worked example
One runtime's telemetry package is about 1,100 lines across six modules. Its parts, and where each
would land:
stream.appendbrain.event_counts, but only over brain's planestream.readTwo of those six are already khive. One (counters) exists for exactly one plane. Three do not exist.
The part that is a design decision, not a port
The channel table is configuration, not code. Each event kind is assigned three things:
durable(goes to the log, readers get everything since their cursor) orephemeral(goes to a bounded ring, a reader that fell behind sees a gap).
stop(a refused append raises out of the emitting task) orgap(a droppedevent surfaces as a gap at reconnect, never as an error).
logorring, which follows from the carrier but is worth stating because itis what a reader's resume logic keys on.
The reason this belongs in a table rather than at the emit site is that the choice is an operator's,
not an emitter's: whether a streaming delta is worth durable storage depends on the deployment, and
an emitter that decides it has hard-coded one deployment's answer. A worked table looks like:
One thing to decide deliberately: whether an unclassified kind defaults to durable or ephemeral,
or is refused. Defaulting to durable makes an unnoticed new kind expensive; defaulting to ephemeral
makes it silently unrecorded; refusing makes adding a kind a config change. The example above picks
ephemeral, and the choice should be khive's, stated in the verb's help, not inherited from whichever
default was easiest.
Proposed verbs
telemetry.emitreturning the carrier it chose is the point: an emitter should be able to seethat its event went to the ring rather than the log, without reading the config itself.
droppedis a field rather than an inference, for the same reasoncachedis on a web fetch.gapon the read side is the contract that makes an ephemeral carrier honest: a reader that fellout of the window is told so and is not handed a shorter list that reads complete.
telemetry.channels()because a table that cannot be read back is a table nobody can debug.How many backends
Three, and only one of them is new:
stream.appendunder the covers.telemetry.counts. Can be computed at read time over the stream, the waybrain.event_countsalready does, which means no new storage. An incremental rollup table is anoptimization to reach for when a measurement says read-time is too slow, not before.
naturally per-process; a cross-process ring is a pub/sub with different operational properties.
If khive would rather not own a broadcast bus, saying so is a fine answer, and the consequence is
that
carrier = "ephemeral"means "the daemon drops it" rather than "the daemon holds a window ofit" — which is still a useful contract, just a weaker one. This is the fork worth ruling on first,
because the rest of the design does not depend on which way it goes.
Ask
A
telemetrypack with the four verbs above, the channel table as config, and an explicit ruling onwhether khive owns an ephemeral broadcast window or only a drop. The durable half already works
through
stream.appendtoday; what a consumer cannot get from khive is the classification and therollup, and those are the two that every consumer otherwise rebuilds.