docs: specify forward poll cursors

openspec/changes/add-forward-poll-cursors/design.md

@@ -0,0 +1,27 @@
+# Design: Forward Poll Cursors
+
+## Public contract and compatibility
+
+New generated `poll_messages` and `poll_threads` operations accept optional string `since` and return `ListMessagesPage` or `ListThreadsPage`, each containing `items` and non-null `next_since`. Existing `list_messages` and `list_threads` routes, output arrays, inputs, ordering, and pagination remain unchanged. This is additive across HTTP, MCP, and CLI; generated descriptions state the bootstrap-and-poll loop and that cursors are opaque, not portable across deployments, and must be replayed unchanged.
+
+`poll_messages` takes an exact thread ID and derives its provider instance from the generic owner registry. `poll_threads` requires an exact configured provider instance. Neither operation permits aggregate multi-provider checkpoints. All query-affecting inputs are fixed by the operation: messages bind operation, authenticated principal, provider instance, and thread ID; threads bind operation, authenticated principal, and provider instance. `limit` is deliberately not bound because it changes page size only, never the continuation position. Future filters, sort controls, or visibility inputs must be bound in canonical form before they can be added to a polling operation.
+
+## Bootstrap, progression, and ordering
+
+A forward-capable provider defines one immutable, totally ordered source position for each pollable change. The order MUST include a deterministic provider-local tie-breaker; timestamps alone are insufficient. Email's order is the monotonically assigned UID within a UIDVALIDITY epoch. A call without `since` is a bootstrap, not a history listing: it captures one consistent source snapshot and returns `items: []` with a checkpoint at that snapshot's final position. Consumers that want current history first use the existing list operation, then bootstrap polling. This prevents a limited historical response from being mistaken for a fully consumed forward stream. A forward-capable provider always returns a bootstrap cursor, including when its mailbox is empty.
+
+A call with `since` reads one consistent source snapshot and returns only source changes strictly after the saved position in that total order. The provider obtains at most `limit + 1` candidate changes, emits at most `limit`, and sets `next_since` to the position of the final emitted change. If no change is emitted, it returns the exact supplied cursor. It MUST NOT advance a checkpoint to a scanned-but-unemitted high-water mark. Concurrent arrivals after the snapshot are included on a later poll; no item at or before the returned position may be emitted again.
+
+`poll_messages` changes are normalized messages. `poll_threads` is a stream of thread-change observations: every source message position that creates or updates a thread produces that thread's then-current normalized snapshot, so one thread may appear more than once in a page. Its continuation advances by the final emitted source-message position, not by `Thread.last_message_at`. Providers that cannot establish this ordered thread-change stream reject `poll_threads` with the generic forward-poll capability error.
+
+## Domain, security, and provider boundary
+
+`iris-core` owns typed page types, a `ForwardPolling` provider capability, and stable structured errors: `forward_poll_unsupported`, `invalid_forward_cursor`, `forward_cursor_epoch_changed`, and `forward_cursor_auth_unavailable`. `MessageProvider` exposes typed forward-poll methods or options; providers that cannot guarantee a monotonic position advertise no `ForwardPolling` capability and reject the call without name-based registry logic.
+
+The cursor is an opaque, URL-safe AEAD ciphertext, not merely signed/base64 data. Iris configuration supplies a current key identifier and 256-bit key; the authenticated associated data contains the contract version, operation, authenticated principal, provider instance, and canonical query binding. Plaintext contains only the versioned opaque provider position. Tokens have strict encoded and decoded size limits. Unknown key IDs, authentication failure, malformed data, expired contract version, or binding mismatch are rejected before provider I/O with `invalid_forward_cursor`; a missing cursor key makes forward polling unavailable with `forward_cursor_auth_unavailable`. Key rotation accepts configured verification keys during a bounded migration and emits new cursors with the current key.
+
+Email's private position contains UIDVALIDITY plus UID. Outer token validation occurs before I/O. A syntactically valid token then performs only the minimal mailbox metadata read needed to compare UIDVALIDITY; a mismatch returns `forward_cursor_epoch_changed` before listing messages. No email, IMAP, UID, mailbox, or cursor fields appear in generated schemas or public error details.
+
+## Projection and verification
+
+`api/operations.yaml` is the source of truth. Hydra-generated CLI, HTTP, and MCP surfaces expose the same operation names, page schemas, descriptions, and error semantics. No provider-specific route, command, or schema is added. Tests cover compatibility of current list arrays; bootstrap for empty/non-empty sources; adjacent bounded pages without skips or duplicates; empty poll cursor retention; token size/AEAD/binding/key-rotation failures with no I/O; UIDVALIDITY invalidation; two named email instances; duplicate thread-change observations; unsupported providers; HTTP/MCP/CLI parity; generated freshness; and a consumer loop using only generated surface data.

openspec/changes/add-forward-poll-cursors/proposal.md

@@ -0,0 +1,23 @@
+# Proposal: Forward Poll Cursors
+
+## Problem
+
+`list_messages` only offers a backwards `before` timestamp and `list_threads` only offers a backwards pagination cursor. Polling consumers therefore repeatedly fetch a recent window and deduplicate it themselves. Email already has a UID-based incremental-fetch primitive internally, but it is not a portable Iris public contract and cannot safely be exposed as a timestamp.
+
+## Proposal
+
+Add provider-agnostic generated polling operations without breaking existing list responses:
+
+1. Add `poll_messages` and `poll_threads`, each accepting an optional opaque `since` cursor and returning a versioned `{items, next_since}` page.
+2. The first poll with no cursor is an explicit bootstrap: it returns no items and a checkpoint at the source high-water mark. Consumers use existing list operations to hydrate history, then persist and replay this checkpoint unchanged to receive later changes.
+3. A cursor is scoped to its operation, configured provider instance, authenticated principal, and canonical query identity. Providers encode their strongest monotonic source position (for example, email UID plus UIDVALIDITY) without leaking that representation publicly.
+4. Existing `list_messages` and `list_threads` routes, output arrays, ordering, and pagination behavior are unchanged.
+
+## Scope
+
+- In scope: typed cursor/page contracts, provider implementations, generated CLI/HTTP/MCP schemas, deterministic cursor validation/error behavior, and public-boundary integration tests.
+- Out of scope: changing provider sync cadence, durable server-side consumer checkpoints, realtime subscriptions, cross-deployment cursor portability, and a live production deployment.
+
+## Motivation
+
+An LLM can discover a simple, safe polling loop from the generated descriptions alone: list to hydrate if needed; bootstrap the corresponding `poll_*` operation; save `next_since`; then provide that opaque value on later calls. This avoids falsely presenting email UID state as an RFC3339 timestamp and retains compatibility for current list clients.

openspec/changes/add-forward-poll-cursors/specs/forward-poll-cursors/spec.md

@@ -0,0 +1,47 @@
+# Forward Poll Cursors Specification
+
+## ADDED Requirements
+
+### Requirement: Additive generated polling operations expose opaque forward cursors
+
+Iris SHALL add `poll_messages` and `poll_threads` without changing the existing list-operation routes or array outputs. Each polling operation SHALL accept optional opaque `since` and return `{items, next_since}` with non-null `next_since`. Generated CLI, HTTP, and MCP descriptions SHALL instruct consumers to hydrate with existing list operations if needed, bootstrap polling, then persist and replay `next_since` unchanged.
+
+#### Scenario: Existing list compatibility
+- **WHEN** an existing client calls `list_messages` or `list_threads`
+- **THEN** it receives the same array-shaped response and pagination behavior as before this change
+
+#### Scenario: Consumer bootstraps polling
+- **WHEN** a consumer calls a `poll_*` operation without `since`
+- **THEN** it receives no items and a high-water checkpoint it can pass unchanged to a later request
+
+### Requirement: Forward polling has no skip-or-duplicate ambiguity
+
+A provider SHALL define an immutable, totally ordered source position for every pollable change, including a deterministic tie-breaker; timestamps alone are insufficient. A continued poll SHALL read one consistent source snapshot and return changes strictly after the saved position in ascending provider-position order. With a limit, `next_since` SHALL represent the final emitted item, never merely the largest scanned item. An empty continued page SHALL return the supplied cursor unchanged. Concurrent post-snapshot changes SHALL appear on a later request.
+
+#### Scenario: Bounded page
+- **WHEN** more changes exist than the requested limit
+- **THEN** the next cursor advances only through the final returned change and the following poll returns the remaining changes without a gap
+
+### Requirement: Cursors bind all query identity and protect their contents
+
+A cursor SHALL be URL-safe AEAD ciphertext with strict size limits. Its associated data SHALL bind contract version, authenticated principal, operation, configured provider instance, and canonical query identity. Message polling binds the thread ID; thread polling binds an exact provider instance and permits no aggregate checkpoint. Iris SHALL use a current key ID to issue cursors and SHALL accept configured prior verification keys only for a bounded rotation period.
+
+#### Scenario: Invalid or mismatched cursor
+- **WHEN** a cursor is malformed, unauthenticated, oversized, wrong-principal, wrong-operation, wrong-provider, or wrong-thread
+- **THEN** Iris returns `invalid_forward_cursor` before provider I/O
+
+### Requirement: Bootstrap and source epochs are explicit
+
+A forward-capable provider SHALL return a checkpoint for bootstrap even when the source is empty. Email positions SHALL include UIDVALIDITY plus UID. After outer cursor validation, Iris MAY perform the minimal mailbox metadata read required to compare UIDVALIDITY; a mismatch SHALL return `forward_cursor_epoch_changed` before listing source messages.
+
+#### Scenario: Mailbox epoch changes
+- **WHEN** an email mailbox returns a different UIDVALIDITY than the saved cursor
+- **THEN** Iris rejects that cursor and does not silently advance or suppress messages
+
+### Requirement: Thread polling represents ordered source changes honestly
+
+`poll_threads` SHALL emit a normalized thread snapshot for each ordered source-message position that creates or updates a thread; the same thread MAY appear more than once. A provider without a reliable monotonic position and thread-change stream SHALL reject polling with `forward_poll_unsupported`; Iris SHALL NOT substitute timestamps or invent a source position.
+
+#### Scenario: Repeated thread updates
+- **WHEN** two newly observed messages update the same thread
+- **THEN** the polling result may contain two ordered snapshots of that thread and advances through their respective source positions

openspec/changes/add-forward-poll-cursors/tasks.md

@@ -0,0 +1,22 @@
+# Tasks: Forward Poll Cursors
+
+## Contract and domain
+
+- [ ] T1: Approve this frozen OpenSpec contract before implementation.
+- [ ] T2: Add typed page, opaque AEAD forward-cursor, key-rotation configuration, `ForwardPolling` capability, and stable errors in `iris-core`.
+- [ ] T3: Extend `MessageProvider` and the generic registry with bootstrap and bounded forward-poll operations; preserve current list interfaces unchanged.
+
+## Provider implementation
+
+- [ ] T4: Implement Email UIDVALIDITY + UID positions, source snapshots, and epoch-invalidation semantics without exposing email-specific fields publicly.
+- [ ] T5: Implement ordered `poll_threads` change observations and honest unsupported behavior for providers without a reliable position/change stream.
+
+## Generated surfaces
+
+- [ ] T6: Update `api/operations.yaml` with additive `poll_messages`/`poll_threads`, page schemas, and agent-readable bootstrap/poll instructions.
+- [ ] T7: Update code generation/runtime bindings and regenerate CLI, HTTP, and MCP artifacts.
+- [ ] T8: Add public-boundary HTTP/MCP/CLI tests, including an array-compatibility regression and a generated-surface-only consumer polling loop.
+
+## Verification
+
+- [ ] T9: Run `cargo build --all-targets`, `cargo test --all-targets`, `cargo clippy --all-targets -- -D warnings`, `cargo fmt --all -- --check`, and `cargo run -p iris-codegen --bin iris-codegen -- check`.