Skip to content

channels: add durable attachment storage - #106

Open
XnLemon wants to merge 18 commits into
mainfrom
codex/98-native-media
Open

channels: add durable attachment storage#106
XnLemon wants to merge 18 commits into
mainfrom
codex/98-native-media

Conversation

@XnLemon

@XnLemon XnLemon commented Aug 31, 2026

Copy link
Copy Markdown
Owner

What changed

Updates #98. This PR now completes the native media MVP across the channel, storage, Gateway, model, and Outbox boundaries.

  • Added protocol-neutral attachment references for image, video, audio, and document, with tenant-scoped storage, immutable byte verification, event binding, idempotent upload behavior, expiry cleanup, and no provider URL/token exposure to Runner.
  • Extended Gateway dispatch to bind attachments to durable message_event records before loading verified trpcmodel.ContentParts for Runner.
  • Added OpenAI Responses mapping for text, image, file, and supported audio inputs; video is preserved as an attachment/fallback and intentionally does not claim model video understanding.
  • Added Telegram bounded authenticated media download for native inbound media plus native image/document outbound delivery with deterministic text fallback.
  • Added WeCom encrypted image, file, voice, and video callback normalization after signature/AES/ReceiveID/AgentID verification, plus native image/file outbound delivery through temporary media upload.
  • Evolved ReplyOutbox with durable kind + attachment ref + fallback fields via migration 0016_runtime_reply_media.up.sql.
  • Updated channel, storage, deployment, and runnable example docs for limits, rollout, retention, and production object-store follow-up.

Why

Channel-native media needs to be replay-safe, tenant-scoped, and provider-secret-safe. The Runner receives only controlled content parts and metadata; provider file IDs, WeCom media download URLs, Telegram download URLs, access tokens, and channel secrets stay inside the channel/storage boundary.

Issue #98 MVP ledger

  • Extend the inbound contract with validated media attachments and provider metadata that does not expose secrets or untrusted fetch URLs to Runner.
  • Telegram inbound media preserves provider file identity/caption, performs bounded authenticated download through an owned client, enforces limits, and persists through the attachment boundary.
  • WeCom inbound media verifies/decrypts supported callbacks and normalizes them into the same attachment contract; unsupported or incomplete types fail closed.
  • Outbound providers can send supported media/rich replies through durable Outbox segments, with deterministic fallback when the destination cannot represent the event.
  • Attachment lifecycle, tenant isolation, replay/idempotency, cancellation, and cleanup semantics are covered without logging tokens, raw provider errors, or message payloads.
  • Deterministic fake-based unit/integration E2E and runnable examples are updated; CI does not require production media credentials or public provider endpoints.
  • Channel and deployment documentation covers storage, limits, retention, rollout guidance, and production video/object-store considerations.

Scope and compatibility

  • Existing text-only inbound and outbound paths remain compatible.
  • Media callbacks without a configured attachment store/downloader fail closed before Runner; outbound media without native support or attachment reader uses persisted fallback text.
  • PostgreSQL migrations 0014_wecom_aibot_channel.up.sql, 0015_runtime_attachments.up.sql, and 0016_runtime_reply_media.up.sql add attachment storage and media reply descriptors while preserving existing text reply rows via defaults.
  • Runtime stores that implement the attachment store are wired automatically by bootstrap for Telegram and WeCom.

Security and tenant isolation

  • Attachment references are tenant scoped and bound to a durable event before Runner load.
  • Runner never receives provider download URLs, WeCom media_id fetch URLs, access tokens, Bot tokens, or channel secrets.
  • Provider metadata excludes secrets and fetch instructions.
  • Payload bytes and raw provider/storage errors are not surfaced through Gateway, logs, or audit records.
  • WeCom ignores PicUrl and uses only the verified binding credential context for authenticated media fetch.

Testing

  • ./scripts/format.sh --check
  • bash -lc 'export PATH="$(go env GOPATH)/bin:$PATH"; ./scripts/lint.sh' (go vet ./... and golangci-lint v2.1.6 passed)
  • bash ./scripts/build.sh
  • go test ./trpcservice/channels/telegram ./trpcservice/channels/wecom ./trpcservice/bootstrap ./trpcservice/runtime/storage ./trpcservice/runtime/storage/inmemory ./trpcservice/runtime/storage/postgres ./trpcservice/gateway ./trpcservice/attachment ./migrations -count=1
  • go test ./... -count=1
  • bash ./scripts/coverage.sh (total 91.6%; local patch executable-line estimate 91.9%)
  • bash ./scripts/race.sh
  • python -m mkdocs build --strict --config-file docs/mkdocs.yml
  • git diff --check

Residual risks and follow-ups

  • PostgreSQL currently stores bounded bytes through the existing runtime object boundary. Before production video or high-volume large-file rollout, add an S3/COS-style streaming object store backend.
  • Live Telegram/WeCom media E2E was not run locally; media behavior is covered by deterministic fake SDK/downloader/provider tests.
  • Video is safely stored/sent/fallbacked where channel support exists, but model video understanding, frame extraction, OCR, ASR, and video analysis remain explicit non-goals/follow-ups.

Issue #111 Tool-driven media reply

Closes #111. This PR now closes the service-layer bridge between a revision-authorized agent tool and the existing native media Outbox transport.

  • A published revision can explicitly allow send_test_image through ToolAuthorization; unlisted tools are not exposed and required unavailable tools fail closed during Runner construction.
  • A deterministic fake model/Runner path invokes the tool and writes a structured image ReplyOutbox segment.
  • The durable outbox row contains image kind, event-bound attachment metadata, fallback, trusted target, request ID, trace ID, and trace parent.
  • Existing Telegram and WeCom providers deliver supported structured media natively; their established persisted fallback path remains the unsupported-destination behavior.
  • Model-visible tool output contains only a queued status/message. It never contains attachment IDs, object keys, raw bytes, provider IDs/URLs, tokens, or channel secrets.
  • Tool/model errors create the deterministic execution fallback while retaining durable request/trace correlation.
  • Contract tests cover allowlisted/unapproved execution, actual Runner tool invocation, structured materialization, fallback, idempotent materialization, duplicate callback rejection, function-call history, and Responses streaming.
  • tool.allowed and tool.executed audit facts are emitted without raw arguments/results; existing tool trace callbacks remain in the Runner observability path.

Scope

send_test_image writes a fixed valid PNG and proves the complete secure transport path. It is not image generation, arbitrary file access, OCR, ASR, video understanding, or a general external-agent registry. Future tools use the same context-bound factory/reply-intent boundary.

Additional validation

  • ./scripts/format.sh --check
  • bash -lc 'export PATH="$(go env GOPATH)/bin:$PATH"; ./scripts/lint.sh'
  • bash ./scripts/build.sh
  • go test ./... -count=1
  • bash ./scripts/race.sh
  • python -m mkdocs build --strict --config-file docs/mkdocs.yml
  • git diff --check

Current-base revalidation

main (including merged PR #118 and the gateway cancellation fix) was merged into this branch. The migration history and PostgreSQL reply receipt fixtures were revalidated on the current base.

Current validation

  • go test ./... -count=1
  • bash ./scripts/race.sh
  • ./scripts/format.sh --check
  • bash ./scripts/lint.sh (0 issues)
  • bash ./scripts/build.sh
  • python -m mkdocs build --strict --config-file docs/mkdocs.yml
  • git diff --check

@XnLemon
XnLemon deployed to telegram-e2e August 31, 2026 20:40 — with GitHub Actions Active
@XnLemon
XnLemon deployed to telegram-e2e August 31, 2026 20:44 — with GitHub Actions Active
@XnLemon
XnLemon deployed to telegram-e2e August 31, 2026 20:49 — with GitHub Actions Active
Comment thread trpcservice/runtime/storage/inmemory/inmemory.go Outdated
@XnLemon
XnLemon deployed to telegram-e2e August 31, 2026 23:13 — with GitHub Actions Active
@XnLemon
XnLemon deployed to telegram-e2e August 31, 2026 23:26 — with GitHub Actions Active
@XnLemon
XnLemon deployed to telegram-e2e September 1, 2026 00:07 — with GitHub Actions Active
@XnLemon
XnLemon deployed to telegram-e2e September 1, 2026 00:33 — with GitHub Actions Active
Comment thread trpcservice/channels/wecom/wecom.go Outdated
@XnLemon
XnLemon deployed to telegram-e2e September 1, 2026 01:03 — with GitHub Actions Active

@XnLemon XnLemon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@XnLemon
XnLemon deployed to telegram-e2e September 1, 2026 11:01 — with GitHub Actions Active
@XnLemon
XnLemon deployed to telegram-e2e September 1, 2026 11:23 — with GitHub Actions Active
@XnLemon
XnLemon deployed to telegram-e2e September 1, 2026 12:20 — with GitHub Actions Active
@XnLemon
XnLemon deployed to telegram-e2e September 1, 2026 18:39 — with GitHub Actions Active
Comment thread trpcservice/tool/media_reply.go
@XnLemon
XnLemon deployed to telegram-e2e September 1, 2026 18:59 — with GitHub Actions Active
Comment thread trpcservice/tool/media_reply.go Outdated
@XnLemon
XnLemon deployed to telegram-e2e September 1, 2026 19:09 — with GitHub Actions Active

@XnLemon XnLemon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

…-lease-fix

# Conflicts:
#	docs/docs/index.md
#	docs/mkdocs.yml
#	migrations/history_test.go
#	migrations/migration_test.go
#	trpcservice/bootstrap/environment.go
#	trpcservice/gateway/dispatch.go
#	trpcservice/gateway/dispatch_test.go
@XnLemon
XnLemon deployed to telegram-e2e September 3, 2026 14:05 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

agent: add tool-driven media reply path

1 participant