Skip to content

multipart uploads, tool-server routes, and the General Legal submission - #103

Merged
TeoSlayer merged 2 commits into
mainfrom
feat/multipart-ipc
Sep 3, 2026
Merged

TeoSlayer merged 2 commits into
mainfrom
feat/multipart-ipc

Conversation

@Alexgodoroja

Copy link
Copy Markdown
Collaborator

Three things in dependency order: two platform capabilities the General Legal
app needs, then the app itself.


1. Multipart uploads

A partner endpoint that takes a file could not be expressed at all. The obvious
encoding does not fit, and this is the constraint the whole design turns on:

ipc.MaxFrameSize caps one envelope at 1 MiB. base64 inflates by 4/3, so
inline encoding tops out near 740 KiB of real file — against General Legal's
20 MiB limit. A 20 MiB document needs a 27 MiB envelope, 27× the frame. That is
not a constant to tune.

So the file never travels in one envelope. The agent stages it in frame-sized
chunks, the adapter reassembles it under $APP/blobs, and only then builds one
multipart body. TestBase64InOneEnvelopeExceedsIPCFrame asserts the premise
rather than leaving it as a comment.

Declaring multipart: on a route generates <ns>.upload_begin,
upload_chunk and upload_abort, the blob store, and the form builder.

Three broker changes

Was Now
forward hardcoded application/json, stripping the boundary Content-Type forwarded verbatim for allow-listed types only
tenancy could only read JSON, so it refused multipart outright parses the form to ownership-check refs, same fail-closed stance
oversize bodies silently truncated 413, with a per-app cap

The allow-list is deliberate rather than a passthrough: the request media type
selects which parser the partner runs, and letting a caller choose that freely
is the same lever tenancy already refuses for duplicate keys.

New boot check: an app that forwards multipart while declaring no
tenancy.body_refs now fails to load. An upload names its resource in a form
field
, never the path, so that config would ownership-check nothing on exactly
the route that needs it most — and param_types alone satisfied the old check,
which is how it would have slipped through.

Bugs this surfaced

The silent truncation was pre-existing, not something multipart introduced.
Also fixed: an off-by-one that rejected a body sitting exactly on the part
budget, and a duplicate-field ban that was too broad — repeated names are legal
multipart (multi-file uploads), and no security decision rests on fields nobody
checks, so it now applies to ref fields only.


2. Tool-server routes (mcp:)

Maps a method onto a JSON-RPC tool server over Streamable HTTP, as a second
backend
alongside base_url. Handles the session handshake, SSE or plain
JSON replies, and structuredContent unwrapping.

The separation is the point: the generated client takes no credentials and has
no field to hold any, so a REST API key cannot reach the tool server's host.
TestGeneratedToolServerNeverSeesTheRestKeyE2E asserts that from the outside
against a running adapter — including checking the REST side does carry the
key first, so the negative isn't vacuous.

Only the no-credential mode is implemented; backend.mcp.auth is rejected
rather than silently ignored.


3. io.pilot.generallegal

Contract review (byo key) plus Delaware company formation (open), 19 methods,
all four signed bundles, product demo and next-steps graph.

The graph is two strongly connected components — contract review and formation
— joined by two bridges, with the formation entry reachable from all 11
contract-review nodes.

A new billable field

Methods carry billable (what the call costs), separate from gated. Gated
means your plan does not include this; these methods work and charge you.
Conflating them tells an agent something false in both directions. <ns>.help
renders billable methods under their own heading.


Verification

  • 69 multipart tests, green under -race; full suite green, go vet clean
  • Generated adapter built and run for real over a socket: 3 MiB staged in
    chunks (every envelope asserted under the frame limit), sha256 intact at the
    partner, plus blob single-use, checksum mismatch, aborted blob and path-param
    cases
  • Full topology adapter → broker → partner, including an upload into an
    unowned matter refused with the opaque 404 and never reaching the partner
  • A drift guard keeps the emitted blob store byte-identical to the tested
    reference in internal/multipartkit — verified it actually fails when the
    copy diverges
  • All 4 bundles pass pilot-app verify; no AppleDouble junk in the macOS tars
  • Demo scores 84.6 against a 60 gate

Note for the reviewer

meter_test.go has one deliberate edit outside this feature. TestRunMeter_OneTick
seeded enough credit to survive ~1.19s of metering against a 2s deadline — under
a second of headroom, so any unrelated work added to the package could flake it,
and this branch did (1 in 10 vs 0 in 20 on main). It now seeds 1 micro-dollar so
the first tick exhausts it deterministically. 0 failures in 20 runs after.

Companion website PR: pilot-protocol/website#226

Alex Godoroja added 2 commits August 29, 2026 17:29
Three things, in dependency order: two platform capabilities the General
Legal app needs, then the app itself.

## Multipart uploads

A partner endpoint that takes a file could not be expressed at all. The
obvious encoding does not fit: ipc.MaxFrameSize caps one envelope at 1 MiB,
so base64 tops out near 740 KiB of real file against a 20 MiB partner limit.
That is not a constant to tune — a 20 MiB document needs a 27 MiB envelope.

So the file is staged in frame-sized chunks, reassembled under $APP/blobs,
and sent as one multipart body. Declaring `multipart:` on a route generates
the three staging methods, the blob store and the form builder.

Three broker changes were needed:

- the forward hardcoded application/json, which strips the boundary and makes
  the body undecodable. Content-Type is now forwarded for allow-listed types
  only — the media type selects the partner's parser, and letting a caller
  choose that freely is the lever tenancy already refuses for duplicate keys.
- tenancy could only read JSON bodies, so it refused multipart outright. It
  now parses the form to ownership-check refs, with the same fail-closed
  stance.
- oversize bodies were silently TRUNCATED rather than refused, surfacing as a
  bogus 401 or an opaque 404. Now 413, with a per-app cap.

An app that forwards multipart while declaring no tenancy.body_refs now fails
the boot: an upload names its resource in a form field, so that config would
ownership-check nothing on exactly the route that needs it most.

## Tool-server routes

`mcp:` maps a method onto a JSON-RPC tool server over Streamable HTTP, as a
second backend alongside base_url. The generated client takes no credentials
and has no field to hold any, so a REST key cannot reach the tool server's
host — asserted from the outside against a running adapter, not just by
construction.

## The General Legal submission

Contract review (byo key) plus Delaware company formation (open), 19 methods.
Includes the product demo, a next-steps graph shaped as two strongly connected
components, and all four signed platform bundles.

Methods now carry `billable` — distinct from `gated`, which means "your plan
does not include this". These work and charge you; conflating the two
misleads in both directions.

## Verification

- 69 multipart tests, all green under -race
- generated adapter built and run for real over a socket: 3 MiB staged in
  chunks, sha256 intact at the partner
- full topology adapter -> broker -> partner, including an upload into an
  unowned matter being refused
- a drift guard keeps the emitted blob store byte-identical to the tested
  reference (verified it fails when the copy diverges)
- all 4 bundles pass `pilot-app verify`, no AppleDouble junk
- demo scores 84.6 against a 60 gate
staticcheck ST1013. The mock partner servers wrote numeric literals for
422 and 400; 422 in particular reads as noise next to http.StatusNotFound
a few lines away. No behaviour change.

Verified with the same command CI runs (staticcheck ./...), which I should
have done before pushing rather than after.
@TeoSlayer
TeoSlayer merged commit a5a6035 into main Sep 3, 2026
7 checks passed
@TeoSlayer
TeoSlayer deleted the feat/multipart-ipc branch September 3, 2026 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants