(mcp) Enforce hard response budgets

decisions/decisions/adr-128-hard-mcp-response-budgets.md

@@ -0,0 +1,139 @@
+---
+schema_version: 1
+id: RAC-01K8Q7MCP411
+type: decision
+---
+# ADR-128: Hard MCP Response Budgets
+
+## Context
+
+ADR-033 established a deterministic character budget for MCP tool payloads,
+but the first implementation retained two overrun cases: summaries were only
+marked, and deep relationship neighborhoods could remain far above the
+configured limit. The stdio binary also had no way to choose a startup budget,
+so the documented configurability existed only in an embedding API.
+
+Those exceptions make the safety property illusory. An agent receiving a
+24,000-character summary or a 60,000-character graph response has still had
+its context flooded, even if the payload says `truncated: true`. The native
+AsDecided server must enforce the boundary at the point where every transport
+serializes a tool result.
+
+## Decision
+
+Every successful MCP tool payload string is at or below its effective
+character budget.
+
+- The default is 10,000 characters. `decided-mcp --budget N` configures the
+  server-wide budget for both stdio and HTTP, and `N` must be at least 128.
+- The `get_artifact` and `retrieve_grounding` per-call `budget` arguments may
+  lower the startup value. A positive value below 128 is rejected as a tool
+  error; a caller cannot use it to produce an invalid oversized response.
+- Repeated collections (`matches`, `items`, `incoming`, `neighborhood`,
+  `decisions`, `attention`, and outgoing relationship targets) are reduced
+  deterministically from the tail at whole-item boundaries. Retrieval
+  excerpts and artifact content may use a deterministic fitting prefix before
+  whole items are removed.
+- `truncated`, `omitted`, and `hint` remain the only response markers. Omitted
+  counts are truthful: they include source overflow and entries removed by the
+  budget pass, while character-prefix reductions report dropped characters
+  where the source shape supports that count.
+- If fixed fields alone cannot fit, the serializer returns a small structured
+  `response_budget_exceeded` error instead of an oversized successful payload.
+- These rules replace the historical summary and deep-neighborhood overrun
+  exceptions. Port parity does not preserve those context-flooding bugs.
+
+ADR-033 remains the governing rationale for deterministic character budgets;
+this decision hardens its implementation contract for the native server.
+
+## Status
+
+Accepted
+
+## Category
+
+Technical
+
+## Consequences
+
+Agents receive a predictable upper bound on every successful result, across
+both transports and every response shape. Broad summaries and graph walks may
+return less context than before, but the marker and hint make the omission
+visible and actionable. Operators can choose a larger bound deliberately at
+startup without changing the default safety posture.
+
+There is no cursor or session state: callers narrow the query or make another
+stateless request when they need omitted context. A 128-character minimum
+leaves room for the structured budget error and avoids accepting unusably tiny
+server configurations.
+
+## Alternatives Considered
+
+### Preserve the oracle's overrun exceptions
+
+Rejected. Marking an oversized success does not protect an agent context, and
+the Rust cutover is the opportunity to retire the known bug-for-bug behavior.
+
+### Remove the configurable startup budget
+
+Rejected. Different clients and deployment contexts need different context
+windows; a deterministic operator-controlled character limit is cheap to
+support and remains transport-neutral.
+
+### Use token counts or an LLM judge
+
+Rejected. Tokenizers vary by model and version, and model judgement would
+break deterministic local enforcement. Character counts and fixed truncation
+rules preserve ADR-032 and ADR-066.
+
+## Code Constraints
+
+```yaml
+version: 1
+eligibility: eligible
+reason: "The native budget boundary and explicit overflow response are stable source contracts."
+rules:
+  - id: mcp-budget-keeps-default
+    kind: require_pattern
+    path_glob: "rust/rac-engine/src/budget.rs"
+    pattern: 'pub const DEFAULT_BUDGET: i64 = 10_000;'
+    message: "The native MCP response budget must retain its 10,000-character default."
+  - id: mcp-budget-keeps-minimum
+    kind: require_pattern
+    path_glob: "rust/rac-engine/src/budget.rs"
+    pattern: 'pub const MIN_BUDGET: i64 = 128;'
+    message: "Configured response budgets must retain the explicit 128-character minimum."
+  - id: mcp-budget-has-explicit-overflow
+    kind: require_pattern
+    path_glob: "rust/rac-engine/src/budget.rs"
+    pattern: 'pub const BUDGET_ERROR: &str = "response_budget_exceeded";'
+    message: "Fixed-field budget failures must return an explicit structured error."
+  - id: mcp-budget-is-configurable-at-startup
+    kind: require_pattern
+    path_glob: "rust/decided-mcp/src/main.rs"
+    pattern: '"--budget"'
+    message: "The native MCP server must expose the response budget at startup."
+```
+
+## Related Decisions
+
+- adr-007
+- adr-032
+- adr-033
+- adr-066
+- adr-121
+
+## Related Requirements
+
+- rac-agent-context-guide
+
+## Applies To
+
+- rust/rac-engine/src/budget.rs
+- rust/decided-mcp/src/main.rs
+- rust/decided-mcp/src/http.rs
+- rust/decided-mcp/src/tools.rs
+- rust/decided-mcp/tests/response_budget.rs
+- rust/decided-mcp/tests/http_transport.rs
+- docs/mcp.md
+- rust/PORT-CONTRACT.d/10-mcp-surface.md

docs/mcp.md

@@ -148,6 +148,25 @@ The `examples/guide/` corpus contains one requirement, decision, design, and
 roadmap for a fictional user management service — enough to explore all four
 tools.
 
+### Response budgets
+
+The native server caps each successful tool payload at 10,000 characters by
+default. Set a different startup budget for either transport with
+`--budget N` (the minimum is 128 characters):
+
+```bash
+decided-mcp --root /path/to/your/repo --budget 20000
+```
+
+The limit is measured on the JSON payload in `content[0].text`, not the outer
+JSON-RPC frame. Collection results are truncated deterministically at whole
+items and carry `truncated`, `omitted`, and `hint` fields. Artifact content
+and retrieval excerpts may be shortened by character prefix. If fixed fields
+alone cannot fit, the tool returns a small `response_budget_exceeded` error
+instead of an oversized success. `get_artifact` and `retrieve_grounding` also
+accept a positive per-call `budget` that can lower the startup value; values
+below 128 are rejected as a tool error.
+
 ## 4. Your first grounded question
 
 Once the server is connected, ask your agent:
@@ -412,7 +431,7 @@ server gains a streamable **HTTP transport** for exactly this
 ([ADR-098](https://github.com/asdecided/core/blob/main/decisions/decisions/adr-098-shared-http-mcp-serving.md)):
 
 ```bash
-decided-mcp --root /path/to/your/repo --transport http --host 127.0.0.1 --port 8000 --path /mcp
+decided-mcp --root /path/to/your/repo --transport http --host 127.0.0.1 --port 8000 --path /mcp --budget 10000
 ```
 
 - **`--transport`** — `stdio` (default) or `http`. Bare `decided-mcp` is unchanged,

rust/PORT-CONTRACT.d/10-mcp-surface.md

@@ -324,36 +324,33 @@ bytes:
 
 - Default budget **10,000 characters**, measured over the serialized
   **payload string** (`content[0].text`), *not* the wire frame (which is
-  ~2× + escaping). Configured only at server construction
-  (`build_server(budget=…)`); the stdio CLI has **no flag** — it is always
-  10,000. ORACLE-NEXT adds per-call `budget` args that may only *lower*
-  it: `effective = server if arg<=0 else min(server, arg)`.
-- Truncation is whole-item, from the tail, deterministic; marker fields are
-  appended (insertion order puts them last): `"truncated": true`,
-  `"omitted": <int>`, `"hint": "<pinned string>"`. `truncated` is **absent**
-  (never `false`) on complete responses. Pinned hints: `HINT_SEARCH`,
-  `HINT_RELATED`, `HINT_CONTENT`, `HINT_SUMMARY` in `budget.py`
-  (+ `HINT_RETRIEVE` = `"Lower top_k, raise the budget, or narrow the
-  task."` on NEXT).
-- Per-shape rule (first matching key wins — order matters):
-  `matches` → drop whole matches; `incoming` → drop whole incoming entries;
-  (NEXT: `items` → binary-search-trim the **last kept item's `excerpt`**
-  first, drop whole items only if an empty excerpt still doesn't fit;
-  `omitted` counts dropped whole items, a pure excerpt trim is
-  `truncated:true, omitted:0`); `content` → binary-search the largest
-  fitting prefix, `omitted` = characters dropped; anything else (summary) →
-  marker added, **nothing dropped**.
-- Verified wire consequences (live corpus, defaults):
-  - `get_artifact` on a >10k artifact → payload **exactly 10,000 chars**,
-    `omitted:14350`, `HINT_CONTENT`.
-  - `search_artifacts "telemetry"` → 9,950 chars, `omitted:15`.
-  - **`get_summary` CAN exceed the budget**: live-corpus payload is 24,346
-    chars with `truncated:true, omitted:0, HINT_SUMMARY` — marked, not cut.
-  - **`get_related` with `depth>1` CAN massively exceed the budget
-    (landmine):** the truncator only shrinks `incoming`; `neighborhood` is
-    not truncatable, so `depth:3` on ADR-001 served a 62,609-char payload
-    with `truncated:true, omitted:10, HINT_RELATED`. This is the oracle's
-    real behavior — port it bug-for-bug; do not "fix" it in the port.
+  ~2× + escaping). The native server accepts `--budget N` on both stdio and
+  HTTP; `N` must be at least **128**. The default remains 10,000 when the
+  flag is omitted.
+- ORACLE-NEXT per-call `budget` arguments may only lower the configured
+  server budget: `effective = server if arg<=0 else min(server, arg)`. A
+  positive per-call value below 128 is a tool error, never an oversized
+  successful response.
+- Truncation is deterministic and whole-item wherever a repeated collection
+  is involved. Marker fields are appended (insertion order puts them last):
+  `"truncated": true`, `"omitted": <int>`, `"hint": "<pinned string>"`.
+  `truncated` is **absent** (never `false`) on complete responses. Pinned
+  hints: `HINT_SEARCH`, `HINT_RELATED`, `HINT_CONTENT`, `HINT_SUMMARY`, and
+  `HINT_RETRIEVE`.
+- The serializer applies stable shape rules: `matches`, `incoming`,
+  `neighborhood`, `items`, `decisions`, `attention`, and outgoing relationship
+  targets are reduced from the tail as whole entries. Retrieve `items` may
+  first shorten the last kept `excerpt`; artifact `content` may be shortened
+  to the largest fitting prefix. Omitted counts include source overflow and
+  every dropped item; a pure excerpt/content reduction reports dropped
+  characters where the source shape supports it.
+- **Every successful payload string is at or below the configured budget.**
+  If fixed fields alone cannot fit, the server returns the small structured
+  error `{"error":"response_budget_exceeded", "hint":"..."}` (or its
+  shorter form at unusually small direct serializer limits), rather than an
+  oversized success. This replaces the historical summary and deep-
+  neighborhood overrun exceptions; the native engine does not preserve those
+  bugs for compatibility.
 - The budget serializer re-measures with `_dumps` (spaces included) — the
   10,000 counts those spaces.
 

rust/decided-mcp/src/http.rs

@@ -77,6 +77,7 @@ HTTP (ADR-084).",
 /// model and audit recorder remain serialised behind mutexes, while request
 /// parsing happens outside those locks so a slow client cannot block the next
 /// client from being accepted.
+#[allow(clippy::too_many_arguments)]
 pub fn serve_http(
     root: &str,
     state: ServerState,
@@ -85,6 +86,7 @@ pub fn serve_http(
     port: u16,
     path: &str,
     allowed_origins: &[String],
+    server_budget: i64,
 ) -> ! {
     let listener = match TcpListener::bind((host, port)) {
         Ok(l) => l,
@@ -129,7 +131,15 @@ stateless per call; authentication belongs to the deployment proxy, ADR-085)."
                 let allowed_origins = Arc::clone(&allowed_origins);
                 std::thread::spawn(move || {
                     let _permit = ConnectionPermit { active };
-                    handle_connection(&root, &state, &recorder, &path, &allowed_origins, s);
+                    handle_connection(
+                        &root,
+                        &state,
+                        &recorder,
+                        &path,
+                        &allowed_origins,
+                        server_budget,
+                        s,
+                    );
                 });
             }
             Err(_) => continue,
@@ -174,6 +184,7 @@ fn handle_connection(
     recorder: &Mutex<Option<audit::Recorder>>,
     path: &str,
     allowed_origins: &[String],
+    server_budget: i64,
     mut stream: TcpStream,
 ) {
     if stream.set_read_timeout(Some(HTTP_IO_TIMEOUT)).is_err()
@@ -199,7 +210,7 @@ fn handle_connection(
     let response = {
         let mut state = state.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
         let mut recorder = recorder.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
-        route(root, &mut state, &mut recorder, path, &req)
+        route(root, &mut state, &mut recorder, path, server_budget, &req)
     };
     respond(&mut stream, &response);
 }
@@ -355,6 +366,7 @@ fn route(
     state: &mut ServerState,
     recorder: &mut Option<audit::Recorder>,
     path: &str,
+    server_budget: i64,
     req: &Request,
 ) -> Response {
     if req.path() != path {
@@ -367,7 +379,7 @@ fn route(
         // SDK, which opens an idle stream).
         "GET" => Response { status: "405 Method Not Allowed", body: None },
         "DELETE" => Response { status: "405 Method Not Allowed", body: None },
-        "POST" => route_post(root, state, recorder, req),
+        "POST" => route_post(root, state, recorder, server_budget, req),
         _ => Response { status: "405 Method Not Allowed", body: None },
     }
 }
@@ -376,6 +388,7 @@ fn route_post(
     root: &str,
     state: &mut ServerState,
     recorder: &mut Option<audit::Recorder>,
+    server_budget: i64,
     req: &Request,
 ) -> Response {
     // Accept must be present and admit JSON (json_response mode): absent -> 406.
@@ -448,6 +461,7 @@ fn route_post(
         &message,
         recorder.as_mut(),
         principal.as_deref(),
+        server_budget,
     );
     let status = if era == protocol::Era::Current
         && !protocol::current_method_supported(method)