Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions docs/superpowers/plans/2026-07-13-runtime-capability-awareness.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Runtime Capability Awareness Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Make Zeus use and accurately report live runtime capabilities, real objective-specific plans, authoritative model/context status, and normalized cache-token telemetry.

**Architecture:** Rust owns the canonical capability snapshot and native browser/memory dispatch. React owns presentation of the authoritative turn result and removes duplicate runtime/tool assumptions. Provider usage is normalized once into a typed frontend model used by both status and session panels.

**Tech Stack:** Rust 1.95, Tauri 2, React 18, TypeScript, Vitest, rusqlite-backed runtime state.

## Global Constraints

Do not expose secrets or memory contents in capability prompts. Do not bypass approvals. Do not fabricate plan progress. Missing cache telemetry is unknown, not zero. Preserve stable prompt-prefix ordering and all existing native-loop limits.

---

### Task 1: Runtime capability snapshot

**Files:** Modify `src-tauri/src/engine/mod.rs`, `src-tauri/src/lib.rs`; test in the same Rust modules.

**Interfaces:** Produce `render_capability_snapshot(&RuntimeCapabilitySnapshot) -> String` and idempotent `inject_capability_snapshot(&mut Vec<ChatMessage>, &str)`.

- [ ] Add failing tests asserting healthy snapshots advertise file/shell/test/browser/memory capabilities, omit stale “static-only” claims, render unavailable reasons, contain manifest-backed tool names, and inject once.
- [ ] Run focused Rust tests and confirm the expected assertion failures.
- [ ] Implement typed snapshot construction from engine manifest, browser driver availability, runtime status, filesystem scope, and approval policy; inject it before native provider dispatch.
- [ ] Re-run focused tests to green.

### Task 2: Native browser and memory tools

**Files:** Modify `src-tauri/src/engine/mod.rs`, `src-tauri/src/lib.rs`, `src/providers/toolDispatch.ts`; extend existing Rust native-loop tests.

**Interfaces:** Add manifest/dispatch names `browser`, `retrieveMemory`, and `upsertMemory`, delegating to `AgentRuntimeService` and the persistent memory store.

- [ ] Add failing dispatch tests proving each tool reaches its existing runtime service and returns a structured observation.
- [ ] Add tool schemas to the canonical manifest, pass runtime/project context into dispatch, and remove the duplicate static frontend tool list.
- [ ] Run Rust and TypeScript focused tests to green.

### Task 3: Objective-linked plan states

**Files:** Modify `src/providers/planner.ts`, `src/agentRuntimeDeepLoop.ts`, `src/components/PlanProgressPanel.tsx`, `src/App.tsx`; update colocated tests.

**Interfaces:** Add explicit panel states `conversation`, `planning`, `ready`, and `unavailable`; bind plan results to an objective ID.

- [ ] Replace tests expecting generic boilerplate with failing tests for “No execution plan needed,” “Planning…,” objective-specific steps, stale-result rejection, and “Plan unavailable.”
- [ ] Remove heuristic fallback plans, carry objective IDs through planner state, and render only real plan steps.
- [ ] Run planner, deep-loop, panel, and App tests to green.

### Task 4: Authoritative model/context and cache telemetry

**Files:** Create `src/providers/tokenUsage.ts` and test; modify `src/App.tsx`, `src/components/StatusBar.tsx`, `src/views/HomeView.tsx`, `src/views/InspectorPanel.tsx` and their tests.

**Interfaces:** Produce `normalizeTokenUsage(usage: unknown) -> NormalizedTokenUsage | null` with `input`, `output`, `cacheRead`, and `cacheWrite`; persist the response model and normalized usage on assistant messages.

- [ ] Add failing normalizer tests for OpenAI, Anthropic, compatible nested fields, absent telemetry, and cache percentage clamping.
- [ ] Add failing UI tests for authoritative response model, actual last-turn context, projected-next estimate, cached-read percentage, cache-write display, and “not reported.”
- [ ] Implement normalization, message persistence, StatusBar actual/estimated labeling, and Session cache percentages.
- [ ] Run focused TypeScript tests to green.

### Task 5: Full verification and visible acceptance

**Files:** No new production files unless verification exposes a defect.

- [ ] Run `npm run typecheck`, `npm test`, `npm run build`, `cargo fmt --manifest-path src-tauri/Cargo.toml -- --check`, `cargo test --manifest-path src-tauri/Cargo.toml --all-targets`, `cargo clippy --manifest-path src-tauri/Cargo.toml --all-targets -- -D warnings`, and `bash scripts/check-tauri-capabilities.sh`.
- [ ] Launch the Tauri app and verify the screenshot scenario: conversational limitation question shows no fake plan, the status bar shows the returned model and actual/estimated context labels, and Session shows cache percentage or “not reported.”
- [ ] Review `git diff --check`, changed files, and repository status before completion.
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Runtime Capability Awareness Design

## Objective

Make Zeus describe and use its actual runtime capabilities instead of producing stale, generic limitations such as “static-only,” “cannot run tests,” or “no persistent memory.” Capability claims must be derived from registered runtime services and callable native tools on every agent turn.

## Current mismatch

The React system prompt states that Zeus can read and edit files, execute commands, and run tests. A second static prompt lists workspace tools. Rust separately owns the canonical engine manifest, runtime status, browser service, persistent memory, approvals, and native observe-and-replan loop. These sources can drift. Browser and memory commands are callable from the UI bridge but are not first-class tools in the normal chat loop, so the model cannot reliably verify or use everything the application advertises.

## Authoritative capability snapshot

Rust will build a concise capability snapshot immediately before each native agent turn. The snapshot will be injected into the first system message after user-authored identity guidance and before any skill-specific instructions. It will contain the canonical callable tool names, filesystem and approval policy, browser availability, persistent-memory availability and current entry count, runtime session/tool-run state, and explicit inherent limitations.

The snapshot will instruct the model to distinguish three states: available and callable now, registered but currently unavailable with a concrete reason, and inherently limited. Zeus must attempt a relevant tool or use a runtime status result before claiming an available capability is missing. It must not describe itself as static-only when shell or file tools are registered, claim that tests are invisible when `runTest` or `runCommand` is callable, or claim there is no persistent memory when the runtime memory store is active.

Inherent limitations remain honest: finite context, imperfect intent inference, possible API uncertainty, and the need for approval on gated actions. The snapshot will name the mitigations Zeus can use: source inspection, test/build execution, browser verification, bounded observations, plans, transactional patches, checkpoints, and persistent memory.

## Native browser and memory tools

The canonical engine manifest and native tool dispatcher will add `browser`, `retrieveMemory`, and `upsertMemory` tools. Browser accepts the existing semantic actions and delegates to `AgentRuntimeService::browser_tool`. Memory tools delegate to the existing persistent runtime store and remain scoped by project/session identifiers.

Tool schemas in the prompt will come from the same manifest entries used by dispatch. The implementation will not expose a tool merely because a Tauri command exists; the tool must be reachable through the native agent loop and covered by a dispatch test.

Browser availability will be checked without launching a browser for unrelated turns. The snapshot reports browser tooling as available when the shipped driver resource exists and the runtime service is healthy; an actual driver-start error is returned as a structured tool observation when the model invokes it.

## Prompt ownership

React retains identity, response style, compaction, and optional terse/minimal-code instructions. It will stop maintaining a duplicate list of runtime tools. Rust owns the capability snapshot and appends it idempotently so retries and re-planning do not duplicate the block.

The snapshot builder will be a pure Rust function over typed capability data. This keeps wording testable and prevents live secrets, full memory contents, file contents, or unbounded runtime state from entering the system prompt.

## Live model and context status

The bottom status bar will use the model identifier and normalized usage returned by the latest completed native turn. It will clearly distinguish actual last-turn input from the locally projected next prompt. The active configured model remains a temporary preflight value only until the provider returns an authoritative model identifier.

The projected count must include every frontend-known system block, workspace hint, chat message, compact anchor, and composer draft. The UI will label it as an estimate because Rust may inject capability, skill, memory, and tool-observation context after the frontend projection. After a response, the status bar displays the provider-reported input count against the returned model's registered context window and retains the projected next-turn estimate separately.

Unknown model identifiers will visibly use the conservative fallback window instead of appearing authoritative. Tests will cover configured-model fallback, authoritative response-model replacement, actual versus estimated labels, and context-window calculation.

## Objective-linked plan progress

The Plan Progress panel will never synthesize a generic five-step plan. A substantive execution objective displays only provider-generated or runtime-generated objective-specific steps. A conversational question or explanation request displays a concise “No execution plan needed” state. While a substantive objective is waiting for its plan, the panel displays “Planning…” without fabricated progress. Planner failure displays “Plan unavailable” and the objective, not boilerplate TODOs.

Plan results will be bound to an objective/request identifier so a late planner response cannot replace the plan for a newer user turn. Progress updates will apply only to the active objective's real steps.

## Cache-token visibility and diagnosis

Provider usage will be normalized into input, output, cache-read, and cache-write token counts. Normalization will recognize OpenAI-compatible `prompt_tokens_details.cached_tokens`, Anthropic `cache_read_input_tokens` and `cache_creation_input_tokens`, and equivalent nested fields returned by compatible providers. Missing cache telemetry is represented as “not reported,” not zero.

The Session panel will display cached-read tokens and cached-read percentage of input tokens, plus cache-write tokens when supplied. Its tooltip/status copy will distinguish “0% reported” from “provider did not report cache usage.” The percentage formula is `cacheRead / input * 100`, clamped to 0–100.

The implementation will not claim that a low percentage is a Zeus defect without telemetry. Stable system-prefix ordering will be preserved to maximize provider cache reuse; per-turn dynamic capability values will be placed after the stable capability/tool description so runtime counters do not invalidate the reusable prefix.

## Error handling

If runtime status cannot be collected, the turn continues with a degraded snapshot that states which status probe failed. A failed optional capability must not erase confirmed capabilities. Tool execution errors remain observations; they do not mutate the persistent capability description for later turns unless the underlying health check changes.

## Testing

Tests will first fail against the current behavior. Rust unit tests will assert that the capability snapshot advertises execution, tests, browser, and persistent memory when healthy; rejects the stale limitation phrases; renders unavailable capabilities with reasons; contains only manifest-backed callable tools; and is injected exactly once. Dispatcher tests will prove browser and memory calls reach the existing runtime services.

TypeScript tests will assert that the frontend no longer injects a duplicate tool list while preserving identity and response-style instructions. Existing bridge, approval, redaction, bounded-output, and native-loop tests must remain green.

Completion requires `npm run typecheck`, `npm test`, `npm run build`, `cargo fmt --check`, `cargo test --manifest-path src-tauri/Cargo.toml --all-targets`, `cargo clippy --manifest-path src-tauri/Cargo.toml --all-targets -- -D warnings`, and `bash scripts/check-tauri-capabilities.sh`.

## Non-goals

This change does not promise unlimited context, perfect intent inference, or infallible dependency knowledge. It does not add autonomous permission escalation, silently bypass approvals, preload memory contents into every prompt, or probe/launch a real browser on every turn.
41 changes: 37 additions & 4 deletions src-tauri/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -110,15 +110,48 @@ fn inject_skill(app: &tauri::AppHandle, request: &ChatRequest) -> Result<Vec<Cha
let root = resolve_skills_dir(app)?;
select_auto_skill_messages(&root, request)?
};
if skill_messages.is_empty() {
return Ok(request.messages.clone());
}
let mut out = Vec::with_capacity(request.messages.len() + skill_messages.len());
let capability_message = runtime_capability_message();
let mut out = Vec::with_capacity(request.messages.len() + skill_messages.len() + 1);
out.push(capability_message);
out.extend(skill_messages);
out.extend(request.messages.iter().cloned());
Ok(out)
}

/// Build the authoritative capability prefix from the same manifest used by
/// the native executor. Keeping this server-side prevents the UI prompt and
/// executable tool set from drifting apart.
fn runtime_capability_message() -> ChatMessage {
let tools = engine::tool_manifest()
.into_iter()
.map(|tool| tool.name)
.collect::<Vec<_>>()
.join(", ");
ChatMessage {
role: "system".to_string(),
content: serde_json::Value::String(format!(
"# Zeus runtime capabilities\nThis is authoritative for the current runtime. You are not static-only: you can inspect and edit files, run commands and tests, use Git, search code, apply transactional patches, and perform web search through native tools. Available native tools: {tools}. Use tools to verify runtime behavior when the objective requires it. Do not claim that you cannot run code, access the workspace, or test changes. Be honest about inherent limits: you cannot directly see the user's screen unless an image is attached, runtime observations only exist after you execute a tool, and persistent memory is only available when explicitly surfaced by the host."
)),
}
}

#[cfg(test)]
mod runtime_capability_tests {
use super::*;

#[test]
fn capability_message_is_derived_from_executable_manifest() {
let message = runtime_capability_message();
let text = message_text(&message.content);
assert_eq!(message.role, "system");
for tool in engine::tool_manifest() {
assert!(text.contains(tool.name), "missing tool {}", tool.name);
}
assert!(text.contains("not static-only"));
assert!(text.contains("run commands and tests"));
}
}

fn strip_wrapping_quotes(value: &str) -> String {
value
.trim()
Expand Down
Loading
Loading