Skip to content

fix(bench): read current Pier candidate contract#622

Merged
drewstone merged 1 commit into
mainfrom
fix/pier-plan-bundle-digest
Jul 25, 2026
Merged

fix(bench): read current Pier candidate contract#622
drewstone merged 1 commit into
mainfrom
fix/pier-plan-bundle-digest

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Problem

@tangle-network/agent-bench@0.4.0 reads the pre-#547 candidate layout, so every Runtime 0.105 Pier task fails before execution (plan.bundleDigest is absent).

Change

  • extend the existing Python Pier bridge to read the signed run cell, embedded benchmark task/suite, and profile activation
  • preserve fail-closed bindings for digests, coordinates, limits, model, container, retry policy, harness, and activated profile bytes
  • accept only the fixed executable PATH already signed by Runtime
  • publish agent-bench 0.4.1 and update its real failure/success proof for the current receipt layout

Checked before editing: the bridge already existed in bench/pier_agents; this updates it rather than adding another executor.

Proof

  • real Pier pair: failure 0/1, exact repair 1/1, fresh-process recovery 1/1
  • both arms: 0 model calls, 0 input/output tokens, $0
  • 30/30 focused Python tests
  • 61/61 agent-bench TypeScript files plus Pier bridge
  • packed consumer: agent-bench 0.4.1 with Runtime 0.105.0 on TypeScript 5.9.3 and 6.0.3
  • pnpm verify:bench
  • clean merge tree against current main after fix(bench): isolate evaluator Git from candidate config #620

A genuine-timeout trace-reporting defect remains separate; it does not affect completed trials or this release.

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — 1e83914c

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-25T01:32:52Z

@drewstone
drewstone merged commit 1afea6d into main Jul 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants