Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,9 @@ jobs:
- name: Golden M10 replay fixture
run: python -m unittest tests.test_m10_repair_study

- name: Golden Phase 2 REPAIR acceptance
run: python -m unittest tests.mcp.test_runtime.MCPRuntimeTest.test_complete_repair_cycle_uses_only_control_plane_tools

package:
name: Build and verify package
runs-on: ubuntu-latest
Expand Down Expand Up @@ -131,6 +134,7 @@ jobs:
bwork init
bwork doctor
bwork --json root
python -c "from benchwork.mcp.tool_registry import tool_names; assert len(tool_names()) == 38"

- name: Upload distributions
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
Expand Down
46 changes: 43 additions & 3 deletions docs/en/PHASE2_ACCEPTANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,21 @@ Phase 2 uses the
unavailable optional Host and a correctly denied disclosure are explicit
states, not product failures.

## M17 seal verdict

```yaml
version: 0.3.0rc1
phase2:
implementation: COMPLETE
acceptance: PASS_WITH_ACCEPTED_EXCEPTIONS
contracts: FROZEN_ALPHA
phase3_rfc_gate: OPEN
```

This verdict does not convert optional or unauthorized Host work into PASS.
It records required gates separately from accepted environment and disclosure
states.

## Release acceptance

| Boundary | Status | Release effect | Evidence |
Expand All @@ -27,9 +42,9 @@ External Review remains pending until a researcher approves the exact source,
files, disclosure flags, and destination. No diff was uploaded during this
acceptance run.

The hardened deterministic gate completed with 135 unittest cases (one
environment-dependent subprocess skip), 134 pytest cases, 167 subtests, and
81% branch-aware coverage. Ruff and mypy completed without findings.
The M17 deterministic gate completed with 144 unittest cases, 144 pytest
cases, 174 subtests, and 81% branch-aware coverage on Python 3.13. Ruff and
mypy completed without findings, and the SciPy numerical oracle passed.

## Milestone evidence

Expand All @@ -43,6 +58,7 @@ environment-dependent subprocess skip), 134 pytest cases, 167 subtests, and
| M14 — The Wards | startup, pre-tool, and post-tool hooks; direct state-write denial | `tests/hooks/test_hooks.py` |
| M15 — The Working | canonical writes, preview/commit Seals, Runs, Alembic, seven workflows | runtime tests and lifecycle/Alembic suites |
| M16 — Trial of the Familiar | marketplace package, fresh CLI discovery, accepted Host exception | plugin validation, interactive contract tests, install/upgrade/rollback trials |
| M17 — The Instrumentarium Seal | frozen Alpha contracts, Tool Registry, release metadata, golden REPAIR, rc1 package | registry parity tests, golden scenario, package smoke, this matrix |

The deterministic suite remains the release gate:

Expand All @@ -63,6 +79,30 @@ an accepted `bench.study.audit` proposal, replacement Protocol preview/commit,
registered implementation and Runs, Alembic analysis, Assessment, and a
human-sealed `REPAIR` Decision through Chronicle replay.

## 2026-07-30 M17 rc1 trial

- The Python wheel and source distribution built as `0.3.0rc1`; Twine accepted
both. A force-installed wheel initialized a fresh project, passed Deep
Doctor, loaded all packaged schemas, and discovered 38 registry tools.
- The local marketplace installed
`0.3.0-rc.1+codex.20260730080053`. The source manifest was restored to the
release version `0.3.0-rc.1` after cache-busted installation.
- A fresh ephemeral Codex CLI `0.145.0` read-only conversation explicitly
selected `$benchwork-orchestrate`. `benchwork_status` and
`benchwork_next_actions` both returned `ok=true` with
`mcp-tool-result/1.0`.
- A second fresh conversation implicitly selected
`benchwork:benchwork-orchestrate` for “What should we do next for this
study?” and made the same two read-only MCP calls.
- Neither CLI trial used shell or file-edit tools. The temporary project's
Chronicle remained at zero events after plugin installation and both calls,
and the source repository had no `.benchwork/` directory.
- VS Code `1.122.0` and the `openai.chatgpt` extension were installed, but no
display variable or graphical extension Host was available. IDE execution
therefore remains `HOST-IDE-001: BLOCKED_BY_ENVIRONMENT`.
- No external diff review was attempted or disclosed. External Review remains
`WAITING_FOR_DISCLOSURE_AUTHORIZATION`.

## 2026-07-30 Host trial

- Codex CLI `0.145.0` loaded Benchwork `0.3.0-alpha.1` from
Expand Down
19 changes: 19 additions & 0 deletions docs/en/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,3 +117,22 @@ Codex CLI acceptance is complete. IDE validation is an accepted
`BLOCKED_BY_ENVIRONMENT` Tier 2 exception until a graphical extension Host is
available; external diff review remains
`WAITING_FOR_DISCLOSURE_AUTHORIZATION`.

## M17: The Instrumentarium Seal

Completed for `0.3.0rc1`: Phase 2 API and protocol freeze; 38-tool
machine-readable MCP Registry; versioned Plugin and Skill compatibility
metadata; Host Support Matrix; Review Disclosure Policy; deterministic golden
audit-to-`REPAIR` acceptance; migration and release documentation; and
release-gated CI coverage.

Phase 2 is sealed with Kernel, MCP, Plugin, and Codex CLI gates passing.
`HOST-IDE-001` remains an accepted `BLOCKED_BY_ENVIRONMENT` Tier 2 exception,
External Review remains `WAITING_FOR_DISCLOSURE_AUTHORIZATION`, and Claude Code
CLI remains `PENDING_HOST_VALIDATION`. These states are not reclassified as
failures or fabricated PASS results.

Phase 3 — The Sanctum may now begin through new RFCs for sandboxing,
filesystem isolation, execution policy, patch promotion, experiment executors,
GPU/remote jobs, and Artifact storage. Phase 3 must not weaken the frozen
Phase 2 scientific control-plane boundary.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ version = "0.3.0rc1"
description = "An auditable workbench for agent-assisted computational research."
readme = "README.md"
requires-python = ">=3.11"
license = {text = "MIT"}
license = "MIT"
authors = [{name = "Endofthestars"}]
dependencies = [
"jsonschema>=4.18",
Expand Down
Loading