Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.42.0] - 2026-08-16

### Added
- Public goal-completion evaluation contracts for multi-tool executions,
including milestone, dependency-order, binding, final-state, policy, and
recovery checks.
- Reproducible Arazzo long-horizon evaluation across 1,000-tool catalogs and
3-, 10-, and 30-call workflows, with paired OpenAPI-only and
OpenAPI-plus-Arazzo evidence.

### Changed
- OpenAPI ingestion now resolves component references per path with shared
reference and schema-field caches, avoiding whole-document expansion for
large specifications.

### Fixed
- OpenAPI ingestion preserves literal `$ref` properties, decodes escaped JSON
Pointer segments, and emits serializable stubs for circular references.

## [0.41.0] - 2026-08-16

### Added
Expand Down Expand Up @@ -682,7 +701,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Tests**: 32 tests passing across all modules
- **Example**: `quickstart.py` demonstrating full workflow

[Unreleased]: https://github.com/SonAIengine/graph-tool-call/compare/v0.40.0...HEAD
[Unreleased]: https://github.com/SonAIengine/graph-tool-call/compare/v0.42.0...HEAD
[0.39.0]: https://github.com/SonAIengine/graph-tool-call/compare/v0.38.0...v0.39.0
[0.38.0]: https://github.com/SonAIengine/graph-tool-call/compare/v0.37.0...v0.38.0
[0.37.0]: https://github.com/SonAIengine/graph-tool-call/compare/v0.36.0...v0.37.0
Expand Down Expand Up @@ -721,3 +740,5 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
[0.8.0]: https://github.com/SonAIengine/graph-tool-call/compare/v0.5.0...v0.8.0
[0.28.0]: https://github.com/SonAIengine/graph-tool-call/compare/v0.27.0...v0.28.0
[0.40.0]: https://github.com/SonAIengine/graph-tool-call/compare/v0.39.0...v0.40.0
[0.41.0]: https://github.com/SonAIengine/graph-tool-call/compare/v0.40.0...v0.41.0
[0.42.0]: https://github.com/SonAIengine/graph-tool-call/compare/v0.41.0...v0.42.0
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ selected target produced:
| Candidate binding support | 14.3% | **100%** |
| Target Recall@5 | - | **100%** |

The [case-level v0.41.0 artifact](benchmarks/results/releases/v0.41.0/dependency-chain-evidence.json)
The [case-level v0.42.0 artifact](benchmarks/results/releases/v0.42.0/dependency-chain-evidence.json)
records fixture hashes, every expected target and producer, and replay commands:

```bash
Expand All @@ -239,7 +239,7 @@ make launch-evidence-check
```

The separate
[observability artifact](benchmarks/results/releases/v0.41.0/observability-evidence.json)
[observability artifact](benchmarks/results/releases/v0.42.0/observability-evidence.json)
checks that tracing leaves engine inputs unchanged, replays deterministically,
scrubs secrets, explains every decision, and stays below the documented
`5ms/span` p95 capture-cost gate:
Expand Down
50 changes: 25 additions & 25 deletions benchmarks/results/arazzo_long_horizon_0.42.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"benchmark": "Arazzo Long-Horizon Paired Evaluation",
"methodology": "paired_deterministic_retrieve_plan_execute_goal_state",
"model": "none",
"graph_tool_call_version": "0.41.0",
"graph_tool_call_version": "0.42.0",
"catalog_size": 1000,
"workflow_lengths": [
3,
Expand Down Expand Up @@ -30,16 +30,16 @@
},
"latency_ms": {
"build": {
"p50": 455.804,
"p95": 481.3
"p50": 394.394,
"p95": 404.765
},
"retrieve": {
"p50": 97.714,
"p95": 100.802
"p50": 93.644,
"p95": 97.273
},
"plan_execute": {
"p50": 6.079,
"p95": 7.683
"p50": 6.277,
"p95": 7.275
}
}
},
Expand Down Expand Up @@ -333,9 +333,9 @@
]
},
"latency_ms": {
"build": 465.411,
"retrieve": 102.285,
"plan_execute": 5.312
"build": 409.959,
"retrieve": 99.292,
"plan_execute": 5.352
},
"token_budget_used": 73
},
Expand Down Expand Up @@ -632,9 +632,9 @@
"failure_reason_codes": []
},
"latency_ms": {
"build": 455.804,
"retrieve": 97.714,
"plan_execute": 6.033
"build": 388.415,
"retrieve": 91.597,
"plan_execute": 6.277
},
"token_budget_used": 190
},
Expand Down Expand Up @@ -1122,9 +1122,9 @@
]
},
"latency_ms": {
"build": 475.985,
"retrieve": 97.232,
"plan_execute": 5.541
"build": 393.437,
"retrieve": 94.572,
"plan_execute": 5.458
},
"token_budget_used": 118
},
Expand Down Expand Up @@ -1646,9 +1646,9 @@
"failure_reason_codes": []
},
"latency_ms": {
"build": 452.269,
"retrieve": 95.358,
"plan_execute": 6.079
"build": 394.394,
"retrieve": 93.644,
"plan_execute": 5.975
},
"token_budget_used": 291
},
Expand Down Expand Up @@ -2716,9 +2716,9 @@
]
},
"latency_ms": {
"build": 458.637,
"retrieve": 98.179,
"plan_execute": 6.067
"build": 393.102,
"retrieve": 99.029,
"plan_execute": 5.531
},
"token_budget_used": 125
},
Expand Down Expand Up @@ -3880,9 +3880,9 @@
"failure_reason_codes": []
},
"latency_ms": {
"build": 484.133,
"retrieve": 101.145,
"plan_execute": 7.861
"build": 405.917,
"retrieve": 97.676,
"plan_execute": 7.386
},
"token_budget_used": 301
},
Expand Down
29 changes: 29 additions & 0 deletions benchmarks/results/releases/v0.42.0/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# v0.42.0 release evidence

This directory contains the model-free release artifacts for graph-tool-call
0.42.0.

- `dependency-chain-evidence.json` records the deterministic target and
prerequisite-producer regression used by the README.
- `observability-evidence.json` records result invariance, deterministic replay,
secret scrubbing, reason coverage, serialized trace size, and measured trace
capture overhead.
- `../../arazzo_long_horizon_0.42.json` records paired OpenAPI-only and
OpenAPI-plus-Arazzo evaluation for 3-, 10-, and 30-call workflows in
1,000-tool catalogs.

Regenerate and validate the release artifacts with:

```bash
make launch-evidence
make launch-evidence-check
make observability-evidence
make observability-evidence-check
make arazzo-long-horizon-benchmark \
OUT=benchmarks/results/arazzo_long_horizon_0.42.json
```

These deterministic benchmarks do not use an LLM. The observability latency
value is a local Python microbenchmark and does not measure exporter backend or
service network latency. The Arazzo result measures engine workflow evidence,
planning, binding, and execution-order behavior rather than model reasoning.
Loading
Loading