Skip to content

refactor(converter): own the ATIF conversion; harbor becomes a public-API dependency - #6

Merged
laithalsaadoon merged 3 commits into
mainfrom
feature/own-conversion
Sep 11, 2026
Merged

laithalsaadoon merged 3 commits into
mainfrom
feature/own-conversion

Conversation

@laithalsaadoon

@laithalsaadoon laithalsaadoon commented Sep 11, 2026

Copy link
Copy Markdown
Owner

atif-converter no longer calls harbor's private methods. The Claude Code and Codex conversions are ours, ported from harbor 0.22.0 under Apache-2.0 and built on the two things harbor actually documents: the ATIF data classes in harbor.models.trajectories (RFC 0001) and harbor.utils.trajectory_validator.

Why

harbor documents the data classes and the validator, and nothing that converts a native session log. We were calling ClaudeCode._convert_events_to_trajectory and Codex._convert_events_to_trajectory (1,029 and 741 lines), through a symlink-staging layer that rebuilt the directory shape those methods expect. The two upstream files took 26 and 18 commits between June and September 2026. That was the drift we were exposed to, and it was why the pin had a minor-version ceiling and a runtime probe with its own exit code.

What changed

  • Conversion is ours. domain/claude_code_conversion.py and domain/codex_conversion.py are parity ports (attributed), reached through infrastructure/claude_code_converter.py and infrastructure/codex_converter.py, which read the files directly. Staging, temp dirs, the private-API probe, HarborPrivateApiMissing and the exit 127 handlers are gone (the code stays in the table so it never renumbers).
  • Parity is measured. harbor's private converters survive in one place, tests/harbor_oracle.py, as the oracle: asserted deterministic, frozen to goldens for both synthetic fixtures (the live oracle must equal the frozen one, so an upstream behavior change becomes a named JSON-path diff rather than a mystery), and diffed against our converters over the newest N local sessions with ATIF_PARITY_LIMIT (0 = all).
  • The boundary is a gate. An ast guard pins production imports to the two public modules; import-linter cannot express a sub-module of an external package. Proven to fail by dropping a harbor.agents import into src/.
  • Dependencies. harbor>=0.22.0,<1; litellm>=1.92.0 declared directly, since harbor's converters lazily import it for per-call cost and the port keeps that.
  • Two fidelity "gaps" retired because their mechanism is gone: Claude Code's workflow_subagents_missed (our converter discovers every side file itself) and Codex's single_rollout_per_directory (one file in, one trajectory out). Values are not reused. The data-loss gaps stay on purpose: this is a parity port, and fixing one is a decision to diverge from the oracle, recorded in the fidelity policy.

Measured

Check Result
Synthetic fixtures vs live harbor, both agents 0 divergent paths
Frozen goldens vs our converters 0 divergent paths
Live parity, Claude Code, ATIF_PARITY_LIMIT=0 2,437 sessions, 0 divergences
Live parity, Codex, ATIF_PARITY_LIMIT=0 78 rollouts, 0 divergences
mise run check green, 1,080 tests
mise run docs:gate green

Preserved harbor behavior worth a later decision (not changed here)

  • agent.extra cwds / git_branches / agent_ids are built from Python sets, so list order for multi-element values is hash-seed dependent across processes. Pre-existing in every corpus on disk; a follow-up can sort them once we decide to diverge.
  • Orphan tool_result steps carry the same metadata dict twice (extra.metadata and extra.tool_result_metadata).
  • Codex web_search_call yields an empty tool_call_id and an empty observation result.
  • Codex bundled-step timestamp is the first normalized event, which is the tool call whose output arrived first rather than the earliest call.

Docs updated: CONTRACT, CONTRIBUTING (the pin section is now the oracle section), AGENTS.md, README, RELEASING, the PR template, and every generated-docs passage that named the private method.

After a pre-merge review (second commit)

The review found one real defect: litellm loads its pricing table from GitHub at import time, and the frozen goldens carry per-call cost_usd priced through it, so a remote price edit or an unreachable GitHub in CI would have failed the never-skipped golden gate with no change here. The converter conftest now pins LITELLM_LOCAL_MODEL_COST_MAP=true (the table bundled with the locked litellm), the Codex fixture model moved to one that table prices so the cost path stays frozen, and the goldens were re-frozen. Measured: the bundled table had no entry for the previous fixture model and the golden tests failed under it.

Also from the review: explicit utf-8 on the Codex reader to match its sibling, comments recording the set-ordered agent extras as harbor behavior kept on purpose, four doc sentences that still advertised exit 127 or the retired gap as live, and a test that an unreadable side file becomes a ConversionError carrying the OSError.

Follow-ups this PR does not take

  • Every convert process still pays litellm's import (about 2 s measured) and, unless LITELLM_LOCAL_MODEL_COST_MAP is set, a GitHub fetch for the pricing table. That is harbor's cost path carried over. Pinning the bundled table in production too would make convert offline-safe; it is a pricing-freshness decision, so it is left for its own change.
  • Sorting agent.extra cwds / git_branches / agent_ids would make multi-valued sessions reproducible across processes, at the price of a deliberate divergence from the oracle.
  • The install-weight question in RELEASING stands: the port makes vendoring or a slimmer harbor distribution tractable.

… frozen goldens

The port away from harbor's private conversion API needs a fixed point to be
measured against. This is it: harbor's two private converters reached from
tests only, deterministic (asserted), frozen to goldens for both synthetic
fixtures (asserted equal to the live oracle, so an upstream behavior change
surfaces as a named diff), and a live-corpus parity test over the newest N
local sessions of each agent that skips where there is no corpus.
…-API dependency

atif-converter used to call two PRIVATE harbor methods,
ClaudeCode._convert_events_to_trajectory and Codex._convert_events_to_trajectory
(1,029 and 741 lines, 26 and 18 upstream commits since June), through a
symlink-staging layer that rebuilt the directory shape they expected. harbor
documents only the ATIF data classes (harbor.models.trajectories, RFC 0001) and
the validator; that is now all production code imports from it.

The conversion is ours: domain/claude_code_conversion.py and
domain/codex_conversion.py are parity ports of harbor 0.22.0's converters
(Apache-2.0, attributed), built on the public models, reached through
infrastructure/claude_code_converter.py and infrastructure/codex_converter.py,
which read the files directly. No staging, no temp dirs, no private-API probe,
no HarborPrivateApiMissing, no exit 127 path (the code stays in the table so it
never renumbers). The pin widens to harbor>=0.22.0,<1 and litellm, which harbor
lazily imported for per-call cost, is declared as our own dependency.

Parity is measured, not assumed. harbor's private converters survive in ONE
place, tests/harbor_oracle.py, as the oracle: deterministic (asserted), frozen
to goldens for both synthetic fixtures (the live oracle must equal the frozen
one, so an upstream behavior change becomes a named JSON-path diff), and diffed
against our converters over the newest N local sessions (ATIF_PARITY_LIMIT=0 =
all). Measured: 0 divergent paths over 2,437 Claude Code sessions and 78 Codex
rollouts on this host. An ast guard pins the harbor allowlist to the two public
modules; import-linter cannot express a sub-module of an external package.

Two fidelity "gaps" are retired because their mechanism is gone: Claude Code's
workflow_subagents_missed (our converter discovers every side file itself) and
Codex's single_rollout_per_directory (one file in, one trajectory out). Their
values are not reused; corpora materialized before this carry them still. The
data-loss gaps stay, by choice: this is a parity port, and fixing one is a
decision to diverge from the oracle, to be recorded in the fidelity policy.
…table

A pre-merge review found the frozen goldens depended on a remote file: litellm
loads its pricing table from raw.githubusercontent.com at import time unless
LITELLM_LOCAL_MODEL_COST_MAP is set, and the goldens carry per-call cost_usd
priced through it. A BerriAI price edit, or an unreachable GitHub in CI, would
have failed the never-skipped golden gate with no change in this repo, and the
test's docstring would have blamed harbor. The converter conftest now pins the
table to the one bundled with the locked litellm (setdefault, so a run can
still opt into the remote table), the Codex fixture model moves to one the
bundled table prices so the cost path stays frozen rather than becoming None,
and the goldens are re-frozen. Measured: the bundled table had no entry for
the previous fixture model, and the golden tests failed under it.

Also from the review: the Codex reader opens with an explicit utf-8 like its
Claude Code sibling; the set-ordered agent extras and the fixture's
single-valued cwd/gitBranch are documented as harbor behavior kept on purpose;
four doc sentences that still advertised exit 127 or the retired gap as live
are corrected; and a test pins that an unreadable side file becomes a
ConversionError carrying the OSError.
@laithalsaadoon
laithalsaadoon merged commit 9fd70b5 into main Sep 11, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant