Skip to content

Conformance kit duplicates production semantics by hand — six silent invalidations so far #24

Description

@mortonanalytics

src/bin/ione-conformance.rs restates IONe's contract rules in its own code rather than deriving them from the implementation. Every production fix on PR #23 silently invalidated one of those restatements, and each was caught only by an adversarial review pass rather than by a test.

Six instances, all on one branch:

Production change Kit then said
approval_required made optional FAIL if absent
Panel paths follow nextCursor (#18) FAIL if the peer paginates
Client speaks streamable-HTTP (#15) Could not parse an SSE-framed reply; FAILed 4 of 6 surfaces
hex::decode accepts A-F FAIL on uppercase digest
foreign_roles: null accepted (F5) FAIL on null
registration_endpoint optional (F2) FAIL if absent

Every one of these tells a conforming peer it is broken. That is the worst possible failure direction for this artifact: the kit exists so an external peer (TerraYield) can validate itself without a running IONe, so the cost lands on the peer author, who has no way to know the kit is wrong.

The kit is correct as of PR #23 and now carries 21 unit tests. But the duplication is structural and will keep re-breaking.

Why it is not trivially fixable

The kit deliberately has zero ione:: imports so it stays liftable into a standalone crate a peer can run without this repo. That constraint is what forces the duplication.

Options

  1. Extract a ione-contract crate with no DB/axum deps, holding the pure predicates (signature grammar, cursor termination, URL validation, envelope field rules). Both src/ and the kit depend on it. Preserves liftability — a peer vendors one small crate.
  2. Generate the kit's rule table from a machine-readable form of the contract, so a rule can only be stated once.
  3. Contract-parity tests: for each rule, one test asserting production and the kit agree on the same input vector. Cheapest; catches drift without restructuring.

Option 3 is the smallest thing that would have caught all six.

Acceptance

  • A change to a production contract rule cannot leave the kit stating the opposite without a test failing.
  • The kit keeps zero ione:: imports, or the liftability requirement is explicitly renegotiated.

Context: PR #23.

Metadata

Metadata

Assignees

No one assigned

    Labels

    federationIONe federation / peer integration

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions