feat(api-sync): deterministic spec-sync patcher (Phase A) - #58
Merged
Merged
Conversation
Replaces the AI-driven api-sync pipeline's classification/apply step with a deterministic, stdlib-only patcher (.api-sync/sync.py), mirroring check_contract.py's ast-based style. - .api-sync/spec-map.json: curated mapping from spec constructs (enums, schemas, and nested inline sub-objects via specPath) to SDK symbols. Covers all 84 schemas reachable from paths/webhooks in the current public spec, including the customers and bank_accounts discriminator fan-outs and the shared TrackingPayment/TrackingTransaction/TrackingComplete/ TrackingLiquidity nested-object mappings. ignore.schemas documents the Rfi family, UploadAnalyze*, and the one confirmed-orphan schema (LedgerOperation). - .api-sync/unmodeled.json: honest ledger of currently-absent spec properties and enum-member/value divergences on mapped schemas/enums, each with a reason and owner, seeded from a bootstrap diff against the current spec. - .api-sync/sync.py: --check (state reconciliation against spec-snapshot.json), --apply (reconciles against --spec, hard-fails on removals/required-ness or type changes/new operations or schemas/unmapped targets, applies enum-member and optional-property additions, refreshes the snapshot), --validate-map, --coverage (non-blocking known-gap report). Reachability is computed as the transitive $ref closure of paths+webhooks, so schemas outside it are skipped by construction, not by a hand-maintained list. - tests/test_api_sync.py: literal/TypedDict splicing (incl. the _XRequired total=False two-part pattern and the NotRequired convention), every NEEDS_HUMAN classification, a nested specPath sub-object case, bump classification, idempotency, map validity, and unmodeled.json honoring. - CI: wires an API Sync Check job (map validity, --check, a determinism proof, a non-blocking coverage report) into main.yaml alongside contract-check. Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
Ran .api-sync/sync.py --apply against the current public spec. All three changes were already present in the committed spec-snapshot.json (pending drift the code never caught up to, not new spec surface): - BankingPartner (types.py): spec's banking_partner enum on VirtualAccountOut/CreateVirtualAccountIn has 5 values; the Literal only had 4 (missing "portage"). - PaymentMethod (resources/payins/quotes.py): CreatePayinQuoteInput.payment_method resolves to this file's own local Literal (6 values), not the shared types.py one; the spec's CreatePayinQuoteIn.payment_method enum has 9 (missing "transfers", "pse", "international_swift"). Confirmed against blindpay-v2 and the other 4 SDKs that this is staleness, not a deliberate subset. - refund_wallet_address (resources/quotes/quotes.py): new optional field on QuoteIn, added to CreateQuoteInput. Updated tests/resources/test_quotes.py's fixtures for the new required-key (CreateQuoteInput is a total=True TypedDict, so every call site must supply every key). `.api-sync/sync.py --check` is now green. Two independent applies of this same input produced byte-identical trees (determinism proof). Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
…rmatting cmd_apply was re-serializing the whole spec via json.dumps(..., sort_keys=True) to refresh spec-snapshot.json, which reorders every key in a 1.6MB file on every apply and buries the real (few-line) drift under tens of thousands of unrelated diff lines. Writing spec_path.read_bytes() straight through keeps whatever formatting the upstream filter already produced and makes future applies show only the lines that actually changed. Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
…a duplicate literal client.py hardcoded __version__ = "2.3.0" for the User-Agent header, separate from __init__.py's __version__ = "3.0.0". release-please's extra-files only patches __init__.py, so client.py silently drifted every release. Moves the single source of truth to src/blindpay/_version.py, imported by both __init__.py (public re-export, unchanged for consumers) and client.py (no circular import: _version.py has no dependency on the rest of the package). Updated pyproject.toml's [tool.hatch.version] path and release-please-config.json's extra-files to point at the new file. Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
…elivered spec Regression coverage for the raw-bytes-copy fix in c858f1c: asserts --apply --spec <file> leaves spec-snapshot.json byte-identical to that file even when it uses compact separators, non-ascii content and no trailing newline -- formatting a json.load/json.dump round trip would silently normalize away. Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
Drops the anthropics/claude-code-action step and every CLAUDE_CODE_OAUTH_TOKEN reference. The workflow now: reads .api-sync/spec-current.json from origin/api-sync-data, runs .api-sync/sync.py --apply, and: - exits 0 quietly when there is nothing to apply, - fails loudly (::error:: annotations) when the patcher reports NEEDS_HUMAN, - otherwise commits the regenerated code + refreshed spec-snapshot.json to the api-sync branch, opens/updates the PR with a title prefix derived from the patcher's own bump classification (feat:/fix:) so release-please computes the right version bump, labels it api-sync, and enables auto-merge so the existing required checks are the only gate. Keeps the .github/workflows/ commit guard from the old Claude-driven flow as defense in depth, even though spec-map.json can never point there. api-sync-merged.yml is unchanged: it only clears the api-sync-data baseline files and does not depend on anything replaced here. Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
Adds a workflow that enables GitHub's native auto-merge on the release-please PR as soon as it exists, instead of waiting for a human to click merge. Gating is strict and narrow: only fires when github.event.pull_request.user.login == 'github-actions[bot]' AND the PR carries release-please's own "autorelease: pending" label. This never touches a human-authored PR, and enabling auto-merge does not bypass any required check (lint, typecheck, tests, contract-check, api-sync-check, snyk) -- GitHub still waits for all of them before merging. With this, a merged api-sync PR now flows all the way to PyPI (via publish.yaml's existing release-please -> tag -> uv publish path) with zero human involvement. Note: this repo currently has "Allow auto-merge" OFF at the repository level (gh api repos/blindpaylabs/blindpay-python .allow_auto_merge is false); both this workflow and api-sync.yml's own auto-merge step need it turned on (Settings -> General -> Pull Requests) or `gh pr merge --auto` will fail. Left as a manual follow-up rather than changing repo settings in this PR. Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
Collaborator
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
…ed total=True TypedDict A real PR review caught a breaking-change bug in the applied drift: `class CreateQuoteInput(TypedDict):` has no `total=False`, so every key is structurally required regardless of whether its value type is Optional[...]. The previous choose_field_annotation matched sibling style (plain Optional[str], since that's what CreateQuoteInput's other fields use) and emitted `refund_wallet_address: Optional[str]` -- a REQUIRED key. Any existing caller building a CreateQuoteInput without the new key now fails pyright/mypy, which is exactly the "no breaking change to the published surface" constraint this whole project exists to enforce. It's why the previous commit had to add "refund_wallet_address": None to four call sites. Fix: the emitted annotation now depends only on the target class's totality, never on sibling convention: - total=False (or the non-required half of the `_XRequired` two-part pattern): a bare Optional[T]/T is already optional, matching the existing behavior. - total=True (the default): always NotRequired[Optional[T]] / NotRequired[T], regardless of whether any sibling field already uses NotRequired. Also adds ensure_name_imported(): NotRequired (or Optional) is added to whichever existing `from typing import ...` / `from typing_extensions import ...` line already carries TypedDict, matching that file's own convention, since quotes.py imports Optional from typing but had never needed NotRequired before. CreateQuoteInput.refund_wallet_address is now `NotRequired[Optional[str]]`; the four test_quotes.py call sites that had to explicitly pass `"refund_wallet_address": None` are reverted, since omitting it is exactly what this fix makes safe again. pyright/mypy both pass with the field omitted, proving the regression is closed. Bump stays "patch". Also fixes a related, smaller gap found during the same review pass: the old-vs-new type-change comparison only compared base JSON type (string/integer/etc), never nullability, so a spec property flipping between ["T","null"] and "T" passed silently in either direction. Now flagged as needs-human symmetrically, with one deliberate exception: a property with no "type" key at all on either side (pure metadata, e.g. created_at gaining an explicit type where it previously had none) is treated as compatible, since there is nothing concrete to compare. Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
…y mapped property's type diff_removals_and_changes only ever catches type/nullability drift going forward from a synced baseline -- it cannot see a pre-existing latent mismatch between the spec's declared type and the SDK's annotation that was already there when a field was first modeled. This closes that blind spot the same way state reconciliation replaced event diffing for presence: a full, non-blocking state comparison, reusing the same primitives (coarse_type, SCALAR_TYPE_MAP) instead of a new engine. `--audit-types` walks every mapped `types` entry, compares each modeled property's spec type/nullability against the SDK's actual annotation text, and prints a report. Always exits 0 -- this is Phase C triage material, not a gate; a pile of pre-existing mismatches must not block CI. Flags: - spec nullable, SDK annotation missing Optional[...] (a real gap: a caller could see None where the type says it can't happen); - an enum-constrained property modeled as a bare scalar (no Literal); - a scalar type mismatch (e.g. spec number, SDK str), with integer->float treated as a deliberately compatible widening; - spec array/object vs a non-List/non-nested-TypedDict SDK annotation. Deliberately NOT flagged: SDK wider than the spec needs (Optional where the spec is never null, float where an int would do) -- a legitimate, common, harmless modeling choice, not a bug. A single map entry can list several spec locators asserted to share one shape (e.g. tracking_payment duplicated inline across 6 payout schemas); findings are deduplicated by (file, symbol, field, note) so the same real issue is not reported once per locator. Run against this repo's own current spec: 316 findings, all non-blocking. 293 are "spec nullable, SDK missing Optional" -- overwhelmingly the recent, broad, defensive nullable-almost-everything pass this spec's own created_at/updated_at fields went through (see the c858f1c commit message), not 293 distinct bugs. 21 are enum-constrained properties modeled as a bare str (a mix of legitimate free-form fields and real missing-Literal gaps, e.g. Payin.type/status/payment_method and the shared Tracking*.step fields). Two look like genuine, worth-triaging issues: PayinOut.billing_fee_amount is `number` on the wire but `Optional[str]` in the SDK (a financial amount modeled as a string), and VirtualAccountOut.blockchain_wallet is a real {network, address} object on the wire but collapsed to `Optional[str]` in the SDK (the network/address sub-fields are not modeled at all). None fixed here -- Phase C triage, per the ask. Wired into the API Sync Check CI job as a fourth, non-blocking, always-green step next to the coverage report. Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
This was referenced Aug 4, 2026
ericviana
added a commit
that referenced
this pull request
Aug 4, 2026
Literal ended with "2500000000_plus" (2.5 billion). The spec's enum on CreateCustomerIn, UpdateCustomerIn, CustomerOut and both customer webhook schemas is: ["0_99999", "100000_999999", "1000000_9999999", "10000000_49999999", "50000000_249999999", "250000000_plus"] 250000000_plus (250 million) is the only coherent reading: the band directly below it tops out at 249999999, so the top band has to start at 250000000. A business customer selecting the top revenue band sends a value the API rejects -- the same class of live defect as account_type/BankAccountType (#64). Why .api-sync/sync.py's enum reconciliation did not catch this on its own: it did not compare at all. EstimatedAnnualRevenue was never added to spec-map.json's `enums` list during Phase A (#58) -- that list covers 17 shared, broadly-reused Literals, not the long tail of customer-domain- specific ones customers.py declares (CustomerBusinessType, BusinessIndustry, SourceOfWealth, TaxType, AmlStatus, ProofOfAddressDocType, PurposeOfTransactions, SourceOfFundsDocType, and others -- roughly 15 more Literals with no map entry at all). This is a map-coverage gap, not a comparison-logic flaw: reconcile_enums only ever inspects what spec-map.json lists, so an unmapped Literal is invisible to it regardless of whether it has one extra member, one missing member, or both. Not redesigning or expanding map coverage in this PR -- noting it as a real, separate gap for a future pass. Claude-Session: https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase A of the deterministic API-sync project: replaces the AI-driven (claude-code-action) api-sync step with a deterministic, dependency-free patcher, mirroring
check_contract.py's ast-based style..api-sync/spec-map.json: curated mapping from spec constructs (enums, schemas, and inline nested sub-objects viaspecPath) to SDK symbols. Covers all 84 schemas reachable frompaths/webhooksin the current public spec, including the customers (3-way KYC) and bank_accounts (11-rail) discriminator fan-outs, and the sharedTrackingPayment/TrackingTransaction/TrackingComplete/TrackingLiquiditynested-object mappings shared between payin/payout.ignore.schemasdocuments the Rfi family,UploadAnalyze*, and the one confirmed orphan schema (LedgerOperation, 0$refanywhere in the filtered spec)..api-sync/unmodeled.json: honest ledger (142 entries) of spec properties currently absent from their mapped SDK type, and enum value/member divergences, each with a reason and owner (eric@blindpay.com). Twokinds:propertyandenum..api-sync/sync.py:--check(state reconciliation againstspec-snapshot.json),--apply(reconciles against--spec, hard-fails on removals/required-ness or type changes (including nullability)/new operations or schemas/unmapped targets/ambiguous fan-out targets/unresolvable types, applies enum-member and optional-property additions, refreshes the snapshot by copying the delivered spec's raw bytes),--validate-map,--coverage(non-blocking known-gap report). Reachability is the transitive$refclosure ofpaths+webhooks, computed fresh every run, so schemas outside it are skipped by construction, never by a hand-maintained list -- this is exactly what should prevent a future orphan schema (likeLedgerOperationtoday) from generating phantom work.tests/test_api_sync.py(51 tests): Literal/TypedDict splicing (incl. the_XRequired/total=Falsetwo-part pattern and the totality-drivenNotRequiredrule), import management, every NEEDS_HUMAN classification (including type-change and nullability-change), a nestedspecPathsub-object case (the exact blind spot that hidprovider_reference-- see below), bump classification, idempotency, map validity, unmodeled.json honoring, and a byte-identity regression test for the snapshot refresh.API Sync Checkjob (map validity,--check, a determinism proof, a non-blocking coverage report) wired intomain.yamlalongsidecontract-check..github/workflows/api-sync.yml: rewritten. Dropsanthropics/claude-code-actionand everyCLAUDE_CODE_OAUTH_TOKENreference. Reads.api-sync/spec-current.jsonfromorigin/api-sync-data, runs the patcher, exits quietly on no-op, fails loudly (::error::) on NEEDS_HUMAN, otherwise commits + opens/updates the PR with afeat:/fix:title prefix derived from the patcher's own bump classification and enables auto-merge.api-sync-merged.ymlis untouched (nothing it does depends on what was replaced)..github/workflows/release-auto-merge.yaml(new): enables auto-merge on the release-please PR, gated strictly togithub.event.pull_request.user.login == 'github-actions[bot]'AND theautorelease: pendinglabel. Never touches a human PR; never bypasses required checks.client.pyversion fix (separate commit): moved the single source of truth tosrc/blindpay/_version.py(imported by both__init__.pyandclient.py, no circular import), so the previously-hardcoded, silently-stale__version__ = "2.3.0"User-Agent can't drift from the package version again. Updatedpyproject.toml's[tool.hatch.version]path andrelease-please-config.json'sextra-filesaccordingly.Pending drift applied (Phase A deliverable)
All three were already true of the committed
spec-snapshot.json-- pending drift the code never caught up to, not new spec surface delivered by this PR:refund_wallet_addressadded toCreateQuoteInput(resources/quotes/quotes.py) asNotRequired[Optional[str]]: new optional field onQuoteIn.CreateQuoteInputis atotal=TrueTypedDict, so a bareOptional[str]would have made the key structurally required and broken every existing caller that omits it -- caught in review, see "Post-review fixes" below.BankingPartner(types.py): missing"portage"(spec'sbanking_partnerenum onVirtualAccountOut/CreateVirtualAccountInhas 5 values, the Literal had 4).PaymentMethod(resources/payins/quotes.py-- a local Literal shadowing the sharedtypes.pyone, confirmed via map-validity thatCreatePayinQuoteInput.payment_methodactually resolves to the local symbol): missing"transfers","pse","international_swift"(spec'sCreatePayinQuoteIn.payment_methodenum has 9 values, the local Literal had 6). Confirmed against blindpay-v2 (packages/reference/src/rails.ts) and the other 4 SDKs (node/go/php/swift all already carry all 9) that this was staleness, not a deliberate subset.funding_end_to_end_id/payer_tax_id(onLedgerOperation) and the originally-assumedprovider_referencefield were investigated and are not simple field-adds -- see Findings below.Post-review fixes
Two correctness bugs were caught in review and fixed before this PR was considered done:
CreateQuoteInput(atotal=TrueTypedDict whose other fields happen to be typed as plainOptional[str]), that producedrefund_wallet_address: Optional[str]-- a structurally required key, sincetotal=Truemeans every declared key must be present regardless of whether its value type allowsNone. Existing callers that build aCreateQuoteInputwithout the new key would fail pyright/mypy: a breaking change to the published 3.0.0 surface, which is the project's first constraint. Fixed: the emitted form now depends only on the target class's totality, never on sibling convention --total=False(or the non-required half of the_XRequiredtwo-part pattern) stays bareOptional[T]/T;total=Truealways getsNotRequired[Optional[T]]/NotRequired[T], unconditionally. The patcher also now addsNotRequiredto whichever existing typing/typing_extensions import already carriesTypedDictin that file, sincequotes.pyimportsOptionalfromtypingbut had never neededNotRequired. Verified: pyright and mypy both pass with the new key omitted entirely.["T","null"]and"T"passed silently in either direction -- exactly the silent-divergence failure mode this project exists to prevent. Fixed: nullability changes are now flagged needs-human symmetrically (same treatment as required-ness changes), with one deliberate, tested exception: a property with no"type"key at all on either side (pure metadata-only change) is treated as compatible, since there's nothing concrete to compare -- this is the real, benign shape this spec's owncreated_at/updated_atfields went through.What the type comparison does not cover: it only catches drift going forward from a synced baseline (old spec vs new spec for a property the SDK already models). It does not independently verify that the SDK's pre-existing annotation already matches the spec today, independent of any change -- that would be a materially different, larger audit, not a change-detection pass.
Findings beyond the assigned drift
The map-validity check and the bootstrap diff surfaced several more real, pre-existing gaps while building the map. None were auto-applied; each needed either confirmation of staleness (like the 3 above) or a human decision (recorded below with reason + owner, not silently dropped):
LedgerOperation(holdsfunding_end_to_end_id/payer_tax_id) has zero$refanywhere in the filtered public spec -- an unreachable orphan. Inignore.schemas.provider_referencedoes exist, just nested: it's on the inlinetracking_paymentsub-object ofPayoutOut/PayoutOnEvmOut/4 payout webhook schemas (never$ref'd, duplicated inline 6x). The SDK's sharedTrackingPaymentmodels 6 of the wire's 20 payout-side fields; 14 are unmodeled (provider_reference,provider_integration,provider_error_reason,provider_uetr,provider_imad,provider_clearing_system,recipient_name/tax_id/bank_code/branch_code/account_number/account_type,coelsa_id,end_to_end_id). Recorded inunmodeled.json, recommended as one deliberate, reviewed follow-up PR across all 5 SDKs (exposing the richer payout tracking surface -- including recent UETR/provider_referencecapture work -- is genuinely valuable, just not a blind per-field add).tracking_transaction(missingledger_in_transaction_id/ledger_out_transaction_id/provider_error_reason) andtracking_complete(missingprovider_transaction_id/refund_reason); payin-sidetracking_transactionhas a ~15-field wire shape barely modeled by the generic 4-fieldTrackingTransaction-- and a fullerGetPayinTrackingTransactionTypedDict already exists inpayins.pyas dead code (never referenced).BankAccountTypeenum value mismatch (not a missing member): spec is[checking, saving], SDK Literal is[checking, savings]. A live defect (not just typing) confirmed against blindpay-v2's ownaccount_type === 'saving'branching. Kept mapped (not dropped -- dropping would make it invisible again) with the divergence recorded inunmodeled.json; needs a coordinated rename PR, deliberately not auto-fixed here.KycStatus(customers.py): SDK Literal has 2 values,CustomerOut.kyc_statushas 8. Kept mapped with the divergence recorded; needs a human decision on whether to add the 6 missing members or whether this field was wired to the wrong Literal from the start.CreateWalletIn.name: required by the API, never sent byCreateCustodialWalletInput(onlycustomer_id/network) -- flagged as a likely functional bug, not just a modeling gap.additional_info, etc.) seeded intounmodeled.jsonwith individual reasons.Proof
Determinism proof (two independent copies, applied from a true pre-drift state against the delivered spec, after the NotRequired fix):
(The enum-only and full-3-item determinism/idempotency proofs from the initial commits, also byte-identical across independent copies, are unaffected by this fix -- BankingPartner and PaymentMethod are Literal-member additions, not TypedDict fields.)
Type audit (
--audit-types, non-blocking)diff_removals_and_changesonly catches type drift going forward from a synced baseline -- it can't see a pre-existing latent mismatch that was already there when a field was first modeled. Added--audit-types: a full, non-blocking (always exit 0) state comparison of every mapped property's spec type/nullability against the SDK's current annotation, reusing the existingcoarse_type/SCALAR_TYPE_MAPprimitives rather than a new engine. Wired into the API Sync Check CI job as a fourth, always-green, printed-only step.Run against this repo's current spec: 316 findings, all informational.
Optional[...]". Overwhelmingly a byproduct of a recent, broad, defensive nullable-almost-everything pass this spec's own JSON Schema generation went through (the same shape as thecreated_at/updated_atfields referenced in the snapshot-format fix commit) -- not 293 distinct bugs, but a real, low-urgency, systemic pattern worth someone's attention in Phase C.str(noLiteral) -- a mix of legitimate free-form fields and real missing-Literal opportunities, e.g.Payin.type/.status/.payment_methodand the sharedTracking*.stepfields acrosstypes.py.PayinOut.billing_fee_amount:numberon the wire,Optional[str]in the SDK (Payininpayins.py) -- a financial amount modeled as a string.VirtualAccountOut.blockchain_wallet: a real{network: enum, address: nullable string}object on the wire, collapsed to a bareOptional[str]in the SDK (VirtualAccountinvirtual_accounts.py) -- the network/address sub-fields aren't modeled at all.Deliberately not flagged: SDK wider than the spec needs (
Optional[...]where the spec is never null,floatwhere the spec isinteger) -- common, harmless, legitimate modeling choices. Nothing here is fixed in this PR; it's Phase C triage material, exactly as this audit exists to surface rather than fix.Not done here (deliberately out of scope for Phase A)
BankAccountType/KycStatus/the payout tracking_payment gap -- recorded, not applied, per above.gh api repos/blindpaylabs/blindpay-python --jq .allow_auto_merge->false). Bothrelease-auto-merge.yamlandapi-sync.yml's own auto-merge step need it;gh pr merge --autowill fail until it's turned on in Settings -> General -> Pull Requests.CLAUDE_CODE_OAUTH_TOKENremoval from repo secrets (out of this PR's control; the workflow no longer references it).Per the plan: nothing merges here until reviewed.
https://claude.ai/code/session_01F1stiNzuNtJXoXtiW9ZCbs