Repository navigation
Phase 2 + core loop: corpus, harness, Nemotron agent loop (kill gate met) - #2
Merged
Merged
Conversation
…ation zod model for the MVP subset of EN 16931 with decimal strings only. deriveAmounts computes BT-131, the VAT breakdown (BG-23) and all totals (BG-22) with decimal.js and half-up rounding; category O reports rate 0 as XRechnung BR-DE-14 requires. Refs: CORE-01, EXT-01 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
readCii turns test suite XML into ground truth (MVP subset, keeps #SKONTO# line breaks). buildCii writes schema-ordered CII from model + derived amounts, byte-identical for the same input. Refs: CORE-02 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uite 19 in-scope suite invoices (380, EUR, no allowances/prepaid/rounding, credit transfer). Derived totals match the printed ones (VAT ±0.01 per BR-CO-17); model -> CII -> verifier is ACCEPTABLE for 19/19. CI fetches the suite before tests and runs roundtrip:check. Refs: CORE-01, CORE-02 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Units (2162 Rec 20/21 codes plus German words), countries, VAT categories, currency, dates and decimals. Code lists generated from the pinned KoSIT configuration. Unknown or ambiguous values fail with a reason instead of being guessed; a bare 0 % never becomes a category. Refs: EXT-04 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A value is kept only if its quote occurs on the cited page of the text layer (whitespace/case/dash tolerant, no fuzzy matching) and the printed value occurs inside the quote. Refs: EXT-03 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pdfkit with standard fonts, uncompressed and with a content-derived trailer ID so rebuilds are byte-identical across Node versions. Layouts classic, modern, compact; noise options for number/date format, label wording, block order, small font and page splits. Fields absent from the model are not printed. extractPageTexts reads the text layer with pdfjs-dist. Refs: EVAL-01, EVAL-02 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Placeholders of the test suite are replaced by fictional values. Sets: clean (3 layouts), noisy (2 variants), mutated (13 mutation types, each verified against the verifier), holdout 4 invoices chosen by hash and locked until phase 5. pnpm eval writes JSON and Markdown reports with validity, field accuracy per BT, invented fields, provenance, iterations, cost and runtime. Finding: unit 'Stück' yields BR-CL-23 but the verifier still answers ACCEPTABLE. Refs: EVAL-01, EVAL-03, EVAL-04, EVAL-05, EVAL-06 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nemotron extraction with constrained JSON and one retry; assembly with evidence check and normalization; repair plans from Nemotron Ultra filtered by the patch guard; agent loop with at most 3 rounds that stops when issues stop decreasing; consistency warnings for printed totals; JSONL audit chain (RFC 8785 + SHA-256) and pnpm audit:verify. Refs: EXT-01, EXT-02, EXT-05, CORE-03, REP-01, REP-02, REP-03, REP-04, AUD-01, AUD-02, AUD-03, OPS-04 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ent loop A line whose quantity x price differs from its printed amount triggers repair and otherwise NEEDS_INPUT, never a valid output. Shape checks reject role mix-ups the validator accepts (postcode as city, 'postcode city' as street, non-e-mails). One value+quote may not back fields of different kinds, and a German city must follow its postcode. Lines use one row quote instead of per-field quotes (long invoices were truncated). Tests cover NEEDS_INPUT, user input with provenance 'user' and the line-mismatch stop. Refs: EXT-02, EXT-03, EXT-05, REP-02, REP-04, CORE-03 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…evel errors pnpm eval --pipeline agent runs extraction, repair, verifier and audit per document. Reports add the kill-gate count (valid, BT-112/BT-115 correct, full provenance), invented and wrong fields per document, missing fields and open issues. Corpus hygiene: placeholder invoice numbers, '-' and 'nicht vorhanden' contacts, and long line IDs replaced. Refs: EVAL-04, CORE-01 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
clean 41/45 (91 %) valid with correct totals and full provenance, noisy 23/30, mutated 25/26 as expected. Findings, test-data hygiene and backlog items recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1. After #1 is merged, GitHub retargets this PR to
main.What
Measurement (latest run, non-holdout)
No valid output with wrong amounts since the per-line consistency check.
Requirements
EVAL-01, EVAL-03–06, CORE-01–03, OPS-04, EXT-01–05, REP-01–04, AUD-01–03 (EVAL-02 partial: KoSIT visualization layout moves to phase 4)
🤖 Generated with Claude Code