Backlog processing strategy: volume without compromising quality

[newhoggy]

Backlog processing strategy: volume without compromising quality

Goal

Process the ~80 open issues without compromising code quality, performance, functionality, or
documentation accuracy, using the available hardware (M5 MacBook Pro Max, Mac Mini, Linux x86
with SIMD, minimal EC2 for SVE2).

State snapshot (2026-07-18)

• ~80 open issues clustering into: jq semantics bugs (jq: array slice .[a:b] returns a stream of elements instead of a single sub-array #154–jq: integer literals larger than i64 fail to parse instead of becoming floats #166), YAML semantics bugs
  (YAML: block scalars type-coerce on the streaming path (|- 123 becomes a number) #168–YAML: duplicate mapping keys are first-wins (YAML 1.2 / yq take last or error) #174, yaml: scalar type resolution diverges from the 1.2 core schema, and is duplicated across 5 sites #226), CLI bugs (CLI: yq -I 0 (compact YAML identity) quotes scalars and drops --- document separators #175–CLI: dsv --header is effectively a no-op and cannot be disabled #180, yq: -I 0 changes document values, not just formatting #222), arch-specific SIMD bugs (yaml/simd/x86: signed compare misreads UTF-8 bytes as control chars, panicking yq on non-ASCII #230, yaml: x86 without AVX2 silently misparses YAML (classify_yaml_chars classifies 16 bytes, parser skips 32) #231, YAML to_json() drops multibyte UTF-8 on x86 (signed compare in find_json_escape) #150,
  NEON JSON classifier over-matches boundary bytes (differential vs x86/scalar on invalid JSON) #186, DSV BMI2/SVE2 quote-mask carry is wrong when a quote lands on bit 63 (silent row/field corruption) #149, bug(dsv): select1 underflow on zero-marker words (markers/newlines_select1) #196), robustness (JSON validator: unbounded recursion causes stack-overflow abort on deeply nested input #151–YAML alias resolution: no cycle guard causes infinite recursion abort #153, BitVec::rank0(i) over-counts for i > len (inherits the non-clamping trait default) #187, Structural indices use u32/i16 counters that silently truncate on large/deeply-nested input #188, popcount_words returns u32 which overflows at 512MB of ones #44), test infrastructure (test: land the green test scaffolding (CI arch-matrix, skip-visibility, differential harnesses, refactor snapshots) #192–test: SVE2 SIMD coverage — toggle64 parity reference + differential execution under emulation (split from #191) #194,
  test: the 8 yq comparison tests have never run in CI (skip silently, report green) #227), cleanup/refactor (Refactor: consolidate duplicated helpers between jq_runner and yq_runner #181–Cleanup: YAML SIMD has duplicated scalar helpers, stale doc comments, and a latent terminator mismatch #185, chore(lint): clippy --all-features -D warnings fails (18 pre-existing errors) on clippy 1.96 #197), and perf/research/features (the rest).
• 47 of 50 worktrees are empty scaffolding — zero commits ahead, zero dirty files; the 40
  covering DSV BMI2/SVE2 quote-mask carry is wrong when a quote lands on bit 63 (silent row/field corruption) #149–Structural indices use u32/i16 counters that silently truncate on large/deeply-nested input #188 are also 58 commits behind main. Only three contained real work, and all
  three are now open PRs: bench(bench): add Rust succinct-library comparison and document build-vs-buy decision (ADR-0011) #232 (issue Document evaluation of existing succinct bitvector libraries #47), fix(cli): honor NO_COLOR spec, validate JQ_COLORS, and document environment variables #233 (issue Document environment variables (especially SUCCINCTLY_SVE2) #48), YAML test suite: live conformance harness, conformance docs, issue tracking #229 (issue YAML parser: consider YAML Test Suite and document parser behavior differences #49).
• All three PRs have been reviewed and verified safe to merge (claims reproduced locally,
  including the yaml/simd/x86: signed compare misreads UTF-8 bytes as control chars, panicking yq on non-ASCII #230 SIMD fix under Rosetta and the BitVec overhead correction re-derived from
  the source).
• Five stale PRs untouched since Feb/Mar: refactor(yaml): eliminate seq_items bitvector for memory optimization #104, perf(simd): implement SIMD-accelerated JSON string escaping #115, feat(simd): implement AVX2-accelerated JSON string validation for x86_64 #127, feat(text): implement UTF-8 validation with scalar algorithm, CLI, and benchmarking #132, docs(docs): establish Architecture Decision Records system for project governance #136.
• CI red herrings: the nightly x86_64 leg fails on every fresh PR run because
  test_quiet_mode_no_output asserts empty stderr while current nightly cargo emits a
  future-incompat-report note; https://no-color.org/ intermittently fails the link check.

The core insight

The risk is not fixing bugs wrong — it is fixing them at volume against a test suite that
silently under-reports (#227: 8 yq comparison tests skip silently and report green) and code
that is duplicated (#226: scalar type resolution in 5 sites; #161/#162: eval.rs vs eval_generic
divergence). Spend the first ~10% of effort on harnesses and consolidation; the remaining 90%
becomes safely parallelizable. PR #229's two-sided known-failures manifest is the model: every
future YAML fix is forced to update the conformance record, keeping docs accurate automatically.

Progress update (2026-07-19)

Phase 0 is essentially complete; Phase 1 (the safety net) is largely landed. The 2026-07-18
snapshot above is kept as the historical baseline; this section is the current state.

Phase 0 — done:

• All three real PRs merged: feat(bench): add succinct library comparison benchmarks and ADR-0011 #237 (issue Document evaluation of existing succinct bitvector libraries #47 — ADR-0011 + succinct-library comparison bench;
  superseded the closed bench(bench): add Rust succinct-library comparison and document build-vs-buy decision (ADR-0011) #232), fix(cli): honor NO_COLOR spec, validate JQ_COLORS, and document environment variables #233 (Document environment variables (especially SUCCINCTLY_SVE2) #48), YAML test suite: live conformance harness, conformance docs, issue tracking #229 (YAML parser: consider YAML Test Suite and document parser behavior differences #49).
• CI red herrings fixed: nightly test_quiet_mode_no_output brittleness (PR test(test): refactor JSON validation tests to use pre-built binary #236); the
  no-color.org link-check flake (PR ci: exclude no-color.org from link checker #238).
• YAML to_json() drops multibyte UTF-8 on x86 (signed compare in find_json_escape) #150 closed as resolved by the already-merged yaml/simd/x86: signed compare misreads UTF-8 bytes as control chars, panicking yq on non-ASCII #230 signed-compare fix.
• ci: add commit-message linting workflow (commit-check.yml) #201 and ci(coverage): adopt omni-dev-coverage-check for PR diff coverage + line gate (x86_64 + ARM64), replacing tarpaulin/codecov #198 verified-and-closed — commit-check.yml (via ci: add commit message linting workflow #210) and the omni-dev coverage
  action (via ci: replace codecov with omni-dev coverage check action and document coverage workflow #199) are live in CI.
• Five stale PRs dispositioned:
  • refactor(yaml): eliminate seq_items bitvector for memory optimization #104 (seq_items elimination) — KEEP+REBASE → merged, with the perf: investigate seq_item detection optimizations after bitvector elimination #106 branchless
    mitigation folded in (recovers the 7–15% CPU regression) and stale docs/parsing refreshed.
    Follow-up: sequential yaml_bench + dev bench yq confirmation (branch
    bench-seq-items-elimination-tradeoff).
  • feat(text): implement UTF-8 validation with scalar algorithm, CLI, and benchmarking #132 (utf8-validation-scalar) — KEEP+REBASE → merged; the greenfield text module +
    text validate utf8 CLI baseline that perf(text): SIMD-accelerate the utf8 validator (CLI already landed; scalar remains, overlaps #133/#134) #131/perf(text): Broadword (SWAR) optimizations for scalar UTF-8 validation #133/perf(text): Implement broadword + DFA hybrid UTF-8 validation #134 build on. Benchmark doc marked
    provisional pending re-measurement.
  • perf(simd): implement SIMD-accelerated JSON string escaping #115 (SIMD JSON string escape) — REDIRECT → closed; folded into O5: Extract shared SIMD escape scanning module #125 (extract shared
    escape module) → SIMD optimization for JSON escape scanning in jq output path #91. Salvage on refs/pull/115/head.
  • feat(simd): implement AVX2-accelerated JSON string validation for x86_64 #127 (AVX2 JSON validation) — REDIRECT → closed; SIMD primitives to be harvested into
    feat: SIMD optimization for JSON validator (x86_64 AVX2) #122/perf(json): Implement chunked SIMD validation with register reuse #123/State Machine JSON Validator: Implementation Complete, Optimization Opportunities #130. Salvage on refs/pull/127/head.
  • docs(docs): establish Architecture Decision Records system for project governance #136 (ADR system bootstrap) — OBSOLETE → closed (superseded by the live ADR system
    from docs: adopt Architecture Decision Records (docs/adrs/) #207/docs: adopt Architecture Decision Records practice with corpus and cross-linking #211).
• Worktree pruning: still deferred. Merged/redirect/obsolete worktrees were pruned, but the
  ~40 empty issue-* scaffolding worktrees remain — the background implementation fleet is
  active (it auto-implemented test: the 8 yq comparison tests have never run in CI (skip silently, report green) #227/test: land the green test scaffolding (CI arch-matrix, skip-visibility, differential harnesses, refactor snapshots) #192/test: x86 SIMD coverage — CI leg + differential execution, find_json_escape signed-compare (split from #191) #193 and keeps iterating on issue-* worktrees), so
  the "confirm the fleet is stopped, then prune" precondition is not yet met.

Phase 1 — safety net:

• test: the 8 yq comparison tests have never run in CI (skip silently, report green) #227 — merged (PR test(yaml): make the yq comparison tests actually run — pinned-yq goldens + CI drift check #241): pinned-yq golden conformance suite with goldens sourced from
  the oracle (not the SUT), a two-sided known-failures manifest, a CI drift-check job, and the
  cli feature gate. Design decision recorded on test: the 8 yq comparison tests have never run in CI (skip silently, report green) #227.
• test: land the green test scaffolding (CI arch-matrix, skip-visibility, differential harnesses, refactor snapshots) #192 — merged (PR test(test): add comprehensive refactor guard tests and SIMD CI verification (#192) #242): characterization snapshots + deep-nesting-valid guards +
  CI-fails-loud-on-missing-SIMD-features. Its one remaining item (non-fast -e/--exit-status
  guard) split out as test: guard the non-fast -e/--exit-status path returns correct exit codes (green; #192 Part 4b) #244.
• test: x86 SIMD coverage — CI leg + differential execution, find_json_escape signed-compare (split from #191) #193 — in flight (PR test(simd): implement comprehensive x86 SIMD test coverage and hard feature expectations for CI #245, marked ready): x86 SIMD skip-visibility, SUCCINCTLY_EXPECT_SIMD
  hard contract, per-kernel find_json_escape differential tests, and a product fix for yaml: x86 without AVX2 silently misparses YAML (classify_yaml_chars classifies 16 bytes, parser skips 32) #231
  (SSE2 classify/skip-width mismatch — Rosetta-verified fail-without/pass-with). Review gap: the
  yaml: x86 without AVX2 silently misparses YAML (classify_yaml_chars classifies 16 bytes, parser skips 32) #231 fix has no CI-executable regression guard (the bug is non-AVX2-x86-only and CI runners have
  AVX2) → filed as test(ci): CI-executable regression guard for the x86 SSE2 classify/skip-width path (non-AVX2 fallback) #247.
• chore(lint): clippy --all-features -D warnings fails (18 pre-existing errors) on clippy 1.96 #197 (clippy --all-features -D warnings) — observed clean in recent worktrees off main;
  verify-then-close candidate.

New tracking issues created this session: #244 (#192 Part 4b), #247 (CI regression guard
for the SSE2 classify/skip-width path). #231 is being fixed by PR #245.

Progress update (2026-07-25)

The 2026-07-19 section above is kept as historical baseline; this section is the current state.
Verified against the live issue tracker (gh issue list --state open cross-referenced against every
issue named below), not against the snapshot prose above.

Phases 0–2 and 5 are complete; the safety net is fully in place. Remaining work is Phase 3's two
open lanes (jq, YAML), the Phase 4 cleanups, and the Phase 6 perf/research + feature bulk.

• Phase 0 — done, except worktree pruning. All reconcile items closed. The one open item —
  confirm the fleet is stopped, then prune the empty worktrees — remains deferred: the
  background implementation fleet has not been confirmed stopped, and in-flight worktrees remain
  for the open jq/YAML lanes. Independently prunable now: stale worktrees whose issues have since
  merged (O5: Extract shared SIMD escape scanning module #125, YAML alias resolution: no cycle guard causes infinite recursion abort #153, jq: array index out of bounds errors instead of returning null #157, jq: object comparison/sort interleaves key and value comparison #163, YAML: block scalars type-coerce on the streaming path (|- 123 becomes a number) #168, YAML: leading-dot floats stay strings on the streaming path (.5) #170).
• Phase 1 — safety net: complete. test: the 8 yq comparison tests have never run in CI (skip silently, report green) #227, test: land the green test scaffolding (CI arch-matrix, skip-visibility, differential harnesses, refactor snapshots) #192, test: x86 SIMD coverage — CI leg + differential execution, find_json_escape signed-compare (split from #191) #193 (merged; carried the yaml: x86 without AVX2 silently misparses YAML (classify_yaml_chars classifies 16 bytes, parser skips 32) #231 non-AVX2
  classify/skip-width product fix), and chore(lint): clippy --all-features -D warnings fails (18 pre-existing errors) on clippy 1.96 #197 are all closed. Both 07-19 gap-trackers are closed:
  test: guard the non-fast -e/--exit-status path returns correct exit codes (green; #192 Part 4b) #244 (PR test(test): add comprehensive exit status tests for non-fast path #248) and test(ci): CI-executable regression guard for the x86 SSE2 classify/skip-width path (non-AVX2 fallback) #247 (PR test(yaml): CI-executable regression guard for the x86 SSE2 classify/skip-width path (#247) #251). Every semantics fix now ships against the
  jq/yq/yaml-suite golden + differential harnesses with two-sided known-failures manifests and a CI
  drift job.
• Phase 2 — consolidate: complete. yaml: scalar type resolution diverges from the 1.2 core schema, and is duplicated across 5 sites #226 (5-site scalar type resolution), jq: values/first/last have wrong semantics in eval_generic (divergence from eval.rs) #161/jq: object </> comparisons always false in eval_generic #162 (eval.rs
  ↔ eval_generic divergence), and Refactor: consolidate duplicated helpers between jq_runner and yq_runner #181 (jq_runner/yq_runner helper consolidation) are all closed —
  the jq/YAML bug lanes now fix single sites, not N copies.
• Phase 3 — correctness fleet: CLI, SIMD, and bits lanes fully closed (CLI: yq -I 0 (compact YAML identity) quotes scalars and drops --- document separators #175–CLI: dsv --header is effectively a no-op and cannot be disabled #180/yq: -I 0 changes document values, not just formatting #222/jq: regex builtins (test/match/sub/gsub/scan/splits/capture) unavailable in the shipped CLI #167;
  yaml: x86 without AVX2 silently misparses YAML (classify_yaml_chars classifies 16 bytes, parser skips 32) #231/NEON JSON classifier over-matches boundary bytes (differential vs x86/scalar on invalid JSON) #186/bug(dsv): select1 underflow on zero-marker words (markers/newlines_select1) #196/DSV BMI2/SVE2 quote-mask carry is wrong when a quote lands on bit 63 (silent row/field corruption) #149; BitVec::rank0(i) over-counts for i > len (inherits the non-clamping trait default) #187/Structural indices use u32/i16 counters that silently truncate on large/deeply-nested input #188/popcount_words returns u32 which overflows at 512MB of ones #44/JSON validator: unbounded recursion causes stack-overflow abort on deeply nested input #151–YAML alias resolution: no cycle guard causes infinite recursion abort #153). Remaining open, both in-flight via the fleet:
  • jq lane: jq: comma generator rejected inside index brackets and call arguments #155, jq: 1 == 1.0 is false (Int vs Float use derived PartialEq) #156, jq: try/catch discards the error; catch runs on the input instead of the error value #158, jq: compound/alternative assignment evaluates RHS against the sub-value, not the root #159, jq: // alternative only inspects the first output of its left side #160
  • YAML lane: YAML: integer-valued floats collapse to int on the streaming path (1.0 becomes 1) #169, YAML: merge keys <<: *alias are not resolved #171, YAML: explicit non-scalar keys silently drop the entry #172, YAML: a tab after spaces in indentation is accepted and folded into the key #173, YAML: duplicate mapping keys are first-wins (YAML 1.2 / yq take last or error) #174
• Phase 4 — cleanups: Refactor: DSV has three copies of prefix_xor_scalar and scattered quote-mask logic #182 (DSV prefix_xor ×3) and Cleanup: YAML SIMD has duplicated scalar helpers, stale doc comments, and a latent terminator mismatch #185 (YAML SIMD duplicated helpers) remain
  open, in-flight.
• Phase 5 — SVE2: complete — test: SVE2 SIMD coverage — toggle64 parity reference + differential execution under emulation (split from #191) #194 and DSV BMI2/SVE2 quote-mask carry is wrong when a quote lands on bit 63 (silent row/field corruption) #149 closed.
• Phase 6 — perf/research + features: the remaining bulk, mostly untouched.
  • perf/research (open): Investigate SIMD batch select optimization #40, Reduce bp_to_text memory overhead while maintaining access performance #56, Cursor-based Elias-Fano for bp_to_text with SIMD acceleration #58, feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64, SIMD optimization for JSON escape scanning in jq output path #91, perf: investigate seq_item detection optimizations after bitvector elimination #106, feat: SIMD optimization for JSON validator (x86_64 AVX2) #122, perf(json): Implement chunked SIMD validation with register reuse #123, O4: SIMD escape scanning for jq format functions (@uri, @html, @csv, @dsv) #124, State Machine JSON Validator: Implementation Complete, Optimization Opportunities #130, perf(text): Broadword (SWAR) optimizations for scalar UTF-8 validation #133, perf(text): Implement broadword + DFA hybrid UTF-8 validation #134, BitVec loses to vers-vecs/sux on space, build, rank and select — reconsider its auxiliary use #228.
    (O5: Extract shared SIMD escape scanning module #125/O6: Profile jq string operations for SIMD opportunity #126 closed.)
  • features (open): Evaluate jaq's ValT as interoperability layer for jq/yq evaluation #70, feat: xq - XML query language using jq syntax #85, feat(bench): SSH-based distributed benchmark orchestration #98, M3.1: Lazy streaming keys via JqValue::LazyKeysArray #140, yaml: add opt-in strict validation (yaml validate / syq --validate) #223, yaml: support tags (!!str, !custom, verbatim) #224, yaml: recognize %YAML and %TAG directives #225.
  • Guardrail landed: bench: real-workload input corpus + shape statistics for end-to-end perf validation #301 — merged (PR feat(cli,bench,test): implement real-workload corpus with corpus-stats shape-statistics reporting (#301) #319): real-workload corpus + corpus-stats
    shape-statistics reporting. This supplies the representative inputs for the Phase 6 rule
    "end-to-end measurement before any doc claim," the specific guard against the
    P5/P2.6/P2.8/P3/P8 pattern. Every Phase 6 perf change should now be measured against this
    corpus.
  • jq: A/B-test memchr::memmem for substring builtins (index/rindex/indices/contains/split) — bench-only, guarded adoption (follow-up to #126) #303 (memchr::memmem substring A/B) closed: the bench-only measurement landed (PR feat(bench, build, cli): add jq substring-search A/B benchmark + defer phase 2 #316) and
    phase 2 (guarded adoption) was deliberately deferred.
  • Corpus/suite coverage extended, and perf: investigate seq_item detection optimizations after bitvector elimination #106 re-priced (2026-07-25): bench: corpus has no bare-dash YAML sequence items, so a 16x path is unrepresented #326 (bare-dash corpus
    file + corpus-stats metric, PR feat(cli): establish bare-dash YAML shape existence and frequency baselines for #326 #331) and bench: end-to-end YAML suite generates no flow, anchors, block scalars, explicit keys or multi-doc #327 (generated suite now emits
    flow/anchors/block-scalars/explicit-keys/multi-doc, PR feat(cli): generate flow, anchors, block scalars, explicit keys and multi-doc #330) both merged. The bench: corpus has no bare-dash YAML sequence items, so a 16x path is unrepresented #326 survey
    (docs/benchmarks/yaml-shape-survey.md, 33,575 real YAML files) then corrected two perf
    assumptions: perf: investigate seq_item detection optimizations after bitvector elimination #106's 16× win is on a shape occurring in only 0.042% of real YAML → treat as
    low priority, not rushed; and anchors are ~100× more common (0.494%, 4–11% in Home
    Assistant/Salt/Concourse/GitLab CI), so the corpus anchors: 0 is a sampling artifact and the
    live anchor work is the silent-corruption bug yaml: anchor on a sequence item with a collection value silently corrupts the document #328, not more bare-dash tuning. Findings
    recorded on perf: investigate seq_item detection optimizations after bitvector elimination #106 / bench: end-to-end YAML suite generates no flow, anchors, block scalars, explicit keys or multi-doc #327 / yaml: anchor on a sequence item with a collection value silently corrupts the document #328.

Recommended sequencing from here: with #301 landed, the Phase 6 measurement guardrail is in place —
ADR-gate the architectural / core-structure perf issues (#70, #85, #228, #64, #56/#58) before
implementation, and measure every perf change against the new corpus; let the fleet drain the
jq/YAML lanes and #182/#185 under the now-complete guardrails.

Phase 0 — Reconcile state (one sitting)

• Merge bench(bench): add Rust succinct-library comparison and document build-vs-buy decision (ADR-0011) #232, fix(cli): honor NO_COLOR spec, validate JQ_COLORS, and document environment variables #233, YAML test suite: live conformance harness, conformance docs, issue tracking #229 (done — bench(bench): add Rust succinct-library comparison and document build-vs-buy decision (ADR-0011) #232 superseded by feat(bench): add succinct library comparison benchmarks and ADR-0011 #237; all three merged)
• Fix the nightly brittleness: make test_quiet_mode_no_output immune to cargo's own
  stderr (filter it or exec the built binary directly) — PR test(test): refactor JSON validation tests to use pre-built binary #236
• Add no-color.org to the link-checker exclusions (or accept occasional re-runs) — PR ci: exclude no-color.org from link checker #238
• After YAML test suite: live conformance harness, conformance docs, issue tracking #229 merges: re-test YAML to_json() drops multibyte UTF-8 on x86 (signed compare in find_json_escape) #150's repro — resolved by yaml/simd/x86: signed compare misreads UTF-8 bytes as control chars, panicking yq on non-ASCII #230; YAML to_json() drops multibyte UTF-8 on x86 (signed compare in find_json_escape) #150 closed as duplicate
• Verify-then-close issues already implemented: ci: add commit-message linting workflow (commit-check.yml) #201 (commit-check.yml exists), ci(coverage): adopt omni-dev-coverage-check for PR diff coverage + line gate (x86_64 + ARM64), replacing tarpaulin/codecov #198
  (coverage action already in CI per CLAUDE.md) — both closed
• Decide keep-or-close on the five stale PRs: refactor(yaml): eliminate seq_items bitvector for memory optimization #104 merged, feat(text): implement UTF-8 validation with scalar algorithm, CLI, and benchmarking #132 merged, perf(simd): implement SIMD-accelerated JSON string escaping #115 & feat(simd): implement AVX2-accelerated JSON string validation for x86_64 #127 closed
  (redirect), docs(docs): establish Architecture Decision Records system for project governance #136 closed (obsolete)
• Confirm the background issue-implementation fleet is not running, then prune the 47 empty
  worktrees and their branches — still deferred: the fleet is active (see progress update)

Phase 1 — Build the safety net first

• test: the 8 yq comparison tests have never run in CI (skip silently, report green) #227 — the 8 yq comparison tests must run in CI or fail loudly — merged (PR test(yaml): make the yq comparison tests actually run — pinned-yq goldens + CI drift check #241):
  pinned-yq golden suite (oracle-sourced) + two-sided known-failures manifest + drift-check job
• test: land the green test scaffolding (CI arch-matrix, skip-visibility, differential harnesses, refactor snapshots) #192 — land the green test scaffolding: skip-visibility, characterization snapshots,
  deep-nesting guards — merged (PR test(test): add comprehensive refactor guard tests and SIMD CI verification (#192) #242); residual -e guard split to test: guard the non-fast -e/--exit-status path returns correct exit codes (green; #192 Part 4b) #244
• test: x86 SIMD coverage — CI leg + differential execution, find_json_escape signed-compare (split from #191) #193 — x86 SIMD coverage: CI leg + differential execution — in flight (PR test(simd): implement comprehensive x86 SIMD test coverage and hard feature expectations for CI #245, ready);
  carries the yaml: x86 without AVX2 silently misparses YAML (classify_yaml_chars classifies 16 bytes, parser skips 32) #231 SSE2 classify/skip-width fix (Rosetta-verified); CI-guard gap tracked as test(ci): CI-executable regression guard for the x86 SSE2 classify/skip-width path (non-AVX2 fallback) #247
• chore(lint): clippy --all-features -D warnings fails (18 pre-existing errors) on clippy 1.96 #197 — clippy --all-features -D warnings clean — observed clean in recent worktrees;
  verify-then-close candidate

With jq/yq-as-oracle differential testing in place, every subsequent semantics fix ships with a
machine-checked proof.

Phase 2 — Consolidate before fixing duplicated code

• yaml: scalar type resolution diverges from the 1.2 core schema, and is duplicated across 5 sites #226 first — scalar type resolution is duplicated across 5 sites; fixing YAML: block scalars type-coerce on the streaming path (|- 123 becomes a number) #168–YAML: leading-dot floats stay strings on the streaming path (.5) #170 before
  it means fixing each bug five times
• Settle the eval.rs / eval_generic divergence (jq: values/first/last have wrong semantics in eval_generic (divergence from eval.rs) #161, jq: object </> comparisons always false in eval_generic #162 document it; Refactor: consolidate duplicated helpers between jq_runner and yq_runner #181 is the related
  refactor) before the jq semantics batch, or at minimum require fixes land in both sites
  with shared tests

Phase 3 — The correctness fleet (the bulk, ~35 small bugs)

Parallelize across modules, not within them — one in-flight PR per module lane to avoid
rebase storms:

Lane	Issues	Machine
jq	#154–#166 (post Phase 2)	Any
CLI	#175–#180, #222, #167, #176–#179	Any
YAML	#168–#174, #171–#173 (post #226)	Any
SIMD	#231, #186, #196, #149 (BMI2 half)	Per-arch (see below)
bits	#187, #188, #44, #151–#153	Any; large-tests on Mini

Every PR: failing repro test on main first → fix → differential test where an oracle exists →
clippy → patch coverage → conformance manifest updated where YAML behavior changes.

Phase 4 — Cleanups after their module's bug wave lands

• Refactor: consolidate duplicated helpers between jq_runner and yq_runner #181–Cleanup: YAML SIMD has duplicated scalar helpers, stale doc comments, and a latent terminator mismatch #185, Cleanup: DSV SSE2 backend builds a full BitVec (rank+select) then discards it #183, Cleanup: dead/misleading JSON SIMD code (unused bmi2 module, discarded PCMPISTRM) #184 — refactors sequenced after fixes so they don't conflict

Phase 5 — SVE2 batch (cost-minimized)

• test: SVE2 SIMD coverage — toggle64 parity reference + differential execution under emulation (split from #191) #194 — develop against QEMU emulation on the Linux x86 box (the issue's own suggestion)
• DSV BMI2/SVE2 quote-mask carry is wrong when a quote lands on bit 63 (silent row/field corruption) #149 SVE2 half — same
• One short EC2 session only: after QEMU is green, spin up, validate on real hardware,
  capture artifacts, terminate

Phase 6 — Perf and research (strictly sequential benchmarks)

• BitVec loses to vers-vecs/sux on space, build, rank and select — reconsider its auxiliary use #228 — BitVec vs vers-vecs/sux, in bench-compare/ (ADR-0011 and the succinct-library
  comparison bench from feat(bench): add succinct library comparison benchmarks and ADR-0011 #237 — merged — are the starting point)
• feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64, Cursor-based Elias-Fano for bp_to_text with SIMD acceleration #58, Reduce bp_to_text memory overhead while maintaining access performance #56, perf: investigate seq_item detection optimizations after bitvector elimination #106, feat: SIMD optimization for JSON validator (x86_64 AVX2) #122–O6: Profile jq string operations for SIMD opportunity #126, perf(text): Broadword (SWAR) optimizations for scalar UTF-8 validation #133–perf(text): Implement broadword + DFA hybrid UTF-8 validation #134, SIMD optimization for JSON escape scanning in jq output path #91, Investigate SIMD batch select optimization #40 — one machine at a time, exclusive
  CPU, end-to-end measurement before any doc claim (the project's own record: P2.6, P2.8,
  P3, P8 all showed micro-benchmarks misleading)
• Features last: yaml: add opt-in strict validation (yaml validate / syq --validate) #223–yaml: recognize %YAML and %TAG directives #225 (YAML tags/directives/validation), feat: xq - XML query language using jq syntax #85, feat(bench): SSH-based distributed benchmark orchestration #98, Set up Dependabot for dependency updates #148, perf(text): SIMD-accelerate the utf8 validator (CLI already landed; scalar remains, overlaps #133/#134) #131

Machine assignment

Machine	Role
M5 MacBook	Interactive dev, orchestration, review; NEON bug #186; Rosetta x86 tests
Mac Mini	Background lanes, large-tests, ARM benchmark box (sequential)
Linux x86	x86 SIMD bugs (#231, #149-BMI2, #193), x86 benchmarks, QEMU SVE2 (#194)
EC2 (SVE2)	Single batch validation session after QEMU is green

Benchmarks are never run concurrently with anything; the benchmark machine is quiesced.
Rosetta on the M5 covers x86 correctness checks (as used to verify the #230 fix); it is
never used for x86 performance numbers.

Quality guardrails (definition of done per PR)

1. Failing repro test on main before the fix
2. Differential test against jq/yq oracle where applicable
3. cargo clippy --all-targets --all-features -- -D warnings clean
4. Patch coverage via omni-dev coverage diff
5. Docs updated if the change touches documented behavior; every perf/space claim in docs
   backed by a fresh measurement on the named platform
6. Conventional commit with required scope from .omni-dev/scopes.yaml
7. Worktrees external under $HOME/wrk/work-trees/succinctly/<branch>/, one issue = one
   worktree = one PR

[newhoggy]

De-risk prep: reconcile stale issue bodies against HEAD

While assessing the low-risk backlog (#125, #223, #224, #225, #301, and the "#131"
CLI), several issue bodies turned out to predate the current tree and misdirect
pickup. Reconciling each issue against HEAD — cross-checking its referenced
files/line numbers before starting — is the cheapest per-issue risk reducer.
Findings so far:

• perf(text): SIMD-accelerate the utf8 validator (CLI already landed; scalar remains, overlaps #133/#134) #131 — the CLI + scalar validator + benchmarks already landed (36d2828e,
  13b66886); text validate utf8 is dispatched at
  src/bin/succinctly/main.rs:68. Only the SIMD acceleration remains, which is a
  hot-path perf change overlapping perf(text): Broadword (SWAR) optimizations for scalar UTF-8 validation #133/perf(text): Implement broadword + DFA hybrid UTF-8 validation #134 — not the additive-CLI low-risk
  task the title implies. Re-scoped (see perf(text): SIMD-accelerate the utf8 validator (CLI already landed; scalar remains, overlaps #133/#134) #131).
• O5: Extract shared SIMD escape scanning module #125 — the "Current State" lists only src/jq/stream.rs as the cross-module
  consumer of find_json_escape; src/yaml/light.rs:19 (call site :1886) is also
  a consumer. Design/consumer notes added to O5: Extract shared SIMD escape scanning module #125.
• SIMD optimization for JSON escape scanning in jq output path #91 — premise still valid: src/jq/value.rs:222 escape_json_string is still
  scalar and is a distinct function from the find_json_escape SIMD scanner.
  Depends on O5: Extract shared SIMD escape scanning module #125.
• yaml: add opt-in strict validation (yaml validate / syq --validate) #223 / yaml: recognize %YAML and %TAG directives #225 — verify against the current tests/yaml_test_suite.rs lax:*
  baseline (84 markers) before starting; each fix is auto-scored.

Remaining sweep (not yet reconciled)

• yaml: add opt-in strict validation (yaml validate / syq --validate) #223 — confirm the JSON precedent files still match (src/json/validate.rs,
  src/bin/succinctly/json_validate.rs) and the lax:* baseline count.
• yaml: support tags (!!str, !custom, verbatim) #224 — verify check_unsupported gate location (src/yaml/parser.rs:646) and
  the flow-scalar path (:2661) still hold.
• yaml: recognize %YAML and %TAG directives #225 — verify directive lines still fall through parse_document_line.
• bench: real-workload input corpus + shape statistics for end-to-end perf validation #301 — confirm the sync-script precedents still exist
  (scripts/sync-yaml-test-suite.sh, scripts/sync-yq-golden.sh).

Note: the expensive de-risk scaffolding is already in place

Most of the risk-reduction infrastructure is already built and mostly closed:

• test: add missing coverage exposed by the code review (SIMD differential, numeric equality, overflow, CLI fast paths) #191 / test: land the green test scaffolding (CI arch-matrix, skip-visibility, differential harnesses, refactor snapshots) #192 / test: x86 SIMD coverage — CI leg + differential execution, find_json_escape signed-compare (split from #191) #193 / test: SVE2 SIMD coverage — toggle64 parity reference + differential execution under emulation (split from #191) #194 (closed) — SIMD differential harnesses, refactor
  snapshots (tests/cli_characterization_tests.rs, tests/snapshots/),
  find_json_escape signed-compare guard (tests/yq_cli_tests.rs:1127), SVE2
  toggle64 parity.
• bench: real-workload input corpus + shape statistics for end-to-end perf validation #301 (open) — real-workload corpus + shape statistics, the master de-risker
  for the perf cluster (Investigate SIMD batch select optimization #40/Reduce bp_to_text memory overhead while maintaining access performance #56/Cursor-based Elias-Fano for bp_to_text with SIMD acceleration #58/feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64/SIMD optimization for JSON escape scanning in jq output path #91/perf: investigate seq_item detection optimizations after bitvector elimination #106/feat: SIMD optimization for JSON validator (x86_64 AVX2) #122/perf(json): Implement chunked SIMD validation with register reuse #123/O4: SIMD escape scanning for jq format functions (@uri, @html, @csv, @dsv) #124/State Machine JSON Validator: Implementation Complete, Optimization Opportunities #130/perf(text): Broadword (SWAR) optimizations for scalar UTF-8 validation #133/perf(text): Implement broadword + DFA hybrid UTF-8 validation #134).

So the remaining de-risk gap is this triage discipline, which fits under this
issue's umbrella rather than needing new scaffolding.

[newhoggy]

Progress update (2026-07-30): backlog reconciled against HEAD

Ran the sweep this issue's 2026-07-24 comment called "the cheapest per-issue risk
reducer" — reconciling every open issue against HEAD before pickup. It had not
been re-run since. Verified against a release build of dd531200 (cli,simd),
clean tree.

Result: 48 open -> 42. Five issues were fixed or implemented and nobody had
noticed; a sixth (#413) closed independently mid-sweep when PR #482 merged.

Closed — verified against HEAD, not assumed

Issue	Verdict	Evidence
#397	fixed by 4ad253f3 (PR #439, under #396)	all 6 reproducers correct
#379	fixed by 4ad253f3 — same commit	all 5 reproducers correct
#337	not reproducing	800 KB: 2508 ms -> 22.5 ms; 1.58x/doubling, was 3.90x
#362	not reproducing	4 MB: 3028 ms -> 49.7 ms; 1.83x/doubling, was 3.85x
#91	already implemented	jq output path is SIMD via src/jq/escape.rs:128

For the two perf issues the judged signal is the per-doubling ratio, which is
architecture-independent and is the criterion those issues set themselves
("~3.9x is O(n^2); ~1.9x is linear"). Both are now linear. Output verified
unchanged in each case.

The pattern worth naming

Every one of these was fixed as a side-effect of another issue's work, and
nothing detected it. #397/#379 landed under #396's streaming builtins — and the
diff is verbatim the "Candidate fix" #397 had written out, including the caveat
that it wanted a differential sweep first. That sweep happened under a different
issue's review.

So the failure mode is not stale prose: it is that fixes and issues are not
linked in the direction that closes issues. This will keep happening, and it
costs more than tracker tidiness — it invites re-fixing solved problems against
stale bodies, which is the specific risk this issue exists to manage.

Not closed, and why — three issues carry unmerged evidence

Two neighbours read as close-now but must not be:

• State Machine JSON Validator: Implementation Complete, Optimization Opportunities #130 — the "Rejected at the analysis stage" verdict is sound, but the branch
  is 9 commits ahead of main and none is merged: ADR-0012 itself, a
  JSONTestSuite conformance harness, a cargo-fuzz harness, serde_json differential
  tests, the json_validate_bench-measures-nothing fix, and the kept -9% digit-run
  win. Closing first discards a fuzz + conformance + differential oracle for a
  validator reachable from the CLI on attacker-controlled input. Land, then close.
• feat: SIMD optimization for JSON validator (x86_64 AVX2) #122 — recommended for rejection twice, but ea95cb1a (the corpus extension
  taking the sample from n=35 to n=184, which the whole argument rests on) is the
  lone commit on its branch and is not on main. Land, then close.
• BitVec, RankDirectory, SelectIndex and CompactRank have zero production callers #321 — premise is false on main: src/json/light.rs:77 still holds
  newlines: crate::bits::BitVec, and text::LineIndex does not exist. Blocked
  on BitVec loses to vers-vecs/sux on space, build, rank and select — reconsider its auxiliary use #228's branch (8 commits, unmerged).

ADR-0012 is referenced by #122, #123, #130 and #321 but does not exist on
main — docs/adrs/ stops at adr-0011.md. It is on the #130 branch. Four
issues currently point at a document a reader cannot find.

Also reconciled: #124 — premise holds, but two of its references are stale;
notably src/util/simd/escape.rs is listed as "New:" and already exists, with the
define_escape_scanner! macro (:91) being precisely the seam it proposes to
build.

Revised state (42 open)

Cluster	Count	Note
jq semantics	21	all in src/jq/eval.rs — 22,988 lines
YAML	5	#224, #225, #340, #484, #485
perf / research / infra	16	~9 have a written verdict awaiting a branch landing

The safety-net gap that now dominates the risk

tests/data/jq-golden-known-failures.txt has zero entries — the suite reports
full agreement with jq across 190 cases while ~21 jq divergences reproduce from
the CLI. The corpus simply contains none of the shapes: no case exercises
[1, .], {a:(.x,.y)}, 1,2,3 | . * 2, path(..) or .a.b = 9 on {}. The
190 cases cluster around areas already fixed, because cases were added
alongside those fixes.

And the harness asserts output.status.success(), so a case whose correct jq
behaviour is partial stdout, then error, exit 5 cannot be expressed at all —
ruling out #355, #494, #495, #400, #486, #475, #498, #499.

Contrast yaml-test-suite-known-failures.txt, which carries 87 entries and
forces every YAML fix to delete a line. The jq lane has no equivalent scoreboard.

This is the same insight this issue opened with — "the risk is not fixing bugs
wrong, it is fixing them at volume against a test suite that silently
under-reports" — still true, now about a different suite. Phase 1 was declared
complete when the net was finished for YAML and only nominally for jq.

None of the five closures above has a regression guard, which is recorded on each.

[newhoggy]

De-risking work now captured as issues (2026-07-30)

Following the reconciliation sweep above: the recommended de-risking work existed
only as prose here and in closed-issue comments. Filed as tracked items:

Issue	Work	Blocks / carries
#513	Seed jq-golden-known-failures.txt from the open bug corpus — 11 golden cases + 2 error probes, no harness change	manifest 0 -> 13; the scoreboard the jq lane lacks
#514	Golden harness cannot express a non-zero jq exit (jq_golden_tests.rs:132, and the sync script aborts)	unblocks pinning #355, #400, #494
#515	Six regression cases for the shapes #397/#379 were silently fixed in	nothing currently guards them
#516	No generator emits a childless -	carries #337's closing ask forward
#517	yq_comparison.rs:18 hardcodes 5 of 16 patterns instead of deriving from ALL_PATTERNS	carries #362's closing ask forward
#518	#70 has an unanswered maintainer rebuttal	communication action, six months old

#516 and #517 exist because closing #337 and #362 moved live coverage gaps into
closed issues. The bugs are genuinely fixed, but each issue ended with an
outstanding benchmark-coverage ask, and a note on a closed issue is not a tracked
work item. Filing corrects that rather than reopening.

The case for #513 got stronger while this was being written

#475 was fixed by PR #503. Its own comment had warned that doing so "will light
up a surface that is currently unreachable and therefore untested by goldens."
#505 — a panic in del() on a comma target mixing . with another path —
was filed immediately after. That is the collateral damage a seeded manifest is
for: the lit-up surface would have been pinned in CI rather than found by hand.

Worth noting the practice is already half-right — #501/#503/#504 added ten golden
cases between them. Every one was for the bug that PR fixed. #513 is about
pinning bugs before someone starts on them, which is what makes the manifest a
scoreboard and what makes a neighbouring regression visible.

[newhoggy]

Backlog risk re-triage (2026-08-02)

Ran a full risk re-assessment of the open backlog — every issue reconciled against live HEAD (d13ba795) via three parallel research passes (jq/CLI semantics, YAML, perf/research), not just re-read from its own text. Full write-up, per-issue risk tiers, and a de-risking playbook: https://claude.ai/code/artifact/807ced91-84d4-4ff7-8d50-1f6187e8f651

Context that changed the risk picture: PR #563 merged the same morning — a Partial(Vec<OwnedValue>, Control) architecture threaded through ~400 call sites so a mid-stream error/break keeps outputs already produced (fixes #400, #494; also #562 as a side effect). It explicitly left array/object construction, reduce/update, and computed indexing atomic — several open issues (#354, #392, #534, #483) sit precisely in that carved-out territory, not in code #563 touched.

Closed 5 issues found already resolved, verified against merged PRs / accepted ADRs: #228 (→ PR #528, ADR-0012), #130 (→ PR #521, ADR-0013), #123 (same ADR-0013, duplicate shape of #130), #56 (own restated acceptance criteria met), #58 (superseded by P12/#62; own measurement found EF 34-40x slower for this access pattern). This is the same "reconcile before triage" practice this issue's 2026-07-30 sweep used, applied again — the backlog keeps generating issues that quietly finish without getting closed.

Housekeeping still open from this pass, not yet actioned: #321 is now genuinely unblocked (#228, its cited blocker, has landed) but its own comment still reads as not-yet-actionable — needs a note saying so. #533 stays closed as a confirmed duplicate of #321, no action needed there.

Three genuine code-sharing clusters found in the jq lane (detail in the artifact) that should be designed/sequenced together rather than picked up independently:

• Comma/fanout family: jq: comma operator reorders outputs — document-derived values hoisted ahead of literals #353 → jq: object construction drops all but one output instead of taking the cartesian product #354 (must sequence), jq: if/select collapse a multi-output condition, and the two evaluators disagree how #378 (sibling, triplicated across 3 sites — push_truthiness already exists as the fix), jq: assignment collapses a multi-output RHS — = keeps the first, |= wraps them in an array #392/jq: recurse/recurse(f) drop a null child and flatten a single array-valued child #490/jq: reduce/foreach/until/while silently misfold multi-output init/update/cond/extract instead of jq's per-output fanout #534/jq: repeat(f) silently swallows an error raised inside f — no error, no output, exit 0 #495 (share eval_owned_expr's collapse-to-first policy — a multi-output-preserving sibling helper, on the precedent of the existing eval_owned_input, would de-risk all four at once)
• Path-walker family: jq: path(..)/path(recurse)/path(select) silently return [] instead of the real paths #483 ↔ jq: path() silently accepts a non-path filter where jq raises "Invalid path expression" #530 (opposite halves of one catch-all — design together), perf(jq): computed-key fan-out through '..'/recurse/typeof filters is quadratic in nesting depth #491 (hard prerequisite for jq: path(..)/path(recurse)/path(select) silently return [] instead of the real paths #483 — its fix reuses machinery perf(jq): computed-key fan-out through '..'/recurse/typeof filters is quadratic in nesting depth #491 shows is O(depth²))
• CLI round-trip lossiness (jq: nan does not survive as a number (round-trips to null via JSON) #472/yq/jq: --slurp/--inplace and jq --preserve-input pretty output still collapse duplicate mapping keys #478/path/parent/key builtins lose path context in CLI (eval_generic fallback evaluates builtin in isolation) #554) — thematically related, not code-shared, kept as separate PRs

YAML: #224 (tags)'s own text still cites 5-site scalar-type duplication as a blocking prerequisite — that consolidation (#226) is already closed, so its true remaining scope is smaller than filed: a design note on tag storage/resolution semantics, then 9 call sites (not 5). #340 (CRLF perf recovery) should get one cheap corpus extension (LF-vs-CRLF frequency counter) before its ~25-call-site fast path, to confirm the premise it's staking that complexity on.

Perf/research lane: of #235's original five ADR-gated issues (#70, #85, #228, #64, #56/#58), four have since resolved themselves — #64 is the only one still waiting on a written ADR. Its analysis is already rigorous (self-corrected 3 stale premises) but lives in a GitHub comment, not docs/adrs/. #133/#134 overlap almost entirely and are, structurally, another byte-classification micro-optimization — the shape behind three prior rejections (P2.6, P2.8, P3) — worth a corpus-density check before either, not just before doc claims.

Not yet actioned, needs a human: #518 (reply to jaq maintainer's 6-month-old rebuttal on #70) — two of the rebuttal's three claims are factually wrong and should be retracted in public, but per the issue's own note this needs a human-authored reply, not a generated one.

Full per-issue risk tiers, HEAD-status verification notes, and the complete de-risking playbook are in the linked artifact rather than duplicated here.

[newhoggy]

De-risking work captured as issues (2026-08-02, follow-up to the risk-triage comment above)

Per the same pattern as #513–#518: the playbook's genuinely-standalone prerequisite work
(no existing issue to attach to) is now filed rather than left as prose.

Issue	Work	Blocks / carries
#570	eval_owned_expr collapses multi-output sub-expressions to first-or-array — build a cardinality-preserving sibling (the pattern already exists narrowly as eval_owned_multi/eval_owned_input, just not applied here)	#392, #490, #495, #534
#571	corpus-stats has no LF-vs-CRLF frequency counter	gates #340
#572	seed golden coverage for #534's UPDATE-fold/EXTRACT-fanout/while-until-backtrack shapes (only INIT-forking is pinned, via #544)	prerequisite for #534
#573	yq golden harness can't express a case where yq rejects the input — yq-side analogue of #514	unblocks proper pinning for #519

Everything else in the playbook (sequence #353→#354, design #483/#530 together, land
#491 before #483, route #378 through the existing push_truthiness, #224's/#519's/#64's
own design decisions, #133/#134 dedup) already has an issue to attach to, so it stays as
a note on that issue rather than a new filing.

[newhoggy]

Backlog risk re-triage (2026-08-03)

Re-ran the reconciliation practice this issue's 2026-07-24 comment called "the cheapest per-issue risk reducer" — every open issue checked against live HEAD (72deef9e) via three parallel research passes (jq/CLI semantics, YAML, perf/research/features), not just re-read from its own text. Full write-up, per-issue risk tiers, and the de-risking playbook: https://claude.ai/code/artifact/a00bd3d3-e6f0-490c-bbfe-0abaa1f9f66f

Backlog: ~42 → 30 open since the 2026-07-30 count, 57 commits landed since yesterday's (2026-08-02) triage round.

Five issues are already finished in a worktree with no PR open — the lowest-risk action available right now is opening these PRs, not writing new code: #353 (jq comma-operator ordering, 3 commits clean), #340 (YAML CRLF has_cr fast path, 10 commits, already A/B-measured: M4 Pro regression +4.0%→+0.7%, 7950X +11.0%→+4.7%, byte-identical output across 244 configs), #64 (CS-Poppy select index, 7 commits, own comment already re-derived the math), and #134 (broadword+DFA UTF-8 hybrid, cross-platform measured, DFA correctly rejected per the P2.6/P2.8/P3 pattern). #133 duplicates #134's work independently — same fix, #134 is the superset — recommend closing #133 rather than merging both.

Two prerequisites unblock six issues: #570 (cardinality-preserving sibling to eval_owned_expr) unblocks #392/#490/#534/#575 at once — all four collapse through the same eval.rs:8890-8940 arm. #573 (yq golden harness expected_status/expected_err, mirroring the already-merged #514) is required before #519 can be pinned with any regression guard at all.

Three items are blocked on a human, not more agent time: #354's worktree is stuck mid-rebase with an unresolved conflict in eval.rs (don't let it restart — resolve by hand, sequence after/with #353). #518/#70's maintainer rebuttal has sat unanswered since 2026-01-26. #124's worktree has uncommitted edits — leave untouched.

Stale premises found this pass:

• yq/jq: --slurp/--inplace and jq --preserve-input pretty output still collapse duplicate mapping keys #478 — the --preserve-input pretty-mode criterion is already satisfied on main (Expr::Identity bypasses the IndexMap-collapsing path). Only --slurp/--inplace remain broken; rescope before starting.
• yaml: support tags (!!str, !custom, verbatim) #224 — "duplicated across 5 sites" cites yaml: scalar type resolution diverges from the 1.2 core schema, and is duplicated across 5 sites #226, which is already closed. The real gate is check_unsupported (parser.rs:900-901), 8 call sites. Also found: docs/parsing/yaml.md:607 still describes a has_tag()/tag() API that doesn't match current tag() — doc/code drift, unresolved.
• BitVec, RankDirectory, SelectIndex and CompactRank have zero production callers #321 — "zero callers" holds for BitVec/RankDirectory/CompactRank but not SelectIndex (still used by YAML's WithSelect, bp.rs:83) — becomes fully true once feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64 merges.

Design-before-code, unchanged from prior rounds: #85 (xq/XML, no design doc), #98 full scope (author's own "baseline capture/diff" carve-out is the safer first step), #519 (second instance of a YAML indent-stack corruption family #485 already patched once — needs a generalized design, not another special case), #140 (thorough 8-step design, nothing prototyped).

Remaining risk in the backlog is now concentrated at three central call sites rather than spread evenly: eval.rs's walk_path catch-all (#483/#530, design jointly — literal opposite halves of one arm), eval_owned_expr's collapse (#392/#490/#534/#575, converging on #570), and the YAML indent-stack family (#519).

[newhoggy]

Backlog risk re-triage delta (2026-08-03, ~14.5h after the 03:17 round)

Not a full 3-pass re-derivation — this morning's artifact reconciled the whole backlog against 72deef9e a few hours ago and its shape (3 central choke points: walk_path catch-all, eval_owned_expr collapse, YAML indent-stack family) still holds. This is a delta against current HEAD (becd1b8d, 21 commits since 72deef9e) plus first-pass triage of 9 issues filed after that round.

Resolved since this morning:

• jq: comma operator reorders outputs — document-derived values hoisted ahead of literals #353, perf(yaml): recover the CRLF-support regression with a has_cr const-generic fast path (#324 follow-up) #340 merged and closed.
• perf(text): Broadword (SWAR) optimizations for scalar UTF-8 validation #133 was merged via PR perf(text): broadword ASCII fast path for UTF-8 validation #590 but the PR never referenced the issue, so it stayed open — closed now.
• jq: object construction drops all but one output instead of taking the cartesian product #354 was "stuck mid-rebase with an unresolved conflict" this morning; it's clean and rebased onto current main. Opened PR fix(jq): take the cartesian product in object construction #604.
• feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64 gained 3 more commits (10 total, tree clean, includes docs cross-linking the new F1-F7 issues below) — the "stray commit to squash" caveat from this morning is gone. Opened PR feat(bp): CS-Poppy combined-sampling select support #603.

New wrinkle — needs a human look, not a triage verdict: PR #591 (from #134's branch) is open but now BEHIND main since #590 (#133's branch) already merged overlapping broadword UTF-8 work. This morning's recommendation was "PR from #134 (the superset), close #133" — instead #133's PR merged first. #591 needs a rebase+diff to check whether it still adds anything (the DFA-rejection comparison/writeup) beyond what main already has, rather than merging as-is or closing outright.

Line-number drift: #354's merge touched eval.rs above the sites #570/#483/#530 cite. eval_owned_expr moved 8890-8940 → ~8910-8960, walk_path moved 10126-10153 → ~10146-10173. Same conclusions, re-anchor before editing.

9 new issues since this morning, first-pass triage:

• yaml_bench: anchors/{10,100,1000,5000} panic with UnknownAnchor (generator bug, pre-existing on main) #594 (low) — yaml_bench's anchor-heavy generator has an off-by-one referencing a reference instead of a definition. Pre-existing on main, isolated to a bench helper.
• yaml_bench: x86-only block_scalars build regression coincident with #64, unexplained by its logic #595 (medium, needs investigation) — a real 12-28% x86-only (Zen 4, neutral on M4 Pro) build-time regression on block_scalars/*, found alongside feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64's benchmarking. The issue's own analysis argues it's unlikely to be feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64's logic (feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64 only touches BP select) and the root cause is still open. Worth resolving before trusting feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64's own Zen 4 numbers.
• F1: Drop BP's rank_l2 (largest remaining select/rank memory win) #596-602 (F1-F7) (low) — explicitly "deferred, not scheduled" follow-ups recorded from docs/plan/cspoppy.md §6 while writing up feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64, now that feat(bp): CS-Poppy combined-sampling select support #603 is open for review. All contingent on feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64 landing (several contingent on each other too: F4→F2, F6→feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64). F1 already contains a self-correction — its original premise ("rank_l2 down to one consumer") turned out wrong once feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64 was actually implemented, and the issue records that rather than leaving it stale. F3 is a design-first ADR item, same category as feat: xq - XML query language using jq syntax #85/feat(bench): SSH-based distributed benchmark orchestration #98/M3.1: Lazy streaming keys via JqValue::LazyKeysArray #140/yaml: a doubly out-dented block sequence continuation still corrupts the next mapping entry #519.

De-risking prerequisites are unchanged: #570 (cardinality-preserving eval_owned_expr sibling) still unblocks #392/#490/#534/#575 at one shared arm; #573 (yq golden harness status/err fields) is still required before #519 can be pinned. Still the two highest-leverage single fixes in the backlog.

Backlog: 37 → will drop to 35 once #603/#604 merge (30 this morning + 9 new − #353/#340/#133 closed).

[newhoggy]

Backlog risk re-triage (2026-08-04)

Fresh reconciliation of every open issue against live main (0ab98822 at the start of this round, c32caafa as of this update — PR #617 merged in between). Ran as 3 parallel read-only passes (jq core, YAML, perf/architecture) the same way prior rounds have, plus manual follow-up on #70/#518 which fell outside all three groups. Full per-issue table with file:line evidence: https://claude.ai/code/artifact/f6347222-b502-4909-8c22-168c894cdf27

Backlog: 25 open (was 37 → projected 35 in the last comment here). Net closures since that comment: #483/#530 (#618), #594 (#605), #64 (#603), #598 (F3 ADR, closed without a real decision), #490, #613 (#617), and #599 (closed this round, see below).

Actions taken this round:

• Closed F4: Migrate BitVec's newline bitmaps to a compact rank (contingent on F2) #599 (F4) with a comment — its target (BitVec newline bitmaps) was already superseded by BitVec loses to vers-vecs/sux on space, build, rank and select — reconsider its auxiliary use #228/ADR-0012's Elias-Fano LineIndex, which beats F4's own stated goal by >3x. Nothing left to migrate.
• PR fix(jq): path() now refuses a filter that is not a path expression #621 closed (not merged) — it was an independent duplicate fix for jq: path() silently accepts a non-path filter where jq raises "Invalid path expression" #530 that lost the race to PR fix(jq): recognize more path expressions and reject non-path filters in path() #618, which shipped a superset of the same behavior. Same "two worktrees solved the same problem" pattern as perf(text): Broadword (SWAR) optimizations for scalar UTF-8 validation #133/perf(text): Implement broadword + DFA hybrid UTF-8 validation #134.

Other findings, not yet acted on:

• issue-124's worktree has a 2-day-old uncommitted diff that duplicates already-merged PR fix(jq): render overflowed NumberLiteral as inf/-inf across the real CLI, not just garbage #565 verbatim (eval.rs:3754 on main already has it) — safe to discard, but there's a live process on that worktree so I left it alone. Its own committed SIMD-format work (tip a40aa564) is separately 435 commits behind main and needs a real rebase.
• jq: eval_owned_expr collapses multi-output sub-expressions to first-or-array — build a cardinality-preserving sibling #570's prerequisite (cardinality-preserving eval_owned_expr sibling) already landed via PR fix(jq): recurse doesn't drop a null child or flatten an array-valued one #619 (used to fix jq: recurse/recurse(f) drop a null child and flatten a single array-valued child #490) — this fully unblocks jq: assignment collapses a multi-output RHS — = keeps the first, |= wraps them in an array #392 and jq: break inside while/foreach/repeat raises a spurious error instead of propagating to the enclosing label #575 as small, proven-pattern fixes. jq: reduce/foreach/until/while silently misfold multi-output init/update/cond/extract instead of jq's per-output fanout #534 stays High risk; jq: eval_owned_expr collapses multi-output sub-expressions to first-or-array — build a cardinality-preserving sibling #570's own PR explicitly deferred the fork/fold/backtrack logic it needs.
• F6: Expose WithCsPoppy's sample rate as a tuning knob #601 (F6, CS-Poppy sample-rate knob) is fully unblocked now that feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64 merged — cheapest safe pick in the F1–F7 cluster.
• yaml_bench: x86-only block_scalars build regression coincident with #64, unexplained by its logic #595 (yaml_bench block_scalars x86 regression): PR feat(bp): CS-Poppy combined-sampling select support #603 (feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64) never touches parser/SIMD code, so it structurally can't be the cause. PR perf(yaml): recover the CRLF-support regression with a has_cr const-generic fast path (#324 follow-up) #593 (perf(yaml): recover the CRLF-support regression with a has_cr const-generic fast path (#324 follow-up) #340's CRLF fix, merged 1.5h earlier and a direct ancestor) is a likelier culprit and got conflated into the same before/after measurement. Recommend re-running the A/B as becd1b8d vs 09158571 before spending perf-host time on feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64.
• jq: object construction drops all but one output instead of taking the cartesian product #354/fix(jq): take the cartesian product in object construction #604 and perf: computed slice bounds (Expr::SliceExpr) skip eval_generic's native fast path #615/perf(jq): add native SliceExpr fast path to eval_generic #620 were CONFLICTING/stale when this round started but have since been rebased to MERGEABLE/CLEAN — both ready to merge pending CI.
• Evaluate jaq's ValT as interoperability layer for jq/yq evaluation #70/meta: #70 has an unanswered maintainer rebuttal — reply or close #518: jaq's maintainer explicitly declined to keep engaging with what they flagged as an LLM-authored issue, and meta: #70 has an unanswered maintainer rebuttal — reply or close #518 itself asks for a human-written reply. Left alone for a human decision — not something to auto-resolve.

Full table (all 25 issues, risk tier, evidence, de-risking notes) is in the artifact linked above.

[newhoggy]

Backlog risk re-triage (2026-08-05)

Fresh reconciliation of every open issue against live main (e996a4b3, previous round was c32caafa). Ran as 3 parallel read-only passes (jq/CLI, YAML, perf/architecture) the same way prior rounds have, each re-checking cited file:line evidence and reproducing bugs against a fresh release build where practical. Full per-issue table with risk tiers and evidence: https://claude.ai/code/artifact/aa41162d-4127-491a-9c8a-5fbf49e9068e

Backlog: 20 open (was 25 on 2026-08-04). Risk mix across the 19 non-meta issues: 7 High / 6 Medium / 6 Low.

Actions taken this round:

• Closed perf(jq): computed-key fan-out through '..'/recurse/typeof filters is quadratic in nesting depth #491 — its headline contributor (the eval_owned_expr reindex round-trip) already merged via PR perf(jq): fast-path bare ./.foo/.[n] in eval_owned_expr/eval_owned_input #629 (depth-400 time 0.30s → 0.07s). The remaining cost (deep clones in push_recursive_branches/resolve_recurse) is fully tracked by perf(jq): eval_index_expr/eval_slice_bound eagerly materialize a computed key/bound's full subtree before checking whether it's usable #626, so leaving both open double-booked the same work.

Cheapest wins available, not yet acted on (left for manual review):

• PR fix(jq): assignment no longer collapses a multi-output RHS (#392) #641 (fixes jq: assignment collapses a multi-output RHS — = keeps the first, |= wraps them in an array #392) and PR perf(jq): format values directly instead of round-tripping through JsonIndex #630 (prep for O4: SIMD escape scanning for jq format functions (@uri, @html, @csv, @dsv) #124) are both open, mergeable, and fully green — merging either is zero remaining risk.
• yaml_bench: x86-only block_scalars build regression coincident with #64, unexplained by its logic #595 just needs a re-run of an already-diagnosed A/B (becd1b8d vs 09158571 on terminus) — no code required to close it out.

Other findings, not yet acted on:

• jq: break inside while/foreach/repeat raises a spurious error instead of propagating to the enclosing label #575 has a near-mechanical fix template: edfb0110 (2026-08-04) fixed the identical "Break collapsed into a synthetic EvalError" bug shape one day earlier in eval_slice_bound; eval_owned_input is a second working reference. Applying the same pattern to eval_owned_expr should close it.
• perf(jq): eval_index_expr/eval_slice_bound eagerly materialize a computed key/bound's full subtree before checking whether it's usable #626 and jq: recurse(f)/recurse(f;cond) use breadth-first queue order, diverging from jq's depth-first definition #635 both rewrite the same three functions (push_recursive_branches/resolve_recurse, builtin_recurse_f/cond) — recommend sequencing jq: recurse(f)/recurse(f;cond) use breadth-first queue order, diverging from jq's depth-first definition #635 (smaller, queue→stack) before perf(jq): eval_index_expr/eval_slice_bound eagerly materialize a computed key/bound's full subtree before checking whether it's usable #626 (larger, structural-sharing) rather than parallelizing, to avoid rebasing the harder change on top of the simpler one.
• jq: reduce/foreach/until/while silently misfold multi-output init/update/cond/extract instead of jq's per-output fanout #534's only progress since filing is test scaffolding (PRs test(jq,yq): seed the manifests for the nine issues filed since #513 #544/test(test): seed golden coverage for #534's UPDATE-fold, EXTRACT-fanout and while/until-backtrack shapes #581) pinning the current wrong behavior as golden known-failures — zero implementation progress, still High risk. eval_owned_multi (eval.rs:7939-7948, built for jq: recurse(f; cond)'s cond argument still collapses multi-output via eval_owned_expr, breaks truthiness check #627/jq: resolve_node's Select/If arms still collapse multi-output cond via eval_owned_expr #628/jq: recurse(f; cond) silently prunes on a cond/f error instead of propagating it like jq does #636) is the building block its INIT-forking/EXTRACT-fanout needs, and jq: recurse(f; cond)'s cond argument still collapses multi-output via eval_owned_expr, breaks truthiness check #627's own fix is a close template.
• yaml: support tags (!!str, !custom, verbatim) #224 (YAML tags) turned out worse than filed: 3 of the 5 "tag shorthand as scalar text" corpus cases don't just get rejected, they silently produce wrong output, because the rejection gate (check_unsupported) isn't wired into every scalar-entry parse path (e.g. parser.rs:1061-1081's inline-document-value path explicitly leaves it ungated). Needs a full gate-coverage audit, not just the 10 existing call sites.
• feat: xq - XML query language using jq syntax #85 (xq) and feat(bench): SSH-based distributed benchmark orchestration #98 (SSH bench orchestration) both already have milestone-1 carve-outs proposed in their own threads (a format-agnostic subset for xq leaning on the existing DocumentValue/DocumentCursor seam; a single-machine bench baseline capture/diff for feat(bench): SSH-based distributed benchmark orchestration #98) — recommend scoping to those instead of the full design when picked up.
• In the F1-F7 cluster (F1: Drop BP's rank_l2 (largest remaining select/rank memory win) #596/F2: True Poppy rank layout as a standalone CsPoppy (3.125% vs RankDirectory's 25%) #597/F5: Elias-Fano-compressed select samples (shrink BP select beyond #64's u32) #600/F7: select0 support (complement of select1, no known consumer yet) #602, descendants of feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64's CS-Poppy work): F6: Expose WithCsPoppy's sample rate as a tuning knob #601 (F6) merged this cycle and already delivers most of F5: Elias-Fano-compressed select samples (shrink BP select beyond #64's u32) #600 (F5)'s goal via Config::select_sample_rate, without Elias-Fano's separately-measured 46x random-access latency cost. F2: True Poppy rank layout as a standalone CsPoppy (3.125% vs RankDirectory's 25%) #597 (F2) lost its stated consumer when F4 closed last round. Recommend a deprioritize/close discussion on both F2: True Poppy rank layout as a standalone CsPoppy (3.125% vs RankDirectory's 25%) #597 and F5: Elias-Fano-compressed select samples (shrink BP select beyond #64's u32) #600. F1: Drop BP's rank_l2 (largest remaining select/rank memory win) #596 (F1) is the most load-bearing remaining item but also the riskiest — rank_l2 now has two consumers (rank1 hot path + feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64's select1), not one, so it's a two-path rebalancing act, not a cleanup.

Full artifact has per-issue evidence, worktree/PR status, and the shared choke points (eval_owned_multi, the Control-based break-propagation pattern, #64's select infrastructure, define_escape_scanner!, DocumentValue/DocumentCursor) that de-risk multiple issues at once.

[newhoggy]

Backlog risk re-triage (2026-08-08)

Continuous reconciliation across the 2026-08-07 → 08-08 session, checked directly against live GitHub/git state (HEAD 8f9390cc, previous round ac92b99f on 08-07). Full roster, evidence, and detail cards: https://claude.ai/code/artifact/0bb90b80-f7c5-4813-9274-0ed348f66181

Backlog: 10 non-meta open (same count as 08-07, materially different composition — 4 closed, 4 new filed). Risk mix: 5 High / 3 Medium / 2 in-progress-own-work.

Closed this window:

• feat(bench): SSH-based distributed benchmark orchestration #98 — found fully implemented (all 5 phases, EC2-validated) via PR feat(bench): SSH-based distributed benchmark orchestration #660, which never referenced the issue. Closed manually.
• O4: SIMD escape scanning for jq format functions (@uri, @html, @csv, @dsv) #124 — remaining SIMD scope formally rejected by ADR-0016 on measured input shape (none of @uri/@html/@csv/@dsv/@sh's realistic inputs clear the 32-byte AVX2 threshold). 6th SIMD proposal this codebase has killed at the analysis stage (after P2.6, P2.8, P3, P5, P8). The earlier byte-oriented @uri/@html win (PR perf(jq): format values directly instead of round-tripping through JsonIndex #630, 12-15x) stands.
• yaml_bench: ARM-only long_block_scalars build improvement, unexplained, coincident with #64 #649 — closed via docs-only PR, confirmed-but-unresolved-mechanism, benign as expected.
• jq: measure format-function input-string size/escape-density distribution before instantiating SIMD scanning (precursor to #124) #663 / perf(bp): add a BP-specific rank1 micro-benchmark ahead of #596 (F1) #665 / jq: spike the smallest lazy-keys slice to validate #140's QueryResult/GenericResult invasiveness #666 — the three measurement/audit/spike precursors filed on 08-07 all resolved within ~24h: jq: measure format-function input-string size/escape-density distribution before instantiating SIMD scanning (precursor to #124) #663→ADR-0016 (reject), perf(bp): add a BP-specific rank1 micro-benchmark ahead of #596 (F1) #665→merged rank1 baseline (unblocks F1: Drop BP's rank_l2 (largest remaining select/rank memory win) #596), jq: spike the smallest lazy-keys slice to validate #140's QueryResult/GenericResult invasiveness #666→clean Go verdict (folded straight into M3.1: Lazy streaming keys via JqValue::LazyKeysArray #140's body).

Self-correction, the standout finding this round: #626 was misdiagnosed. PR #670 fixed the actual measured O(d²) blowup — but in eval_index_expr/eval_slice_bound (an unconditional to_owned() before a type check), not in push_recursive_branches/resolve_recurse as originally diagnosed on 08-07. Both functions genuinely do clone per-node; the benchmark that measured the symptom just never reached them — depth-400 time went 11.7ms → 93µs (~99% faster) from a completely different fix. The real, narrower bug is re-filed as #668 (only reachable via path()/write operators, not a bare filter), and a plausibly-more-impactful analog on the CLI's actual hot path was found and filed as #669 (eval_generic::eval_index_expr, needs its own measurement). Lesson: confirming a symptom reproduces isn't the same as confirming a specific code read is its cause — worth checking in every future round that a re-measured bug's benchmark actually exercises the function being blamed.

In progress, own worktrees, not yet PRs to review:

• yaml: audit all scalar-entry parse paths for missing tag-rejection gating (precursor to #224) #664 (YAML gate-coverage audit, precursor to yaml: support tags (!!str, !custom, verbatim) #224) — PR yaml: audit scalar-entry parse paths for missing tag-rejection gating #674 open, CI running, nothing failing.
• feat: xq milestone 1 — XmlIndex/XmlValue/XmlCursor core navigation (MVP carve-out of #85) #667 (xq milestone 1, MVP carve-out of feat: xq - XML query language using jq syntax #85) — 3 real feature commits (XmlIndex/XmlCursor/XmlValue + CLI wiring), not yet pushed.

Still open, unchanged since 08-07: #534, #635 (both High, both have available de-risking templates/sequencing noted in the artifact) — no new evidence this round, standing findings hold.

Now unblocked: #596 (F1, drop BP's rank_l2) — #665's baseline is on record on both architectures; ready to attempt. #140 (lazy keys) — #666's Go verdict measured a real 2.3-2.7x memory reduction at 100K fields, and invasiveness came in lower than feared (QueryResult needs zero changes; the real surface is GenericResult, ~14 mostly-mechanical sites).

Full per-issue evidence, repro commands, and the corrected #626/#668/#669 story are in the artifact.

[newhoggy]

Backlog risk re-triage delta (2026-08-08, same-day follow-up)

Quick delta on top of the round posted a few minutes ago — HEAD moved 8f9390cc → 950bb98a. Artifact updated in place (same link): https://claude.ai/code/artifact/0bb90b80-f7c5-4813-9274-0ed348f66181

#664 merged (PR #674) and found substantially more than expected. Its gate-coverage audit for #224 didn't just confirm the 3-4 previously known holes — it found 13 of 19 scalar/key entry paths ungated, collapsing to 6 root causes:

1. parse_document_line's gate runs before indentation is skipped — only fires at column 0.
2. parse_block_node's scalar-dispatch arms never call the gate at all — includes parse_inline_document_value, the exact mechanism behind the J7PZ corpus case (--- !!omap's tag absorbed as a full document-root scalar, so the following lines start a second document from one input).
3. parse_mapping_entry's next-line plain-scalar branch bypasses its own gate (only the same-line branch is guarded).
4. parse_mapping_entry/parse_compact_mapping_entry check once before testing for an anchor, never re-check after consuming one.
5. parse_explicit_value has zero gate calls.
6. No block-context key parser gates the key itself.

Flow context is already fully gated (#369). 7 new regression tests in tests/yq_cli_tests.rs pin today's wrong output per root cause, and docs/compliance/yaml/limitations.md now has the full gated/ungated table. #224 is unblocked and precisely scoped — updated its body in place with this.

#140's real implementation has started — its worktree moved past #666's spike commit and now has uncommitted changes across 9 files (jq_runner.rs, yq_runner.rs, document.rs, eval_generic.rs, lazy.rs, both light.rss, both CLI test files), consistent with the spike's Go verdict and revised scope. In-flight, left alone.

Roster: 9 non-meta open now (was 10 an hour ago) — 5 High / 3 Medium / 1 in-progress (#667 only; #664 moved to closed). No other issues changed since the last comment.

[newhoggy]

Backlog risk re-triage (2026-08-09)

Fresh reconciliation against live main (HEAD fdd7c251, previous round 950bb98a on 08-08), run as 3 parallel read-only passes (jq correctness/perf; YAML tags; architecture/xq/features), each reproducing bugs and re-reading cited file:line evidence rather than trusting prior issue text. Full roster, evidence, and detail cards: https://claude.ai/code/artifact/0bb90b80-f7c5-4813-9274-0ed348f66181 (same link as prior rounds, updated in place).

Backlog: 9 non-meta open (same count as 08-08, composition fully turned over — 13 closed via 13 merged PRs, 3 new filed). Risk mix: 2 High / 3 Medium / 2 Low / 1 in-progress.

A genuinely productive ~32-hour window: three of 08-08's five High-risk issues shipped as real merged implementations — #140 (lazy streaming keys, PR #678), #534 (reduce/foreach/until/while real per-output fanout, PR #692 + follow-up a5605194), and #635 (recurse(f) rewritten to genuine depth-first recursion, PR #679). Implementing #534's fanout immediately surfaced two more control-flow bugs in the same choke point, both filed and fixed same-day: #693 (?/try conflating a caught error with genuine empty, amplified by #534's change) and #694 (collect_owned()/eval_owned_multi silently dropping a Partial's trailing error/break). #684/#685 extended the same lazy-output work to array keys and syq. Same "fixing X finds Y" shape already on record for the del()-follow-up and stream-builtin-error-swallowing issue families.

Closed #683 — found fully implemented (PR #691 shipped its exact title and every acceptance criterion, just never referenced Closes #683). 4th "done but not closed" finding across 4 consecutive rounds (#491 on 08-05, #98/#124 on 08-08, now #683) — the standing check (diff merged PR bodies against closingIssuesReferences, not just the open-issue list) keeps earning its keep.

New: #682, a silent-wrong-output bug — reproduced this round: [path(recurse(.children[]?))] on a 2-element array returns [[]] instead of jq's [[],["children",0],["children",1]]. Root cause traced to needs_path_prepass (eval.rs:8161) returning false for Iterate/Optional(Iterate), so resolve_seq's dynamic-element scan never fans out. No test or fix attempt exists yet — High risk, no mitigation.

Other movement:

• yaml: support tags (!!str, !custom, verbatim) #224 (YAML tags) moves High → Medium: yaml: audit all scalar-entry parse paths for missing tag-rejection gating (precursor to #224) #664's gate audit (merged 08-08) eliminated the unknown-unknowns, and this round adds a scope correction — the real yq oracle never rejects a tag anywhere, it silently resolves and drops it, so the target is resolution semantics, not gate-extension. A tag-name scanner still doesn't exist at all; only !!str was checked against the oracle.
• F1: Drop BP's rank_l2 (largest remaining select/rank memory win) #596 (drop BP's rank_l2) stays High — baseline (perf(bp): add a BP-specific rank1 micro-benchmark ahead of #596 (F1) #665) still current, but re-confirmed as a genuine two-consumer rebalancing act (rank1 hot path + feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64's CS-Poppy select1), not a cleanup, with a named ARM select1 regression risk from feat: Shrink the BP/YAML select index with combined sampling (CS-Poppy) #64 to watch.
• perf(jq): push_recursive_branches/resolve_recurse still deep-clone O(subtree) per '..'/recurse node in path()/write context #668/perf(jq): eval_generic::eval_index_expr may have the same eager key-materialization bug as eval.rs — needs measurement #669 (recurse-clone / eval_generic key-materialization, both descendants of perf(jq): eval_index_expr/eval_slice_bound eagerly materialize a computed key/bound's full subtree before checking whether it's usable #626's self-correction) stay Medium. perf(jq): eval_generic::eval_index_expr may have the same eager key-materialization bug as eval.rs — needs measurement #669 gained before-fix numbers this round (JSON ~O(d²), YAML ~O(d^2.9)) and a confirmed-constructible fix template, though GenericResult's 8 variants mean ~8 mechanical arms vs. eval.rs's 4. perf(jq): push_recursive_branches/resolve_recurse still deep-clone O(subtree) per '..'/recurse node in path()/write context #668 gained an isolating benchmark (perf(jq): add a benchmark that isolates push_recursive_branches/resolve_recurse's clone cost, ahead of #668 #675) but no before-fix numbers yet, and its two candidate fix strategies (Rc-sharing vs. Cow-threading) remain undecided.
• jq: eval_generic.rs has no lazy map/select for any container type (blocks lazy keys_unsorted | map/select) #686 (generic lazy map/select) stays Low/exploratory — its own requested measurement precursor landed (PR test(bench): add wide JSON pattern + --queries flag to jq_bench for #686 #690), go/no-go verdict not yet recorded.
• feat: xq milestone 1 — XmlIndex/XmlValue/XmlCursor core navigation (MVP carve-out of #85) #667 (xq milestone 1) — builds/tests clean in isolation, but its worktree is now 37 commits behind main, including this window's own GenericResult rename (LazyKeysUnsorted → LazyKeys{sorted}) and new LazyIndexRange variant that its exhaustive matches don't know about. Won't compile on rebase as-is — worth rebasing soon rather than letting a fast-moving shared-core module's drift keep compounding.

Full per-issue evidence, repro commands, and cross-cutting patterns are in the artifact.

[newhoggy]

Backlog risk re-triage (2026-08-11)

Fresh reconciliation against live main (HEAD bfe73084, previous round fdd7c251 on 08-09), run as 3 parallel read-only passes (jq correctness — regex/dates/format/path bugs; jq feature-gaps/perf/architecture/xq; YAML/yq correctness+perf), each reproducing bugs and re-reading cited file:line evidence rather than trusting issue text. Full roster, evidence, repro commands, and de-risking priorities: https://claude.ai/code/artifact/0bb90b80-f7c5-4813-9274-0ed348f66181 (same link as prior rounds, updated in place).

Backlog: 38 non-meta open (vs 9 on 08-09) — 30 issues closed in the same ~25.5h window, but 59 new ones filed, mostly two waves: #700's lazy map/select lineage finishing (PR #740 → #724/#725 closed, spawning #757/#758/#667-rebase follow-ups) and a larger wave of fresh regex/date/YAML-comment/anchor/duplicate-key bug reports. Risk mix: 8 High / 16 Medium / 7 Low / 7 ready-to-land.

Three findings matter more than the raw count:

• jq: arithmetic/comparison operators collapse a multi-output operand to its first value instead of jq's cartesian fanout #768 (arithmetic/comparison operators collapse a multi-output operand instead of jq's cartesian fanout) is the sharpest bug in the batch — silent truncation, not an error, on .[] + 1-style idioms, 11 operators, zero test net. jq: object construction drops all but one output instead of taking the cartesian product #354/jq: assignment collapses a multi-output RHS — = keeps the first, |= wraps them in an array #392 already prove the fork-not-collapse fix pattern for two prior instances of the same anti-pattern in result_to_owned.
• yq: OwnedValue has no room for YAML style metadata — mutating queries drop style on touched AND untouched nodes #739 (yq OwnedValue has no room for style/comment/anchor metadata) is the architectural root cause behind yq: default output never preserves original YAML flow-style/quote-style #707/yq: assignment operators (=, |=, ...) still drop all comments, not just the assigned field #752/yq: YAML output never preserves anchor/alias (&x/*x) syntax, even on plain identity #763(mutation half)/yq: mapping-key trailing comment silently dropped from identity output when value is deferred to next line #765, not four independent bugs — all five funnel through the same to_owned() → JSON-round-trip chokepoint. Recommend an ADR before touching the harder half, with yq: default output never preserves original YAML flow-style/quote-style #707 (cursor still alive, just not threaded into the write path) as a cheap proof-of-pattern first.
• 7 near-zero-cost closures are sitting unclaimed: 3 open, green, mergeable PRs (fix(yaml): preserve trailing YAML line comments through yq (#710) #753→yq: comments are silently dropped from all output #710, yq: implement merge-flag suffixes (*+, *?, *n, *d, *c) on */*= #762→yq: merge-flag suffixes (*+, *?, *n, *d, *c) on */*= are unimplemented #713, yq: add --front-matter, --split-exp, and --eval-all/file_index #770→yq: missing --front-matter, real --split-exp (collides with -s), and eval-all/fileIndex #715) blocked only on review — yq: implement merge-flag suffixes (*+, *?, *n, *d, *c) on */*= #762 incidentally already fixes yq: */*= merge silently discards data when a top-level operand is null or absent #767 too — plus 2 worktrees with verified-correct fixes and no PR opened (yq: alias-sync (#711) skips pipe chains with a trailing non-assign stage #764, json: stream_json_as_yaml double-quotes strings — reads raw_bytes() instead of as_str() #776), plus feat: xq milestone 1 — XmlIndex/XmlValue/XmlCursor core navigation (MVP carve-out of #85) #667 (xq milestone 1), which flipped from last round's "37 commits behind, won't compile" to fully rebased, clean build/clippy/tests, just needing a PR.

Regex cluster (#727/#728/#729/#730) shares one root — build_regex's flag table (eval.rs:5707) — for #727+#730, while #728/#729 each already have a correct sibling implementation two functions away (builtin_gsub_with_flags, builtin_scan) to copy rather than redesign.

Prior-work / de-risking recommendations (full ordered list in the artifact): land the 3 ready PRs and open PRs for the 3 fix-ready worktrees first (cheapest, closes ~7 issues); fix build_regex's flag table once for #727+#730; fix #729's streaming before re-testing #730's n flag; write the #739 ADR before #752/#763/#765; run #758's already-built A/B benchmarks before investing in #757's harder design; sequence #721 before #722 (IN/INDEX reduces to any(...)); rebase #763's stale worktree before starting; isolate path-construction cost in benches/jq_recurse_clone_bench.rs before #701.

Full per-issue evidence, repro commands, and cross-cutting patterns are in the artifact.