Generate and verify shared serial layouts without enabling production routes

.github/workflows/ci.yml

@@ -374,6 +374,16 @@ jobs:
             -- --ignored --exact --nocapture | tee "$RUNNER_TEMP/word-native-replay.log"
           # Cargo also succeeds when a mistyped exact selector runs no tests.
           grep -F 'test result: ok. 1 passed; 0 failed;' "$RUNNER_TEMP/word-native-replay.log"
+      - name: Compare generated serial tables with pinned native processor
+        env:
+          EXIFTOOL_PERL: perl
+        run: |
+          set -euo pipefail
+          export OXIDEX_PINNED_EXIFTOOL="$(dirname "$EXIFTOOL")"
+          cargo test --all-features --test serial_directory_native \
+            real_audio_serial_tables_replay_pinned_native_callbacks \
+            -- --ignored --exact --nocapture | tee "$RUNNER_TEMP/serial-native-replay.log"
+          grep -F 'test result: ok. 1 passed; 0 failed;' "$RUNNER_TEMP/serial-native-replay.log"
       - name: Verify C header is up-to-date
         run: |
           cbindgen --config cbindgen.toml --crate oxidex --output api/oxidex.h.tmp
@@ -569,6 +579,8 @@ jobs:
           # the source-rebinding, complete-inventory and native replay cases.
           export OXIDEX_PINNED_EXIFTOOL="$ROOT"
           export OXIDEX_TABLES_JSON="$DUMP" OXIDEX_SONY_DUMP="$DUMP"
+          export OXIDEX_SERIAL_RUST="$GITHUB_WORKSPACE/src/exiftool_tables/serial_tables.rs"
+          python3 tools/exiftool-tables/verify_serial_directory.py "$OXIDEX_SERIAL_RUST" "$DUMP"
           export OXIDEX_WORD_KEYED_RUST="$GITHUB_WORKSPACE/src/exiftool_tables/keyed_tables.rs"
           export OXIDEX_PROCESSOR_INVENTORY="$PROCESSORS"
           export OXIDEX_WORD_PROCESSOR=Image::ExifTool::CanonCustom::ProcessCanonCustom

docs/AUTOGENERATION-PLAN.md

@@ -37,7 +37,7 @@ areas remain unfinished; they cannot disappear from the denominator.
 | Shared binary strings | Merged in PR #747 at `8f0fdaf4`. It preserves raw bytes in saved state, distinguishes a one-byte default from a remainder string, and carries the bounded CameraInfo repair. | This is a shared capability, not a Canon migration. CameraInfo remains a legacy text-domain adapter, and no Canon manual reader has been removed. The prior full-pair supervisor-status limitation remains recorded. |
 | Keyed-directory schema and compiler | Merged in PR #748 at `ebbe1ece`; all final hosted checks passed at `a422e8de`. | Native parent facts and expression declarations are checked. Shared reporting policy merged in #750; the inactive reader merged in #752 at `634e5616`. No production route is active. |
 | Shared word-directory processor | Merged in PR #754 at `1138a880`; nine tables, 132 rows, 698 Python tests and all five hosted jobs pass. | Four of five unsupported child processors now have generated descriptors. Canon production routing and manual-reader retirement remain unfinished. |
-| Dynamic serial-processor validation | Merged in PR #755 at `93d19e24`; all five hosted checks pass, including 706 native Python tests. | The probe validates native reads and callbacks. The next reviewed source inventory accounts for eight tables and 132 alternatives; shared Rust execution and manual-reader retirement still need proof. |
+| Shared serial processor | Native probe and inventory merged in #755/#756. The next combined checkpoint compiles eight generated tables, accounts for 106 emitted alternatives and 26 omissions, and passes Real AudioV3/V4 native/Rust replay. | Final hosted acceptance and merge remain. A separate V4 draft replaces its manual 31-slot sequence; carrier/corpus acceptance is still required before counting retirement. |
 | Recorded source inventory | Merged in PR #749 at `18a8ef17`; all final hosted checks passed at `72e8e664`. The report accounts for 1,512 table identities and retains 119 tables with no named rows. | This establishes the captured source population. Classifying which rules are generated, manual, unsupported or unclassified remains open; source shape is not automation. |
 | Sony plain generator recovery | PR #745 merged; six tables and 193 rows reproduced | These tables can be rebuilt. This alone does not prove that their behavior is fully automatic. |
 | Sony enciphered recovery | Producer and independent verifier preserved; M4 review found five blockers; not landed | The draft still has a Sony-specific translation layer. Its review remains useful, but it is not the architecture target. |
@@ -150,8 +150,9 @@ Canonical artifacts and ledgers were regenerated successfully with local Perl
 1. Compile sequential fields and lengths that depend on earlier raw values from
    the native serial processor. The [native probe](reference/serial-processor-checkpoint.md)
    is merged in #755. The reviewed JSON inventory captures all eight selected
-   tables and preserves every refusal. A shared Rust emitter and reader are
-   staged separately; combined compilation and native replay remain open.
+   tables and preserves every refusal. The combined Rust emitter and reader
+   compile and pass native Real AudioV3/V4 replay. Complete their final hosted
+   gate, then validate the V4 carrier and remove its manual sequence.
    Account for every condition and conversion before reducing the final
    unsupported-child count from one to zero.
 2. Resolve the four omitted parent rows using shared byte handling. An explicit

docs/AUTOGENERATION-PROGRESS.md

@@ -46,9 +46,10 @@ and zero skips**, in 566.450 seconds. The initial threaded-Perl failure was
 reproduced locally and fixed; the eight focused probe tests pass on both
 threaded and non-threaded Perl 5.38.2.
 
-The next reviewed source batch is committed on
-`codex/ciff-opaque-native-integration-20260913`; its first published complete
-inventory checkpoint is `37c32d58`. It accounts for all eight source-selected
+The source batch merged in PR #756 as `eb700430` at 11:46 UTC. All five
+hosted checks passed at `298917dd`, including 735 canonical Python tests,
+zero failures and zero skips, in 653.244 seconds. Its first published complete
+inventory checkpoint was `37c32d58`. It accounts for all eight source-selected
 serial tables: **130 entries, 132 alternatives, 115 clear at the source gate
 and 17 with explicit refusals**. One table also retains a priority-policy
 blocker. A complete processor grammar rejects changed executable behavior;
@@ -62,13 +63,37 @@ ordering. Final output groups/conversions and actual image digest computation
 remain outside these probes' proof. Warnings are recorded separately from the
 chronological callback trace. The combined checkpoint at `66c430c6` passes **37 focused tests, zero
 failures and zero skips**, on both threaded and non-threaded Perl 5.38.2
-against pinned ExifTool 13.59. Full hosted checks remain required for merge.
-
-The next delivery is one shared serial reader and source-driven Rust emitter,
-followed by native/Rust replay. Their worker commits are staged separately and
-still need combined compilation and review. No production route is enabled;
-**one unsupported Canon child processor, four omitted parent rows and zero
-Canon manual readers retired** remain the production status.
+against pinned ExifTool 13.59. The full hosted checks then passed for #756.
+
+The shared serial reader and Rust emitter are now combined on
+`codex/shared-serial-integration-20260913`, with pipeline checkpoint `9693ef9e`.
+The [runtime checkpoint](reference/serial-runtime-checkpoint.md) records the
+proof and its limits. All eight tables remain accounted for: **106 emitted
+alternatives and 26 explicit omissions**, with zero independent verifier
+mismatches. This is stricter than the source-only 115/17 split because the
+emitter also refuses unproved runtime formats and missing-member conditions.
+Four tables clear the definition gate; native/Rust replay currently covers
+Real AudioV3 and AudioV4 only.
+
+All 64 focused Python tests pass with zero failures and zero skips in
+65.023 seconds. The full `cargo clippy --all-features -- -D warnings` check
+and formatting check pass. All 13 shared-reader unit tests pass. The explicit
+native/Rust test passes
+its ten cases on both threaded and non-threaded Perl 5.38.2, using the actual
+generated tables. Official regeneration passes across 32 declared artifacts;
+all generated Rust is unchanged. The expression ledger initially records the
+local absolute invocation paths. Re-running its producer with the portable
+invocation reproduces the committed ledger exactly. All 607 expressions agree
+on 16,789 applicable comparisons; 14 inputs are inapplicable.
+
+This checkpoint still needs its final hosted gate and merge. No production
+route is enabled; **one unsupported Canon child processor, four omitted parent
+rows and zero Canon manual readers retired** remain the production status.
+The Real AudioV4 retirement draft is separately committed and published at
+`118afac7`. Source review accepts its narrow behavior, but it still needs a
+fresh Rust build, native output comparison, full-corpus pair and upgrade-flow
+proof. Its metadata occurrence API retains the existing group-0/group-2
+limitations; native warning output and V3/V5 activation also remain unfinished.
 
 The first additional retirement candidate is Real AudioV4's manually specified
 31-entry sequence. It uses the same native serial processor and can prove reuse

docs/UPGRADE-NEXT-STEPS.md

@@ -19,7 +19,7 @@ populations. They are implementation history, not work to restart.
 
 | Priority | Remaining work | State | Completion evidence |
 | --- | --- | --- | --- |
-| 1 | Extend the source inventory and complete the next shared directory capability | Sony Tag202a migration/retirement merged in #746; source inventory and artifact join merged in #749/#751. Directory validation and nine word tables merged in #753/#754; canonical regeneration and all final gates pass. One child processor and four parent rows remain unsupported. The serial native probe merged in #755 with all five checks passing. The next source inventory accounts for eight tables / 132 alternatives; staged Rust emission and reading still need combined validation. Real AudioV4 is the next shared-capability retirement candidate. | Classify runtime/manual rules against the recorded inventory; resolve named processor blockers, verify both carriers, then retire the duplicate readers. Details in the [scoreboard](./AUTOGENERATION-PROGRESS.md). |
+| 1 | Extend the source inventory and complete the next shared directory capability | Sony Tag202a migration/retirement merged in #746; source inventory and artifact join merged in #749/#751. Directory validation and nine word tables merged in #753/#754; canonical regeneration and all final gates pass. One child processor and four parent rows remain unsupported. Serial probe/inventory merged in #755/#756. Combined Rust emission/reading passes native V3/V4 replay; eight tables account for 106 emitted alternatives and 26 omissions. Final hosted gate is pending. Real AudioV4 retirement is published separately and still needs carrier/corpus/upgrade-flow acceptance. | Classify runtime/manual rules against the recorded inventory; resolve named processor blockers, verify both carriers, then retire the duplicate readers. Details in the [scoreboard](./AUTOGENERATION-PROGRESS.md). |
 | 2 | Continue generated EXIF directory migration | IFD1, InteropIFD and ExifIFD E-2 landed; Claude's handoff reserves E-3, fallback retirement, IFD0 and numeric-coercion work | Refresh Claude's ownership before taking work; preserve occurrence behavior and measure each activation against its own control |
 | 3 | Join producer and upgrade accounting | Source-to-artifact join merged; runtime and manually maintained rule classification remain open | Generated facts, manual rules, unsupported rules and actual execution remain distinct; ordinary source changes need no new tag rules |
 | 4 | Preserve Sony/Nikon producer recovery evidence | Sony plain landed; enciphered/encrypted recovery preserved and unlanded | Recovery is labeled maintenance; no numbering-only reconstruction or duplicate per-vendor interpreter is promoted as the target |

docs/reference/serial-layout-inventory.md

@@ -1,15 +1,14 @@
 # Staged native serial-layout inventory
 
 `tools/exiftool-tables/serial_directory.py` compiles a captured native
-`ProcessSerialData` table into a versioned JSON descriptor. It is a source
-inventory only: it has no Rust literal, reader, enablement entry or carrier
-route.
+`ProcessSerialData` table into a versioned JSON descriptor. It is a source inventory and source-derived literal emitter only: it has no
+reader, enablement entry or carrier route.
 
-It can also inventory the complete source-selected population rather than a
-hand-picked table:
+It can also render inactive Rust facts from the complete source-selected
+population rather than a hand-picked table:
 
 ```sh
-python3 tools/exiftool-tables/serial_directory.py "$DUMP" --output "$OUT"
+python3 tools/exiftool-tables/serial_directory.py "$DUMP" --output "$OUT" --rust-output "$SERIAL_OUT"
 ```
 
 `$DUMP` is a recorded `dump_tables.pl` JSON capture from the pinned native
@@ -52,11 +51,14 @@ grammar and continue to flow from the captured source facts.
 The committed 13.59 recorded-input report
 `serial-layout-inventory-processserialdata.json` contains eight selected tables:
 130 native entries and 132 native alternatives. Eight tables produce descriptor
-records, 115 alternatives clear the descriptor's row gate, 17 retain named
-refusals, four tables have no row-level Gate A blocker, and no selected table is
-empty. `Real::MediaProps` also retains its native `PRIORITY => 0` as the named
+records, 115 alternatives clear the descriptor's source row gate, 17 retain named
+source refusals, four tables have no row-level Gate A blocker, and no selected
+table is empty. The emitter adds a separate runtime-readiness refusal for the
+four AFInfo/AFInfo2 alternatives whose native missing regex member is an empty
+string; current shared `Cond` semantics do not represent that policy. `Real::MediaProps` also retains its native `PRIORITY => 0` as the named
 table-level blocker `serial_table_priority`; this checkpoint does not discard
 collision/reporting policy. `Real::AudioV3` is the non-AFInfo control: it uses the same resolved
-processor and has 12 source rows with no row-level refusal. This describes
-source facts; it does not prove a Rust reader, parent dispatch, or carrier
-route.
+processor and has 12 source rows with no row-level refusal. The generated literal has `ALL_SERIAL_TABLES` plus sorted native-row/table
+omission sidecars. Table Gate A blocks any partial table from reader execution.
+This describes source facts; it does not prove a Rust reader, parent dispatch,
+or carrier route.