kernel: IngestJob — state machine pending→running→(completed|failed), queryable progress, idempotency via content addressing

[JArmandoAnaya]

Context

The IngestJob is the (potentially long) process Source → Assets in a Batch.

Deliverables

• State machine pending → running → (completed | failed); failure captures a structured
  error report (per-file errors + fatal cause).
• Progress: processed/total counters queryable mid-run (SDK polling contract that the API
  and UI will reuse).
• Idempotency: same source + same params ⇒ same content hashes ⇒ no duplicate assets or
  blobs; re-running a failed job resumes/redoes safely.
• Runs synchronously in-process for the beta (background execution arrives with the server;
  keep the service API future-proof for that).

Acceptance criteria

• Re-run test: second run creates zero new blobs/assets
• Mid-run progress observable in a test (hook/callback)
• Failed run reports which files failed and why

[JArmandoAnaya]

Done in PR #92, squash-merged as 93945d6. M2 is 6/8.

Migration 9 (ingest_job_progress), so FORMAT_VERSION is now 9. VERSION stayed 0.0.1.dev0; no openapi.json drift, no dependency change, and no new error class — refusing to resume a completed run is an ordinary InvalidTransition, because it is a move the table does not have.

What landed

• INGEST_TRANSITIONS in domain/ingest.py, swept by a test that reads the table rather than restating it. It carries the kernel's first backward edge, failed → running: the argument against reopening a batch does not transfer, because a batch pins a schema version and its jobs are already cut against that pin, whereas nothing is pinned against an ingest run. running → running is deliberately absent — a job stuck there is a crashed process, not a reported failure, and the remedy (ingest again, which creates nothing) already exists.
• processed / total, written once per item, each in its own small transaction, so IngestService.get answers "where is it now" from any connection. total is NULL for a clip: VideoMetadata carries no frame count by design, so a total there would be arithmetic presented as fact.
• The per-file report as a JSON column, separate from the fatal error — one broken machine is still not five thousand broken files.
• IngestService.resume(job_id): a redo on the same row, into the batch the first attempt was headed for. batch_name earns its column here, because a run that died mid-decode reached no batch and would otherwise lose the name the caller asked for.
• A job is now created pending and moved to running by whoever picks it up — the vocabulary a queue needs, with every refusal still ahead of the insert.

Two findings worth carrying forward

Repository.update is session.merge(...) — a whole-row replace. #20's ingest() captured the job model in transaction 1 and reused it after the decode; the moment counters are written in between, that stale copy silently undoes all of them. Nothing carries an IngestJob across the decode any more — only job_id travels, and every write re-reads the row inside its own transaction. That is what require_job(uow, id) was made public for.

A migration whose only exercise is through an earlier rebuild is not exercised at all. The fresh-versus-migrated test walks back to generation 1, from where migration 8 re-creates ingest_job whole from _tables — including migration 9's columns — so migration 9 found them present and did nothing. test_migration_nine_alters_a_table_migration_eight_rebuilt stamps a database at generation 8 and runs the real ALTER path instead. Its baseline is produced by dropping the four columns rather than retyping the CREATE TABLE, because these tests compare sqlite_master text and a hand-written baseline differs in whitespace.

Also

require_move was promoted from job_service's module-private helper to domain/transitions.py, and BatchService's second spelling of it deleted — the standing "a gate two services need is promoted, not copied" rule. Message text is byte-identical, so every existing sweep still matches.

854 tests, up from 828.

Next: #21 — thumbnails at ingest and the backfill helper (asset.thumbnail_hash, migration 10, declared after #20's four columns on AssetRow). It serves M5's gallery and lands on a now-settled pipeline. Then #23 closes the milestone and tags v0.0.1-alpha.2.