feat(kernel): VideoProcessor — ffmpeg probing, oriented dimensions, deterministic frame extraction (#17)

docs/README.md

@@ -12,7 +12,7 @@ contracts (kernel purity, headless annotator) are described there and enforced i
 | [batches.md](batches.md) | The unit of annotation work: the state machine, membership frozen at approval, the schema pin, and the exact partition into jobs |
 | [jobs.md](jobs.md) | Annotation jobs: the job and per-asset progress machines, what counts as settled, ordered `next_pending`, and derived progress |
 | [annotations.md](annotations.md) | The labels themselves: the one door, the batch's pinned version, the five hard rejects, attribute values, and progress derived from the annotations |
-| [media.md](media.md) | Decoding raw media: the two processor ports, the accepted formats, the orientation policy, pinned thumbnails and what their determinism does and does not promise |
+| [media.md](media.md) | Decoding raw media: the two processor ports, the accepted image formats, the orientation policy for stills and clips, pinned thumbnails and seek-free frame extraction, and what their determinism does and does not promise |
 | [datasets.md](datasets.md) | The curated trunk: promotion from a completed batch, what `skipped` keeps out, curation without a `confirm=`, and the append-only change log |
 | [releases.md](releases.md) | The immutable artifact: what a manifest is and is not, why two publishes agree byte for byte, hash verification, and the seeded split recipe |
 | [events.md](events.md) | Domain events: subscribing by type, why emission follows the commit, at-most-once delivery, and what an isolated subscriber failure does |

docs/media.md

@@ -5,12 +5,19 @@ decoded. That is what the media ports are for: hand them bytes, and they either
 the picture is or refuse it with a sentence naming the file.
 
 ```python
+from pathlib import Path
+
 from visionset.kernel.services import WorkspaceService
 
 with WorkspaceService.open("./road-signs") as workspace:
     with open("photos/IMG_0043.jpg", "rb") as handle:
         metadata = workspace.image_processor.probe(handle)  # ImageMetadata(width=..., ...)
         preview = workspace.image_processor.thumbnail(handle)  # opaque JPEG bytes
+
+    clip = Path("footage/dashcam.mp4")
+    info = workspace.video_processor.probe(clip)  # VideoMetadata(fps=29.97, ...)
+    for frame in workspace.video_processor.frames(clip, fps=1):
+        ...  # frame.index, frame.timestamp, frame.content — a PNG
 ```
 
 ## Two protocols, not one
@@ -21,8 +28,22 @@ has no thumbnail to serve and a Pillow adapter has no frames to iterate. A share
 force each implementation to declare the other's methods and raise from them — a runtime failure
 where a compile-time absence was available.
 
-Each is declared by the task that implements it. `ImageProcessor` and the Pillow adapter behind
-it exist today; `VideoProcessor` arrives with the ffmpeg one.
+Each is declared by the task that implements it, in its own file, and both exist today:
+`ImageProcessor` behind `PillowImageProcessor`, `VideoProcessor` behind `FfmpegVideoProcessor`.
+
+They also take different inputs, and that asymmetry is deliberate rather than an inconsistency
+waiting to be tidied. `ImageProcessor` takes a **stream**; `VideoProcessor` takes a **path**. A
+video decoder is an out-of-process program that seeks: handed a pipe it cannot say how long a
+clip is without decoding all of it, and cannot revisit a byte it has already read. Nothing is
+lost by requiring a file, because no caller has video bytes without one — a source is a path on
+disk, and a blob in the default blob store is a path too.
+
+| | `ImageProcessor` | `VideoProcessor` |
+| --- | --- | --- |
+| Input | an open binary stream | a `Path` |
+| Reads | `probe` → `ImageMetadata` | `probe` → `VideoMetadata` |
+| Produces | `thumbnail` → JPEG bytes | `frames` → an iterator of `VideoFrame` |
+| Needs | Pillow, a dependency | ffmpeg, a binary on `PATH` |
 
 ## The accepted list is two formats, and widening it is a decision
 
@@ -47,6 +68,19 @@ unsupported.
 **The bytes decide the format, not the filename.** A `.png` holding JPEG bytes reports `jpeg`.
 A suffix is a hint for choosing which files to look at; what a file *is* comes from decoding it.
 
+### There is no `VideoFormat`, on purpose
+
+The video side has no curated list at all: it accepts whatever ffmpeg's demuxer opens, and
+`VideoMetadata.codec` is a plain `str` recording what that turned out to be.
+
+The asymmetry is the argument. **An image is an asset; a video is a source.** Curating
+`ImageFormat` buys something real, because those exact bytes enter the dataset and the promise
+above is made about them. A video's bytes never do — they leave the decoder as PNG frames — so a
+closed list of codecs would gate nothing while going stale every time a camera vendor ships a new
+profile. `codec` therefore *records* what was read instead of *deciding* what may be read, which
+is the same split as `DatasetChange.operation` being a `str` while `DatasetOperation` is the enum
+a writer picks from.
+
 ## Orientation is applied, not reported
 
 `ImageMetadata.width` and `.height` are **as displayed**. A 32×24 JPEG carrying EXIF orientation
@@ -66,6 +100,72 @@ stored dimensions, and that is the ordinary path rather than a fallback.
 The policy is **format-independent**. PNG carries EXIF in an `eXIf` chunk and is oriented on
 exactly the same terms as JPEG, which is the case a hand-rolled tag-274 reader gets wrong.
 
+It is **modality-independent** too. A clip carries its turn in a display matrix rather than in an
+EXIF tag — a phone held upright writes a landscape stream plus a quarter turn — and ffmpeg
+applies that matrix when it decodes. So `VideoMetadata` reports the swapped edges, and the frames
+`frames()` yields come out at exactly those dimensions. Reporting the stored numbers would
+describe a picture nobody will ever see, and would put every extracted frame at odds with the
+metadata stamped beside it. ffprobe spells the same quarter turn as `90`, as `-90` and as `270`
+depending on the file and the build, so the rule is arithmetic on the angle (`rotation % 180`)
+and never a membership test against three literals.
+
+## Frames: the extraction is pinned, and it does not seek
+
+`frames()` streams one frame at a time off a running ffmpeg — nothing buffers a clip — and every
+argument that decides what comes out is fixed in one tuple in the adapter. Four of them are worth
+knowing about:
+
+- **`fps=N:round=up`** is the extraction grid, and it is what makes `frame.timestamp` honest. The
+  filter maps each input frame onto an output slot and the last one to land there wins; under the
+  default `near` rounding the winner is the frame nearest the slot's *midpoint*, so at 1 fps a
+  clip shot at 10 yields the pictures from 0.4 s and 1.4 s while labelling them 0.0 s and 1.0 s.
+  `up` makes the frame at the grid point itself the winner, which puts the reported timestamp
+  within one *source* frame of the pixels — the best any resampler can do. Frame counts are
+  identical either way.
+- **There is no seek.** Input seeking lands on a keyframe and is approximate; output seeking
+  interacts with the filter. Extraction reads the clip from the start every time, which is
+  exactly what makes it reproducible.
+- **`-xerror`** is the least obvious and the most load-bearing. Without it ffmpeg reports a
+  truncated clip on stderr, exits **zero**, and hands back the frames it managed to decode — so a
+  damaged file would ingest as a merely short one and nothing would ever say so. Its opposite
+  number, `-err_detect explode`, is deliberately not used: it rejects perfectly good files.
+- **`-pred none -compression_level 6` and `+bitexact`** pin the PNG encoder, whose bytes become
+  the content hash of the asset each frame turns into.
+
+Two consequences worth stating outright:
+
+- **A frame's `index` counts the extracted sequence, not the source.** A source frame number
+  means nothing for a variable-rate stream and cannot be reproduced without knowing the rate the
+  file was shot at. `timestamp` is the locator that survives — it says where in the clip to look,
+  whatever rate the next decomposition runs at.
+- **Asking for a higher rate than the clip has duplicates frames.** This is documented rather
+  than clamped: clamping would mean probing inside `frames()`, and an ingest content-addresses
+  what it stores, so the duplicates collapse into one asset anyway. The honest fix is not to ask.
+
+### The iterator owns a running program
+
+`frames()` is not itself a generator — it validates its arguments, checks for ffmpeg and returns
+an inner one — so a missing binary or a negative `fps` is reported at the call rather than at the
+first iteration inside whatever loop happened to consume it.
+
+What comes back holds a live decoder until it is exhausted or closed. A `for` loop that runs to
+the end is fine, and so is one that `break`s (closing the generator terminates ffmpeg); a caller
+that stashes the iterator somewhere should close it, `contextlib.closing`-style.
+
+### What video determinism costs, and who pays
+
+The same caveat as thumbnails, with a bigger consequence. Identical clip, identical `fps`,
+identical frames — across runs and across processes, **on one machine with one installed
+ffmpeg**. It does *not* hold across ffmpeg builds.
+
+Frames are content-addressed, so this propagates: **video-derived asset identity is reproducible
+within an ffmpeg build, not across one.** Re-ingesting the same clip after an ffmpeg upgrade
+yields different hashes and therefore new assets beside the old ones. Images do not have this
+property — a JPEG's bytes are its bytes — and video does, because the asset is something we
+computed rather than something we were given. The practical rule is the familiar one: assert
+repeatability, never a hardcoded hash, and do not build a cross-machine identity claim on a
+frame.
+
 ## Thumbnails: one encoding, pinned
 
 Every `thumbnail()` returns a **JPEG**, at most 256 pixels on its longest edge unless the caller
@@ -115,8 +215,8 @@ failure against the item it was reading, and carry on with the next file.
 
 | Error | Means | Remedy |
 | --- | --- | --- |
-| `UnsupportedMedia` | not an image at all, an image in a format outside `ImageFormat`, or one declaring more pixels than the decoder will take | filter the input, convert the file, or ask for the format to be accepted |
-| `CorruptMedia` | an accepted format whose bytes will not decode | re-fetch or re-export the file |
+| `UnsupportedMedia` | not an image at all, an image in a format outside `ImageFormat`, one declaring more pixels than the decoder will take, or a file ffmpeg never opened | filter the input, convert the file, or ask for the format to be accepted |
+| `CorruptMedia` | an accepted format whose bytes will not decode, or a clip that decoded for a while and then ran out | re-fetch or re-export the file |
 
 The split is by **remedy**, which is the only thing an error hierarchy should branch on. "Your
 input folder contains a README" and "one of your JPEGs is truncated" have opposite answers, and
@@ -129,7 +229,31 @@ fails exactly when the decode fails. And splitting `UnsupportedMedia` into "not
 and an exotic one both come back as *I do not know what this is*.
 
 The family is named for **media**, not images, because the video processor raises the same two
-for the same reasons.
+for the same reasons — the split is by remedy, and remedies have no modality. For video the two
+are separated by *when* the decoder gave up, and the order of the calls is what keeps that
+honest. A container ffmpeg cannot open at all is `UnsupportedMedia`, and that includes a clip
+whose index went missing with its tail; a clip that opens, yields frames and then runs out is
+`CorruptMedia`, and it hands back the frames that did decode before it raises.
+
+### `MediaToolUnavailable` is not in the family
+
+The default video adapter needs ffmpeg on the machine, and its absence is a `MediaToolUnavailable`
+— deliberately **not** a `MediaError`, so `except MediaError` will not catch it.
+
+Every error in that family answers *what is wrong with this file?*; this one answers *what is
+wrong with this machine?*. An ingest catches the media family per item and carries on, so if this
+were in it a missing decoder would be recorded five thousand times against five thousand innocent
+files, and the run would report a data problem it does not have. It is the single fatal cause an
+ingest job records beside its per-file report, and its message carries an install hint, because
+the remedy is a package manager.
+
+It is also why nothing checks for ffmpeg at import or when a workspace opens: a machine with no
+ffmpeg still opens workspaces and still ingests images perfectly well, so the check belongs to
+the call that actually needs to decode a video.
+
+Two more refusals sit outside the family on purpose, for the same reason `max_edge < 1` does: a
+missing file is a `FileNotFoundError` and a non-positive `fps` is a `ValueError`. Neither is a
+property of any media.
 
 ### `name` and `reason`
 
@@ -177,21 +301,42 @@ second call sees nothing. A test says so, rather than leaving it to be discovere
 
 ## Composition
 
-The image processor is the fourth port on `WorkspaceService`, reached as
-`workspace.image_processor` and built by a zero-argument `image_processor_factory` on both
-`init` and `open` — the shape the [event bus](events.md) uses, because neither is derived from
-the workspace path. One per open workspace, never a module-level singleton, and nothing to
-close: the decoder holds no state at all.
+The two media processors are the fourth and fifth ports on `WorkspaceService`, reached as
+`workspace.image_processor` and `workspace.video_processor` and built by zero-argument
+`image_processor_factory` / `video_processor_factory` on both `init` and `open` — the shape the
+[event bus](events.md) uses, because none of them is derived from the workspace path. One of each
+per open workspace, never a module-level singleton, and nothing to close: the decoders hold no
+state at all. A live frame iterator *does* own a running program, but that belongs to whoever
+asked for it, not to the workspace.
+
+A new port is appended **last** to `WorkspaceService.__init__`, never inserted: both classmethods
+bind those arguments positionally, so a parameter added in the middle silently re-binds every one
+after it.
+
+No service below the composition point ever names `PillowImageProcessor` or
+`FfmpegVideoProcessor`. That is the rule [workspaces.md](workspaces.md) describes, and it is what
+makes swapping a decoder a change to two functions and to nowhere else.
+
+## ffmpeg is a binary; Pillow is a dependency
+
+Pillow is in `[project].dependencies` and arrives with the wheel. ffmpeg cannot: it is a program,
+not a package, so `pip install visionset` does not put one on the machine and a user may
+legitimately never need it. Hence the lazy check, the install hint, and the fact that video tests
+**skip** locally when it is missing — with one guard, because a silently skipped video test looks
+exactly like a passing one. CI installs ffmpeg and sets `VISIONSET_REQUIRE_FFMPEG=1`, which turns
+that skip into a hard error, so a broken install step goes red rather than quietly shrinking the
+suite.
 
-No service below the composition point ever names `PillowImageProcessor`. That is the rule
-[workspaces.md](workspaces.md) describes, and it is what makes swapping a decoder a change to
-two functions and to nowhere else.
+Neither library needs an import-linter change. The contracts forbid *frameworks* inside the
+kernel — FastAPI, Typer, MCP, uvicorn — not third-party libraries, and ffmpeg is reached through
+`subprocess` and is not an import at all.
 
 ## What is deliberately not here yet
 
-- **No `Asset` field.** `ImageMetadata` is returned, not stored; putting `format` and origin on
-  the asset row belongs with the ingest pipeline.
-- **No blob write.** `thumbnail()` hands back bytes. Storing them content-addressed and
-  recording a `thumbnail_hash` is the thumbnail-cache task.
-- **No video.** `VideoProcessor`, fps probing and frame extraction arrive with the ffmpeg
-  adapter.
+- **No `Asset` field.** `ImageMetadata` and `VideoMetadata` are returned, not stored; putting
+  `format` and origin on the asset row belongs with the ingest pipeline.
+- **No blob write.** `thumbnail()` and `frames()` hand back bytes. Storing them
+  content-addressed, recording a `thumbnail_hash` and writing a frame's `index`/`timestamp` onto
+  an asset are the ingest and thumbnail-cache tasks.
+- **No `Source`.** Nothing yet records that a clip was registered, at what original rate, or with
+  what decomposition parameters. `VideoMetadata.fps` is what that record will be built from.

docs/workspaces.md

@@ -15,15 +15,20 @@ are the whole format — if WAL is ever adopted, `visionset.db-wal` and `-shm` b
 of it and this page has to say so, because a user who copies only `visionset.db` under WAL
 loses committed data.
 
-Two of the four ports have no line in that layout, and that is the point: the [event
-bus](events.md) is in-process and the [image processor](media.md) is a decoder, so neither
-leaves anything behind. They are composed here anyway, because a workspace is what services
+Three of the five ports have no line in that layout, and that is the point: the [event
+bus](events.md) is in-process and the two [media processors](media.md) are decoders, so none of
+them leaves anything behind. They are composed here anyway, because a workspace is what services
 are handed and every port has to arrive with it. One of each per open workspace, built by
-`event_bus_factory` and `image_processor_factory` — never a module-level singleton, which two
-workspaces open at once must not share.
+`event_bus_factory`, `image_processor_factory` and `video_processor_factory` — never a
+module-level singleton, which two workspaces open at once must not share.
+
+A new port is appended **last** to `WorkspaceService.__init__`, never inserted: `init` and `open`
+bind those arguments positionally, so a parameter added in the middle silently re-binds every one
+after it.
 
 `WorkspaceService` is the only place in the kernel that names `SqliteMetadataStore`,
-`FilesystemBlobStore`, `InProcessEventBus` or `PillowImageProcessor`. Everything above it — later
+`FilesystemBlobStore`, `InProcessEventBus`, `PillowImageProcessor` or `FfmpegVideoProcessor`.
+Everything above it — later
 surface, the CLI, MCP — gets an open service and reaches the ports through it, so swapping
 an adapter is a change to two functions and to nowhere else.
 
@@ -224,9 +229,10 @@ Four habits that keep the boundary honest:
   workspace-level rules with it.
 - One `unit_of_work()` per operation, and do the whole operation inside it.
 - Reach the ports through the handle — `workspace.metadata_store`, `workspace.blob_store`,
-  `workspace.event_bus`, `workspace.image_processor`. No service other than `workspace_service`
-  should name `SqliteMetadataStore`, `FilesystemBlobStore`, `InProcessEventBus` or
-  `PillowImageProcessor` — if a second one does, the composition point has stopped being single.
+  `workspace.event_bus`, `workspace.image_processor`, `workspace.video_processor`. No service
+  other than `workspace_service` should name `SqliteMetadataStore`, `FilesystemBlobStore`,
+  `InProcessEventBus`, `PillowImageProcessor` or `FfmpegVideoProcessor` — if a second one does,
+  the composition point has stopped being single.
 - Publish [events](events.md) *after* the `unit_of_work()` block, never inside it. An
   announcement is about work that committed, and a subscriber that raises must have nothing
   left to roll back.

src/visionset/kernel/__init__.py

@@ -34,6 +34,7 @@
     JobNotFound,
     LabelClassNotInSchema,
     MediaError,
+    MediaToolUnavailable,
     MissingRequiredAttribute,
     NoSplitRecipe,
     NotAWorkspace,
@@ -83,6 +84,7 @@
     "JobNotFound",
     "LabelClassNotInSchema",
     "MediaError",
+    "MediaToolUnavailable",
     "MissingRequiredAttribute",
     "NoSplitRecipe",
     "NotAWorkspace",