feat(kernel): VideoProcessor — ffmpeg probing, oriented dimensions, deterministic frame extraction (#17) - #89
Merged
Conversation
…eterministic frame extraction (#17) Declares `ports/video_processor.py` beside #16's `image_processor.py` — one file per port, each declared by the task that implements it — and ships `FfmpegVideoProcessor` behind it. `probe` reports as-displayed dimensions, source rate, duration and codec; `frames` streams PNG frames carrying `(index, timestamp)` off a running ffmpeg, one at a time. The two media errors are reused rather than extended: a container ffmpeg never opens is `UnsupportedMedia`, a clip that decodes for a while and then runs out is `CorruptMedia`. A missing binary is `MediaToolUnavailable`, deliberately outside that family — no file is at fault, so an ingest must not record it against five thousand innocent ones. `workspace.video_processor` is the fifth port, composed the EventBus way. No migration: FORMAT_VERSION stays 6, VERSION stays 0.0.1.dev0, no Asset field.
JArmandoAnaya
added a commit
that referenced
this pull request
Aug 21, 2026
…eterministic frame extraction (#17) (#89) Declares `ports/video_processor.py` beside #16's `image_processor.py` — one file per port, each declared by the task that implements it — and ships `FfmpegVideoProcessor` behind it. `probe` reports as-displayed dimensions, source rate, duration and codec; `frames` streams PNG frames carrying `(index, timestamp)` off a running ffmpeg, one at a time. The two media errors are reused rather than extended: a container ffmpeg never opens is `UnsupportedMedia`, a clip that decodes for a while and then runs out is `CorruptMedia`. A missing binary is `MediaToolUnavailable`, deliberately outside that family — no file is at fault, so an ingest must not record it against five thousand innocent ones. `workspace.video_processor` is the fifth port, composed the EventBus way. No migration: FORMAT_VERSION stays 6, VERSION stays 0.0.1.dev0, no Asset field.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #17.
Declares
ports/video_processor.pybeside #16'simage_processor.py— one file per port,each declared by the task that implements it — and ships
FfmpegVideoProcessorbehind it.workspace.video_processoris the fifth port on the composition point.Acceptance criteria
gives exactly 2 / 10 / 20 frames. That parametrised test is also the canary for the whole
extraction command — almost any change to it moves one of those numbers.
VideoFrame(index, timestamp, content), not inferred downstream.extract to identical SHA-256s. Repeatability, never a hardcoded hash.
Design decisions, and why
Path, notBinaryIO. The one place this port diverges fromImageProcessor, deliberately.A video decoder is an out-of-process program that seeks: handed a pipe it cannot say how long a
clip is without decoding all of it, and cannot revisit a byte. Nothing is lost — a source is a
path and a blob in the default store is a path too.
codecis astr; there is noVideoFormatenum. An image is an asset, a video is asource. Curating
ImageFormatbuys something because those exact bytes enter the dataset; avideo's never do — they leave as PNG frames — so a closed codec list would gate nothing while
going stale. Same split as
DatasetChange.operationvsDatasetOperation.Rotation is applied, not reported. The EXIF rule, on a different mechanism: a phone held
upright writes a landscape stream plus a display matrix, and ffmpeg applies it when decoding. So
VideoMetadatareports the swapped edges and the frames come out at exactly those dimensions.Reporting the stored numbers would put every frame at odds with the metadata stamped beside it.
The two media errors are reused, not extended. A container ffmpeg never opens is
UnsupportedMedia— including a clip whose index went missing with its tail, which is whyprobing before extracting matters; a clip that opens, yields frames and then runs out is
CorruptMedia, and it hands back what decoded before it raises.MediaToolUnavailableis theone new error and is not a
MediaError: no file is at fault, so an ingest catching thefamily per item must not record "ffmpeg missing" against five thousand innocent files. It is
#19's fatal cause, next to the per-file report.
frames()is not a generator — it validates, checks for ffmpeg and returns an inner one — soa missing binary or a negative
fpsis reported at the call rather than at the first iterationinside whatever loop consumed it. What comes back owns a live decoder until exhausted or closed;
an abandoned iterator terminates it, and a test asserts the process is reaped.
Three things ffmpeg taught us, pinned in the adapter
round=upon the fps filter. Under the defaultnear, extracting a 10 fps clip at 1 fpsyields the pictures from 0.4 s and 1.4 s while labelling them 0.0 s and 1.0 s.
upmakes theframe at the grid point the winner, which is what makes
timestamp = index / fpshonest.Frame counts are identical either way.
-xerroris load-bearing. Without it a truncated clip exits zero with the frames itmanaged, so a damaged file would ingest as a merely short one and nothing would say so. Its
opposite number
-err_detect explodeis deliberately not used: it rejects intact files.interacts with the filter. Extraction reads from the start every time.
One consequence #19 and #20 should know about
Frames are content-addressed, and the encoder is pinned, so extraction is deterministic — within
one installed ffmpeg, not across builds. That propagates: video-derived asset identity is
reproducible within an ffmpeg build, not across one. Re-ingesting a clip after an ffmpeg upgrade
yields new hashes and therefore new assets. Images do not have this property; the asset is
something we were given. Video does; the asset is something we computed.
Fixtures
tests/fixtures/media.pygainswrite_corrupt_videoandwrite_rotated_video, andwrite_videonow asks for
-movflags +faststart— that index at the front is exactly what lets a truncatedclip stay partially readable, which is the only way to reach the
CorruptMediabranch.write_rotated_videouses-display_rotation; the older-metadata:s:v rotate=spelling isdropped silently by recent ffmpeg, which would have produced a fixture that tests nothing and
fails nowhere. Both are covered in
tests/fixtures/test_media.py.No migration:
FORMAT_VERSIONstays 6,VERSIONstays0.0.1.dev0, noAssetfield, noopenapi.jsonchange. Pillow is a dependency and ffmpeg is a binary, sopyproject.tomlisuntouched and no import contract moves.
Checks