Skip to content

feat(kernel): preprocessing recipe domain and driver port - #799

Merged
JArmandoAnaya merged 1 commit into
mainfrom
feat/preprocessing-p1-domain
Aug 26, 2026
Merged

feat(kernel): preprocessing recipe domain and driver port#799
JArmandoAnaya merged 1 commit into
mainfrom
feat/preprocessing-p1-domain

Conversation

@JArmandoAnaya

Copy link
Copy Markdown
Contributor

Position 1 of the pre-processing epic (#785): the kernel half of resize and augmentation recipes, with no pixels, no wire and no routes. Everything here would still make sense with no HTTP and no terminal, which is why it is the first branch of the stack.

What changed

  • kernel/domain/preprocessing.pyResizeStep, AugmentOp, AugmentStep, the Step discriminated union, RecipeSpec and PreprocessingRecipe, all frozen with extra="forbid". The spec's cross-field rules: at most one resize step and it comes first, an augmentation step requires at least one variant, a variant count requires at least one augmentation step, and no augmentation appears twice. recipe_hash names a spec over the manifest's own canonical_bytes, so key order and spelled-out defaults cannot make two equal specs hash apart.
  • kernel/domain/release.pycanonical_bytes now takes any pydantic model instead of Manifest only. The encoder is the same; the manifest's key set and bytes are pinned by the existing tests and unchanged.
  • kernel/domain/preprocessing_transform.pyletterbox_fit (the one spelling of the letterbox arithmetic), the TransformedView / TransformedFile / TransformedAnnotation view models, and transform_manifest, which moves every annotation the way the pixels will move: stretch and letterbox scale, hflip mirrors and keeps a polyline's order, brightness and contrast touch no geometry, rot90 rotates box corners and polygon points and refuses a polyline. Classification tags are unchanged by every step.
  • kernel/ports/preprocessing.pyPreprocessingDriver, the port a pixel engine implements: step_kinds and apply(step, image, *, seed, variant). Drivers do pixels only; the kernel takes instances, never names.
  • kernel/errors.pyAugmentationRequiresSplit and PreprocessingStepUnsupportedGeometry (the latter carrying step, geometry and asset_id as keyword-only attributes, still constructible from one message). Both are mapped as 409 in server/errors.py and listed in the docs/content/api.md table; no route raises them yet, and the exact-correspondence gate is what requires the rows now.

The determinism contract

Variant k of an image is seeded by sha256(f"{recipe_hash}:{content_hash}:{k}"). Every draw reads a fixed position of that digest: hflip applies iff bit 0 is set, the brightness and contrast factors come from 32-bit words 1 and 2 (uniform in [1 - amount, 1 + amount]), and the rot90 quarter turns from word 3 (always 1, 2 or 3, counter-clockwise). Positions are fixed rather than sequential so adding a step to a recipe changes the recipe hash and nothing about how the other draws are read. Variant 0 is the base image and draws nothing; variants exist for train-fold assets only, and a spec that augments a release published without a split recipe is refused. Geometry arithmetic is exact everywhere; byte stability of the pixels is a promise the driver phase makes within one environment only.

Two decisions the dispatch left to the implementation

  • An asset the manifest recorded without pixel dimensions cannot have its coordinates scaled, mirrored or rotated. The transform raises ExportSourceUnreadable for it, reading "the release names bytes an export cannot use" one step wider rather than adding a third error code the phase was not given. Tag-only assets need no dimensions and pass through.
  • The rot90 refusal fires only where the step applies: a polyline on a validation or test asset is left alone, because no variant is ever made of it.

Stack

Position 1 of the pre-processing stack (#785); based on main, which now holds PR #788. Phase P2 (Pillow drivers) branches from this branch and calls letterbox_fit and the three draw functions; Phase P3 consumes TransformedView.

Checks

Targeted, all green on the final tree: uv run mypy src/visionset (180 source files), uv run lint-imports (4 contracts kept), uv run ruff check . / uv run ruff format ., uv run pytest tests/kernel tests/server tests/architecture tests/formats (3090 passed, 8 skipped).

Full local gate, bash scripts/check.sh, run once on the final tree:

Timing
     37s  python tests
      0s  ruff (lint)
      0s  ruff (format)
      0s  mypy
      0s  import contracts
      6s  frontend build
     29s  frontend tests
     11s  frontend lint
      1s  openapi drift
      0s  generated client drift
      1s  mcp tool reference drift
      0s  version sync
     47s  annotator + app e2e (chromium)
     26s  browser cycle, real server (chromium)
    158s  total

All checks passed.

check.sh: PASSED  ran=python,frontend,generated,browser  skipped=docs

Found, not fixed

  • tests/server/test_errors.py says MediaError is "the only kernel error with a constructor"; LossyExportNotConsented and ExportTargetNotFound already have one, and this branch adds a third. The comment's intent — one-argument constructibility — still holds and is what the test asserts.

@JArmandoAnaya
JArmandoAnaya force-pushed the feat/preprocessing-p1-domain branch 2 times, most recently from 8d383d5 to ba0f33a Compare August 26, 2026 11:33
A pre-processing recipe is a frozen value: a RecipeSpec holds at most one
resize step, first, followed by augmentation steps applied at most once
each, and says how many augmented variants each train-fold asset gets.
recipe_hash names a spec through the manifest's own canonical encoder,
whose parameter widens from Manifest to any pydantic model so the recipe
reuses it rather than copying it; the manifest's bytes are untouched.

Everything random about a variant is derived from
sha256(f"{recipe_hash}:{content_hash}:{k}") at fixed positions — bit 0
for hflip, words 1 and 2 for the brightness and contrast factors, word 3
for the quarter turns — so the geometry transform and a pixel driver read
the same draw. transform_manifest produces a TransformedView: per file
the transformed size, the moved annotations (variants suffixed -aug{k}),
the source content hash, the variant index and the fold. Variants exist
for the train fold only; a spec that augments against a release without
a split recipe raises AugmentationRequiresSplit, and rot90 over a
polyline raises PreprocessingStepUnsupportedGeometry. letterbox_fit is
the one spelling of the letterbox arithmetic both sides call.

PreprocessingDriver is the port a pixel engine implements: step kinds
and apply(step, image, seed=, variant=). Both new errors are mapped in
server/errors.py and listed in the api.md table — no route raises them
yet — so the exact-correspondence gate stays total.
@JArmandoAnaya
JArmandoAnaya force-pushed the feat/preprocessing-p1-domain branch from ba0f33a to 9d324ff Compare August 26, 2026 11:38
@JArmandoAnaya
JArmandoAnaya merged commit e311b84 into main Aug 26, 2026
15 checks passed
@JArmandoAnaya
JArmandoAnaya deleted the feat/preprocessing-p1-domain branch August 26, 2026 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant