Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .agents/skills/backend/kernel-architecture/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,10 @@ Four import-linter contracts in `pyproject.toml`, plus a fresh-process test unde

1. **Kernel purity** — `visionset.kernel` never imports `visionset.server`, `visionset.cli`,
`visionset.mcp`, `visionset.formats`, `visionset.wire`, `visionset.jobs`,
`visionset.inference`, nor `fastapi` / `typer` / `mcp` / `uvicorn`. The kernel decides what
exists; publication shapes, background handlers and the inference composition root all sit on
the other side of that line.
`visionset.inference`, `visionset.preprocessing`, nor `fastapi` / `typer` / `mcp` /
`uvicorn`. The kernel decides what exists; publication shapes, background handlers, the
inference composition root and the pre-processing driver registry all sit on the other side
of that line.
2. **Delivery clients are siblings** — `server`, `cli`, and `mcp` never import each other.
Shared logic moves down into the kernel, never sideways.
3. **Job handlers are below the surfaces** — `visionset.jobs` imports no delivery package.
Expand Down
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ See `README.md` for the monorepo map and `CONTRIBUTING.md` for the full check li

1. **Kernel purity** — `visionset.kernel` never imports a delivery package (`visionset.server`,
`visionset.cli`, `visionset.mcp`), nor `visionset.formats`, `visionset.wire`,
`visionset.jobs`, `visionset.inference`, nor `fastapi`/`typer`/`mcp`/`uvicorn`. Enforced by
`visionset.jobs`, `visionset.inference`, `visionset.preprocessing`, nor
`fastapi`/`typer`/`mcp`/`uvicorn`. Enforced by
four import-linter contracts in `pyproject.toml` plus a fresh-process test in
`tests/architecture/`; the full contract list and its reasoning are in the
`kernel-architecture` skill.
Expand Down
19 changes: 16 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,11 @@ nothing was being distributed. This is the first version that is.
and named `<hash>-aug<k>`; a recipe that augments refuses a release published without a split
(`AUGMENTATION_REQUIRES_SPLIT`), and a step refuses a geometry it cannot move
(`PREPROCESSING_STEP_UNSUPPORTED_GEOMETRY`). Workspace format version 17 adds the
`preprocessing_recipes` table.
`preprocessing_recipes` table. In the browser, **Pre-processing** is the Dataset section's
fourth view, between Assets and Releases: the project's recipes as a list, an editor whose
resize step is preselected from the chosen target's hints, and a preview of three sample
assets through the export's own path; the Export dialog gains a **Pre-processing recipe**
control, `None` by default.

- **Export targets: a release is exported for the model it will train.** Part of the
export-targets epic (#784). Every installed format declares the targets it writes for, and the
Expand All @@ -41,7 +45,10 @@ nothing was being distributed. This is the first version that is.
its trainer has a task for - the drop is reported, consented through `allow_lossy`, and honoured
in the output - and the compatibility report, the export result and the job payload all record
`target` beside `format`. The target table in `docs/content/releases.md` is generated from the
catalog by `scripts/export_target_catalog.py`, behind a drift gate.
catalog by `scripts/export_target_catalog.py`, behind a drift gate. The browser's Export
dialog chooses a **Target model** rather than a format - the catalog grouped by family, each
option saying what the model accepts - and its lossy consent names the target and what the
release would lose to it.

- **A schema version records which kind of work published it** (#368). New nullable
`provenance` on `AnnotationSchema`: `curated` for a version somebody sat down and designed,
Expand Down Expand Up @@ -111,9 +118,15 @@ nothing was being distributed. This is the first version that is.
`pydantic-settings` object. The default is **one** worker, and that is a property of the store
rather than a cautious guess: SQLite has a single writer and a run writes progress as it goes.

### Deprecated

- **The `yolo` format name.** `ultralytics` is its name now; `yolo` is accepted as an alias for
this release only and is removed in the next. `visionset export --format yolo` prints a
deprecation line on stderr and continues, and the browser's export dialog never offers it.

### Changed

- **`yolo` format renamed `ultralytics`; `yolo` accepted as an alias until the next release.**
- **`yolo` format renamed `ultralytics`.**
Part of the export-targets epic (#784). The dialect now derives its task from the release -
segment when it holds a polygon, classify when it holds only classification tags, detect
otherwise - and its `data.yaml` carries `path: .`, one key per fold present, and `names` as a
Expand Down
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ without a server, an account, or your pixels leaving the machine.
| **Annotate** | boxes, polygons and classification tags in the browser, with undo/redo, keyboard-first tools, and a headless engine underneath that the UI is only one renderer of. |
| **Version** | schema versions are immutable and every label records the one it was judged against. A release freezes the whole thing into a manifest; publish twice from unchanged data and the bytes are identical. |
| **Split** | a stored recipe rather than a materialised assignment, keyed on **content hash** — so two copies of one image cannot straddle a train/test boundary. |
| **Export** | YOLO, COCO, Pascal VOC and [the lane family](docs/content/releases.md#the-lane-formats), each declaring what it can carry. VisionSet works out exactly what a format would drop *before* writing anything, and refuses to drop it silently. |
| **Export** | addressed to [the model you will train](docs/content/releases.md#export-targets) — the Ultralytics YOLO line, YOLOv7, COCO, Pascal VOC, classification and [the lane family](docs/content/releases.md#the-lane-formats) — each format declaring what it can carry. VisionSet works out exactly what a target would drop *before* writing anything, and refuses to drop it silently. |
| **Pre-process** | a named [recipe](docs/content/preprocessing.md) applied at export: resize every image, and write augmented variants of the training images beside their sources. The release stays untouched, and the export report records the recipe it ran with. |
| **Auto-label** | a model you configure and fetch yourself, never one that arrives on its own. Click a point and SAM 2 proposes the shape under it; type words and Grounding DINO finds what they name. Every suggestion is a proposal until you accept it, and an accepted one records which model produced it. |
| **Automate** | one SDK under everything, reachable as a Python API, a REST API, a CLI, and an MCP server an agent can drive. |

Expand Down Expand Up @@ -58,7 +59,7 @@ labelling:
"env": { "VISIONSET_WORKSPACE": "/path/to/workspace" } } } }
```

The whole cycle as tools, plus the two deletions that are offered only when the server is started
The whole cycle as tools, plus the four deletions that are offered only when the server is started
with `--allow-destructive` — because a `confirm` parameter is documented in the same listing an
agent reads before choosing, and four of four measured runs sent it on the first call. See
[docs/content/mcp.md](docs/content/mcp.md) for how a client is configured and why each tool exists,
Expand All @@ -76,7 +77,7 @@ visionset batch approve "$BATCH" --jobs-of 100 && visionset batch start "$BATCH"
# …annotate, then…
visionset batch complete "$BATCH" && visionset batch promote "$BATCH"
visionset release publish --tag v1.0 --project road-signs --split 0.7,0.15,0.15
visionset export --project road-signs --release v1.0 --format yolo --out ./out --allow-lossy
visionset export --project road-signs --release v1.0 --target yolo11 --out ./out --allow-lossy
```

Every command takes `--json` for scripting, and the shapes are the REST API's. See
Expand Down Expand Up @@ -109,9 +110,11 @@ src/visionset/ Single Python distribution (one wheel, one import namesp
wire/ The JSON shapes the CLI and MCP publish (gated against the REST models)
server/ FastAPI — exposes the SDK via REST; openapi.json is a committed contract
cli/ Typer CLI (`visionset` console script)
mcp/ MCP server (stdio) — 38 agent tools over the same SDK
formats/ Exporter plugins: yolo, coco, voc, classification and the five lane
formats (entry-point group `visionset.formats`)
mcp/ MCP server (stdio) — 56 agent tools over the same SDK, four more on request
formats/ Exporter plugins: ultralytics, yolov5-yaml, coco, voc, classification and
the five lane formats (entry-point group `visionset.formats`)
preprocessing/ Pre-processing drivers: Pillow resize and augmentation behind the
`PreprocessingDriver` port (entry-point group `visionset.preprocessing`)
jobs/ Handlers for work that outlives a request: ingest, export, weights
inference/ Where a model connection becomes a running model (optional runtime)
_static/ Compiled UI bundle lands here at build time (ships in the wheel)
Expand Down
2 changes: 1 addition & 1 deletion docs/content/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ If you are new to VisionSet, start with [install.md](install.md), then continue
| [annotations.md](annotations.md) | The labels themselves: the one door, the batch's pinned version, the five hard rejects, attribute values, progress derived from the annotations, and the editor's shortcut table |
| [media.md](media.md) | Decoding raw media: the two processor ports, the accepted image formats, the orientation policy for stills and clips, pinned thumbnails and seek-free frame extraction, and what their determinism does and does not promise |
| [datasets.md](datasets.md) | The curated trunk: promotion from a completed batch, what `skipped` keeps out, curation without a `confirm=`, and the append-only change log |
| [releases.md](releases.md) | The immutable artifact: what a manifest is and is not, why two publishes agree byte for byte, hash verification, and the seeded split recipe |
| [releases.md](releases.md) | The immutable artifact: what a manifest is and is not, why two publishes agree byte for byte, hash verification, and the seeded split recipe - then exporting: targets and the dialects they resolve to, what a format can carry and what a target narrows, the consent gate, and the report every export writes |
| [preprocessing.md](preprocessing.md) | The optional stage at export: resize and augmentation recipes, the grammar, what moves geometry and what moves pixels, determinism and its scope, the train-only rule, and the report |
| [events.md](events.md) | Domain events: subscribing by type, why emission follows the commit, at-most-once delivery, and what an isolated subscriber failure does |
| [persistence.md](persistence.md) | The metadata store: repositories, unit of work, table layout, migrations and `format_version` |
Expand Down
10 changes: 8 additions & 2 deletions docs/content/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,10 +105,16 @@ GET /releases/{release_id}
GET /releases/{release_id}/manifest bytes
GET /releases/{release_id}/verify
GET /releases/{release_id}/assignment
GET /releases/{release_id}/export-compatibility ?target=|format=, exactly one
POST /releases/{release_id}/export ?target=|format=&allow_lossy=, launch
GET /releases/{release_id}/export-compatibility ?target=|format=, exactly one; &recipe=
POST /releases/{release_id}/export ?target=|format=&allow_lossy=&recipe=, launch
GET /formats
GET /export-targets the models a release can be exported for
GET /projects/{project_id}/preprocessing-recipes the project's pre-processing recipes
POST /projects/{project_id}/preprocessing-recipes
GET /projects/{project_id}/preprocessing-recipes/{name}
PUT /projects/{project_id}/preprocessing-recipes/{name} whole-value; a rename is `name` in the body
DELETE /projects/{project_id}/preprocessing-recipes/{name} no confirmation gate
POST /projects/{project_id}/preprocessing-preview one asset through a spec; never cached

GET /inference/connections
POST /inference/connections
Expand Down
11 changes: 9 additions & 2 deletions docs/content/architecture/backend/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# The backend

The Python distribution in [`src/visionset/`](../../../../src/visionset/) contains
eight packages. They are not peers. The core defines the domain, the next layer
nine packages. They are not peers. The core defines the domain, the next layer
makes that domain usable by callers, and the outer layer exposes protocols.

## The stack
Expand All @@ -19,6 +19,7 @@ flowchart TB
Jobs["jobs\nbackground handlers"]
Inference["inference\nmodel adapters"]
Formats["formats\nexporter plugins"]
Preprocessing["preprocessing\npixel drivers"]
end

Kernel["kernel\ndomain · ports · services · adapters"]
Expand All @@ -31,13 +32,18 @@ flowchart TB
Cli --> Formats
Jobs --> Formats
Jobs --> Inference
Server --> Preprocessing
Cli --> Preprocessing
Mcp --> Preprocessing
Jobs --> Preprocessing
Server --> Kernel
Cli --> Kernel
Mcp --> Kernel
Wire --> Kernel
Jobs --> Kernel
Inference --> Kernel
Formats --> Kernel
Preprocessing --> Kernel
```

Read the diagram as a dependency graph: an arrow is "may import". Nothing points
Expand All @@ -52,6 +58,7 @@ upward, and the three surfaces do not point at each other.
| [`cli`](../../../../src/visionset/cli/) | Typer. The whole cycle from a shell. | [cli.md](cli.md) |
| [`mcp`](../../../../src/visionset/mcp/) | The MCP tool surface, for agents. | [mcp.md](mcp.md) |
| [`formats`](../../../../src/visionset/formats/) | Exporter plugins, discovered by entry point. | [formats.md](formats.md) |
| [`preprocessing`](../../../../src/visionset/preprocessing/) | Pre-processing drivers - the Pillow resize and augmentation engines behind the `PreprocessingDriver` port - discovered over the `visionset.preprocessing` entry-point group the way exporters are. | [formats.md](formats.md#the-sibling-group-preprocessing-drivers) |
| [`wire`](../../../../src/visionset/wire/) | The JSON shapes the CLI and MCP publish. | [wire.md](wire.md) |
| [`jobs`](../../../../src/visionset/jobs/) | Handlers for work that outlives a request. | [jobs.md](jobs.md) |
| [`inference`](../../../../src/visionset/inference/) | Where a model connection becomes a running model, and which model families could run next. | [inference.md](inference.md) |
Expand All @@ -64,7 +71,7 @@ the graph above. They are run by `uv run lint-imports`, which

| Contract | What it forbids |
| --- | --- |
| Kernel purity | `visionset.kernel` importing `server`, `cli`, `mcp`, `formats`, `wire`, `jobs`, `inference`, or `fastapi` / `typer` / `mcp` / `uvicorn` |
| Kernel purity | `visionset.kernel` importing `server`, `cli`, `mcp`, `formats`, `wire`, `jobs`, `inference`, `preprocessing`, or `fastapi` / `typer` / `mcp` / `uvicorn` |
| Delivery clients are siblings | `server`, `cli` and `mcp` importing each other |
| Job handlers are below the surfaces | `visionset.jobs` importing any delivery package or web framework |
| Inference adapters are below the surfaces | `visionset.inference` importing any delivery package, or `visionset.jobs` |
Expand Down
8 changes: 5 additions & 3 deletions docs/content/architecture/backend/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,11 @@ downward instead - which is the whole reason [`visionset.wire`](wire.md) is its
package rather than a `cli/_json.py`.

Resolving a format name to an exporter plugin happens *here*, not in the kernel:
`visionset.formats.registry.exporter(name)`, never a dict lookup, because a
`KeyError` is outside the `VisionSetError` tree and would answer a typo with a
traceback.
`visionset.formats.registry.pick(installed, name)` for a format and the port's
`resolve_target` for a target, never a dict lookup, because a `KeyError` is outside
the `VisionSetError` tree and would answer a typo with a traceback. The
pre-processing drivers an export applies are resolved the same way, through
`visionset.preprocessing.registry`.

## Related

Expand Down
32 changes: 30 additions & 2 deletions docs/content/architecture/backend/formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ The kernel never resolves a name. `ReleaseService.export` takes an `Exporter`
**instance**, because the `Kernel purity` contract forbids
`visionset.kernel` importing `visionset.formats` - a plugin registry is discovery,
and the kernel is the part that must not do any. So the surface holding the name
does the lookup, always through `registry.exporter(name)` rather than a dict, so a
typo answers a `VisionSetError` rather than a `KeyError` and a traceback.
does the lookup - `registry.pick(installed, name)` for a format, which also says
whether the name was a deprecated alias, and the port's own `resolve_target` for a
target - never through a dict, so a typo answers a `VisionSetError` rather than a
`KeyError` and a traceback.

## What ships

Expand Down Expand Up @@ -55,6 +57,32 @@ states rather than two, because a boolean answers "is this written?" and "is thi
written intact?" with one word - and a caller consenting to lose three annotations
would receive two of them back as boxes.

Beside them sits `targets`: the models the format writes for, each a frozen
`ExportTarget` with its tasks, the geometries an export addressed to it carries, and
the pre-processing hints a recipe editor preselects. A format that is not a trainer's
declares one target named after itself, so every surface renders one control. The
registry validates the declarations at the scan - a target promising a geometry the
format never writes, or one name declared by two formats, is refused there - and the
kernel derives the catalog `GET /export-targets`, `visionset target list` and
`list_export_targets` all render. [`docs/content/releases.md`](../../releases.md#export-targets)
carries the catalog and the narrowing rule.

## The sibling group: preprocessing drivers

[`src/visionset/preprocessing/`](../../../../src/visionset/preprocessing/) is the
same mechanism one port over. `PreprocessingDriver`
([`kernel/ports/preprocessing.py`](../../../../src/visionset/kernel/ports/preprocessing.py))
is the port a pixel engine implements - `step_kinds` and `apply(step, image, *, seed,
variant)` - and `preprocessing.registry` scans the `visionset.preprocessing`
entry-point group, keeps what satisfies the port and keys it by step kind, with
`drivers()`, `pick()`, `driver_for()` and `driver()` mirroring the format registry's
shape. The two built-in drivers, `pillow-resize` and `pillow-augment`, live in
[`pillow/`](../../../../src/visionset/preprocessing/pillow/). The kernel takes driver
instances through `ReleaseService.export(..., drivers=)` and never a name, and the
purity contract forbids it importing this package for the reason it forbids
`formats`. [`docs/content/preprocessing.md`](../../preprocessing.md) covers what a
recipe is and what the drivers promise.

A plugin also gets a `ContentReader` and never a `BlobStore`: a reader can read
where the port could also `put`, and a plugin that could write into the content
store could give a release bytes nobody published.
Expand Down
Loading
Loading