From 135dae1765af978a4041149ea7eb1580c8b7876e Mon Sep 17 00:00:00 2001 From: Jesus Armando Anaya <1445792+JArmandoAnaya@users.noreply.github.com> Date: Wed, 26 Aug 2026 05:50:35 -0700 Subject: [PATCH] docs: the export-targets and pre-processing epics, read back against the code Every page the two epics touched or should have touched, corrected where it disagreed with main: the tutorial and README export by target, the changelog gains the browser half of both epics and a Deprecated entry for the yolo alias, the MCP counts and the REST route list catch up, and the architecture pages, AGENTS.md and the kernel-architecture skill learn that visionset.preprocessing exists and that the kernel may not import it. --- .../backend/kernel-architecture/SKILL.md | 7 ++-- AGENTS.md | 3 +- CHANGELOG.md | 19 +++++++++-- README.md | 15 +++++---- docs/content/README.md | 2 +- docs/content/api.md | 10 ++++-- docs/content/architecture/backend/README.md | 11 +++++-- docs/content/architecture/backend/cli.md | 8 +++-- docs/content/architecture/backend/formats.md | 32 +++++++++++++++++-- docs/content/architecture/backend/kernel.md | 29 ++++++++++------- docs/content/architecture/backend/mcp.md | 2 +- docs/content/architecture/backend/server.md | 2 +- docs/content/architecture/cross-cutting.md | 5 +-- docs/content/background-jobs.md | 3 +- docs/content/mcp.md | 12 ++++--- docs/content/preprocessing.md | 2 +- docs/content/releases.md | 3 +- docs/content/tutorial.md | 7 ++-- docs/content/ui/product-principles.md | 2 +- 19 files changed, 124 insertions(+), 50 deletions(-) diff --git a/.agents/skills/backend/kernel-architecture/SKILL.md b/.agents/skills/backend/kernel-architecture/SKILL.md index cbe0f6ac..5cdc71cc 100644 --- a/.agents/skills/backend/kernel-architecture/SKILL.md +++ b/.agents/skills/backend/kernel-architecture/SKILL.md @@ -40,9 +40,10 @@ Four import-linter contracts in `pyproject.toml`, plus a fresh-process test unde 1. **Kernel purity** — `visionset.kernel` never imports `visionset.server`, `visionset.cli`, `visionset.mcp`, `visionset.formats`, `visionset.wire`, `visionset.jobs`, - `visionset.inference`, nor `fastapi` / `typer` / `mcp` / `uvicorn`. The kernel decides what - exists; publication shapes, background handlers and the inference composition root all sit on - the other side of that line. + `visionset.inference`, `visionset.preprocessing`, nor `fastapi` / `typer` / `mcp` / + `uvicorn`. The kernel decides what exists; publication shapes, background handlers, the + inference composition root and the pre-processing driver registry all sit on the other side + of that line. 2. **Delivery clients are siblings** — `server`, `cli`, and `mcp` never import each other. Shared logic moves down into the kernel, never sideways. 3. **Job handlers are below the surfaces** — `visionset.jobs` imports no delivery package. diff --git a/AGENTS.md b/AGENTS.md index 19da2040..24e6d351 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,7 +73,8 @@ See `README.md` for the monorepo map and `CONTRIBUTING.md` for the full check li 1. **Kernel purity** — `visionset.kernel` never imports a delivery package (`visionset.server`, `visionset.cli`, `visionset.mcp`), nor `visionset.formats`, `visionset.wire`, - `visionset.jobs`, `visionset.inference`, nor `fastapi`/`typer`/`mcp`/`uvicorn`. Enforced by + `visionset.jobs`, `visionset.inference`, `visionset.preprocessing`, nor + `fastapi`/`typer`/`mcp`/`uvicorn`. Enforced by four import-linter contracts in `pyproject.toml` plus a fresh-process test in `tests/architecture/`; the full contract list and its reasoning are in the `kernel-architecture` skill. diff --git a/CHANGELOG.md b/CHANGELOG.md index 80f0b8e6..efe5c79e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,7 +29,11 @@ nothing was being distributed. This is the first version that is. and named `-aug`; a recipe that augments refuses a release published without a split (`AUGMENTATION_REQUIRES_SPLIT`), and a step refuses a geometry it cannot move (`PREPROCESSING_STEP_UNSUPPORTED_GEOMETRY`). Workspace format version 17 adds the - `preprocessing_recipes` table. + `preprocessing_recipes` table. In the browser, **Pre-processing** is the Dataset section's + fourth view, between Assets and Releases: the project's recipes as a list, an editor whose + resize step is preselected from the chosen target's hints, and a preview of three sample + assets through the export's own path; the Export dialog gains a **Pre-processing recipe** + control, `None` by default. - **Export targets: a release is exported for the model it will train.** Part of the export-targets epic (#784). Every installed format declares the targets it writes for, and the @@ -41,7 +45,10 @@ nothing was being distributed. This is the first version that is. its trainer has a task for - the drop is reported, consented through `allow_lossy`, and honoured in the output - and the compatibility report, the export result and the job payload all record `target` beside `format`. The target table in `docs/content/releases.md` is generated from the - catalog by `scripts/export_target_catalog.py`, behind a drift gate. + catalog by `scripts/export_target_catalog.py`, behind a drift gate. The browser's Export + dialog chooses a **Target model** rather than a format - the catalog grouped by family, each + option saying what the model accepts - and its lossy consent names the target and what the + release would lose to it. - **A schema version records which kind of work published it** (#368). New nullable `provenance` on `AnnotationSchema`: `curated` for a version somebody sat down and designed, @@ -111,9 +118,15 @@ nothing was being distributed. This is the first version that is. `pydantic-settings` object. The default is **one** worker, and that is a property of the store rather than a cautious guess: SQLite has a single writer and a run writes progress as it goes. +### Deprecated + +- **The `yolo` format name.** `ultralytics` is its name now; `yolo` is accepted as an alias for + this release only and is removed in the next. `visionset export --format yolo` prints a + deprecation line on stderr and continues, and the browser's export dialog never offers it. + ### Changed -- **`yolo` format renamed `ultralytics`; `yolo` accepted as an alias until the next release.** +- **`yolo` format renamed `ultralytics`.** Part of the export-targets epic (#784). The dialect now derives its task from the release - segment when it holds a polygon, classify when it holds only classification tags, detect otherwise - and its `data.yaml` carries `path: .`, one key per fold present, and `names` as a diff --git a/README.md b/README.md index 0ad62ba3..b1a8d501 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,8 @@ without a server, an account, or your pixels leaving the machine. | **Annotate** | boxes, polygons and classification tags in the browser, with undo/redo, keyboard-first tools, and a headless engine underneath that the UI is only one renderer of. | | **Version** | schema versions are immutable and every label records the one it was judged against. A release freezes the whole thing into a manifest; publish twice from unchanged data and the bytes are identical. | | **Split** | a stored recipe rather than a materialised assignment, keyed on **content hash** — so two copies of one image cannot straddle a train/test boundary. | -| **Export** | YOLO, COCO, Pascal VOC and [the lane family](docs/content/releases.md#the-lane-formats), each declaring what it can carry. VisionSet works out exactly what a format would drop *before* writing anything, and refuses to drop it silently. | +| **Export** | addressed to [the model you will train](docs/content/releases.md#export-targets) — the Ultralytics YOLO line, YOLOv7, COCO, Pascal VOC, classification and [the lane family](docs/content/releases.md#the-lane-formats) — each format declaring what it can carry. VisionSet works out exactly what a target would drop *before* writing anything, and refuses to drop it silently. | +| **Pre-process** | a named [recipe](docs/content/preprocessing.md) applied at export: resize every image, and write augmented variants of the training images beside their sources. The release stays untouched, and the export report records the recipe it ran with. | | **Auto-label** | a model you configure and fetch yourself, never one that arrives on its own. Click a point and SAM 2 proposes the shape under it; type words and Grounding DINO finds what they name. Every suggestion is a proposal until you accept it, and an accepted one records which model produced it. | | **Automate** | one SDK under everything, reachable as a Python API, a REST API, a CLI, and an MCP server an agent can drive. | @@ -58,7 +59,7 @@ labelling: "env": { "VISIONSET_WORKSPACE": "/path/to/workspace" } } } } ``` -The whole cycle as tools, plus the two deletions that are offered only when the server is started +The whole cycle as tools, plus the four deletions that are offered only when the server is started with `--allow-destructive` — because a `confirm` parameter is documented in the same listing an agent reads before choosing, and four of four measured runs sent it on the first call. See [docs/content/mcp.md](docs/content/mcp.md) for how a client is configured and why each tool exists, @@ -76,7 +77,7 @@ visionset batch approve "$BATCH" --jobs-of 100 && visionset batch start "$BATCH" # …annotate, then… visionset batch complete "$BATCH" && visionset batch promote "$BATCH" visionset release publish --tag v1.0 --project road-signs --split 0.7,0.15,0.15 -visionset export --project road-signs --release v1.0 --format yolo --out ./out --allow-lossy +visionset export --project road-signs --release v1.0 --target yolo11 --out ./out --allow-lossy ``` Every command takes `--json` for scripting, and the shapes are the REST API's. See @@ -109,9 +110,11 @@ src/visionset/ Single Python distribution (one wheel, one import namesp wire/ The JSON shapes the CLI and MCP publish (gated against the REST models) server/ FastAPI — exposes the SDK via REST; openapi.json is a committed contract cli/ Typer CLI (`visionset` console script) - mcp/ MCP server (stdio) — 38 agent tools over the same SDK - formats/ Exporter plugins: yolo, coco, voc, classification and the five lane - formats (entry-point group `visionset.formats`) + mcp/ MCP server (stdio) — 56 agent tools over the same SDK, four more on request + formats/ Exporter plugins: ultralytics, yolov5-yaml, coco, voc, classification and + the five lane formats (entry-point group `visionset.formats`) + preprocessing/ Pre-processing drivers: Pillow resize and augmentation behind the + `PreprocessingDriver` port (entry-point group `visionset.preprocessing`) jobs/ Handlers for work that outlives a request: ingest, export, weights inference/ Where a model connection becomes a running model (optional runtime) _static/ Compiled UI bundle lands here at build time (ships in the wheel) diff --git a/docs/content/README.md b/docs/content/README.md index bff959d2..0c8734cb 100644 --- a/docs/content/README.md +++ b/docs/content/README.md @@ -36,7 +36,7 @@ If you are new to VisionSet, start with [install.md](install.md), then continue | [annotations.md](annotations.md) | The labels themselves: the one door, the batch's pinned version, the five hard rejects, attribute values, progress derived from the annotations, and the editor's shortcut table | | [media.md](media.md) | Decoding raw media: the two processor ports, the accepted image formats, the orientation policy for stills and clips, pinned thumbnails and seek-free frame extraction, and what their determinism does and does not promise | | [datasets.md](datasets.md) | The curated trunk: promotion from a completed batch, what `skipped` keeps out, curation without a `confirm=`, and the append-only change log | -| [releases.md](releases.md) | The immutable artifact: what a manifest is and is not, why two publishes agree byte for byte, hash verification, and the seeded split recipe | +| [releases.md](releases.md) | The immutable artifact: what a manifest is and is not, why two publishes agree byte for byte, hash verification, and the seeded split recipe - then exporting: targets and the dialects they resolve to, what a format can carry and what a target narrows, the consent gate, and the report every export writes | | [preprocessing.md](preprocessing.md) | The optional stage at export: resize and augmentation recipes, the grammar, what moves geometry and what moves pixels, determinism and its scope, the train-only rule, and the report | | [events.md](events.md) | Domain events: subscribing by type, why emission follows the commit, at-most-once delivery, and what an isolated subscriber failure does | | [persistence.md](persistence.md) | The metadata store: repositories, unit of work, table layout, migrations and `format_version` | diff --git a/docs/content/api.md b/docs/content/api.md index f097c97d..7ab679c3 100644 --- a/docs/content/api.md +++ b/docs/content/api.md @@ -105,10 +105,16 @@ GET /releases/{release_id} GET /releases/{release_id}/manifest bytes GET /releases/{release_id}/verify GET /releases/{release_id}/assignment -GET /releases/{release_id}/export-compatibility ?target=|format=, exactly one -POST /releases/{release_id}/export ?target=|format=&allow_lossy=, launch +GET /releases/{release_id}/export-compatibility ?target=|format=, exactly one; &recipe= +POST /releases/{release_id}/export ?target=|format=&allow_lossy=&recipe=, launch GET /formats GET /export-targets the models a release can be exported for +GET /projects/{project_id}/preprocessing-recipes the project's pre-processing recipes +POST /projects/{project_id}/preprocessing-recipes +GET /projects/{project_id}/preprocessing-recipes/{name} +PUT /projects/{project_id}/preprocessing-recipes/{name} whole-value; a rename is `name` in the body +DELETE /projects/{project_id}/preprocessing-recipes/{name} no confirmation gate +POST /projects/{project_id}/preprocessing-preview one asset through a spec; never cached GET /inference/connections POST /inference/connections diff --git a/docs/content/architecture/backend/README.md b/docs/content/architecture/backend/README.md index 61c60af3..5af9949e 100644 --- a/docs/content/architecture/backend/README.md +++ b/docs/content/architecture/backend/README.md @@ -1,7 +1,7 @@ # The backend The Python distribution in [`src/visionset/`](../../../../src/visionset/) contains -eight packages. They are not peers. The core defines the domain, the next layer +nine packages. They are not peers. The core defines the domain, the next layer makes that domain usable by callers, and the outer layer exposes protocols. ## The stack @@ -19,6 +19,7 @@ flowchart TB Jobs["jobs\nbackground handlers"] Inference["inference\nmodel adapters"] Formats["formats\nexporter plugins"] + Preprocessing["preprocessing\npixel drivers"] end Kernel["kernel\ndomain · ports · services · adapters"] @@ -31,6 +32,10 @@ flowchart TB Cli --> Formats Jobs --> Formats Jobs --> Inference + Server --> Preprocessing + Cli --> Preprocessing + Mcp --> Preprocessing + Jobs --> Preprocessing Server --> Kernel Cli --> Kernel Mcp --> Kernel @@ -38,6 +43,7 @@ flowchart TB Jobs --> Kernel Inference --> Kernel Formats --> Kernel + Preprocessing --> Kernel ``` Read the diagram as a dependency graph: an arrow is "may import". Nothing points @@ -52,6 +58,7 @@ upward, and the three surfaces do not point at each other. | [`cli`](../../../../src/visionset/cli/) | Typer. The whole cycle from a shell. | [cli.md](cli.md) | | [`mcp`](../../../../src/visionset/mcp/) | The MCP tool surface, for agents. | [mcp.md](mcp.md) | | [`formats`](../../../../src/visionset/formats/) | Exporter plugins, discovered by entry point. | [formats.md](formats.md) | +| [`preprocessing`](../../../../src/visionset/preprocessing/) | Pre-processing drivers - the Pillow resize and augmentation engines behind the `PreprocessingDriver` port - discovered over the `visionset.preprocessing` entry-point group the way exporters are. | [formats.md](formats.md#the-sibling-group-preprocessing-drivers) | | [`wire`](../../../../src/visionset/wire/) | The JSON shapes the CLI and MCP publish. | [wire.md](wire.md) | | [`jobs`](../../../../src/visionset/jobs/) | Handlers for work that outlives a request. | [jobs.md](jobs.md) | | [`inference`](../../../../src/visionset/inference/) | Where a model connection becomes a running model, and which model families could run next. | [inference.md](inference.md) | @@ -64,7 +71,7 @@ the graph above. They are run by `uv run lint-imports`, which | Contract | What it forbids | | --- | --- | -| Kernel purity | `visionset.kernel` importing `server`, `cli`, `mcp`, `formats`, `wire`, `jobs`, `inference`, or `fastapi` / `typer` / `mcp` / `uvicorn` | +| Kernel purity | `visionset.kernel` importing `server`, `cli`, `mcp`, `formats`, `wire`, `jobs`, `inference`, `preprocessing`, or `fastapi` / `typer` / `mcp` / `uvicorn` | | Delivery clients are siblings | `server`, `cli` and `mcp` importing each other | | Job handlers are below the surfaces | `visionset.jobs` importing any delivery package or web framework | | Inference adapters are below the surfaces | `visionset.inference` importing any delivery package, or `visionset.jobs` | diff --git a/docs/content/architecture/backend/cli.md b/docs/content/architecture/backend/cli.md index fc1df1ef..ca468f3a 100644 --- a/docs/content/architecture/backend/cli.md +++ b/docs/content/architecture/backend/cli.md @@ -40,9 +40,11 @@ downward instead - which is the whole reason [`visionset.wire`](wire.md) is its package rather than a `cli/_json.py`. Resolving a format name to an exporter plugin happens *here*, not in the kernel: -`visionset.formats.registry.exporter(name)`, never a dict lookup, because a -`KeyError` is outside the `VisionSetError` tree and would answer a typo with a -traceback. +`visionset.formats.registry.pick(installed, name)` for a format and the port's +`resolve_target` for a target, never a dict lookup, because a `KeyError` is outside +the `VisionSetError` tree and would answer a typo with a traceback. The +pre-processing drivers an export applies are resolved the same way, through +`visionset.preprocessing.registry`. ## Related diff --git a/docs/content/architecture/backend/formats.md b/docs/content/architecture/backend/formats.md index 57753c64..75447f50 100644 --- a/docs/content/architecture/backend/formats.md +++ b/docs/content/architecture/backend/formats.md @@ -20,8 +20,10 @@ The kernel never resolves a name. `ReleaseService.export` takes an `Exporter` **instance**, because the `Kernel purity` contract forbids `visionset.kernel` importing `visionset.formats` - a plugin registry is discovery, and the kernel is the part that must not do any. So the surface holding the name -does the lookup, always through `registry.exporter(name)` rather than a dict, so a -typo answers a `VisionSetError` rather than a `KeyError` and a traceback. +does the lookup - `registry.pick(installed, name)` for a format, which also says +whether the name was a deprecated alias, and the port's own `resolve_target` for a +target - never through a dict, so a typo answers a `VisionSetError` rather than a +`KeyError` and a traceback. ## What ships @@ -55,6 +57,32 @@ states rather than two, because a boolean answers "is this written?" and "is thi written intact?" with one word - and a caller consenting to lose three annotations would receive two of them back as boxes. +Beside them sits `targets`: the models the format writes for, each a frozen +`ExportTarget` with its tasks, the geometries an export addressed to it carries, and +the pre-processing hints a recipe editor preselects. A format that is not a trainer's +declares one target named after itself, so every surface renders one control. The +registry validates the declarations at the scan - a target promising a geometry the +format never writes, or one name declared by two formats, is refused there - and the +kernel derives the catalog `GET /export-targets`, `visionset target list` and +`list_export_targets` all render. [`docs/content/releases.md`](../../releases.md#export-targets) +carries the catalog and the narrowing rule. + +## The sibling group: preprocessing drivers + +[`src/visionset/preprocessing/`](../../../../src/visionset/preprocessing/) is the +same mechanism one port over. `PreprocessingDriver` +([`kernel/ports/preprocessing.py`](../../../../src/visionset/kernel/ports/preprocessing.py)) +is the port a pixel engine implements - `step_kinds` and `apply(step, image, *, seed, +variant)` - and `preprocessing.registry` scans the `visionset.preprocessing` +entry-point group, keeps what satisfies the port and keys it by step kind, with +`drivers()`, `pick()`, `driver_for()` and `driver()` mirroring the format registry's +shape. The two built-in drivers, `pillow-resize` and `pillow-augment`, live in +[`pillow/`](../../../../src/visionset/preprocessing/pillow/). The kernel takes driver +instances through `ReleaseService.export(..., drivers=)` and never a name, and the +purity contract forbids it importing this package for the reason it forbids +`formats`. [`docs/content/preprocessing.md`](../../preprocessing.md) covers what a +recipe is and what the drivers promise. + A plugin also gets a `ContentReader` and never a `BlobStore`: a reader can read where the port could also `put`, and a plugin that could write into the content store could give a release bytes nobody published. diff --git a/docs/content/architecture/backend/kernel.md b/docs/content/architecture/backend/kernel.md index f6fa9123..c68f99f2 100644 --- a/docs/content/architecture/backend/kernel.md +++ b/docs/content/architecture/backend/kernel.md @@ -29,7 +29,10 @@ flowchart LR `ports` declares protocols and nothing else - a `MetadataStore`, a `BlobStore`, an `ImageProcessor`, a `VideoProcessor`, an `EventBus`, a `JobQueue`, an `AuthProvider`, a `ProgressReporter`, an `Exporter`, an `Importer`, a -`ModelProvider`. `adapters` holds the implementations this distribution ships. +`ModelProvider`, a `PreprocessingDriver`. `adapters` holds the implementations this +distribution ships for the storage and media ports; an `Exporter` or a +`PreprocessingDriver` is implemented outside the kernel, in `visionset.formats` and +`visionset.preprocessing`, and handed in as an instance. A service names a port, never an adapter, which is what lets a test drive a service against an object literal. @@ -39,23 +42,24 @@ service against an object literal. | --- | --- | --- | | [`domain/`](../../../../src/visionset/kernel/domain/) | pydantic models, the transition tables, the capability tables | Pure values. Imports nothing from the rest of the kernel. | | [`ports/`](../../../../src/visionset/kernel/ports/) | `Protocol` declarations | Signatures name domain types and standard-library types. Nothing else. | -| [`services/`](../../../../src/visionset/kernel/services/) | the twelve services | The only way to change anything. Take an open `WorkspaceService` and reach ports through it. | +| [`services/`](../../../../src/visionset/kernel/services/) | the fifteen services | The only way to change anything. Take an open `WorkspaceService` and reach ports through it. | | [`adapters/`](../../../../src/visionset/kernel/adapters/) | SQLite, the filesystem, Pillow, ffmpeg | The only place a third-party library is named. | ## The purity contract The kernel may not import `visionset.server`, `visionset.cli`, `visionset.mcp`, -`visionset.formats`, `visionset.wire`, `visionset.jobs` or `visionset.inference`, -nor `fastapi`, `typer`, `mcp` or `uvicorn`. +`visionset.formats`, `visionset.wire`, `visionset.jobs`, `visionset.inference` or +`visionset.preprocessing`, nor `fastapi`, `typer`, `mcp` or `uvicorn`. The obvious half is that a domain must not depend on a delivery mechanism. The -less obvious half is why `formats`, `wire`, `jobs` and `inference` are on the list -when none of them is a web framework: each is a place where a *decision about the -outside world* is made - which plugin exists, what gets published, what runs in a -worker, which model is loaded - and a kernel that could reach one could reach the -thing behind it. `ReleaseService.export` takes an `Exporter` **instance** for -exactly this reason: resolving a format name to a plugin is discovery at runtime, -and the kernel is the part that must not do any. +less obvious half is why `formats`, `wire`, `jobs`, `inference` and `preprocessing` +are on the list when none of them is a web framework: each is a place where a +*decision about the outside world* is made - which plugin exists, what gets +published, what runs in a worker, which model is loaded, which engine moves the +pixels - and a kernel that could reach one could reach the thing behind it. +`ReleaseService.export` takes an `Exporter` **instance**, and its `drivers=` takes +`PreprocessingDriver` instances, for exactly this reason: resolving a name to a +plugin is discovery at runtime, and the kernel is the part that must not do any. Enforced twice, and the second one is the one that catches a lazy import: @@ -76,7 +80,8 @@ This page is about arrangement. What the services actually do has its own pages: [schemas](../../schemas.md), [sources](../../sources.md), [ingest](../../ingest.md), [batches](../../batches.md), [jobs](../../jobs.md), [annotations](../../annotations.md), [datasets](../../datasets.md), -[releases](../../releases.md), [events](../../events.md), +[releases](../../releases.md), [preprocessing](../../preprocessing.md), +[events](../../events.md), [persistence](../../persistence.md) and [media](../../media.md). The [`kernel-architecture`](../../../../.agents/skills/backend/kernel-architecture/SKILL.md) diff --git a/docs/content/architecture/backend/mcp.md b/docs/content/architecture/backend/mcp.md index 47c3b5ba..faa8613b 100644 --- a/docs/content/architecture/backend/mcp.md +++ b/docs/content/architecture/backend/mcp.md @@ -12,7 +12,7 @@ flowchart TB Guard["guarded()\nwraps every body"] Doc["inspect.cleandoc\ndescription="] Ann["ToolAnnotations\nread / write hints"] - Tools["projects · schemas · sources · batches\njobs · assets · annotations\ndatasets · releases · formats"] + Tools["projects · schemas · sources · batches\njobs · assets · annotations\ndatasets · releases · formats · preprocessing · inference"] Dtools["DESTRUCTIVE_TOOLS\nregistered only on request"] Main --> Guard diff --git a/docs/content/architecture/backend/server.md b/docs/content/architecture/backend/server.md index c97f8d8b..e8c76e71 100644 --- a/docs/content/architecture/backend/server.md +++ b/docs/content/architecture/backend/server.md @@ -39,7 +39,7 @@ invented at a call site. | Module | Holds | | --- | --- | | [`main.py`](../../../../src/visionset/server/main.py) | `create_app()`, the static bundle mount, the SPA deep-link fallback | -| [`routes/`](../../../../src/visionset/server/routes/) | one module per resource - projects, schemas, sources, ingest, batches, jobs, annotations, assets, datasets, releases, formats, background jobs, inference | +| [`routes/`](../../../../src/visionset/server/routes/) | one module per resource - projects, schemas, sources, ingest, batches, jobs, annotations, assets, datasets, releases, formats and export targets, preprocessing recipes, background jobs, inference | | [`models.py`](../../../../src/visionset/server/models.py) | the pydantic request and response models `openapi.json` is generated from | | [`errors.py`](../../../../src/visionset/server/errors.py) | `ERROR_RULES` - every domain error's status and code | | [`dependencies.py`](../../../../src/visionset/server/dependencies.py) | which workspace a request serves, and the bearer-token gate | diff --git a/docs/content/architecture/cross-cutting.md b/docs/content/architecture/cross-cutting.md index eef9900a..234f5a54 100644 --- a/docs/content/architecture/cross-cutting.md +++ b/docs/content/architecture/cross-cutting.md @@ -11,8 +11,9 @@ change that fights one is wrong - the boundary does not move to make a build pas ### Kernel purity `visionset.kernel` never imports `visionset.server`, `visionset.cli`, -`visionset.mcp`, `visionset.formats`, `visionset.wire`, `visionset.jobs` or -`visionset.inference`, nor `fastapi`, `typer`, `mcp` or `uvicorn`. +`visionset.mcp`, `visionset.formats`, `visionset.wire`, `visionset.jobs`, +`visionset.inference` or `visionset.preprocessing`, nor `fastapi`, `typer`, `mcp` or +`uvicorn`. - **Contract** - `[[tool.importlinter.contracts]]` in [`pyproject.toml`](../../../pyproject.toml), run by `uv run lint-imports`. diff --git a/docs/content/background-jobs.md b/docs/content/background-jobs.md index cebbd41e..dec295fa 100644 --- a/docs/content/background-jobs.md +++ b/docs/content/background-jobs.md @@ -193,7 +193,8 @@ each had to argue it in its own module. ## Why `visionset.jobs` is not in the kernel -The export handler resolves an `Exporter` through `visionset.formats.registry`, +The export handler resolves an `Exporter` through `visionset.formats.registry` (and +the pre-processing drivers a recipe needs through `visionset.preprocessing.registry`), and import-linter forbids `visionset.kernel` from importing it - the same wall that makes `ReleaseService.export` take an instance rather than a format name. So handlers sit one package out, beside `visionset.formats` and `visionset.wire`. diff --git a/docs/content/mcp.md b/docs/content/mcp.md index 332ee2e8..45aca761 100644 --- a/docs/content/mcp.md +++ b/docs/content/mcp.md @@ -79,7 +79,7 @@ it, and what twelve real agent runs did with it - see ## The tools -Forty-nine tools are offered by default, in the order an agent meets them, plus the three +Fifty-six tools are offered by default, in the order an agent meets them, plus the four below that are offered only on request — see [above](#destructive-tools-are-not-offered-unless-you-ask). [mcp-tools.md](mcp-tools.md) is the complete listing, generated from the server itself; this @@ -362,16 +362,18 @@ out of the object to pick the variant, and omitting it fails. Always send ## What is not here, and why Fifty candidate tools were recorded across the four REST tasks; thirty of them shipped and -twenty did not. Twenty-seven have been added since, every one of them because a surface grew a +twenty did not. Thirty have been added since, every one of them because a surface grew a capability an agent had no way to reach. The larger groups say what that looks like: the four batch-composition tools above; the seven inference-connection tools, closing the Models page's SDK-first parity; the four schema-draft tools above, because composing a schema across several -calls needs somewhere to hold a class before it is finished; the three deletions, which are +calls needs somewhere to hold a class before it is finished; the four deletions, which are advertised only on request; the pre-labeling trio, `pre_label_job` beside the two fan-outs, closing the last capability declared with no consumer; `check_export`, the plan-before-apply -half of an export on the `preview_schema_change` precedent; and `list_export_targets`, because +half of an export on the `preview_schema_change` precedent; `list_export_targets`, because `export_release` takes a target name and an agent has to be able to read the catalog it comes -from. That is fifty-four offered by default and fifty-seven in all. The parity rule means +from; and the three recipe tools, because `export_release` takes a recipe by name and an agent +has to be able to write one and read the names back. That is fifty-six offered by default and +sixty in all. The parity rule means *evaluated*, not *implemented* — tool-selection accuracy degrades with count, so a tool ships only when an agent has a reason to reach for it that no neighbour covers. diff --git a/docs/content/preprocessing.md b/docs/content/preprocessing.md index 52cf620a..d8804985 100644 --- a/docs/content/preprocessing.md +++ b/docs/content/preprocessing.md @@ -53,7 +53,7 @@ fields were written in. | Step | Fields | What it does | | --- | --- | --- | | `resize` | `strategy` (`letterbox` or `stretch`), `width`, `height` (32 to 8192), `pad_value` (0 to 255, default 114) | Every exported image, base and variant, lands at `width × height`. `stretch` scales each axis on its own; `letterbox` keeps the aspect, scales to fit, and pads the rest with `pad_value` - the grey YOLO trainers letterbox with themselves. | -| `augment` | `op` (`hflip`, `brightness_contrast`, `rot90`), `amount` (0 to 0.5, default 0.2) | One augmentation applied when generating variants. `amount` bounds the brightness and contrast factors and means nothing to the other two. | +| `augment` | `op` (`hflip`, `brightness_contrast`, `rot90`), `amount` (above 0 and at most 0.5, default 0.2) | One augmentation applied when generating variants. `amount` bounds the brightness and contrast factors and means nothing to the other two. | Cross-field rules, refused with the rule named: at most one `resize` step, and it comes first; each `op` at most once; an `augment` step needs `variants_per_asset` of at least 1; and diff --git a/docs/content/releases.md b/docs/content/releases.md index 12af8a49..8bbe328b 100644 --- a/docs/content/releases.md +++ b/docs/content/releases.md @@ -862,7 +862,8 @@ carries the identical body under `detail.compatibility`. ## In the browser `@visionset/ui-core`'s dataset screen carries the stats, the release timeline, the -publish dialog and export. Three things it takes from this page rather than +publish dialog, the project's [pre-processing recipes](preprocessing.md) and export - +addressed to a target model, with a recipe chosen beside it. Three things it takes from this page rather than re-deciding: a release is immutable, so nothing offers an edit; verification is **on demand**, because it re-reads every blob; and a split's fractions are compared with the same tolerance the kernel uses, since `0.7 + 0.15 + 0.15` is not `1.0`. diff --git a/docs/content/tutorial.md b/docs/content/tutorial.md index ef537591..dce04b36 100644 --- a/docs/content/tutorial.md +++ b/docs/content/tutorial.md @@ -213,11 +213,14 @@ boxes and polygons (`yolo` is still accepted as an alias of `ultralytics` for on command rather than trusting this listing — it reads installed entry-point metadata, so a third-party plugin appears in it too. -Then export: +An export is addressed to the model you will train rather than to a format. `visionset target +list` prints the catalog - `yolo11` resolves to `ultralytics`, `yolov7` to `yolov5-yaml`, and +every other format to a target named after itself - and `--target` takes one of those names in +place of `--format`; exactly one of the two is given. Then export: ```bash visionset export --project road-signs --release v1.0 \ - --format ultralytics --out ./yolo --allow-lossy + --target yolo11 --out ./yolo --allow-lossy ``` `--allow-lossy` is required here and the refusal without it is not bureaucracy. A YOLO label row diff --git a/docs/content/ui/product-principles.md b/docs/content/ui/product-principles.md index 8b743d2b..61ba6ca1 100644 --- a/docs/content/ui/product-principles.md +++ b/docs/content/ui/product-principles.md @@ -143,7 +143,7 @@ row's metrics for nothing. to nothing else** — one declaration, and a consumer must not add a gap of its own. The primitive owns it because that is the direction nobody can forget. It knows two values: the segmented switch keeps Nova's 8px, and a `line` bar — navigation over a page's content, as on -the dataset's three views or the project's strip below `lg` — takes the layout unit, 16px. +the dataset's four views or the project's strip below `lg` — takes the layout unit, 16px. Asserted by measurement rather than by a class string. No panel repeats its own tab's name as a heading: the tab already labels the panel, so a