Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion docs/content/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -340,6 +340,13 @@ GET /background-jobs/{job_id} → 200 { "state": "running", "proce
GET /background-jobs/{job_id}/artifact → 200 application/zip
```

`format=yolo`, the former name of `ultralytics`, is still accepted on this route and on
`export-compatibility` until the release after next: it resolves to the same plugin, the job's
`result.format` and the compatibility report's `format` read `ultralytics`, and no deprecation
text appears anywhere in the response. Only the CLI warns, on stderr. The MCP `export_release`
tool behaves as this route does. Address the export by `target` where you can; the alias is a
grace period, not a second name.

The two surfaces are separate because they describe different things. An ingest job knows what it
is *about* - a source, a batch - and publishes those as fields a client can navigate. A background
job is about whatever its payload says, so it publishes `type` and `result` instead. What they
Expand Down Expand Up @@ -552,7 +559,11 @@ Both are reachable on the same route, and they differ in `detail`:
payload parsed, and a kernel rule rejected it. `detail` is usually `null`.

Most malformed input arrives as the first: a `LabelClass` that cannot be constructed never
reaches a service to be refused by one.
reaches a service to be refused by one. The export address is one more of the first kind:
`POST /releases/{id}/export` and `GET /releases/{id}/export-compatibility` take exactly one of
`target` and `format`, and both or neither is a 422 `VALIDATION_ERROR` whose single error has
`loc: ["query"]`, `msg: "give exactly one of target and format"` and an `input` echoing the two
values as sent.

A refusal from a **bulk write** carries `detail.index` - the position in the array you sent of
the item that caused it:
Expand Down
10 changes: 8 additions & 2 deletions docs/content/architecture/backend/formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,14 @@ Beside them sits `targets`: the models the format writes for, each a frozen
`ExportTarget` with its tasks, the geometries an export addressed to it carries, and
the pre-processing hints a recipe editor preselects. A format that is not a trainer's
declares one target named after itself, so every surface renders one control. The
registry validates the declarations at the scan - a target promising a geometry the
format never writes, or one name declared by two formats, is refused there - and the
registry validates the declarations at the scan, and three rules are enforced there
rather than at export time: an exporter declares at least one target, because a
format with none is installed yet unreachable through the one control a surface
renders; each target's geometries stay within the union of the exporter's
`supported_geometries` and `degraded_geometries`, so the catalog never promises a
file the format does not write; and a target name is declared by exactly one
installed format, so resolving it is never a guess. The first two refuse with
`InvalidExportTarget` and the third with `ExportTargetConflict`, and the
kernel derives the catalog `GET /export-targets`, `visionset target list` and
`list_export_targets` all render. [`docs/content/releases.md`](../../releases.md#export-targets)
carries the catalog and the narrowing rule.
Expand Down
2 changes: 1 addition & 1 deletion docs/content/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -336,7 +336,7 @@ quietly leaving the impression that a terminal can label images.
| Annotate | `add_annotations` with every edge multiplied by `scale`, then `set_asset_progress` for the rest, then `complete_job` |
| Trunk | `complete_batch` → `promote_batch` → `dataset_stats` |
| Release | `publish_release`, `list_releases`, `verify_release` |
| Export | `list_formats` → `export_release(dest=…)` - a directory, not an archive |
| Export | `list_formats` → `export_release(format="dummy", dest=…)` - a directory, not an archive; a `format` addresses no trainer, so this is the plain-format call and the Recipe row below is the same release addressed by `target` |
| Recipe | `create_preprocessing_recipe`, `list_preprocessing_recipes`, then `export_release(target="yolo11", recipe="yolo-640", allow_lossy=True, dest=…)` - the result's `preprocessing` names the recipe under its hash and maps the train fold's `-aug1` variant to its source, and both files are on disk |
| Refusal | `publish_release` on the same tag → a **result** carrying an error envelope, `retry_with` null |

Expand Down
22 changes: 14 additions & 8 deletions docs/content/releases.md
Original file line number Diff line number Diff line change
Expand Up @@ -356,7 +356,7 @@ The catalog this build ships, generated from the declarations by
| `yolo26` | YOLO26 | `ultralytics-yolo` | `ultralytics` | classify, depth, detect, obb, pose, segment, semantic | bbox, classification_tag, polygon | 640×640 | letterbox |
| `yolov10` | YOLOv10 | `ultralytics-yolo` | `ultralytics` | detect | bbox | 640×640 | letterbox |
| `yolov3` | YOLOv3 | `ultralytics-yolo` | `ultralytics` | detect | bbox | 640×640 | letterbox |
| `yolov5` | YOLOv5 | `ultralytics-yolo` | `ultralytics` | detect | bbox | 640×640 | letterbox |
| `yolov5` | YOLOv5 | `ultralytics-yolo` | `ultralytics` | classify, detect, segment | bbox, classification_tag, polygon | 640×640 | letterbox |
| `yolov6` | YOLOv6 | `ultralytics-yolo` | `ultralytics` | detect | bbox | 640×640 | letterbox |
| `yolov7` | YOLOv7 | `community-yolo` | `yolov5-yaml` | detect | bbox | 640×640 | letterbox |
| `yolov8` | YOLOv8 | `ultralytics-yolo` | `ultralytics` | classify, detect, obb, pose, segment | bbox, classification_tag, polygon | 640×640 | letterbox |
Expand Down Expand Up @@ -392,8 +392,12 @@ Two formats write a YOLO dataset, and they differ only in the grammar of `data.y
*dialect*: the wire identifier `format_name` names the descriptor grammar, and the model a person
will train - the *target* - resolves to exactly one dialect. `ultralytics` is what every trainer
from YOLOv3 to YOLO26 in the Ultralytics line reads; `yolov5-yaml` is the older grammar YOLOv7
reads. `yolo`, the former name of `ultralytics`, is accepted as an alias for one release and then
removed; `visionset export --format yolo` says so on stderr and continues.
reads. `yolo`, the former name of `ultralytics`, is accepted as an alias until the release after
next, and then removed. Every surface resolves it to the same plugin and reports `format_name` as
`ultralytics`; only the CLI warns, because `visionset export --format yolo` has a stderr to say
so on. `POST /releases/{id}/export?format=yolo` and `export_release(format="yolo")` accept it
silently, the response carrying no deprecation text at all - a warning has no field to land in
on a 202 or in a tool result.

The layout both share:

Expand Down Expand Up @@ -644,11 +648,13 @@ will not write two lanes claiming the same one of its four mask slots. Both name
both are the same `ExportSourceUnreadable` the YOLO exporter raises for a class the schema does
not declare.

**YOLO, COCO and VOC carry no polyline at all**, and that is checked rather than assumed
(`test_the_three_general_formats_declare_polyline_truthfully`). YOLO and VOC reduce a *polygon*
to its bounding box, which is defensible because a polygon encloses an area a box approximates;
an open path encloses nothing, so a box drawn round it would be an invention. COCO's
`segmentation` is a closed ring and it has no open-path primitive. All three therefore report a
**The YOLO dialects, COCO and VOC carry no polyline at all**, and that is checked rather than
assumed (`test_the_general_formats_declare_polyline_truthfully`). `yolov5-yaml` and `voc` reduce a
*polygon* to its bounding box, which is defensible because a polygon encloses an area a box
approximates; an open path encloses nothing, so a box drawn round it would be an invention.
`ultralytics` writes a polygon as its vertices - its presence is what selects the `segment` layout,
and its `degraded_geometries` is empty - but has no row for an open path either. COCO's
`segmentation` is a closed ring and it has no open-path primitive. All four therefore report a
polyline class as **dropped**, and their label files contain no trace of one.

### The destination is the caller's
Expand Down
7 changes: 6 additions & 1 deletion src/visionset/formats/ultralytics/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,12 @@ def _target(
_EVERYTHING,
),
_target("yolov6", "YOLOv6", frozenset({Task.DETECT}), frozenset({GeometryType.BBOX})),
_target("yolov5", "YOLOv5", frozenset({Task.DETECT}), frozenset({GeometryType.BBOX})),
_target(
"yolov5",
"YOLOv5",
frozenset({Task.DETECT, Task.SEGMENT, Task.CLASSIFY}),
_EVERYTHING,
),
_target("yolov3", "YOLOv3", frozenset({Task.DETECT}), frozenset({GeometryType.BBOX})),
}
)
Expand Down
19 changes: 11 additions & 8 deletions src/visionset/kernel/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -811,14 +811,17 @@ class ExportTargetConflict(VisionSetError):


class InvalidExportTarget(VisionSetError):
"""An exporter declares a target it cannot deliver.

A target's ``supported_geometries`` must stay within the declaring
exporter's own, because the target is a promise about that exporter's
output — one claiming a geometry the exporter never writes would make the
catalog describe files that do not appear. Raised by ``validate_targets``,
which is a check on the *declaration*: nothing about the caller's request
is wrong, the installed plugin is.
"""An exporter's target declaration is defective.

Either it declares no target at all — the target control is the one
gesture every surface renders, so such a format is installed yet
unreachable — or a target's ``supported_geometries`` reach outside the
declaring exporter's own. The second matters because the target is a
promise about that exporter's output, and one claiming a geometry the
exporter never writes would make the catalog describe files that do not
appear. Raised by ``validate_targets``, which is a check on the
*declaration*: nothing about the caller's request is wrong, the installed
plugin is.
"""


Expand Down
15 changes: 11 additions & 4 deletions src/visionset/kernel/ports/exporter.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,18 +170,25 @@ def export(
def validate_targets(exporter: Exporter) -> None:
"""Check an exporter's target declarations against the exporter itself.

Every target's ``supported_geometries`` must stay within what the exporter
writes at all — its ``supported_geometries`` and its
There must be at least one, because the target control is the one gesture
every surface renders and an exporter declaring none would be installed
yet unreachable. Every target's ``supported_geometries`` must stay within
what the exporter writes at all — its ``supported_geometries`` and its
``degraded_geometries`` together — so a defective declaration is refused
where it can be named rather than surfacing as a catalog entry whose
exports are missing what it promised. Degraded counts because a target
carries what reaches the output, reduced or not: a format writing every
lane resampled still has a target that carries lanes.

Raises:
InvalidExportTarget: a target claims a geometry the exporter does not
write.
InvalidExportTarget: the exporter declares no target, or a target
claims a geometry the exporter does not write.
"""
if not exporter.targets:
raise InvalidExportTarget(
f"format {exporter.format_name!r} declares no export target, so nothing "
f"can address it; declare at least one"
)
written = exporter.supported_geometries | exporter.degraded_geometries
for target in exporter.targets:
undeliverable = target.supported_geometries - written
Expand Down
90 changes: 87 additions & 3 deletions tests/formats/test_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,24 @@
from __future__ import annotations

from collections.abc import Iterable
from importlib.metadata import EntryPoint
from pathlib import Path

import pytest

from visionset.formats._targets import self_target
from visionset.formats.registry import exporter, exporters, pick
from visionset.kernel.domain import Annotation, GeometryType, Manifest, Release
from visionset.kernel.errors import ExportFormatNotFound
from visionset.kernel.ports import ContentReader
from visionset.kernel.domain import (
Annotation,
ExportTarget,
GeometryType,
Manifest,
Release,
TargetFamily,
Task,
)
from visionset.kernel.errors import ExportFormatNotFound, InvalidExportTarget
from visionset.kernel.ports import ContentReader, Exporter


class _AnImporter:
Expand Down Expand Up @@ -68,6 +77,57 @@ def test_a_discovered_exporter_declares_its_targets() -> None:
assert target.tasks == frozenset()


#: Every YOLO trainer this build addresses, with the tasks each accepts. The
#: catalog table in `docs/content/releases.md` is generated from the same
#: declarations, so a dropped or narrowed target would only ever show up
#: there as a diff nobody reads; this is the assertion that fails instead.
YOLO_TARGETS = {
"yolo26": {
Task.DETECT,
Task.SEGMENT,
Task.SEMANTIC,
Task.DEPTH,
Task.CLASSIFY,
Task.POSE,
Task.OBB,
},
"yolo12": {Task.DETECT, Task.SEGMENT, Task.CLASSIFY, Task.POSE, Task.OBB},
"yolo11": {Task.DETECT, Task.SEGMENT, Task.CLASSIFY, Task.POSE, Task.OBB},
"yolov10": {Task.DETECT},
"yolov9": {Task.DETECT, Task.SEGMENT},
"yolov8": {Task.DETECT, Task.SEGMENT, Task.CLASSIFY, Task.POSE, Task.OBB},
"yolov7": {Task.DETECT},
"yolov6": {Task.DETECT},
"yolov5": {Task.DETECT, Task.SEGMENT, Task.CLASSIFY},
"yolov3": {Task.DETECT},
}


def test_the_yolo_targets_are_exactly_these_ten_with_these_tasks() -> None:
declared = {
target.name: set(target.tasks)
for plugin in exporters().values()
for target in plugin.targets
if target.family is not TargetFamily.OTHER
}

assert declared == YOLO_TARGETS


def test_a_yolo_target_carries_a_geometry_for_each_task_the_dialect_lays_out() -> None:
"""``segment`` without polygons, or ``classify`` without tags, is a task no export can reach."""
behind = {
Task.DETECT: GeometryType.BBOX,
Task.SEGMENT: GeometryType.POLYGON,
Task.CLASSIFY: GeometryType.CLASSIFICATION_TAG,
}
for plugin in exporters().values():
for target in plugin.targets:
for task, geometry in behind.items():
if task in target.tasks:
assert geometry in target.supported_geometries, (target.name, task)


def test_every_installed_exporter_stays_discovered() -> None:
"""Discovery filters on the port, so one missing member silently drops a
plugin from every surface — this is what would say which one."""
Expand Down Expand Up @@ -179,6 +239,30 @@ def export(
assert not isinstance(_Outdated(), Exporter)


class _Targetless(_AnExporter):
"""Carries every member of the port, and declares nothing under ``targets``."""

format_name = "targetless"
targets: frozenset[ExportTarget] = frozenset()


def test_a_plugin_declaring_no_target_is_refused_at_the_scan(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""The port's ``isinstance`` filter cannot see an empty set; the scan's validation must."""
from visionset.formats import registry

assert isinstance(_Targetless(), Exporter)
shipped = tuple(registry.entry_points(group="visionset.formats"))
defective = EntryPoint(
name="targetless", value=f"{__name__}:_Targetless", group="visionset.formats"
)
monkeypatch.setattr(registry, "entry_points", lambda *, group: (*shipped, defective))

with pytest.raises(InvalidExportTarget, match="'targetless'"):
registry.exporters()


def test_a_plugin_missing_the_targets_member_is_not_an_exporter() -> None:
"""An exporter with no target would be a format no target control can reach.

Expand Down
84 changes: 81 additions & 3 deletions tests/formats/test_ultralytics.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,13 @@
import pytest
from tests.fixtures.media import write_image

from visionset.formats.ultralytics import DATA_FILENAME, UltralyticsExporter
from visionset.formats.ultralytics import DATA_FILENAME, TARGETS, UltralyticsExporter
from visionset.kernel import ExportSourceUnreadable
from visionset.kernel.domain import (
Annotation,
BboxGeometry,
ClassificationGeometry,
ExportTarget,
GeometryType,
LabelClass,
PolygonGeometry,
Expand Down Expand Up @@ -120,8 +121,10 @@ def publish(self, tag: str = "v1", *, split: SplitRecipe | None = None) -> UUID:
dataset_id = self.projects.get_dataset(self.project.id).id
return self.releases.publish(dataset_id, tag, split=split).id

def export(self, release_id: UUID, dest: Path) -> Path:
self.releases.export(release_id, UltralyticsExporter(), dest, allow_lossy=True)
def export(self, release_id: UUID, dest: Path, *, target: ExportTarget | None = None) -> Path:
self.releases.export(
release_id, UltralyticsExporter(), dest, allow_lossy=True, target=target
)
return dest

def close(self) -> None:
Expand Down Expand Up @@ -359,6 +362,81 @@ def test_a_release_holding_only_tags_is_written_as_a_class_tree(tmp_path: Path)
assert len(list((out / "train" / "time-of-day").iterdir())) == 1


def _target(name: str) -> ExportTarget:
(found,) = (one for one in TARGETS if one.name == name)
return found


def test_a_tags_only_release_addressed_to_a_detect_target_is_not_a_class_tree(
tmp_path: Path,
) -> None:
"""The task follows the target, not the dialect.

The dialect can lay out ``classify``, but ``yolov10`` has no such task and
carries no tag, so the service hands the plugin a manifest with no tag in
it and the export is the detect layout with nothing on its images.
"""
fixture = Fixture(tmp_path)
fixture.label({0: [_tag()], 1: [_tag()]})
out = fixture.export(fixture.publish(), tmp_path / "out", target=_target("yolov10"))
fixture.close()

assert (out / DATA_FILENAME).exists()
assert not (out / "train").exists()
labels = sorted((out / "labels" / "train").iterdir())
assert len(labels) == 3
assert all(path.read_text(encoding="utf-8") == "" for path in labels)


def test_a_polygon_release_addressed_to_a_detect_target_is_written_as_detect(
tmp_path: Path,
) -> None:
"""A polygon selects ``segment`` only when the target carries it; ``yolov10`` does not."""
fixture = Fixture(tmp_path)
lane = Annotation(
asset_id=uuid4(),
label_class="lane",
schema_version=1,
geometry=PolygonGeometry(points=[(8.0, 12.0), (24.0, 12.0), (16.0, 36.0)]),
provenance="human",
)
fixture.label({0: [lane, _box(x=8, y=12, width=16, height=24)]})
out = fixture.export(fixture.publish(), tmp_path / "out", target=_target("yolov10"))
fixture.close()

rows = [
path.read_text(encoding="utf-8")
for path in sorted((out / "labels" / "train").iterdir())
if path.read_text(encoding="utf-8")
]
(written,) = rows
assert written.splitlines() == ["0 0.250000 0.500000 0.250000 0.500000"]


def test_a_polygon_release_addressed_to_a_segment_target_keeps_its_vertices(
tmp_path: Path,
) -> None:
fixture = Fixture(tmp_path)
lane = Annotation(
asset_id=uuid4(),
label_class="lane",
schema_version=1,
geometry=PolygonGeometry(points=[(8.0, 12.0), (24.0, 12.0), (16.0, 36.0)]),
provenance="human",
)
fixture.label({0: [lane]})
out = fixture.export(fixture.publish(), tmp_path / "out", target=_target("yolov5"))
fixture.close()

rows = [
path.read_text(encoding="utf-8")
for path in sorted((out / "labels" / "train").iterdir())
if path.read_text(encoding="utf-8")
]
(written,) = rows
assert written.splitlines() == ["1 0.125000 0.250000 0.375000 0.250000 0.250000 0.750000"]


def test_a_class_that_cannot_name_a_directory_is_refused_by_name(tmp_path: Path) -> None:
fixture = Fixture(tmp_path)
fixture.schemas.create_version(
Expand Down
Loading
Loading