Skip to content
Merged
35 changes: 22 additions & 13 deletions docs/content/inference.md
Original file line number Diff line number Diff line change
Expand Up @@ -595,8 +595,8 @@ POST /inference/suggest
"contour": [[404.0, 221.0], …]
}
],
"applied": {"detail": "balanced"},
"parameters": ["detail"]
"applied": {"tolerance": 1.0},
"parameters": ["tolerance"]
}
```

Expand Down Expand Up @@ -643,18 +643,19 @@ model.

1. **Which pieces** of the mask become shapes.
2. **Closing the gaps** in them that are narrower than a reach.
3. **Tracing** the boundary of what is left.
4. **Simplifying** that boundary to a vertex count somebody can edit.
3. **Tracing** the boundary of what is left along the pixels' edges, and smoothing it once.
4. **Simplifying** that boundary to within a tolerance you choose.

The geometry branch happens after the second step: a polygon class takes steps 3 and 4 on the
piece you pointed at, a box class takes one extent over every piece that survived. A box
therefore does not move when `detail` does.
therefore does not move when `tolerance` does.

| Setting | What it moves | Applies to |
| --- | --- | --- |
| `detail` | `coarse`, `balanced` or `fine` - how much of the outline survives | polygon |
| `tolerance` | a distance in the asset's pixels; every point of the traced outline lies within it of the polygon | polygon |

It is optional, and omitting it gives what this route always gave: `balanced`.
It is optional and defaults to `1.0`. It is refused outside `0.25` to `16`, never clamped: a
clamped value would report a tolerance the server did not apply.

**Two settings used to be here and are not** (#557). How wide a gap gets closed and how many
pieces become shapes are still decided, at fixed defaults nobody asks for. As controls they
Expand All @@ -663,10 +664,18 @@ shape - so they read as knobs wired to nothing, and could only be got wrong on t
one. Their value is in the default rather than in the choice. They come back as settings if
a real need for the choice appears.

**The tolerance is relative, which is what makes one setting work everywhere.** It is a fraction
of the region's own size rather than a pixel count, so it does the same thing to a thing eight
pixels across and a thing eight hundred across, and `balanced` keeps a typical object in the
10-40 vertex range.
**The tolerance is a distance, and that is the whole promise.** Every point of the traced
outline lies within `tolerance` pixels of the polygon you get back, so the number means the
same thing on a thing eight pixels across and a thing eight hundred across, and you know what
you will get before you move it. One pixel follows the mask closely on any object; sixteen
gives a rough shape to nudge into place. In the editor, `[` doubles it and `]` halves it, and
the slider runs on a doubling track between the two ends.

**The outline is the mask's edge, smoothed.** The trace runs along the boundary between lit
and unlit pixels — so a single pixel is its unit square rather than a point — and one pass of
corner cutting over those unit edges turns the staircase a pixel grid imposes into a smooth
line while moving no real corner by more than half a pixel. That smoothed ring, reduced once at
a quarter pixel, is the `contour` every answer carries.

**Specks are dropped first, and a click never becomes a cleanup job.** A mask routinely carries
more than one separate piece - a scrap of antialiasing along an edge, a reflection, a patch of
Expand All @@ -691,7 +700,7 @@ the largest piece alone cuts the object off at the occlusion, and a box per piec
thing twice.

**`parameters` says which settings apply here**, for the kind of shape your `allowed_geometries`
will produce. A box has no outline, so `detail` has nothing to do to one and the list comes back
will produce. A box has no outline, so the tolerance has nothing to do to one and the list comes back
**empty** - which is how a client is told to offer no adjustments at all. A client renders what
this lists and works none of it out for itself.

Expand All @@ -704,7 +713,7 @@ makes an outline ragged. Its reach grows with the piece and stops at a few pixel
gap is a feature of the shape rather than an artefact of tracing it.

**`contour` is the outline the shape was reduced from**, in the asset's own pixels, and it is
there so a client can re-run `detail` without asking again. It is the same points the server
there so a client can re-run the tolerance without asking again. It is the same points the server
reduced, which matters:
simplification is not nested, so a client starting from anything else could not be held to the
server's answer. A box carries none, because it is an extent rather than something reduced from
Expand Down
27 changes: 15 additions & 12 deletions docs/content/ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -551,7 +551,7 @@ The gesture:
| left-click | adds a point on the object, and asks again |
| alt-click | adds a point that is **not** on the object, and asks again |
| `↵` | accepts the proposal as an annotation |
| `[` / `]` | coarser or finer, without opening anything |
| `[` / `]` | coarser or finer - doubles or halves the tolerance, without opening anything |
| `Esc` | closes the adjustments; then clears the points; then puts the tool away |

Every click sends **all** the points placed so far - the route is stateless - and
Expand Down Expand Up @@ -581,31 +581,34 @@ class and the model's confidence beside it, and is in neither the document nor t
undo history. `Esc` is its undo. Switching class, switching frames or leaving the
page discards it, and nothing is written.

**Its vertices are drawn the whole time it is up**, which is what makes the detail
**Its vertices are drawn the whole time it is up**, which is what makes the tolerance
setting something you can see rather than a number that changes. A committed shape
shows its vertices only while it is selected; a proposal is not selected and shows
them anyway, because choosing how much outline to keep is exactly a question about
where the points are.
them anyway, because choosing how far the polygon may drift from the mask is
exactly a question about where the points are.

**The shape can be adjusted before it is accepted**, from a section inside the
same card - never a second panel over the picture, which would cover the thing
being adjusted. It is closed until you ask for it, because the default is right
most of the time.

One setting, and whether it appears is the server's answer rather than the
editor's guess (`docs/content/inference.md`). **Detail** is a three-position slider -
coarse, balanced, fine - with a label beside it naming the step and what it costs,
`Fine · 41 pts`. `[` and `]` move it without opening anything. Either way it costs
no request at all: the answer carried the outline it was reduced from, and the
editor re-simplifies it here, so the shape and its vertices move under a held key.
editor's guess (`docs/content/inference.md`). **Detail** is a slider on a
doubling track over the tolerance, from `0.25` px to `16` px, defaulting to
`1.0`, with a label beside it naming the value and what it costs,
`2.0 px · 23 pts`. `[` doubles it and `]` halves it without opening anything.
Either way it costs no request at all: the answer carried the outline it was
reduced from, and the editor re-simplifies it here, so the shape and its
vertices move under a held key. On a small object a coarse tolerance can leave
nothing to show; step finer and the shape comes back.

Pressing the slider never takes focus off the canvas, so `[`, `]`, `Esc` and `↵`
keep working while you drag it. Tab still reaches it, for driving it from the
keyboard on purpose.

On a class that stores a box the section does not appear at all, because detail
changes an outline and a box has none. The editor does not know that; the answer
says so, by naming no settings.
On a class that stores a box the section does not appear at all, because the
tolerance changes an outline and a box has none. The editor does not know that;
the answer says so, by naming no settings.

**Two settings were here and are not.** Closing the gaps in the mask and proposing
every separate piece are still done, at fixed defaults. As controls they did
Expand Down
4 changes: 2 additions & 2 deletions docs/content/ui/annotator.md
Original file line number Diff line number Diff line change
Expand Up @@ -446,8 +446,8 @@ badges: `classColor` in `frontend/annotator/src/adapters/react/paint.ts`. `ui-co

**The suggestion preview is a third visual state, not a shape marked selected.** Selection
carries the panel row, the delete key and the keyboard rules a proposal must not have. Its
vertices are up the whole time it is on screen, undecimated at every detail step, because
where precision was gained or lost *is* what the detail control is about. Its outline is
vertices are up the whole time it is on screen, undecimated at every tolerance change, because
where precision was gained or lost *is* what the tolerance control is about. Its outline is
dashed and an accepted annotation's is solid, which is what tells proposed from committed
at a glance.

Expand Down
4 changes: 2 additions & 2 deletions frontend/annotator/src/adapters/react/TransientLayer.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -225,8 +225,8 @@ function SuggestedShape({
is not one — selection carries the panel row, the delete key and the
keyboard rules a preview must not have. So it gets its own rule: the
vertices are up the whole time the preview is, undecimated at every step,
because where precision was gained or lost *is* what `detail` is about and
a counter alone made it a blind control (#557).
because where precision was gained or lost *is* what the tolerance is about and
a counter alone made it a blind control.

Polygons only. A box has no vertex list — its corners are grips, and a
preview has nothing to drag.
Expand Down
2 changes: 1 addition & 1 deletion frontend/annotator/src/adapters/react/paint.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ function answerOf(...suggestions: readonly Suggestion[]): Answer {
modelRef: MODEL_REF,
confidence: suggestions[0]?.confidence ?? null,
suggestions,
parameters: ["detail"],
parameters: ["tolerance"],
};
}

Expand Down
5 changes: 4 additions & 1 deletion frontend/annotator/src/adapters/react/paint.ts
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,9 @@ export interface PaintedSuggestion {
* would flicker on every press". The engine kept its half of that and the renderer
* threw it away, so the flicker it argues against was happening on every press.
* A held suggestion is the best answer anyone has until a better one arrives,
* whichever status is carrying it.
* whichever status is carrying it. The one status that does decide is `none`,
* which holds the shapes the current tolerance reduced to nothing so that a
* finer one can re-derive them — held to be re-simplified, not to be drawn.
*
* The **points are carried whatever the status**, which is why the caller checks
* for them separately: the dots are what makes a refine click legible, and they
Expand All @@ -309,6 +311,7 @@ export function paintSuggestions(
// of guard as the one below: what the type allows, not what the machine does.
const labelClass = state.labelClass;
if (labelClass === null) return [];
if (state.status === "none") return [];
const color = classColor(declared, labelClass);
const painted: PaintedSuggestion[] = [];
for (const suggestion of state.suggestions) {
Expand Down
102 changes: 53 additions & 49 deletions frontend/annotator/src/core/geometry/simplify.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,35 +3,40 @@
*
* `tests/fixtures/simplification.json` is written by the kernel and kept current
* by `tests/inference/test_simplification_fixture.py`. This proves the port
* reproduces it — exactly, point for point, at every step — which is what lets
* the editor re-simplify locally while the kernel stays authoritative on what is
* written.
* reproduces it — exactly, point for point, at every tolerance — which is what
* lets the editor re-simplify locally while the kernel stays authoritative on
* what is written.
*
* **Exact equality, deliberately.** A tolerance on the comparison would let a
* genuine divergence through: the two implementations either run the same
* arithmetic in the same order or they will disagree about a vertex somewhere,
* and "somewhere" is what a golden fixture exists to find. Both languages hold
* IEEE-754 doubles and `Math.sqrt` is Python's `** 0.5`, so equality is
* achievable rather than optimistic.
* and "somewhere" is what a golden fixture exists to find.
*/

import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";

import type { Point } from "../types";
import { DETAIL_STEPS, EPSILON, MINIMUM_TOLERANCE, polygonAt, simplified, steppedDetail, toleranceFor } from "./simplify";
import type { Detail } from "./simplify";
import {
DEFAULT_TOLERANCE,
MAXIMUM_TOLERANCE,
MINIMUM_TOLERANCE,
polygonAt,
simplified,
steppedTolerance,
} from "./simplify";

interface Case {
readonly name: string;
readonly contour: readonly (readonly number[])[];
readonly tolerance: Readonly<Record<string, number>>;
readonly polygon: Readonly<Record<string, readonly (readonly number[])[] | null>>;
}

interface Fixture {
readonly minimum_tolerance: number;
readonly epsilon: Readonly<Record<string, number>>;
readonly default_tolerance: number;
readonly maximum_tolerance: number;
readonly tolerances: readonly number[];
readonly cases: readonly Case[];
}

Expand All @@ -41,54 +46,45 @@ const fixture = JSON.parse(readFileSync(FIXTURE_URL, "utf8")) as Fixture;
const points = (rows: readonly (readonly number[])[]): Point[] =>
rows.map((row) => [row[0]!, row[1]!] as Point);

/** The fixture keys a polygon by Python's spelling of the float: `1.0`, not `1`. */
const keyed = (tolerance: number): string =>
Number.isInteger(tolerance) ? `${tolerance}.0` : String(tolerance);

describe("the constants travel rather than being restated", () => {
it("floors the tolerance where the kernel does", () => {
it("floors, defaults and caps the tolerance where the kernel does", () => {
expect(MINIMUM_TOLERANCE).toBe(fixture.minimum_tolerance);
});

it("means the same thing by each step", () => {
expect({ ...EPSILON }).toEqual(fixture.epsilon);
});

it("names the steps the kernel names", () => {
expect([...DETAIL_STEPS].sort()).toEqual(Object.keys(fixture.epsilon).sort());
expect(DEFAULT_TOLERANCE).toBe(fixture.default_tolerance);
expect(MAXIMUM_TOLERANCE).toBe(fixture.maximum_tolerance);
});
});

describe.each(fixture.cases)("$name", (found) => {
it.each([...DETAIL_STEPS])("resolves the same tolerance at %s", (step) => {
expect(toleranceFor(points(found.contour), step)).toBe(found.tolerance[step]);
});

it.each([...DETAIL_STEPS])("keeps exactly the same vertices at %s", (step) => {
const expected = found.polygon[step];
const actual = polygonAt(points(found.contour), step);
if (expected === null || expected === undefined) {
it.each([...fixture.tolerances])("keeps exactly the same vertices at %s px", (tolerance) => {
const expected = found.polygon[keyed(tolerance)];
expect(expected).toBeDefined();
const actual = polygonAt(points(found.contour), tolerance);
if (expected === null) {
expect(actual).toBeNull();
return;
}
expect(actual).toEqual(points(expected));
expect(actual).toEqual(points(expected!));
});
});

describe("the gate would notice a port that ignored its input", () => {
it("has a case whose vertex count differs at every step", () => {
// Without one, a `polygonAt` that returned the contour unchanged — or that
// used a fixed tolerance — would satisfy every straight-edged case, which
// answers four corners at all three settings and is right to.
it("has a case whose vertex count moves with the tolerance", () => {
const moving = fixture.cases.filter((found) => {
const counts = [...DETAIL_STEPS]
.map((step) => found.polygon[step])
const counts = Object.values(found.polygon)
.filter((value): value is readonly (readonly number[])[] => Boolean(value))
.map((value) => value.length);
return new Set(counts).size === DETAIL_STEPS.length;
return new Set(counts).size >= 4;
});
expect(moving.length).toBeGreaterThan(0);
});

it("has a case that cannot be a polygon at all", () => {
const refused = fixture.cases.filter((found) =>
[...DETAIL_STEPS].every((step) => found.polygon[step] === null),
Object.values(found.polygon).every((value) => value === null),
);
expect(refused.length).toBeGreaterThan(0);
});
Expand Down Expand Up @@ -120,24 +116,32 @@ describe("simplification on its own", () => {
});
});

describe("stepping through the vocabulary", () => {
it("moves one step at a time", () => {
expect(steppedDetail("balanced", 1)).toBe("fine");
expect(steppedDetail("balanced", -1)).toBe("coarse");
describe("stepping the tolerance", () => {
it("doubles for coarser and halves for finer", () => {
expect(steppedTolerance(1, -1)).toBe(2);
expect(steppedTolerance(1, 1)).toBe(0.5);
});

it("stops at each end rather than wrapping", () => {
// Held down, a wrapping control takes somebody from the coarsest straight to
// the finest without their having asked for anything in between.
expect(steppedDetail("coarse", -1)).toBe("coarse");
expect(steppedDetail("fine", 1)).toBe("fine");
expect(steppedTolerance(MAXIMUM_TOLERANCE, -1)).toBe(MAXIMUM_TOLERANCE);
expect(steppedTolerance(MINIMUM_TOLERANCE, 1)).toBe(MINIMUM_TOLERANCE);
});

it("visits every step on the way across", () => {
const walked: Detail[] = ["coarse"];
while (walked[walked.length - 1] !== "fine") {
walked.push(steppedDetail(walked[walked.length - 1]!, 1));
it("walks the fixture's ladder from the floor to the ceiling", () => {
const walked: number[] = [MINIMUM_TOLERANCE];
while (walked[walked.length - 1]! < MAXIMUM_TOLERANCE) {
walked.push(steppedTolerance(walked[walked.length - 1]!, -1));
}
expect(walked).toEqual([...DETAIL_STEPS]);
expect(walked).toEqual([...fixture.tolerances]);
});

it("lands on the ceiling from a value that is not on the ladder", () => {
expect(steppedTolerance(12, -1)).toBe(MAXIMUM_TOLERANCE);
expect(steppedTolerance(0.3, 1)).toBe(MINIMUM_TOLERANCE);
});

it("snaps a value between stops to the nearest one before stepping", () => {
expect(steppedTolerance(1.19, -1)).toBe(2);
expect(steppedTolerance(1.19, 1)).toBe(0.5);
});
});
Loading
Loading