Skip to content

Interactive auto-labeling's in-flight feedback is in the wrong place and has no delay threshold #541

Description

@JArmandoAnaya

The suggest tool arms with S, and a click on the canvas asks a segmentation model to propose a
shape for whatever sits under the point, through POST /inference/suggest. It shipped as slice 3b
of the inference work, in the pull request that added the interactive tool and its ephemeral
preview (#451).

While that request is in flight the editor does say something, and this is worth stating plainly
because it changes what needs building: SuggestPanel has an asking reading that renders a
spinning icon, the title "Looking at that…", and the sentence "The first click on a frame is the
slow one — refining after it is quick." So the feedback exists. Two things about it are wrong.

It is in the wrong place. That card lives in the notice column at the top-right of the editor.
The person who just clicked is looking at the point they clicked, in the middle of the picture,
where nothing happens at all — no marker changes, no cursor changes. On a cold start, which is
model load plus inference on the CPU and can run well over a second, the canvas is silent for the
whole wait.

It has no delay threshold. The card flips the instant the session's status becomes asking, so
a warm response — which is most of them, because a segmenter reads the image once and answers every
later click from that reading — produces a brief flash of a different card and then flips back. The
refine gesture is the worst case: the panel is showing "A shape for …", the click replaces it with
"Looking at that…" for a fraction of a second, and it comes back. The cold-start sentence has the
same problem from the other end. It is true of the first click on a frame and it is shown on every
click, including the fast ones it is not describing.

The result is that the two states a person actually needs to tell apart — this is working and
this is broken — are not distinguished where they are looking, and the reassurance that is
offered arrives when it is not needed.

What this asks for is one timing rule governing both surfaces: nothing at all below a threshold, an
indicator anchored at the click point above it, and the cold-start sentence held back until the
wait is long enough to be about a cold start.

Two constraints are worth recording before anyone picks this up, because both are deliberate and
neither should be undone in passing. Refining while a request is in flight is supported on purpose
withPoint bumps a serial and keeps the previous preview so the canvas never blanks, and a stale
answer is dropped by that serial rather than by cancelling anything — so an indicator must
re-anchor rather than block the click. And the annotator package ships headless, with no Tailwind,
no design tokens and no stylesheet, so anything drawn on the canvas is drawn in the package's own
SVG vocabulary.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    frontendannotator / ui-core / app packagesui-corefrontend/ui-core workuxUser experience / interaction design

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions