Skip to content

Commit 752e9b5

Browse files
angusbezzinaclaude
andcommitted
docs(navigation): record selection navigation + frame anchoring, cover them in the probe
DECISIONS.md gains two rows and two sections: why Parent/Child replaced the one-way "Widen", why descent replays history instead of re-querying a live tree, why prepending the frontier child is what makes the round trip hold both ways, the `component` consequence that follows from prepending (first SEEDED rung above the BOUND one, read from the identifier — an unseeded id is a slash-joined path that greps to nothing), and the open question F1 left about whether the parent chain stays seeded-only. Frame anchoring records why the drawn box outranks the resolved element visually, why the element is named in the composer rather than drawn, why navigating reveals it, and why the hover gate lives in the session. PARITY.md gains rows for child navigation (one shared pure ChildNavigationRule on all three adapters; each supplies only candidates) and for frame-mode anchoring / hover gating (shared session + shared view, so no code asymmetry — but the live-tree probe is macOS-only, recorded as a verification gap). AnnotKitOverlayProbe grows Phase 8: round trip up (climb 3, descend 3, land on the original), round trip down through a real ChildNavigationSource query, history-not-re-query (proved by the path depth NOT growing on a second descent), the component fix against the live tree (unseeded child, unseeded parent, component resolves to the seeded grandparent with no slash), an inert frame-mode hover on a point proven live in point mode, and the frame anchor surviving until navigation drops it while the drawn rect persists. Behaviour is unchanged; the probe reports nine phases, all PASS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent d7cc0eb commit 752e9b5

3 files changed

Lines changed: 407 additions & 2 deletions

File tree

DECISIONS.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@ Resolves the open decisions from the plan (planning/annotkit in the cli repo). P
1111
| Default element source | Accessibility hierarchy | The only strategy that surfaces SwiftUI `accessibilityIdentifier` values. |
1212
| Annotation target rule | Deepest actionable, else deepest meaningful; anchor the selector to the nearest identifier | One rule on both platforms. Supersedes the earlier macOS "deepest meaningful" and iOS "nearest identified" split. See below. |
1313
| Marquee target rule | Largest meaningful element ≥85% surrounded; else the tightest element enclosing the drawn frame | Rect selection, the deliberate inverse of the point rule's deepest-wins. See below. |
14+
| Selection navigation | Bidirectional Parent/Child over one path; descent replays history and only queries the source at the deepest rung | Replaces the one-way "Widen". Prepending the frontier child shifts every rung, so a note's `component` is the first SEEDED rung above the BOUND one. Whether the parent chain stays seeded-only is OPEN. See below. |
15+
| Frame mode anchoring | The frame the user DREW anchors the overlay until they navigate; the resolved element is NAMED in the composer, not drawn on the canvas | Hover is point-mode-only, gated in the session rather than the view. See below. |
1416
| Opt-in element source | View tree (NSView/UIView) | Surfaces concrete view class names; richer for AppKit/UIKit hosts. Collapses to hosting views in pure SwiftUI. |
1517
| `pathname` mapping | Host-supplied route, inferred fallback | A native app has no URL routes; the host sets a route, else infer from the key window title or identifier. |
1618
| Overlay coverage | Primary screen (MVP) | The overlay covers the primary display; SwiftUI-local points map to AX screen coordinates there. Full multi-display placement is deferred (cli-a99qm.4.2). |
@@ -129,6 +131,88 @@ the optional `MarqueeTargetSource` capability, which returns a component-widenin
129131
ladder identical in contract to `ComponentLadderSource`, so widening and the
130132
note's `component` field work unchanged.
131133

134+
## Selection navigation (VRT-mijf.1)
135+
136+
The composer's one-way "Widen" button is replaced by Parent and Child over a
137+
single path: index 0 is the deepest rung known so far, ascending indices are
138+
progressively broader, and one index marks the rung the note is bound to.
139+
140+
The rename is not cosmetic. "Widen" named the MECHANISM — the highlighted area
141+
gets bigger — when the act is choosing which component the note is filed against.
142+
A button that makes things bigger implies no inverse, so a user who overshot, or
143+
whom the target rule bound coarser than they meant, had nothing to press.
144+
145+
**Descent prefers HISTORY over re-querying.** Above the deepest rung, Child just
146+
steps the index back down what the user climbed. Only AT the deepest rung does it
147+
ask the source for children, and it then PREPENDS the one it takes, so index 0
148+
still means "deepest known rung". Re-querying on every press would be less code
149+
and wrong: the source's answer is a heuristic over a LIVE tree, so a hover state
150+
resolving or a list reflowing between two presses makes the same key produce a
151+
different result. Prepending is what makes the round trip hold in BOTH
152+
directions — after descending to child C, Parent returns to the original target
153+
and Child returns to C ITSELF rather than re-running the heuristic against a tree
154+
that has moved on.
155+
156+
**The consequence that bit us.** Prepending shifts every existing rung up one, so
157+
"the note's `component` is the rung above the target" stopped being true: index 1
158+
is now the ORIGINAL target, which is frequently unseeded. `component` is
159+
therefore the first SEEDED rung strictly above the BOUND rung, and it is read
160+
from that rung's IDENTIFIER, never from its `Element.id`. An unseeded element's
161+
`id` is a slash-joined path (`AXWindow[0]/AXGroup[0]/AXStaticText[1]`); exported
162+
as a `component` it hands the consuming agent a grep target that matches nothing
163+
while looking entirely plausible in the note — a silent miss, not a visible one.
164+
The same path is rooted differently depending on which entry point produced the
165+
element (`snapshot()` roots at the window, the hit-test and marquee paths at the
166+
application), so the id is not even stable for one node, which is a second reason
167+
it can never be a code locator.
168+
169+
**Open, pending dogfood: should the parent chain stay seeded-only?** It is today —
170+
every rung above the target is an identified component, so every rung locates
171+
code and no press can bind a note to something that names nothing. The cost is
172+
that it skips structural levels the user can SEE: a row inside an unseeded stack
173+
offers no rung for the stack, so Parent jumps from the row straight to the card
174+
and the level the user was aiming at is unreachable. Admitting unseeded rungs
175+
would fix the navigation and degrade the notes. Which failure is worse is not
176+
decidable from the design; it needs real use, so this is recorded as unresolved
177+
rather than settled.
178+
179+
## Frame mode anchoring (VRT-mijf.2)
180+
181+
When a drawn frame resolves to a real element, the overlay anchors its highlight,
182+
composer and pin to the FRAME the user drew — not to the element — until the user
183+
presses Parent or Child, at which point the bound element becomes the anchor and
184+
the frame stays on screen, dimmed.
185+
186+
**Why the frame outranks the resolved element.** The user drew a box, so the box
187+
is the truth of the selection until they say otherwise. Anchoring to the
188+
resolution instead makes the rectangle vanish the instant the mouse comes up and
189+
the highlight snap to a card that was never swept, which reads as the tool having
190+
ignored the gesture.
191+
192+
**Why the element is NAMED rather than DRAWN.** A note must never be captured
193+
against a target the user could not see, so the binding has to appear somewhere.
194+
But a second rectangle on the canvas is exactly what "show me only the frame I
195+
drew" rules out, and two boxes of different shapes leave it ambiguous which one
196+
the note records. The composer header carries the name behind a `Frame →` prefix,
197+
so it reads as what the frame RESOLVED to rather than as a label for the
198+
rectangle, and the prefix disappears the moment navigation puts a named element
199+
back on the canvas — the name is never qualified in two places at once.
200+
201+
**Why navigating reveals the element.** Pressing Parent or Child IS the question
202+
"which element is this filed against?", so the answer has to become visible;
203+
moving the binding while the highlight stays on the drawn rect would give no
204+
feedback at all. The frame survives, weaker, because it is still what the note
205+
records (`regionRect`) even once it no longer decides the binding.
206+
207+
**Why hover is gated in the SESSION, not the view.** Frame mode selects from a
208+
swept rectangle, so a hover highlight there advertises a click-selection no press
209+
in that mode can produce — the dogfooding report was a whole card lit up with its
210+
name tag while nothing had been drawn. The view keeps its own guard for the
211+
narrower during-the-drag case; the MODE gate belongs one level down because there
212+
it is unit-testable without a window, no future UI path can reintroduce it, and
213+
it removes a cross-process AX hit-test per pointer-motion event. It is a cost
214+
decision as much as a visual one.
215+
132216
## IP hygiene (carried into the F7 legal gate)
133217

134218
- Do not copy original Agentation source (PolyForm Shield 1.0.0, non-compete). Only the `AGENTATION_NOTES.md` file format is reused, reimplemented clean-room.

PARITY.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ row; each asymmetry is closed by code or has a tracked mitigation.
1414
| Annotation target rule | shared `AnnotationTargetRule` over an AX candidate chain | shared `AnnotationTargetRule` over a UIView candidate chain | none — both build a `[TargetCandidate]` chain and apply the SAME rule (deepest actionable, else deepest meaningful). Closes the earlier split (macOS "deepest meaningful" vs iOS "nearest identified"), cli-got28.2 |
1515
| Component widening | `ComponentLadderSource` (AX chain) | `ComponentLadderSource` (UIView chain) | none — same ladder (target, then enclosing identified components) |
1616
| Marquee selection (drawn frame → element) | `MarqueeTargetSource`: shared `MarqueeTargetRule` over `[MarqueeCandidate]` read from the AX tree | `MarqueeTargetSource`: shared `MarqueeTargetRule` over `[MarqueeCandidate]` read from the UIView tree | none — the DECISION is one pure rule (largest ≥85%-surrounded element, else the tightest enclosing one); the adapters differ only in how they read candidates out of their own tree. Both do ONE walk from a single root so depth (the rule's tie-break) is numbered comparably, both collect the subtree WHOLE (an intersects-the-frame filter would discard the enclosing-pass candidates), and both return the SAME target-first, broadest-last ladder as `ComponentLadderSource`, so the session's widening and `component` field work unchanged from a framed selection |
17+
| Child navigation (select child) | `ChildNavigationSource` over the AX tree; the opt-in view-tree source implements it over `NSView` | `ChildNavigationSource` over the `UIView` tree | none — the ORDERING is the one pure `ChildNavigationRule` (contains the gesture's hint, then seeded, then larger area, then lowest index) on all three adapters, which supply only `[ChildCandidate]`. All three also share the cost shape the protocol mandates: re-find the bound element by descending its containing window along its own frame centre (bounded by tree DEPTH, not tree size), then collect the NEAREST meaningful descendants, each branch stopping at its first meaningful node — so a SwiftUI host's unidentified layout wrappers are descended THROUGH rather than offered as children, which is what keeps the Child control alive under pure SwiftUI on both platforms. macOS additionally skips window chrome and its own overlay window during that descent; UIKit has no chrome, and the overlay is already excluded by the shared window lookup |
18+
| Frame-mode anchoring + hover gating | shared `AnnotationSession` (`selectionAnchorFrame`, the `tool == .point` hover gate, `setTool` clearing `hovered`) rendered by the shared `OverlayView` | same | none in the code — all of it is session-level and platform-free, and one SwiftUI view renders it. ASYMMETRIC VERIFICATION, recorded as a gap rather than closed: `AnnotKitOverlayProbe` Phase 8 drives navigation, the note's `component`, the hover gate and the frame anchor against a REAL accessibility tree, and it is macOS-only (`#if os(macOS)`, AppKit + `AXUIElement`), so the iOS adapter's live behaviour is covered only by unit tests over the pure rules. Mitigated, not fixed, by the fact that everything Phase 8 asserts about anchoring and hover lives in the shared session; what remains unverified on iOS is the ADAPTER's candidate collection. Note the hover gate is also moot on touch-only iOS — hover exists there only with a trackpad or pencil — so the reported symptom cannot arise without a pointer |
1719
| Marquee drag threshold | cursor slop (a mouse does not move on a deliberate click) | larger touch slop | ASYMMETRIC BY DESIGN, owned by the drag UI, not the adapters: a finger rolls several points on a deliberate tap, so the macOS threshold on iOS would turn taps into marquees. Below the threshold both platforms route the gesture to the point path (`select(atAXPoint:)`), per the caller contract on `select(inAXRect:)` |
1820
| Overlay excluded from element lookup | AX window identifier (`AXIntrospection.overlayWindowIdentifier`) filtered out of every `kAXWindows` read | `PassThroughWindow` TYPE identity filtered out of `IOSElementSource.windows()` | ASYMMETRIC BY NECESSITY — the hosts are different window kinds. macOS's overlay is a separate `NSPanel` matched by the identifier the controller stamps on it; iOS's is a `UIWindow` in the HOST's scene sharing its pid, so no pid/scene filter separates it and a type check (internal to the module) cannot drift the way an identifier convention can. Both filter in the shared window lookup, so snapshot / hit-test / region-anchor / marquee agree; leaving it in would let a marquee bind the user's note to AnnotKit's own UI |
1921
| Coordinate space | Cocoa bottom-left to AX top-left flip | UIKit top-left native | iOS needs no flip; shared `ScreenSpace` used only on macOS |

0 commit comments

Comments
 (0)