feat(marquee): iOS port + parity (VRT-cne0.8)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: angusbezzina

PARITY.md

@@ -13,6 +13,9 @@ row; each asymmetry is closed by code or has a tracked mitigation.
 | Hit test primitive | `AXUIElementCopyElementAtPosition` + NSView `hitTest` | `UIView.hitTest(_:with:)` | iOS has no global AX point query; uses view hitTest. Tracked: F5.2 |
 | Annotation target rule | shared `AnnotationTargetRule` over an AX candidate chain | shared `AnnotationTargetRule` over a UIView candidate chain | none — both build a `[TargetCandidate]` chain and apply the SAME rule (deepest actionable, else deepest meaningful). Closes the earlier split (macOS "deepest meaningful" vs iOS "nearest identified"), cli-got28.2 |
 | Component widening | `ComponentLadderSource` (AX chain) | `ComponentLadderSource` (UIView chain) | none — same ladder (target, then enclosing identified components) |
+| Marquee selection (drawn frame → element) | `MarqueeTargetSource`: shared `MarqueeTargetRule` over `[MarqueeCandidate]` read from the AX tree | `MarqueeTargetSource`: shared `MarqueeTargetRule` over `[MarqueeCandidate]` read from the UIView tree | none — the DECISION is one pure rule (largest ≥85%-surrounded element, else the tightest enclosing one); the adapters differ only in how they read candidates out of their own tree. Both do ONE walk from a single root so depth (the rule's tie-break) is numbered comparably, both collect the subtree WHOLE (an intersects-the-frame filter would discard the enclosing-pass candidates), and both return the SAME target-first, broadest-last ladder as `ComponentLadderSource`, so the session's widening and `component` field work unchanged from a framed selection |
+| Marquee drag threshold | cursor slop (a mouse does not move on a deliberate click) | larger touch slop | ASYMMETRIC BY DESIGN, owned by the drag UI, not the adapters: a finger rolls several points on a deliberate tap, so the macOS threshold on iOS would turn taps into marquees. Below the threshold both platforms route the gesture to the point path (`select(atAXPoint:)`), per the caller contract on `select(inAXRect:)` |
+| Overlay excluded from element lookup | AX window identifier (`AXIntrospection.overlayWindowIdentifier`) filtered out of every `kAXWindows` read | `PassThroughWindow` TYPE identity filtered out of `IOSElementSource.windows()` | ASYMMETRIC BY NECESSITY — the hosts are different window kinds. macOS's overlay is a separate `NSPanel` matched by the identifier the controller stamps on it; iOS's is a `UIWindow` in the HOST's scene sharing its pid, so no pid/scene filter separates it and a type check (internal to the module) cannot drift the way an identifier convention can. Both filter in the shared window lookup, so snapshot / hit-test / region-anchor / marquee agree; leaving it in would let a marquee bind the user's note to AnnotKit's own UI |
 | Coordinate space | Cocoa bottom-left to AX top-left flip | UIKit top-left native | iOS needs no flip; shared `ScreenSpace` used only on macOS |
 | Screenshot | ScreenCaptureKit / `cacheDisplay` | `UIGraphicsImageRenderer` + `drawHierarchy` | both capture own hierarchy only; no cross-window or secure overlays |
 | Overlay host | resizing `NSPanel` (toolbar corner idle, full screen annotating) | pass-through `UIWindow` | both interactive; selection via the shared SwiftUI catcher, not a global monitor |

Sources/AnnotKit/iOS/IOSElementSource.swift

@@ -260,10 +260,30 @@ public final class IOSElementSource: ElementSource, ComponentLadderSource {
 
     // MARK: - Window helpers
 
+    /// Every inspectable HOST window — AnnotKit's own overlay is never one.
+    ///
+    /// On macOS the overlay is a separate `NSPanel` filtered out of `kAXWindows`
+    /// by identifier; on iOS the risk is live rather than theoretical, because
+    /// ``PassThroughWindow`` sits in the SAME scene as the host and is returned by
+    /// `UIWindowScene.windows` like any other window. Its chrome is genuinely
+    /// identified and genuinely meaningful, so it passes
+    /// ``TargetCandidate/isEligibleMeaningful`` cleanly: leave it in and a marquee
+    /// (which by construction spans screen area the overlay draws across) can bind
+    /// the user's note to AnnotKit's own UI, and a snapshot/selector lists our
+    /// toolbar as if it were app content.
+    ///
+    /// Excluded by TYPE identity, not by name, identifier, or window level: the
+    /// type is internal to this module so the check cannot be defeated by a naming
+    /// convention drifting, and a pid filter is useless here because AnnotKit runs
+    /// in the host's process. Filtering in this ONE helper is what makes
+    /// ``snapshot()``, ``keyWindow``, `hitTest`, `componentLadder(at:)`, and the
+    /// marquee path agree — matching the macOS side, which excludes the overlay in
+    /// every window lookup rather than only in the one that motivated it.
     private static func windows() -> [UIWindow] {
         UIApplication.shared.connectedScenes
             .compactMap { $0 as? UIWindowScene }
             .flatMap(\.windows)
+            .filter { !($0 is PassThroughWindow) }
     }
 
     static var keyWindow: UIWindow? {
@@ -276,4 +296,155 @@ public final class IOSElementSource: ElementSource, ComponentLadderSource {
         return window.convert(inWindow, to: nil)
     }
 }
+
+// MARK: - Marquee (drawn frame -> view)
+
+extension IOSElementSource: MarqueeTargetSource {
+    /// The ladder for a frame the user DREW, over the `UIView` tree: the view the
+    /// frame binds to per ``MarqueeTargetRule`` first, then each enclosing
+    /// identified component, broadest last — the SAME target-first contract as
+    /// ``componentLadder(at:)``, because the session assumes `ladder[0]` IS the
+    /// bound target for both widening and the note's `component` field.
+    ///
+    /// Structurally identical to the macOS path (`AXIntrospection.marqueeLadder`):
+    /// both platforms only differ in how they read candidates out of their tree,
+    /// and the decision itself is the one shared pure rule, so an identical drag
+    /// over an identical layout resolves identically by construction rather than
+    /// by two implementations being kept in step by hand.
+    ///
+    /// Cost: one full walk of the hit window's view tree per drag RELEASE — never
+    /// during the drag and never on touch-move. That rate is what makes a
+    /// whole-tree walk affordable here, where the point path must stay on the
+    /// ancestor chain.
+    public func marqueeLadder(in rect: CGRect) -> [Element] {
+        // Standardize before anything geometric: a right-to-left / bottom-to-top
+        // drag arrives with negative extents, where `contains` degenerates and the
+        // window lookup below would silently find nothing.
+        let marquee = rect.standardized
+        guard let root = Self.marqueeRoot(containing: CGPoint(x: marquee.midX, y: marquee.midY)) else { return [] }
+
+        // ONE recursive walk from that single root, so every candidate's depth is
+        // measured from the SAME origin (window = 0, its children = 1, …). Depth is
+        // the rule's tie-break between geometrically indistinguishable candidates;
+        // numbering assembled from several differently-rooted traversals would turn
+        // that tie-break into noise.
+        //
+        // The subtree is collected WHOLE — deliberately not pre-filtered to views
+        // intersecting the drawn frame. The rule's second pass needs the candidates
+        // whose frames CONTAIN the frame (the user drew INSIDE something), and an
+        // intersects-the-marquee filter is exactly what discards them.
+        var views: [UIView] = []
+        var candidates: [MarqueeCandidate] = []
+        Self.collectMarqueeCandidates(root, depth: 0, views: &views, candidates: &candidates)
+
+        guard let resolution = MarqueeTargetRule.resolve(marquee: marquee, in: candidates) else { return [] }
+        let target = views[resolution.index]
+
+        // The widening rungs are anchored at the TARGET's frame centre, not the
+        // drawn frame's: a sloppy marquee can spill outside the element it bound
+        // to, and a container that does not contain the target is not a component
+        // the user could widen to. Same value the point path passes.
+        let targetFrame = Self.screenFrame(of: target)
+        let targetCentre = CGPoint(x: targetFrame.midX, y: targetFrame.midY)
+        return [Self.element(for: target)]
+            + Self.enclosingComponents(of: target, containing: targetCentre).map { Self.element(for: $0) }
+    }
+
+    /// The frontmost visible non-overlay window containing `point`, or nil when the
+    /// drag happened over nothing of ours.
+    ///
+    /// `UIWindowScene.windows` has no documented front-to-back order, so the
+    /// frontmost is derived rather than assumed: highest `windowLevel` first, ties
+    /// broken by the LATER array position, which is UIKit's own within-level
+    /// ordering. Getting this wrong on a host that presents an alert or
+    /// share-sheet window would walk the window BEHIND the one the user is looking
+    /// at and bind the note to an element they cannot see.
+    ///
+    /// AnnotKit's own overlay cannot be picked here because ``windows()`` never
+    /// returns it — see that helper for why the exclusion lives there and not in
+    /// this method.
+    private static func marqueeRoot(containing point: CGPoint) -> UIWindow? {
+        windows()
+            .filter { !$0.isHidden && $0.alpha > 0.01 }
+            .enumerated()
+            .sorted { lhs, rhs in
+                lhs.element.windowLevel.rawValue == rhs.element.windowLevel.rawValue
+                    ? lhs.offset > rhs.offset
+                    : lhs.element.windowLevel.rawValue > rhs.element.windowLevel.rawValue
+            }
+            .first { screenFrame(of: $0.element).contains(point) }?
+            .element
+    }
+
+    /// Depth-first walk collecting a PARALLEL pair per view — the live `UIView` and
+    /// its pure ``MarqueeCandidate`` — so ``MarqueeTargetRule/Resolution/index``
+    /// maps straight back to a live view.
+    ///
+    /// The candidate is built with the same ``candidate(for:)`` the point path
+    /// uses, so container-root classification — which is what the rule's
+    /// eligibility filter reads — is identical for a tap and a drag by
+    /// construction, not by two call sites agreeing today.
+    ///
+    /// Hidden and effectively transparent subtrees are skipped WHOLE: they keep
+    /// real frames, so a marquee would happily "surround" a view the user cannot
+    /// see, and being a large such frame it could win pass 1 outright. The point
+    /// path gets this filtering free from `UIView.hitTest`, which a whole-tree walk
+    /// never goes through.
+    private static func collectMarqueeCandidates(
+        _ view: UIView,
+        depth: Int,
+        views: inout [UIView],
+        candidates: inout [MarqueeCandidate]
+    ) {
+        views.append(view)
+        candidates.append(
+            MarqueeCandidate(element: candidate(for: view), frame: screenFrame(of: view), depth: depth)
+        )
+        guard depth < maxDepth else { return }
+        for subview in view.subviews where !subview.isHidden && subview.alpha > 0.01 {
+            collectMarqueeCandidates(subview, depth: depth + 1, views: &views, candidates: &candidates)
+        }
+    }
+
+    /// The identified components that geometrically ENCLOSE `target` at `point`,
+    /// smallest-first — the widening rungs above a bound target, mirroring the
+    /// macOS `enclosingComponents(of:containing:in:)` so a drag widens through the
+    /// same kind of components on both platforms.
+    ///
+    /// The scan is GEOMETRIC, not pure ancestry (DECISIONS.md → "Component
+    /// containment is GEOMETRIC"): a card's identified background surface is
+    /// routinely a SIBLING of the card's content rather than its ancestor, so it
+    /// never appears in an ancestor chain. Scanning each ancestor PLUS its direct
+    /// subviews reaches those surfaces without a second whole-tree walk. Deduped by
+    /// identifier because the same surface is reachable from several ancestors, and
+    /// the `>= targetArea` floor keeps a SMALLER identified sibling that merely
+    /// happens to cover the target's centre out of a ladder that is supposed to
+    /// only ever widen.
+    ///
+    /// Container roots are excluded, matching
+    /// ``AnnotationTargetRule/wideningLadder(in:)`` stopping at the window: the
+    /// window encloses everything, so it would be the top rung of every ladder
+    /// while naming no component an agent could act on.
+    private static func enclosingComponents(of target: UIView, containing point: CGPoint) -> [UIView] {
+        let targetFrame = screenFrame(of: target)
+        let targetArea = targetFrame.width * targetFrame.height
+        var containers: [(view: UIView, area: CGFloat)] = []
+        var seen = Set<String>()
+        for ancestor in ancestorChain(from: target) {
+            for view in [ancestor] + ancestor.subviews {
+                guard !(view is UIWindow), view !== target else { continue }
+                guard !view.isHidden, view.alpha > 0.01 else { continue }
+                let identifier = view.accessibilityIdentifier ?? ""
+                guard !identifier.isEmpty, !seen.contains(identifier) else { continue }
+                let frame = screenFrame(of: view)
+                let area = frame.width * frame.height
+                guard frame.width > 0, frame.height > 0, frame.contains(point), area >= targetArea else { continue }
+                seen.insert(identifier)
+                containers.append((view, area))
+            }
+        }
+        containers.sort { $0.area < $1.area }
+        return containers.map(\.view)
+    }
+}
 #endif