feat(dom): name clickables from text, onclick, icon class, role + bump 3.1.0 - #51
Merged
Merged
Conversation
# Conflicts: # .github/workflows/publish.yml
chore(release): 2.1.0 → main
release: v2.2.0 (agent executor)
release: v2.3.0 (parser hotfix)
release: v2.4.0 (vision feedback)
release: v2.4.1 (anthropic ImageUrl fix)
release: v2.5.0 (CDP DomExtractor)
release: v2.6.0 (agent DOM grounding)
release: 2.7.0 - grounding chain closes
release: develop → main (2.7.x line: agent fixes, type_text index, clippy clean, README 2.x)
release: v3.0.0 — PerimeterX press-and-hold solver + CDP primitives
Previously the LLM received clickables as bare `[N] tag`, named only when an aria-label/alt/title/name/placeholder/value attribute was present. Anchor text, JS click handlers, and icon-only buttons arrived anonymous, leaving the screenshot as the sole disambiguator. - Detection unions the attribute heuristic with CDP's native isClickable set, catching JS-attached listeners, not just markup. - ax_name falls back through: aria/alt/title/name/placeholder -> descendant text -> onclick handler name (humanized, verb only) -> FontAwesome icon class -> role. - role overrides the displayed tag, so <i role="button"> renders as button. - clickable_map: raise cap 80 -> 200 and sort visible-first so on-screen elements survive truncation. - Split snapshot_parser into snapshot_parser/clickables/clickable_naming to satisfy the 200-LOC module cap. Verified live against headless Chrome via the new (ignored) e2e test clickable_names_resolve_text_onclick_icon_and_role. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Clickable elements sent to the LLM were bare
[N] tag, named only when an aria/alt/title/name/placeholder/value attribute existed. Anchor text, JS click handlers, and icon-only buttons arrived anonymous.isClickableset (catches JS-attached listeners).ax_namefallback chain: aria/alt/title/name/placeholder → descendant text → onclick handler name (humanized, verb only) → FontAwesome icon class → role.roleoverrides the displayed tag (<i role="button">→ button).clickable_map: cap 80 → 200, visible-first sort so on-screen elements survive truncation.snapshot_parserintosnapshot_parser/clickables/clickable_namingfor the 200-LOC cap.Verification
clickable_names_resolve_text_onclick_icon_and_role(ignored test) — anchor text, nested span text, onclick-derived name, role→button all confirmed.🤖 Generated with Claude Code