Run on any CDP browser, and read the whole page - #3
Conversation
connect(endpoint) is now the whole contract: a ws:// or http(s):// address
in, a Browser bound to one page out. Lexmount becomes a thin layer on top
(lexmount_session, moli_session kept as an alias), and the lexmount package
an optional extra. Checked end to end on Moli, Lexmount's Chrome image,
Cloudflare Kitesurf and Browser Run, Browserbase, Chrome, headless-shell,
Lightpanda, Obscura, and self-hosted browserless, Steel, chromedp, Kernel
and Selenium Grid.
What hosted and non-Chromium browsers needed, found by running them:
- A browser that starts with no page gets one created at once, instead of
a 15s wait for a page that never comes.
- An address advertised from inside a container (ws://0.0.0.0:3000, a
container IP, no port) is pointed back at the address that answered.
- headers= for services that authenticate the handshake; user:pass@ in the
URL is sent as Basic auth.
- A 429 on connect is waited out when it asks for seconds and reported when
it asks for hours (Cloudflare's exhausted daily quota: Retry-After 57691).
- No websocket keepalive: a slow engine busy on a long page cannot answer a
ping, and the connection was dropped mid-run.
- selenium_session() for a Grid, which only hands out CDP per session.
Reading the page:
- Page text is retrieved, not truncated. The snapshot returns every row
(a table row stays one row), and evidence.py keeps the rows that bear on
the goal up to JEV_EVIDENCE_CHARS (20,000). A 6,000-character prefix
missed the answer on four of four long articles.
- Same-name links to the same destination are one control, not "1 of 5".
- Goal words that name a kind of target ("article", "link") no longer
rank candidates.
- Dead elements are found with one selector query instead of an ancestor
walk per element; output is identical on five pages, and the snapshot
costs 30-45% less on Kitesurf, which meters CPU per page.
Acting on it:
- A submit button is treated like a link: wait for the next document
before reading. On Lightpanda the agent read the page being left and
spent twenty steps re-clicking.
- A link that opens on a new page target (Kitesurf) is followed there.
- A field the page replaces after it is typed into (Wikipedia's search box
on Kitesurf) is found again and typed into again, so the form does not
submit an empty search.
- One snapshot per step: the read taken after an action is reused as the
next step's observation.
- Decision requests retry dropped connections and 500/502/504, which ended
one run in ten on otherwise healthy browsers.
- The text model is only required when a step types.
Signed-off-by: waple0820 <feng.wang@lexmount.com>
|
Claude finished @waple0820's task in 3m 30s —— View job PR Review
Two bugs found; no security issues. 1. raise RuntimeError(f"Model unavailable ({last})")This line is dead code. On the final attempt ( 2. self.session, self.target = attached["sessionId"], page["targetId"]
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Claude review completed. Review run: https://github.com/lexmount/jev-nolayout/actions/runs/35972709266. Please check Claude's inline feedback and summary comment for details.
The last line of post() could never run: every attempt returns or raises, so the reason gathered across retries was never reported. A retried status now says how many attempts it took, and the loop ends in an explicit unreachable assertion. Following a link onto a new page target replaced the session without detaching the old one, leaking a CDP session per followed link. Detach it first; a browser that caps sessions would otherwise refuse a later attach. Signed-off-by: waple0820 <feng.wang@lexmount.com>
|
Both findings were real, and both are fixed in the latest commit:
|
|
Claude finished @waple0820's task in 2m 16s —— View job PR Review
Both bugs from the prior review are fixed in this push. No new issues found.
No other bugs or security issues identified in the diff. |
There was a problem hiding this comment.
Automated Claude review completed. Review run: https://github.com/lexmount/jev-nolayout/actions/runs/35973399596. Please check Claude's inline feedback and summary comment for details.
1 participant
What
connect(endpoint)takes any CDP browser — aws://URL or anhttp(s)://address — and returns aBrowserbound to one page. Lexmount becomes a thin layer on top (lexmount_session, withmoli_sessionkept), andlexmountan optional extra.Checked end to end
Five goals, each run twice, success judged by the final URL:
What each browser needed
ws://0.0.0.0:3000, container IP, no port): point it at the address that answered.headers=;user:pass@in the URL becomes Basic auth.selenium_session().Reading the page
evidence.py, 20,000 chars). A 6,000-char prefix missed the answer on 4/4 long articles.