perf: don't download media files

[nikitachapovskii-dev]

Closes #109.

This went bigger than I expected

• createRequest marks media-file URLs with skipNavigation; both request handlers return an empty result with Skipped media file instead of parsing a page.
• New src/media.ts: isMediaUrl (extension of the last path segment) and blockMediaRequests, a page.route handler aborting image/media/font.
• The blocker is registered in the Playwright pre-navigation hook and again after enableBlockingInPage, because the Ghostery handler continues everything it doesn't block.
• Pulled the skipped-result building out of checkValidResponse into pushSkippedResult, shared by both call sites. It now takes the status code from the caller — the old response.status() doesn't exist on the raw-http response.
• Docs in both Actor READMEs, unit tests for isMediaUrl, three standby tests using a request counter on the test image.

Applied to both url2m and rag

[nikitachapovskii-dev]

Scope question before I finalise this: should we also drop URLs that are media files
(e.g. query=https://example.com/photo.png), or only block images/video/fonts loaded by the page?

also should we mention this in readme?
@nicklamonov, @ruocco-l

[nicklamonov]

Headline

Metric	Requests	% of all
All requests	2,537,469	100%
Search terms	1,280,564	50.47%
URLs	1,256,905	49.53%
Media-file URLs	8,641	0.34%
— of which images/video/audio only	1,536	0.06%

Breakdown

Category	Requests	% of all	Top extensions
Documents	7,015	0.277%	pdf 6,826 · xlsx 112 · docx 39 · doc 18 · xls 14
Images	1,512	0.060%	jpg 746 · png 381 · svg 202 · avif 77 · webp 49 · jpeg 46
Archives/binaries	90	0.004%	exe 22 · gz 20 · zip 17 · deb 10 · dmg 10
Video	16	0.0006%	mp4 15 · m3u8 1
Audio	8	0.0003%	ogg 3 · mp3 3 · m4a 2

The .mp4/.jpg case from your example is far rarer than PDFs — PDF alone is 79% of all media hits. Actual video is 16 requests in 11 weeks.

Two adjacent buckets not in the 8,641, in case you want them counted:

• 596 requests (0.024%) reference a media file inside the URL's query string rather than the path (e.g. ...?outImageAddress=https%3A%2F%2F...png). Upper bound including these: ~9,237 (0.36%).
• 54 requests are search phrases (with spaces) ending in a media extension — not URLs.

Trend

Media share is growing: 0.14% in mid-May → 0.46–0.48% in late July (~3x), with absolute volume up from ~230 to ~1,450 per week. Small in share, but the direction is consistent.

Summary

Before I read the "Trend" part I was thinking to drop media requests. But then it got me thinking that maybe we shouldn't.

At the same time, we can't really convert an image to markdown. So I would probably block media even if it's a URL (not docs though).

If it's not a one-liner - let's skip it for now. If it is, let's drop media support for URLs too (images and videos only for now).

[nikitachapovskii-dev]

Great, that's exactly what the PR does + skipping audio
Documents are untouched.

[nicklamonov]

I will test on some beta version, when it's released. So approving for now.

[ruocco-l]

Good job with this! 🚀