Add /load-shareurl proxy for hosts blocked on some school networks

[jpolitz]

Joe says:

We really want to make sure that shareurl works from raw.githubusercontent.com. Notably this proxies only that domain for now, specifically to make sure we support this common use case. In general I expect that less constrained users (e.g. ugrad) just fetch code happily with CORS from the browser in the long run.

This also comes with some improvements to proxying, extracted into a helper:

• Real size limits
• Streaming, not blocking waiting for the whole response
• CSP headers so if someone tries to load something executable (e.g. in SVG) it really can't run
• Some filtering on redirects to avoid weird cases of redirecting to 127.0.0.1 inside the server

Since we have existing proxying for images it's nice to improve this and have a good helper.

Claude says (lightly edited by Joe):

Some classrooms will likely load curriculum via #shareurl from raw.githubusercontent.com on networks that block GitHub. The new /load-shareurl endpoint fetches an allowlisted URL on the browser's behalf and streams the response back, so those classrooms get a working path.

A small window.fetch shim in beforePyret.js routes any fetch to an allowlisted host through the proxy automatically. That covers every browser fetch path in one place: drive.js's makeUrlFile, the url/url-file import prefetches in cpo-main.js, and the Pyret runtime's F.fetch via cross-fetch.

/downloadImg moves to share the same streaming-fetch helper, picking up nosniff

• sandbox response headers, an IP-filtering HTTP agent (request-filtering-agent), a response size cap, and a request timeout.

[blerner]

Seems reasonably straightforward.

Do we want to proxy raw.githubusercontent automatically, or only if it fails first?

Do you intend to have different timeouts in the two proxy usages? Or should those be made constants instead of magic numbers?

[jpolitz]

I'll do some magic # fixup, that's good feedback.

Re: eager vs. fallback proxying – the problem with fallback is that in the failure case it can take a long time for the first request to fail on a school content filter. So the experience is lousy.

If we end up proxying a lot, we can do some heuristic thing where we try for 2s or something and give up and proxy if that doesn't work.

[blerner]

Maybe we start proxying automatically, and then in the background, we try to request a specific file that we own, without a proxy: if it goes through, we can disable the proxy?

[jpolitz]

Good point. Can even just do proxy/non-proxy in parallel on the first request, and then disable the proxy if the regular request succeeds.

If we add more domains that becomes a dictionary of "proxy-for" different ones. But that's not the worst thing ever, and would adapt to firewall rules at schools...