Shared outbound fetch helper with connect-time SSRF checks - #821
Merged
Merged
Conversation
fetchOutbound() downloads a third-party URL for a user: every hop (the URL and each redirect target) passes the SSRF guard, the socket connects only to addresses the guard checked, no env proxy, only the caller's headers (Authorization/Cookie dropped on a cross-origin redirect), one deadline for the whole exchange, a byte cap on Content-Length and on the bytes read, non-2xx as errors, and URLs without query strings in messages. ssrf.util gains ssrfGuardedLookup() and createSsrfGuardedAgents() so other callers can close the same gaps; assertSafeOutboundHost keeps its behaviour and messages. Needed by #645 (multipart file parts fetched from a URL).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
fetchOutbound()inpackages/backend/src/common/outbound-fetch.util.ts, the shared helper promised in #645 for downloading a URL that comes from a user or a model (first user: the multipart file parts in #818).What it does:
http/httpsagents whoselookupapplies the guard at connect time, so the socket only reaches addresses the guard checked. These are exported fromssrf.utilasssrfGuardedLookup()andcreateSsrfGuardedAgents().Authorization/Cookieif a redirect goes to another origin.Content-Lengthand on the bytes actually read. The body comes back as aBuffer, so it can be resent on a 401 retry.OutboundFetchErrorwithreason: 'status'. Messages useredactUrl()(origin + path, no query string, no userinfo).SsrfBlockedError, so the existing allowlist hint keeps working.assertSafeOutboundHost()is refactored onto the same internal check and keeps its behaviour and error messages.Tests: 16 new cases with local servers (redirect to an internal address, DNS answer that changes between check and connect, non-http redirect, redirect loop, cross-origin header drop, non-2xx, Content-Length and streamed cap, deadline, env proxy). Removing the guarded agents, the per-hop check or
proxy: falseeach makes at least one of them fail. Full backend suite passes locally.