HTTP fetcher vulnerable to SSRF and redirect loops - no URL validation or redirect limits

[Elshayib]

Bug Report

Issue

In fetcher/http.go, the HTTP.Fetch method has no protection against SSRF (Server-Side Request Forgery) or infinite redirect loops.

Location

fetcher/http.go lines 30-55

Details

1. SSRF Vulnerability:
The fetcher blindly follows any URL without validation:

func (h *HTTP) Fetch(ctx context.Context, source string) (io.ReadCloser, error) {
    req, err := http.NewRequestWithContext(ctx, http.MethodGet, source, nil)
    // ...
    resp, err := h.httpClient().Do(req)
    // ...
}

The code even documents this issue in comments: "HTTP performs no URL filtering; it will follow any URL it receives, including internal or private network addresses. Applications that accept URLs from untrusted input should supply a Client whose http.Transport rejects private/loopback addresses at dial time"

However, the default client (http.DefaultClient or a new client with just timeout) does not reject private addresses. This means:

• http://localhost:8080 - accessible
• http://169.254.169.254/latest/meta-data/ - AWS metadata endpoint accessible
• http://10.0.0.1/admin - internal network accessible
• file:///etc/passwd - file protocol potentially accessible

2. No Redirect Limit:
The default http.Client follows redirects indefinitely (up to 10 by default in Go, but can be configured). There's no explicit limit set, and no protection against redirect loops.

3. No Response Size Limit:
Large responses can cause memory exhaustion.

Reproduction

fetcher := &fetcher.HTTP{}
// SSRF - access internal metadata
fetcher.Fetch(ctx, "http://169.254.169.254/latest/meta-data/iam/security-credentials/")
// Redirect loop - if server redirects to itself
fetcher.Fetch(ctx, "http://evil.com/redirect-loop")
// Large response - memory exhaustion
fetcher.Fetch(ctx, "http://evil.com/10gb-file")

Suggested Fix

func (h *HTTP) httpClient() *http.Client {
    if h.Client != nil {
        // Apply security hardening even to custom clients
        return h.Client
    }
    return &http.Client{
        Timeout: DefaultHTTPTimeout,
        Transport: &http.Transport{
            DialContext: (&net.Dialer{
                Timeout:   30 * time.Second,
                KeepAlive: 30 * time.Second,
            }).DialContext,
            ForceAttemptHTTP2:     true,
            MaxIdleConns:          100,
            IdleConnTimeout:       90 * time.Second,
            TLSHandshakeTimeout:   10 * time.Second,
            ExpectContinueTimeout: 1 * time.Second,
            // Block private addresses
            DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
                // Parse host:port
                host, _, err := net.SplitHostPort(addr)
                if err != nil {
                    host = addr
                }
                // Block private/loopback ranges
                if isPrivateIP(host) {
                    return nil, fmt.Errorf("blocked private address: %s", host)
                }
                return (&net.Dialer{Timeout: 30 * time.Second}).DialContext(ctx, network, addr)
            },
        },
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            if len(via) >= 10 {
                return fmt.Errorf("too many redirects")
            }
            // Optionally: block redirects to private addresses
            if isPrivateURL(req.URL) {
                return fmt.Errorf("redirect to private address blocked")
            }
            return nil
        },
    }
}

Impact

High - if this library is used to fetch user-supplied URLs, it can lead to:

• Internal service enumeration
• Cloud metadata service access (AWS/GCP/Azure)
• File system access via file:// protocol
• Denial of service via large responses or redirect loops

[sonupreetam]

Thanks for the report @Elshayib. I believe this is largely addressed by #68, which resolved #52. cc @jpower432 @eddie-knight flagging for your call.

[jpower432]

Hi @Elshayib Thanks for the report. As @sonupreetam mentioned this may have been resolved through the #68.

In the PR we added documentation and a runnable example (ExampleURI_ssrfSafe) showing how to configure dial-time SSRF mitigation with a custom transport

We decided intentionally not to embed SSRF policy in the fetcher itself as the right set of blocked address ranges can depend on the caller's environment, so this is left to the caller via the Client field.

Is there something specific you'd like to see changed beyond what #68 covered or perhaps we can better surface the example? Happy to discuss if you think there's still a gap.