UPSTREAM PR #17088: HTTP client: simplify and improve HTTP(S) proxy support

[loci-dev]

Note

Source pull request: openssl/openssl#17088

This fixes minor bugs and handles a design flaw with the bio_update_fn connect/disconnect callback function of type OSSL_HTTP_bio_cb_t not having a parameter of type OSSL_HTTP_REQ_CTX *, which causes trouble with connecting via an HTTPS proxy during TLS connection setup because this needs access to the proxy and no_proxy information. Moreover, the callback needs access to the SSL_CTX and to the server hostname for setting the Server Name Indication (SNI). All of this has been done so far in a pretty cumbersome and not entirely correct way.

Make sure that OSSL_HTTP_proxy_connect() is called if and only if needed with TLS.
So far, this did not correctly take into account the environment variables https_proxy and no_proxy.
Doing this fix unfortunately requires generalizing the callback function such that it can take into account whether a proxy is actually being used during connection setup.
This is done in a binary backward compatible way: by adding to OSSL_HTTP_bio_cb_t an rtcx parameter that can be used unless the details parameter is 1 on connect, which is tried first for compatibility with OpenSSL 3.0.
The generalization also makes the design of OSSL_HTTP_bio_cb_t more future-proof.

Also introduce OSSL_HTTP_REQ_CTX_proxy_connect(), an improved variant of OSSL_HTTP_proxy_connect(), which strongly simplifies the use of the callback function, e.g., in apps/.

Update of Feb 9, 2026:
I recently found a way of working around the missing OSSL_HTTP_REQ_CTX pointer that does not require adding a parameter to OSSL_HTTP_bio_cb_t, which would have incurred an API break.
This is possible by introducing OSSL_HTTP_REQ_CTX_proxy_connect(), a variant of OSSL_HTTP_proxy_connect()
that uses a OSSL_HTTP_REQ_CTX pointer, and calling this function (in case an SSL/TLS connection is being opened)
already from OSSL_HTTP_open(), where the pointer is available, rather than letting the callback do this.
Moreover, using SSL_CTX_{set,get}_ex_data() to convey the server host name via the SSL_CTX
to the callback function avoids having to pass the server name via the generic callback arg.
As a result, the callback arg can be reduced to the bare minimum: a pointer to the SSL_CTX.

Implementing this improvement I noticed that that userinfo (user name and password) that can be provided with the proxy URI was not used, neither for the HTTP nor the HTTPS proxy case. I added this.

This PR also includes several fixes:

• added several missing failure checks to OSSL_HTTP_proxy_connect()

• fix OSSL_HTTP_open() to include in the Host: header line the server port, which is needed for non-default ports

• fix the CMP app to provide the right proxy usage info

• Fix cleanup of TLS BIO via bio_update_fn callback function.
  Make app_http_tls_cb() tidy up on disconnect the SSL BIO it pushes on connect.
  Make OSSL_HTTP_close() respect this.

• OSSL_HTTP_proxy_connect(): Fix glitch in response HTTP header parsing.

It would be good to have CI tests for using an HTTP(S) proxy, but this would require providing within OpenSSL a test proxy that supports HTTP and HTTPS connections.
This PR meanwhile contains client-side unit tests for the provided improvements of HTTP(S) proxy use,
which has become possible after lifting some not really needed argument restrictions of OSSL_HTTP_open().
I tested the improved implementation also with external both HTTP and HTTPS proxies, including proxy client credential (user name and password) usage.

This PR also includes a commit that fixes an omission documenting the ok parameter of OSSL_HTTP_close().