feat(upstream): verify the upstream certificate against configurable CAs

[nic-6443]

Description

upstream.tls.verify has been in the schema for a while, but only the kafka scheme ever read it — for an https/grpcs upstream APISIX connected without ever checking the certificate it was handed. This makes the field mean what it says for those schemes, and adds upstream.tls.ca_certs so the trust anchors can be picked per upstream instead of only globally through ssl_trusted_certificate.

{
  "scheme": "https",
  "type": "roundrobin",
  "nodes": { "127.0.0.1:8443": 1 },
  "tls": {
    "verify": true,
    "ca_certs": ["<content of ca.crt>"]
  }
}

Both settings are applied with the apisix-nginx-module upstream API (set_ssl_verify / set_ssl_trusted_store), so this needs APISIX-Runtime — on a plain OpenResty build a route carrying either field gets the usual "need to build APISIX-Runtime" error instead of a nil call.

A few details worth calling out:

• verify loses its false default. Unset now means "fall back to the nginx configuration", which is what happens today; the kafka scheme reads a nil exactly the way it read a false, so nothing changes there.
• grpcs cannot apply these in set_by_route. Both settings live in the apisix-nginx-module request context, and ngx.exec("@grpc_pass") goes through ngx_http_named_location(), which zeroes r->ctx — the same reason the client certificate is already re-applied from grpc_access_phase. So grpcs applies them there.
• @grpc_pass also gains grpc_ssl_name $upstream_host. nginx reads it only once verification is on, and without it the upstream certificate would be checked against the name of the upstream block, apisix_backend, so grpcs verification could never pass. It mirrors proxy_ssl_name $upstream_host on the proxy_pass side and is a no-op while verify is unset.
• The parsed X509_STORE is cached under the ca_certs array itself rather than under the config version. In standalone mode every resource loaded in one reload shares a modifiedIndex, so a version-keyed entry would hand one upstream's CA certificates to another.

APISIX-Runtime 1.3.17

ca_certs without a client certificate needs api7/apisix-nginx-module#126: ngx_http_apisix_set_upstream_ssl() used to apply the trusted store only inside its ctx->upstream_cert != NULL branch, and nginx loads proxy_ssl_trusted_certificate into the SSL_CTX only when proxy_ssl_verify is on at configuration time, which APISIX never sets — so an upstream that asked for verification and supplied its own CA ended up with an empty trust store.

That is the field's main use, so the runtime bump travels with this PR rather than separately; otherwise this could merge onto 1.3.16 and quietly do nothing. 1.3.17 also picks up ngx_http_ffi_client v0.1.2/v0.1.3.

Which issue(s) this PR fixes:

N/A

Checklist

• I have explained the need for this PR and the problem it solves
• I have explained the changes or the new features added to this PR
• I have added tests corresponding to this change
• I have updated the documentation to reflect this change
• I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

Tests

t/node/upstream-tls2.t covers verify on its own over https, then ca_certs with no client certificate anywhere — an upstream on :8768 whose certificate proxy_ssl_trusted_certificate does not cover, rejected with verify alone and accepted once ca_certs names its CA. That pair fails on 1.3.16 and passes on 1.3.17. The file ends with grpcs blocks for both fields; they fail without the grpc_access_phase half of the change.

t/node/upstream-mtls2.t covers ca_certs alongside a client certificate: a non-certificate entry rejected by the Admin API, a CA that does not match, one that does, and several CAs where only the last one anchors the chain.

t/node/upstream-mtls.t TEST 21 and TEST 23 set tls.verify on an https upstream to check that it does not disturb the client certificate path. That field is no longer inert there, and the test upstream serves a self-signed certificate issued for another name, so they now ask for false — what they were written to assert is unchanged.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[membphis]

[P1] Isolate upstream TLS keepalive pools by verification policy

tls.verify and tls.ca_certs are applied only during a new TLS handshake, but the keepalive pool key does not include either the verification mode or CA identity. A connection established with verify=false or a different CA can therefore be reused after the policy is tightened, so the new request does not perform the promised certificate verification. Please include the effective verification policy and a stable digest of ca_certs in the HTTPS/gRPCS pool key (or disable keepalive until isolation is implemented), and add a regression that warms a connection under verify=false/CA-A, switches the same endpoint and SNI to verify=true/CA-B, and verifies that a new handshake applies the new policy.