Skip to content

tls: verify server_name on OpenSSL, opt-in - #8

Open
david-yu wants to merge 1 commit into
ipv6/06-tls-snifrom
ipv6/07-tls-verify-server-name
Open

david-yu wants to merge 1 commit into
ipv6/06-tls-snifrom
ipv6/07-tls-verify-server-name

Conversation

@david-yu

@david-yu david-yu commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Part 7 of 10 of the IPv6 series, split out of #1 and rebased on current scylladb/seastar master. Based on #7, so the diff here is only this PR's commit.

Why

GnuTLS checks the peer certificate against server_name. OpenSSL only checks the chain (SSL_VERIFY_PEER with no expected host), so it accepts any trusted certificate for any name. The two backends disagree, and once callers pass IP literals as server_name (previous PR) there is no way to have OpenSSL check an IP SAN.

What

  • tls: verify the peer certificate against server_name on OpenSSL, opt-in (96fad948b) — tls_options::verify_server_name, default false. On OpenSSL it sets the expected host or IP on the session's X509_VERIFY_PARAM, so a mismatch surfaces through the existing verify() path. GnuTLS keeps its always-on check; the header documents the difference. The test certificate gains an IP:::1 SAN.

The question for review

Should OpenSSL instead match GnuTLS and verify whenever server_name is set? That would break clients that connect by address to certificates without an IP SAN, so this keeps it opt-in.

Testing

Matching DNS names and IP literals (127.0.0.1, ::1, [::1]) pass; a wrong name or address raises verification_error. With the new check removed, test_verify_server_name_dns_mismatch reports "Should have gotten validation error", i.e. OpenSSL accepts any trusted certificate for any name today.

On the core-aws VM (Ubuntu 24.04, GCC 14, C++23, RelWithDebInfo, GnuTLS and OpenSSL in one tree), this PR's head builds and passes dns, ipv6, socket, rpc, httpd, httpd_openssl, tls, tls_openssl, network_interface, websocket and unix_domain with SEASTAR_TEST_REQUIRE_IPV6=1, so none of their IPv6 cases skipped.

Stack

  1. tests, ci: fail instead of skip when IPv6 is expected #2 tests, ci: fail instead of skip when IPv6 is expected
  2. dns: reach nameservers over IPv6 #3 dns: reach nameservers over IPv6
  3. dns: literals for any family, c-ares error text, link-local nameservers #4 dns: literals for any family, c-ares error text, link-local nameservers
  4. net, http, rpc: dual-stack listeners (IPV6_V6ONLY, IPv4-mapped peers) #5 net, http, rpc: dual-stack listeners (IPV6_V6ONLY, IPv4-mapped peers)
  5. net: IPv6 zones and strict literal parsing #6 net: IPv6 zones and strict literal parsing
  6. tls: do not send IP literals in the SNI extension #7 tls: do not send IP literals in the SNI extension
  7. tls: verify server_name on OpenSSL, opt-in #8 tls: verify server_name on OpenSSL, opt-in ← this PR
  8. net: ipv6_addr equality and hash, ipv4_addr(socket_address) throws instead of terminating #9 net: ipv6_addr equality and hash, ipv4_addr(socket_address) throws instead of terminating
  9. net: listen on an unspecified address binds any family #10 net: listen on an unspecified address binds any family
  10. http: build the Host header from an address in authority form #11 http: build the Host header from an address in authority form

The GnuTLS backend passes server_name to gnutls_certificate_verify_peers3,
so a certificate that chains to a trusted CA but is issued for a different
host (or a different IP SAN) fails verification. The OpenSSL backend only
ever checked the chain: SSL_VERIFY_PEER with no expected host, so any
trusted certificate was accepted for any server_name.

Add tls_options::verify_server_name. On OpenSSL it registers the expected
identity on the session's X509_VERIFY_PARAM — an IP literal (brackets and
zone stripped) via X509_VERIFY_PARAM_set1_ip_asc, a DNS name via
SSL_set1_host without partial wildcards — so the mismatch surfaces through
SSL_get_verify_result and the existing verify() path. It defaults to false:
clients that connect by address to servers whose certificates carry no IP
SAN keep working until they opt in. GnuTLS keeps its always-on check; the
header documents the difference.

The test certificate gains an IP SAN for ::1 next to 127.0.0.1, and
test_alt_names asserts both values. New cases run the echo test with the
flag set: matching DNS name and IP literals (127.0.0.1, ::1, [::1]) pass,
a wrong DNS name and wrong addresses raise verification_error.
@david-yu
david-yu force-pushed the ipv6/07-tls-verify-server-name branch from 470cf68 to 96fad94 Compare September 25, 2026 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant