UPSTREAM PR #30967: Unicode codepoints fit in 32 bit values.

[loci-dev]

Note

Source pull request: openssl/openssl#30967

Using "unsigned long" for these makes no sense, and has the result of using a value that is potentially
far too big on some platforms, and resulting in "always true" warnings on others if you check the range,
as noticed by @mbroz.

Instead of using the "whatever size the platform feels like" along with prayer and clean living, how about
we just use a uint32_t for everything.

The only vestiges of unsigned long then end up being the undocumented UTF8_[putc|getc] APIs,
which have been around since the days of yore when any function you wrote should be made public
because that was the way.. Therefore we deprecate these functions with no planned replacement,
if someone screams they must have them we can expose the internal uint32_t using interface in a
follow on, but let's see if this can be flensed.

Checklist

• documentation is added or updated
• tests are added or updated

[loci-review]

Overview

Analysis of 20,084 functions across OpenSSL binaries reveals type safety refactoring (unsigned long → uint32_t for Unicode) with minimal performance impact. Modified: 14 functions (0.07%), New: 2, Removed: 0.

Power Consumption:

• libcrypto.so: +0.005% (+13.78 nJ) - negligible
• libssl.so: 0.0% (no change)
• openssl: 0.0% (no change)

Commits: e117289 "Unicode codepoints fit in 32 bits. Use uint32_t" and 6630c34 (Perl build adjustments) by Bob Beck.

Function Analysis

Significant Improvements:

• UTF8_putc (libcrypto.so): Response time -13.4% (-9.5 ns), throughput time -91.0% (-64.8 ns). Refactored to wrapper calling optimized ossl_utf8_putc_internal (58 ns vs 71 ns baseline).
• out_utf8 (libcrypto.so): Response time -12.4% (-12.2 ns). Benefits from faster internal UTF-8 encoding function.
• cpy_utf8 (libcrypto.so): Response time -11.9% (-11.7 ns). Same UTF-8 optimization benefit.
• do_buf.constprop.0 (libcrypto.so): Response time -2.9% (-103 ns). UTF-8 function improvements (10-13 ns per call) accumulate across multiple invocations.

Notable Regressions:

• UTF8_getc (libcrypto.so): Response time +17.4% (+15.4 ns), but throughput time -70.9% (-62.8 ns). Wrapper overhead acceptable for modular design; actual work efficiently delegated to internal function.
• ASN1_mbstring_ncopy (libcrypto.so): Response time +1.5% (+61 ns), throughput time +8.0% (+66 ns). Despite UTF-8 callback improvements, error handling overhead (+4-5 ns per function) and entry block costs offset gains.

Source Code Correlation: Type migration to uint32_t enables 32-bit register optimizations and semantic correctness. API consolidation (UTF8_getc/putc → ossl_utf8_getc_internal/ossl_utf8_putc_internal) yields 11-18% per-call improvements. Changes confined to ASN.1 encoding module (certificate parsing, string conversion), not cryptographic hot paths.

Other analyzed functions showed minor changes consistent with type safety improvements.

Flame Graph Comparison

Function: UTF8_getc@@OPENSSL_4.0.0 (libcrypto.so) - illustrates wrapper refactoring pattern

Base version:

Target version:

Base version executes all UTF-8 decoding inline (88.6 ns self-time). Target delegates to ossl_utf8_getc_internal (78 ns), introducing 15 ns wrapper overhead but achieving 70.9% throughput reduction through modular design. The 17.4% response time increase is acceptable for improved maintainability and backward compatibility.

Additional Findings

No Critical Path Impact: All changes are in ASN.1 string processing (certificate parsing, PKCS#12 conversion), not in performance-critical cryptographic operations (AES, SHA, SSL record processing). Modified functions execute during certificate validation and import/export—infrequent operations where nanosecond changes are negligible compared to millisecond-scale I/O and validation costs.

Code Quality Benefits: uint32_t provides correct Unicode semantics (max U+10FFFF fits in 21 bits), improves portability across 32/64-bit platforms, and enables compiler optimizations. Internal API consolidation supports OpenSSL 4.1 deprecation strategy while maintaining backward compatibility.

💬 Questions? Tag @loci-dev