Repository navigation
tests/tls: set 1000-day validity for self-signed CA cert - #276
Closed
travisdowns wants to merge 634 commits into
Closed
travisdowns wants to merge 634 commits into
travisdowns wants to merge 634 commits into
Conversation
The documentation in front of file::dup() says
/// \note Use on read-only files.
Enforce this requirement in the code.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Now we have two possible different handles -- one for real posix file impl and one for blockdev file impl. Currently both render the same posix_file_handle_impl out of their dup() methods, but those handles should be different types. Templatizing the posix_file_handle_impl solves this. Now the specific file impl's dup() calls posix_file_impl::do_dup<T> specifying which posix_file_handle_impl<T> it should create. The handle_impl<T> then uses T as a parameter to make the specific file impl. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
It's mostly unused now. Except for rename_scheduling_group(), but it can use internal::scheduling_group_index helper, like other similar functions do. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
So they can be used for constexpr definitions. Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Deprecated 4+ years ago with 7173966. Safe to be dropped. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Add an empty, default constructed std::source_location object to the task object and getter / setter.
Add calls to update `resume_point` variable with location of next resume location to all `await_suspend` functions and `then` functions.
Add resume point locations to `tasktrace` object. Update `formatter::format` to print source location of next resume alone with task type.
Add printing source_location of resume point for each task, that `do_dump_task_queue` logs.
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
It is used to construct a vector of temporary_buffers to pass them later into output_stream::write() zero-copy. This patch collects the vector of temporary_buffers directly, without the help of scattered message. The scattered_message, in turn, will be deprecated. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
There's another one, that gets sg_data from scheduling_group itself. The only caller of the index-based overload has sched group at hand and can use it. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Add support for fuzz testing using libFuzzer: - Add 'fuzz' build mode to configure.py and seastar_cmake.py - Add fuzzer and fuzzer_main sanitizer components - Add seastar_add_fuzz_test() CMake function - Add tests/fuzz/ subdirectory (built only in Fuzz mode) - Document fuzz testing in doc/testing.md Configure with: ./configure.py --mode fuzz --compiler clang++ Build with: ninja -C build/fuzz fuzz_tests
Add a libFuzzer-based differential fuzzer for sstring that uses std::string as an oracle. The fuzzer interprets input bytes as a sequence of operations, applying each to both strings and verifying equivalence after every operation. Coverage includes: - Construction (copy, move, fill, iterator, raw data) - Mutation (append, resize, erase, replace, insert, swap) - Search (find, find_last_of, starts_with, ends_with, contains) - Comparison (==, !=, <, >, <=, >=, compare) - Access (at, front, back, operator[], c_str, data) - Iteration (forward, reverse, distance) The fuzzer uses multiple string slots for multi-string operations and generates test data in three modes (zeros, low entropy, high entropy) to maximize code path coverage.
Add test-args input to test.yaml to allow passing additional arguments to test.py. Use this to run fuzz tests via the standard test workflow with mode=fuzz and filtering to Seastar.fuzz.* tests.
…scylladb#3210) Sometimes a performance metrics is inherently volatile and fluctuates rapidly. The great example of such a metrics is length of task queues or IO queues. Nowadays those values are exported as gauges. When monitoring system periodically "scrapes" (polls) those values, the collected data point is merely a snapshot of the queue state at that precise moment that doesn’t really show much information about what really happens in that area. Scraping happens once every several minutes, while queue length changes almost every millisecond. The extracted information is less than one thousand's part of a percent. Negligibly little information it is. In the old days the very same problem existed for IO request latencies. Back then the latency of requests in a class was tracked by saving the latency of the last completed request and exporting it as gauge. Similarly -- almost all the information about request latencies was lost. The 803e790 fixed that and introduced a monotonically growing total_queue_time value exported as counter and monitored using promql `rate()` function. This PR extends this approach onto IO queue length by introducing the integrated_length<T> wrapper. It makes little sense just to add-up the queue-lengths whenever the queue gets modified. Instead, the integrator maintains the sum of `length * duration` products taken at specific checkpoints. In it the `length` is the length of the queue when being checkpoint-ed, and the `duration` is the time passed since last checkpoint. If drawing the XY plot where X is time and Y is queue length, the accumulated value is then the square of the area below the plot. If, as promql rate does, getting two integrated values, subtracting the latest one from previous and dividing it by the duration (difference of X points) -- the result would apparently be the average Y value between two scrape points. Which is _much_ more information compared to a randomly scraped point. It's not perfect, of course. Short spikes are averaged-out and are thus swallowed by this metrics. The shorter the spike is the less noticeable it comes. Nonetheless, this gives much more information about how queue length changes over time, as compared to randomly taken gauge snapshots.
… Travis Downs Initial fuzzing support, only supports fuzzing stuff which doesn't require the reactor. Adding reactor support now. The only bug the fuzzer found for sstring was the `npos` one (see scylladb#3236), so that's good I guess. I did test it by introducing a few subtle bugs and it found them. - Add fuzz testing infrastructure using libFuzzer - Add differential fuzzer for sstring using std::string as oracle - Add 'fuzz' build mode to configure.py - Add fuzzer sanitizer components to FindSanitizers.cmake - Add `seastar_add_fuzz_test()` CMake function - Document fuzz testing in doc/testing.md The fuzzer interprets input bytes as a sequence of operations, applying each to both an sstring and std::string, then verifying equivalence. Coverage includes: - Construction (copy, move, fill, iterator, raw data) - Mutation (append, resize, erase, replace, insert, swap) - Search (find, find_last_of, starts_with, ends_with, contains) - Comparison (==, !=, <, >, <=, >=, compare) - Access (at, front, back, operator[], c_str, data) - Iteration (forward, reverse, distance) ```bash ./configure.py --mode fuzz --compiler clang++ ninja -C build/fuzz fuzz_tests ./build/fuzz/tests/fuzz/sstring_fuzz -runs=10000 ``` Closes scylladb#3237 * github.com:scylladb/seastar: test: add fuzz tests to CI workflow test: add sstring differential fuzzer test: add fuzz testing infrastructure
… Pavel Emelyanov The method in question is special-purpose helper for posix stack, that uses it to create file_desc for socket. It's hardly a good fit for the public API, pollable_fd-s are seastar internal entities. The posix stack can create file_desc on its own, without reactor help. However, it needs to guess whether or not to use SOCK_NONBLOCK flag for it, which depends on the backend in use, so some interaction with reactor is still needed. Closes scylladb#3240 * github.com:scylladb/seastar: reactor: Deprecate make_pollable_fd() net/posix: Create file_desc for sockets in-place reactor,net: Keep sock_need_nonblock boolean on posix_network_stack net/posix: Re-format constructor initializer lists
There are two issues with posix_file_impl::dup() First, despite being documented that file-handle can only be obtainer for read-only files, posix files generate handles for any file. Second, when a file-handle is converted back to a file, the result is always posix_file_real_impl, even though the handle could be obtained from e.g. blockdev_file_impl. Unfortunately, there's no easy way to test how blockdev file handles work, because block devices are not open()able for non-root. So this PR only comes with some basic tests for regular files dup()-ing Closes scylladb#3249 * github.com:scylladb/seastar: file: Templatize posix_file_handle_impl file: Don't dup() non-read-only files file: Split ..._impl::dup() implementations test: Add a simple test for dup()
…w Cybulski Adds source location of next resume point to `task` objects. When constructing `tasktrace` object, source location of resume points will be added to it. When printing, the source location of next resume point will be printed as well, hopefully improving debug experience. Example: ``` 0x2b3899 /home/y/work/scylladb-test/build/Debug/seastar/libseastar.so+0x454c364 ... -------- test_app/main.cpp:15:15 seastar::continuation<seastar::internal::promise_base_with_type<void>, ... ``` Partially fixes scylladb#2381 Closes scylladb#2707 * github.com:scylladb/seastar: coroutine.hh: disable source_location for GCC to avoid ICE reactor: improve do_dump_task_queue reporting Use source_location in `do_dump_task_queue` Update backtrace with source locations of resume points Add calls to update resume_point Add a std::source_location (resume_point) to task object.
reactor_backend_selector determines which reactor backends are available on the system. The selected backend is stored in reactor::_backend, but there is currently no way to query the selected backend type at runtime. Add reactor::get_backend_name(), which exposes the backend name via engine().get_backend_name(). Note: While smp::configure() does log the selected backend using reactor_opts, a runtime getter is useful for user visibility. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
When an executable is built on one machine and distributed, it encodes the full path (esp. with cmake) to the source files. This rarely matches the paths to the source files for whoever ends up debugging core dumps from that executable. Allow using -ffile-prefix-map to relativize the source paths, improving the debugging experience a tiny bit. The feature is exposed using a new cmake variable.
Fixes scylladb#3267 seastar::async(...) (i.e. thread_impl context) is not propagated through nested "then(...)" calls. The impl of SEASTAR_FIXTURE_THREAD_TEST_CASE need to keep the invocation of the actual test code on top level, thus cannot call the helper, but must do stuff manually. Extended test to verify this. Closes scylladb#3268
The function tries two paths for sched_wakeup_granularity_ns: first the legacy /proc/sys one, then a debugfs path. The second attempt mistakenly uses legacy_path instead of debug_fs_path. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
The function receives an std::exception_ptr argument but prints std::current_exception() to the logger instead. It's benign as it's called fron inside catch block and the same std::current_exception is passed as the argument, but the intenti was to print the argument exception anyway. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
The _shard_to_numa_node_mapping is first resize()-d to smp::count elements and is thus filled with zeros. Then the subsequent loop push_back()-s the same number more entries, resulting in a two times larger vector than needed, having wrong first half. Presumably, the intent was to _reserve_ the vector, then populate. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
The three prometheus_http_test cases drove a raw loopback socket and called input.read() once per request, expecting the entire response (headers + chunked body) to fit in a single buffer. That was a latent fragility — chunked-encoded responses can be delivered in multiple buffers, in which case only a prefix would be examined and the body substring checks would fail. Switch the client side to seastar's http::client, which parses chunked framing and lets us read the full response body via util::read_entire_stream_contiguous(). Status is checked via the parsed http::reply object instead of substring matching on the raw bytes. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
http_chunked_data_sink_impl::put(span<temporary_buffer>) used to take the fallback per-buffer path which memcpy'd every user buffer through the underlying output_stream's small _buf. Replace it with a zero-copy implementation: write the chunk-size header through the small buffered write (so it piggy-backs on output_stream's mixed-mode facility), then hand the user buffers over to output_stream::write(span) which splices them into _zc_bufs without copying, then write the trailing CRLF. The span is non-owning and only valid synchronously, so the user buffers are first moved into an owned vector that's captured by the continuation across the suspension point in write_size(). A unit test, test_chunked_sink_two_chunks_two_bufs, drives the chunked output stream with two flush()-separated writes of two buffers each and checks that the resulting wire bytes contain exactly two chunks framed with the expected size headers and CRLFs. With the previous fallback implementation the same test produces four chunks. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Make http_chunked_data_sink_impl (API ≥ 9) splice user buffers into the underlying output_stream's zero-copy queue instead of memcpy'ing them through its small buffer. Chunk-size header and trailing CRLF still go through the buffered write, piggy-backing on output_stream's mixed-mode facility. First it rewrites httpd/content_length_limit and prometheus tests on top of http::client to fix a latent fragility (single input.read() per response) that the new sink would otherwise expose. The second commit is the sink change itself, plus a unit test. Fixes scylladb#2916 Closes scylladb#3375 * github.com:scylladb/seastar: http: Make chunked data sink zero-copy tests/prometheus_http: Rewrite on top of http::client tests/httpd: Rewrite content_length_limit on top of http::client
Turns out, perfect forwarding is sufficient to support both rvalue and lvalue references without needing a separate overload for `const value_type&`. Additionally, it fixes accidental copies or moves when returning references from continuations by properly propagating the return type using `decltype(auto)`.
This is useful if you want to rename something but only if the destination file does not exist.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com> Closes scylladb#3339
Fixes the following compilation error when building with `-std=c++26`:
src/http/response_parser.rl:172:58: error: use of overloaded operator '+' is ambiguous (with operand types 'sstring' (aka 'basic_sstring<char, unsigned int, 15>') and 'string' (aka 'basic_string<char>'))
172 | _error_message = sstring("Parsing error at offset ") + std::to_string(offset) + "..."
Prepare for multiple Seastar instances per process by removing reliance on the global static smp::count. This commit adds the core infrastructure and converts the core smp module. New API: - smp::_shard_count member and shard_count() accessor - smp::_this_smp thread-local and static this_smp() accessor - smp::all_shards() returning std::views::iota(0u, _shard_count) - Free functions this_smp(), this_smp_shard_count(), and this_smp_all_shards() in smp.hh Deprecated: - smp::count (still written to in configure() for backward compat) - smp::all_cpus() In reactor.cc, smp::configure() and non-static member functions use _shard_count directly (since _this_smp is only set partway through configure). Static member functions like poll_queues() and pure_poll_queues() use this_smp_shard_count(). The log_aiocbs() method is changed from static to non-static so it can access _shard_count (it's called from configure() before _this_smp is set). The definition `unsigned smp::count = 0;` is kept for backward compatibility; no pragma is needed since [[deprecated]] only warns on use, not on definition.
Mechanically replace all remaining smp::count usages across headers, networking, apps, demos, and tests with this_smp_shard_count(), this_smp_all_shards(), or this_smp().all_shards().
On larger shard systems (128+) we are running into: ``` Apr 08 17:37:44 ip-172-31-25-67 rpk[11696]: WARN 2026-04-08 17:37:44,098 [shard 126:main] seastar - Exceptional future ignored: std::__1::system_error (error OpenSSL:1074528519, Failed to obtain new BIO index: error:400C0107:lib(128)::operation fail), backtrace: 0x238d4c2 0x6be7fff 0x6be82dd 0x239781c 0x22eb02f 0x22ebddb 0x2027766 0x2ec2b70 0x6b112b2 /opt/redpanda/lib/libc.so.6+0x94ac2 /opt/redpanda/lib/libc.so.6+0x12684f ``` which would then lead to SSL connections silently not working. This is because newer ossl versions only accept 127 entries. Previous versions would silently accept up to 255 with some silent possible benign corruption (openssl/openssl@d78519e). Share the BIO methods across shards. They don't contain state so are safe to share. Co-authored-by: Alexey Bashtanov <alexey.bashtanov@redpanda.com> Co-authored-by: Tyler Rockwood <rockwood@redpanda.com> Closes scylladb#3351
Some of those tests are copies of each other with minor changes, wrapping common parts into a helper is worth it. Also, two tests are removed, as they become exact copies of yet another one after variadic futures extinction. Closes scylladb#3342 * github.com:scylladb/seastar: tests: Generalize async connection-based scheduling RPC tests tests: Generalize sync connection-based scheduling RPC tests tests: Remove redundant variadic/nonvariadic RPC tuple tests tests: Generalize max timeout RPC tests
When a request entry sits at the front of the outgoing queue and its
timeout deadline expires before send_entry() is called, the timer
callback fires but does nothing -- withdrawal is guarded by
'it != _outgoing_queue.begin()'. The entry stays in the queue with
a stale _expiry. send_entry() then computes
expire - timer<rpc_clock_type>::clock::now()
which is a negative duration. Assigning the negative int64_t result
to uint64_t left wraps around to a near-infinite value (~2^63), which
the server receives and interprets as a timeout far in the future,
causing the server to sleep for an effectively infinite period waiting
for a deadline that already passed on the client.
Fix: in send_entry(), if the timer was set and the deadline has already
passed, drop the entry without sending -- equivalent to the timer
callback having arrived just a little later than it did. The server
never receives the corrupted timeout, and the client-side reply handler
times out normally and delivers timeout_error to the caller.
The expiration check is placed outside the 'buf.size && _propagate_timeout'
block intentionally: whether a timed-out entry should be dropped is a
client-side liveness decision, independent of whether the timeout value
is encoded into the wire frame. Even when timeout propagation is
disabled, sending an already-expired request only makes the server do
work for a caller that will receive timeout_error regardless.
For the bug to surface two rare events must coincide:
1. The future returned by the previous entry's send completes in a
way that schedules the next entry's continuation as a new task
rather than calling it immediately (i.e. it is not a ready future
at the .then() site).
2. That scheduling happens at a preemption point, so the reactor
runs a poll cycle -- advancing the lowres clock and firing pending
timer callbacks -- before the continuation calls uncancellable()
and send_entry().
The bug cannot be deterministically reproduced without modifying the
production code-flow, because the .then() continuation on a ready
future is called inline in the current implementation (condition 1 is
never met in practice without an artificial yield). To surface it
in a test one would need to inject a force_poll()+yield() step into
connection::send() between the two continuations, and run a
CPU-hogging scheduling group that busy-spins long enough to advance
the lowres clock past the request deadline. That approach requires
modifying production code for test purposes, so no automated test is
added here; the fix is a straightforward defensive check.
Fixes scylladb#672
Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Apparently, the ninja-build package on Ubuntu 26.04 provides a ninja command but not ninja-build. On the other hand cmake apparently looks for ninja-build. Detect this situation and steer cmake in the right direction when it happens. Closes scylladb#3376
smp::count is now a static variable as a convenience - it can be accessed from anywhere without requiring any object to access it. But that means we can only have one smp::count per process. This stands in the way for running multiple Seastar instances in a single process, which is desirable for testing (cluster-in-a-box). Remove this limitation by making smp::count a private non-static member named _shard_count. smp::count is kept for compatibility, but deprecated. Similarly smp::all_cpus() gains a non-static equivalent smp::all_shards(). To simplify access from within reactor threads, which is the vast majority of accesses, we add convenience this_smp(), this_smp_shard_count(), and this_smp_all_shards(). Two uses of smp::count before it is initialized are converted to FIXMEs. The tree is converted to use the new accessors. Closes scylladb#3355 * github.com:scylladb/seastar: smp: convert all smp::count usages to instance-aware alternatives smp: add per-instance shard_count and this_smp() infrastructure disk_params: document pre-init smp::count access with explicit 0 reactor_backend: document pre-init smp::count access with explicit 0 tests: alien_test: pass shard count to alien thread explicitly
vla.hh uses malloc() and uintptr_t, but doesn't include their respective headers. Fix this. Closes scylladb#3378
OpenSSL's SSL_get_error relies on the per-thread error queue to classify failures. In seastar's cooperative scheduling model, multiple TLS sessions share the same thread. If one session leaves stale errors on the queue and then yields, another session's SSL_get_error call may misclassify the error — e.g. reporting SSL_ERROR_SYSCALL instead of SSL_ERROR_SSL or vice versa — because SSL_get_error peeks at the oldest error on the queue to decide the classification (ssl_lib.c, ossl_ssl_get_error). We have seen error queue contamination cause issues in practice that we had to patch in the past. The current code is believed to be correct — all error paths drain the queue before scheduling points — but we add these checks to aid debugging if a regression is introduced in the future. Add verify_clean_error_queue() checks before every SSL_do_handshake, SSL_write_ex, SSL_read_ex and SSL_shutdown call. If stale errors are found, the first entry is logged at warn level and an assertion fires to surface the problem in tests.
…esztegi-Nagy OpenSSL uses a per-thread error queue to communicate failure details. In seastar's cooperative scheduling model multiple TLS sessions share the same thread, so if stale errors are left on the queue across a scheduling point, `SSL_get_error` on an unrelated session can misclassify the result (e.g. reporting `SSL_ERROR_SYSCALL` instead of `SSL_ERROR_SSL`) because it peeks at the oldest error on the queue to decide the classification. We have hit this in practice at redpanda and it resulted in hard-to-debug customer issues. This PR hardens the OpenSSL error queue handling. See the individual commit messages for the details. **Drain stale errors on error paths** - `net/tls: use make_openssl_error for BIO creation failure`. Aligns the `BIO_new` failure with every other error path in the session constructor. - `net/tls: drain error queue on unexpected error codes`. Ensures the default switch cases in error handlers drain the queue rather than silently discarding errors. **Drain stale errors after successful operations** - `net/tls: clear error queue after successful SSL_CTX_new`. `SSL_CTX_new` can return success while leaving errors from partially-failed system config parsing (e.g. an invalid `Ciphersuites` value in `openssl.cnf`). - `net/tls: clear error queue after successful SSL operations`. `SSL_shutdown`, `SSL_do_handshake`, `SSL_write_ex`, and `SSL_read_ex` can return success while leaving errors from internal BIO write failures that OpenSSL absorbed. Root cause is a return-value translation gap in `bio_write_intern` where our `bwrite` callback's 0-means-failure is misclassified as success by `tls_retry_write_records`. See the commit message for detailed call chains. **Assert the queue is clean** - `net/tls: assert clean error queue before SSL operations`. Adds `verify_clean_error_queue` checks before every `SSL_do_handshake`, `SSL_write_ex`, `SSL_read_ex`, and `SSL_shutdown` call as a safety net to surface regressions in tests and to make issues easier to debug in production. Closes scylladb#3369 * github.com:scylladb/seastar: net/tls: assert clean error queue before SSL operations net/tls: clear error queue after successful SSL operations net/tls: clear error queue after successful SSL_CTX_new net/tls: drain error queue on unexpected error codes net/tls: use make_openssl_error for BIO creation failure
The value is going to become more than yes/no switch and this patch prepares for that. For now only two values -- yes and no. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Continuation of the previous patch -- now the filesystem is also prepared to export its "nowait" facility as enum class. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
This is the third state, which allows RWF_NOWAIT flag for read requests only. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
The nowait mode to use depends on three sources of information: the filesystem itself, the kernel version and the reactor option (CLI switch). Previously the selection was between yes and no modes only; this patch wires the read_only mode detection and activation. The --linux-aio-nowait CLI option default is changed from the old kernel-version whitelist to an empty value to distinguish between "user explicitly set the CLI switch" and "user left it at default" later. When auto-detecting, the kernel version determines the mode: - kernel >= 7.0: yes - kernel >= 6.0: read_only - kernel >= 4.13: yes - otherwise: no Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
… Pavel Emelyanov Whether or not to set RWF_NOWAIT flag for iocb is controlled by the nowait_works bit that travels a long journey from reactor option via fs_info, then file, then struct request (or more exactly -- one of its united sub-structures). Nowadays this bit is boolean, but we've seen that it might be ultimately broken for writes and work for reads. This PR turns the nowait_works bit on file and fs_info to be a tri-state enum. The request struct still carries it as bit (because they are independent for reads and writes) as well as reactor option remains boolean. "By default" (i.e. -- on modern kernels with no --linux-aio-nowait option) the mode activates to read_only for XFS. As a result, reads are submitted with the flag and writes are submitted without it. With that we trade guaranteed write-retry for potential delay in the kernel submitting it. Hopefully, chances for the latter are low enough. refs: scylladb#2974 Closes scylladb#2987 * github.com:scylladb/seastar: file: Change nowait_works mode detection file: Introduce read-only nowait_mode filesystem: Make nowait_works bit a enum class too file: Make nowait_works bit a enum class
parallel_for_each_state holds pending futures in a std::vector<future<>>, so we pay a heap allocation for the buffer even when the range has just one element. coroutine::parallel_for_each already sidesteps this with an inline-capacity small_vector; do the same here. With boost::container::small_vector<future<>, 5>, ranges of up to 5 elements keep their pending futures inline on the state object -- no separate heap allocation for the vector. Larger ranges still spill to the heap, so n > 5 looks the same as before. Quick numbers from parallel_for_each_perf (release, single shard): suspend_1: 4.000 -> 3.000 allocs/elem (vector buffer gone) suspend_2: 2.500 -> 2.000 allocs/elem; 32.65 ns -> 30.57 ns (-6%) suspend_10/100: unchanged (n > 5, falls back to heap) The timing win on n=1 is in the noise; the alloc-count reduction is the durable improvement. There is still a ~30 ns gap vs coroutine::parallel_for_each, but that's architectural: parallel_for_each_state uses a promise<>/future<> completion bridge that adds a schedule() round-trip, while the coroutine variant holds the consumer's coroutine_handle and calls .resume() on it inline. Worth a separate look. Signed-off-by: Kefu Chai <k.chai@proxmox.com>
The function's sole purpose was releasing tokens back to the token bucket (_group.release_capacity(cap)), implementing the dispatch -> completion backlink that rate-limited dispatch to the observed completion rate. Commit 557017f removed that call when the 2-bucket backlink was replaced by the flow-rate monitor in io_queue, leaving the function permanently empty with an ignored parameter. In this change, we * Remove the declaration, the out-of-line definition in fair_queue.cc, * Remove all call sites in io_queue.cc and the tests, * Remove the now-unused lambda captures in fair_queue_perf.cc. Signed-off-by: Kefu Chai <k.chai@proxmox.com> Closes scylladb#3382
Two redundant clock reads were removed from the hot scheduling path. * end_task_run() drop unused parameter cpu_stall_detector::end_task_run() accepted a sched_clock::time_point argument that its body never used -- only the signal fence and the _last_tasks_processed_seen store are there. The caller in run_some_tasks() was therefore evaluating now() (one vDSO RDTSCP + multiply/shift) and discarding the result. Remove the parameter. Note: the compiler already eliminated the dead clock_gettime call within the same TU at -O2, so this is a correctness/API fix rather than a runtime win -- future callers in other TUs could not rely on that optimisation. * Thread the quantum-start timestamp into run_tasks_impl() run_some_tasks() called now() for start_task_run(), then run_tasks() called now() again as its first action to initialise t_run_completed. These two reads were back-to-back with no task execution between them, so they measured the same instant twice. Introduce a private run_tasks_impl(sched_clock::time_point t_initial) that accepts the already-computed timestamp instead of calling now() again. run_some_tasks() shares one now() value across both start_task_run() and run_tasks_impl(). The public virtual run_tasks() keeps its zero-argument signature (required by the sched_entity interface) and simply calls run_tasks_impl(now()). On Linux with clocksource=tsc, steady_clock::now() resolves to a vDSO fast path: RDTSCP + multiply/shift in userspace with no syscall. The cost is ~20-40 cycles per call. This change eliminates one such call per scheduling quantum from the run_some_tasks() entry point. Signed-off-by: Kefu Chai <k.chai@proxmox.com> Closes scylladb#3383
Certificate compression compresses a 3.5k cert using 350k worth of buffers in zlib-ng. Since Seastar applications don't favor large allocations, just disable compression. Since Seastar applications favor long-lived connections, this has little impact on actual data movement. Since this was added only in openssl 3.2, we guard it with a #ifdef. Fixes scylladb#3364 Closes scylladb#3385
The mtls CA cert generated by `seastar_sign_cert` (when no CA is supplied) omits `-days`, which makes openssl default to a 30-day validity. Leaf certs signed by this CA already use `-days 1000`, but the CA expires first and signature verification fails with "certificate has expired" once the build directory is older than 30 days. Fresh CI builds never hit this because ninja is satisfied as long as the .crt file exists — there is no rebuild trigger when a cert expires in place. Persistent dev/build trees are the ones that break. Set the CA's validity to 1000 days to match the leaf certs. Signed-off-by: Travis Downs <travis.downs@redpanda.com>
Member
Author
|
(this was meant for upstream) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The mtls CA cert generated by
seastar_sign_cert(when no CA is supplied) omits-days, which makesopenssl req -new -x509default to a 30-day validity. The leaf certs signed by this CA already use-days 1000, but the CA itself expires first, so signature verification fails withcertificate has expiredonce the build directory is older than 30 days.Fresh CI builds never hit this because ninja considers the rule satisfied as long as the
.crtfile exists — there is no trigger to regenerate a cert that has expired in place. Persistent dev/build trees are the ones that break, with errors like:This sets the CA's validity to 1000 days, matching the leaf certs.
Test plan
./test.py --mode dev --name tls_opensslpasses after regenerating the certs (rm build/dev/tests/unit/mtls_*.{crt,key,csr} && ninja -C build/dev mtls_certs)openssl x509 -in build/dev/tests/unit/mtls_ca.crt -noout -dates→notAfter=Jan 30 ... 2029 GMT