Make the CUDA stream pool per-thread and per-device

[vuule]

Description

libcudf's stream pool was a single global pool of 32 streams shared by every thread, so concurrent threads calling fork_streams were handed overlapping streams. That creates false dependencies between unrelated work and serializes it. Each thread now owns a pool per device, created empty and grown on demand up to a cap configurable with LIBCUDF_STREAM_POOL_SIZE (default 32). A request grows the pool to twice the requested count and hands out streams starting at a rotating offset, so a nested fork_streams generally avoids the streams its caller already holds. Pools are returned to a per-device free list when a thread exits and adopted by the next thread that needs one, so applications that create and destroy many threads do not accumulate streams.

fork_streams and join_streams keep their signatures. Streams may still be used from other threads, which libcudf does in the host compression, CSV, and JSON read paths, but they must not outlive the thread that acquired them; all current call sites join before the acquiring frame returns.

Multithreaded parquet reads improve by 7-12% in GPU time at 8 threads (parquet_multithreaded_read_decode_mixed 27.4 ms to 25.5 ms, parquet_multithreaded_read_decode_string 29.5 ms to 26.8 ms), with run-to-run noise dropping from 9-11% to 4-6%. Single- and two-thread configs, ORC, and groupby are unchanged, and peak memory is unchanged.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.