Skip to content

⚡ perf: Optimize high concurrency test by wrapping payload in Arc#135

Open
panayang wants to merge 1 commit into
mainfrom
perf-arc-encoded-payload-13273604940859237615
Open

⚡ perf: Optimize high concurrency test by wrapping payload in Arc#135
panayang wants to merge 1 commit into
mainfrom
perf-arc-encoded-payload-13273604940859237615

Conversation

@panayang

Copy link
Copy Markdown
Member

💡 What:
Replaced Vec<u8> with Arc<Vec<u8>> for the encoded payload in tests/high_concurrency.rs. This involved updating PartialReader's data field and run_worker's argument type, along with wrapping the serialized payload in an Arc inside both test functions.

🎯 Why:
The high concurrency test was performing a deep clone of the encoded payload vector inside the task spawning loop (encoded.clone()). In a tight loop spawning 5,000,000 tasks, this leads to massive, expensive memory allocations that drastically affect performance. Wrapping the payload in an Arc (Atomic Reference Count) ensures only the reference count is incremented upon cloning, avoiding deep copies of the byte array and saving significant CPU and memory allocation overhead.

📊 Measured Improvement:
Running cargo test --release --test high_concurrency before and after the change showed significant performance improvements:

Baseline:

  • Batching: Processed 5000000 tasks in 8.17s (612166.94 tasks/sec)
  • No Batching: Processed 5000000 tasks in 23.58s (212024.57 tasks/sec)

After Optimization:

  • Batching: Processed 5000000 tasks in 5.67s (881996.51 tasks/sec) -> ~44% faster
  • No Batching: Processed 5000000 tasks in 7.85s (637342.51 tasks/sec) -> ~200% faster

PR created automatically by Jules for task 13273604940859237615 started by @panayang

Avoid expensive Vec clone in no-batching high concurrency test loops by
wrapping `encoded` payload in an `Arc`. This avoids massively allocating when
spawning concurrent async tasks.

Performance measurement (release mode):
Before:
  batching: 612166.94 tasks/sec (8.17s)
  no batching: 212024.57 tasks/sec (23.58s)
After:
  batching: 881996.51 tasks/sec (5.67s)  -> ~44% improvement
  no batching: 637342.51 tasks/sec (7.85s) -> ~200% improvement

Co-authored-by: panayang <223123845+panayang@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant