Skip to content

optimize GPU execution and eliminate CPU sync bottlenecks - #14

Open
BehradFM wants to merge 2 commits into
ViCCo-Group:masterfrom
BehradFM:master
Open

optimize GPU execution and eliminate CPU sync bottlenecks#14
BehradFM wants to merge 2 commits into
ViCCo-Group:masterfrom
BehradFM:master

Conversation

@BehradFM

Copy link
Copy Markdown

Hi there

I was training this model on my GPU and noticed some performance bottlenecks caused by frequent CPU-GPU synchronizations.
I've refactored parts of the code to keep tensors on the device and maximize hardware utilization.

These optimizations resulted in a massive performance boost, making the overall training process approximately 2.5x faster!

  • Batching Optimization: Move the identity matrix ("I") used for batching directly to the device (CPU,CUDA).
  • Native Operations: Rewrote "choice_accuracy" function.
  • Asynchronous Execution: "Disabled CUDA_LAUNCH_BLOCKING" to allow PyTorch to execute operations asynchronously.

I have also verified that the code remains fully compatible and works on both CPU and GPU device.

I would be really happy to see this merged so that others in the community can benefit from the faster training times!
Thanks for the great repository!

@BehradFM

Copy link
Copy Markdown
Author
  • I have just recently Refactored l1_regularization function to dynamically initialize the accumulator tensor on the target device.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant