Refactor activation functions for optimization

[LoserCheems]

Summary

• Optimize activation functions by implementing faster exp2 and sigmoid computations.

Root Cause

• Previous implementations used standard library functions that were not optimized for performance.

Changes

• Refactored activation functions to utilize optimized hardware approximations for exp2 and sigmoid.

Reproduction

• No specific bug to reproduce; performance improvements can be validated through benchmarks.

Tests

• Performance tests should be conducted to ensure optimizations yield expected improvements.

Compatibility

• No breaking changes introduced; existing functionality remains intact.

Checklist

• Linked issue provided [FEATURE REQUEST] Next-Generation Trainable Sparse Attention Mechanism #219 [FEATURE REQUEST] Triton-based efficient multi-platform, multi-variant attention #222
• Adds or updates tests
• Updates docs if needed
• No perf regressions