local attention and cache system#291

Merged

LoserCheems merged 47 commits into

quantfrom

May 24, 2026

Commits on May 16, 2026

Refactor block min/max functions to use window size parameters directly
LoserCheems
committed
Refactor apply_mask function to streamline window size parameters and improve masking logic
LoserCheems
committed
Fix normalization logic in combine kernels to handle zero sums correctly
LoserCheems
committed
Fix parameter naming for query heads in softmax and gate threshold functions
LoserCheems
committed
Enhance num_splits_heuristic to ensure a minimum return value of 1 and add window_sizes_heuristic function for computing partitioned window sizes.
LoserCheems
committed
Fix parameter naming for query heads in launch configuration functions
LoserCheems
committed
Add window size parameters to block min/max functions and update return values
LoserCheems
committed
Refactor masking logic to prevent simultaneous application of causal and local masks
LoserCheems
committed
Refactor kernel parameters to use window size variables and correct query heads naming
LoserCheems
committed
Refactor kernel parameters to include window size variables and correct query heads naming
LoserCheems
committed
Refactor kernel parameters to include window size variables and correct query heads naming
LoserCheems
committed
Refactor _bwd_inner_dense_kernel to use window size parameters directly
LoserCheems
committed
Refactor _bwd_inner_sparse_kernel to use window size parameters directly
LoserCheems
committed
Refactor _bwd_inner_gated_kernel to use window size parameters directly
LoserCheems
committed
Refactor _fwd_dense_kernel and related functions to incorporate window size parameters for improved local attention handling
LoserCheems
committed
Refactor _fwd_sparse_kernel and _flash_sparse_attn functions to integrate window size parameters for enhanced local attention processing
LoserCheems
committed
Refactor _fwd_gated_kernel and _flash_gated_attn functions to incorporate window size parameters for enhanced local attention processing
LoserCheems
committed
Refactor _bwd_dense_kernel and related functions to incorporate window size parameters for improved local attention processing
LoserCheems
committed
Refactor _bwd_sparse_kernel and related functions to incorporate window sizes for improved local attention processing
LoserCheems
committed
Refactor _bwd_gated_kernel and _flash_gated_attn functions to incorporate window size parameters for enhanced local attention processing
LoserCheems
committed
Refactor attention functions to replace window_size parameter with is_local for improved local attention handling
LoserCheems
committed
Fix condition in _fwd_sparse_kernel to correctly handle local attention processing
LoserCheems
committed

Commits on May 19, 2026

Commits on May 20, 2026

Commits on May 24, 2026