Flash Attention 4

[karimknaebel]

Given that torch's FlexAttention just added a FA4 backend, do you expect NATTEN with the FlexAttention backend to become faster than your builtin kernels? :)

[alihassanijr]

It is possible. I am excited about that possibility but haven't had a chance to check out the new Flex backend yet.
If the perf improvement we expect to get from FA4 exceeds the difference between Flex's overhead with our mask and our C++ implementation, then yes there could be some improvement.