How about the implementation of Mamba-3?

[Syaoran036]

https://openreview.net/pdf?id=HwCvaJOiCj

I'm really looking forward to this

[swfsql]

Not wanting to promote here, but I did give it a try with an AI in here - didn't test tho.

note: the forward is not using the semi-separable decomposition (ssd), just the same as the step; this is how ssd looks like for Mamba-2, I think.

[payk24]

@Syaoran036, here is a PR for it
#850

[zhuconv]

@Syaoran036, here is a PR for it #850

Thanks for your excellent work! Could you share how you tested your mamba3 code? Actually I trained for several steps and see the loss curve but but mamba3 left behind mamba2 and i do not know what is the problem.