Skip to content

attn decode support mtp and refine code#38

Open
shaochangxu wants to merge 1 commit intoTencent:mainfrom
shaochangxu:feature/attn_decode_mtp
Open

attn decode support mtp and refine code#38
shaochangxu wants to merge 1 commit intoTencent:mainfrom
shaochangxu:feature/attn_decode_mtp

Conversation

@shaochangxu
Copy link
Copy Markdown
Contributor

@shaochangxu shaochangxu commented Apr 2, 2026

  1. support mtp 1 and mtp 2 for bf16/fp8 decode attn
  2. support num_heads_q / num_heads_kv == 8
  3. refine attn decode code

@shaochangxu shaochangxu force-pushed the feature/attn_decode_mtp branch from dfeca33 to b7a9f3e Compare April 2, 2026 13:48
@shaochangxu shaochangxu force-pushed the feature/attn_decode_mtp branch from b7a9f3e to e64c734 Compare April 3, 2026 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant