Misc. bug: 10% slower prefill start speed than baseline? (using Q8 KV)

[fantasyz]

Name and Version

Beellama:
0.4.6 main

LLama.cpp:
version: 0.4.0-dev (build 10910, commit 1dfe94e)
built with GNU 15.2.0 for Linux x86_64

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

#!/bin/sh
# Launch script exported with Grun for profile: Qwen-27B-MTP-TP
CUDA_VISIBLE_DEVICES=0,1 ./llama-server \
  --api-key sk-ik-llama \
  --batch-size 2048 \
  --cache-ram 24480 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --chat-template-file /mnt/AI/models/qwen-chat-template-froggeric-22.1.jinja \
  --checkpoint-min-step 1024 \
  --ctx-checkpoints 32 \
  --ctx-size 192000 \
  --fit off \
  --gpu-layers all \
  --image-min-tokens 1024 \
  --load-mode none \
  --main-gpu 1 \
  --min-p 0.0 \
  --mmproj /mnt/AI/models/Qwen3.8-27B-mmproj-BF16.gguf \
  --model /mnt/AI/models/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-MID-HIGH.gguf \
  --parallel 1 \
  --presence-penalty 0.0 \
  --repeat-penalty 1.0 \
  --spec-draft-n-max 4 \
  --spec-draft-n-min 0 \
  --spec-draft-ngl all \
  --spec-draft-p-min 0.3 \
  --spec-draft-type-k q8_0 \
  --spec-draft-type-v q8_0 \
  --spec-type draft-mtp,ngram-map-k4v \
  --split-mode tensor \
  --temp 0.9 \
  --tensor-split 50,50 \
  --top-k 20 \
  --top-p 0.95 \
  --n-gpu-layers-draft all \
  --no-prefill-assistant \
  --reasoning-preserve

Problem description & steps to reproduce

With the same launch arguments, I noticed that prefill speed is always starting at ~2700 tps on BeeLlama and ~29xx tps on Llama.cpp. Tested with first prompt after new llama-server launch. About 30k identical system prompt + initial instruction.

First Bad Commit

No response

Relevant log output

Logs (Llama.cpp)

./run_Qwen-27B-MTP-TP.sh
00:00.26 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
00:00.26 I srv    load_model: loading model '/mnt/AI/models/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-MID-HIGH.gguf'
00:13.67 I cmn          init: llama threadpool init, n_threads = 6
00:13.83 I common_speculative_init_result: creating MTP draft context against the target model '/mnt/AI/models/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-MID-HIGH.gguf'
00:14.27 I srv    load_model: loaded multimodal model, '/mnt/AI/models/Qwen3.8-27B-mmproj-BF16.gguf'
00:14.32 I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 192000, kv_unified = 'false'
00:14.32 W set_sampler: backend sampling not supported with SPLIT_MODE_TENSOR; using CPU
00:14.32 W spec common_specu: backend offload failed for seq_id=0; using CPU sampler
00:14.34 I srv  llama_server: model loaded
00:14.34 I srv  llama_server: listening on http://127.0.0.1:8080
00:14.34 W srv  llama_server: NOTICE: server default port will be changed to :9931 in a future release
00:14.34 W srv  llama_server:         ref: https://github.com/ggml-org/llama.cpp/pull/26508
00:16.30 I slot- 0 get_availa | task -1 | selected slot by LRU, t_last = -1
00:16.31 I slot- 0 launch_slo | task 0 | processing task, is_child = 0
00:19.96 I slot- 0 print_timi | task 0 | PP, n_tokens =  10240, progress = 0.30, t =   3.5 s / 2955.5 t/s
00:20.68 I slot- 0 print_timi | task 0 | PP, n_tokens =  12288, progress = 0.36, t =   4.2 s / 2941.9 t/s
00:21.41 I slot- 0 print_timi | task 0 | PP, n_tokens =  14336, progress = 0.42, t =   4.9 s / 2923.3 t/s
00:22.16 I slot- 0 print_timi | task 0 | PP, n_tokens =  16384, progress = 0.48, t =   5.6 s / 2903.2 t/s
00:22.91 I slot- 0 print_timi | task 0 | PP, n_tokens =  18432, progress = 0.54, t =   6.4 s / 2881.9 t/s
00:23.68 I slot- 0 print_timi | task 0 | PP, n_tokens =  20480, progress = 0.60, t =   7.2 s / 2859.1 t/s
00:24.47 I slot- 0 print_timi | task 0 | PP, n_tokens =  22528, progress = 0.66, t =   7.9 s / 2834.7 t/s
00:25.28 I slot- 0 print_timi | task 0 | PP, n_tokens =  24576, progress = 0.72, t =   8.7 s / 2811.0 t/s
00:26.09 I slot- 0 print_timi | task 0 | PP, n_tokens =  26624, progress = 0.78, t =   9.6 s / 2787.1 t/s
00:26.38 I slot- 0 print_timi | task 0 | PP, n_tokens =  27245, progress = 0.80, t =  10.0 s / 2715.8 t/s
00:27.38 I slot- 0 print_timi | task 0 | PP, n_tokens =  29293, progress = 0.86, t =  10.9 s / 2698.9 t/s
00:28.23 I slot- 0 print_timi | task 0 | PP, n_tokens =  31341, progress = 0.92, t =  11.7 s / 2683.4 t/s
00:29.09 I slot- 0 print_timi | task 0 | PP, n_tokens =  33389, progress = 0.98, t =  12.5 s / 2663.1 t/s
00:29.20 I slot- 0 print_timi | task 0 | PP, n_tokens =  33555, progress = 0.98, t =  12.9 s / 2607.9 t/s
00:29.61 I slot- 0 print_timi | task 0 | PP, n_tokens =  34067, progress = 1.00, t =  13.1 s / 2597.7 t/s
00:32.87 I slot- 0 print_timi | task 0 | n_gen =    251, tg =  82.49 t/s, tg_3s =  82.81 t/s
00:33.28 I slot- 0 print_timi | task 0 | prompt eval time =   13.5 s / 34071 tok (  0.4 ms/tok, 2516.8 t/s)
00:33.28 I slot- 0 print_timi | task 0 |        eval time =    3.4 s /  293 tok ( 11.8 ms/tok,  85.0 t/s)
00:33.28 I slot- 0 print_timi | task 0 |       total time =   17.0 s / 34364 tok
00:33.28 I slot- 0 print_timi | task 0 |    graphs reused =         78
00:33.28 I slot- 0 print_timi | task 0 | draft acceptance =  53.3% ( 204 /  383 accepted), mean len =  3.4
00:33.28 I slot- 0    release | task 0 | stop processing: n_tokens = 34363, truncated = 0
00:33.52 I slot- 0 get_availa | task -1 | selected slot by LCP similarity, f_sim_best = 0.850 (> 0.100 thold), f_keep = 0.998

Logs (BeeLlama)

./run_Qwen27B-MTP-TP.sh 
00:00.41 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
00:00.42 I srv    load_model: loading model '/mnt/AI/models/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-MID-HIGH.gguf'
00:17.36 I cmn          init: llama threadpool init, n_threads = 6
00:17.55 I common_speculative_init_result: creating MTP draft context against the target model '/mnt/AI/models/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-MID-HIGH.gguf'
00:18.28 I srv    load_model: loaded multimodal model, '/mnt/AI/models/Qwen3.8-27B-mmproj-BF16.gguf'
00:18.28 I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 192000, kv_unified = 'false'
00:18.29 W set_sampler: backend sampling not supported with SPLIT_MODE_TENSOR; using CPU
00:18.29 W spec common_specu: backend offload failed for seq_id=0; using CPU sampler
00:18.29 I srv  llama_server: model loaded
00:18.29 I srv  llama_server: listening on http://127.0.0.1:8080
00:18.29 W srv  llama_server: NOTICE: server default port will be changed to :9931 in a future release
00:18.29 W srv  llama_server:         ref: https://github.com/ggml-org/llama.cpp/pull/26508
00:45.89 I slot- 0 get_availa | task -1 | selected slot by LRU, t_last = -1
00:45.89 I slot- 0 launch_slo | task 0 | processing task, is_child = 0
00:48.98 I slot- 0 print_timi | task 0 | PP, n_tokens =   8192, progress = 0.27, t =   3.0 s / 2713.6 t/s
00:49.71 I slot- 0 print_timi | task 0 | PP, n_tokens =  10240, progress = 0.34, t =   3.7 s / 2736.4 t/s
00:50.45 I slot- 0 print_timi | task 0 | PP, n_tokens =  12288, progress = 0.41, t =   4.5 s / 2739.9 t/s
00:51.20 I slot- 0 print_timi | task 0 | PP, n_tokens =  14336, progress = 0.48, t =   5.2 s / 2738.3 t/s
00:51.97 I slot- 0 print_timi | task 0 | PP, n_tokens =  16384, progress = 0.55, t =   6.0 s / 2728.8 t/s
00:52.76 I slot- 0 print_timi | task 0 | PP, n_tokens =  18432, progress = 0.61, t =   6.8 s / 2716.8 t/s
00:53.56 I slot- 0 print_timi | task 0 | PP, n_tokens =  20480, progress = 0.68, t =   7.6 s / 2701.3 t/s
00:54.37 I slot- 0 print_timi | task 0 | PP, n_tokens =  22528, progress = 0.75, t =   8.4 s / 2683.5 t/s
00:55.20 I slot- 0 print_timi | task 0 | PP, n_tokens =  24576, progress = 0.82, t =   9.2 s / 2665.5 t/s
00:56.04 I slot- 0 print_timi | task 0 | PP, n_tokens =  26624, progress = 0.89, t =  10.1 s / 2647.2 t/s
00:56.35 I slot- 0 print_timi | task 0 | PP, n_tokens =  27252, progress = 0.91, t =  10.4 s / 2613.6 t/s
00:57.37 I slot- 0 print_timi | task 0 | PP, n_tokens =  29300, progress = 0.98, t =  11.4 s / 2574.6 t/s
00:57.50 I slot- 0 print_timi | task 0 | PP, n_tokens =  29509, progress = 0.98, t =  11.6 s / 2543.9 t/s
00:57.89 I slot- 0 print_timi | task 0 | PP, n_tokens =  30021, progress = 1.00, t =  12.0 s / 2506.7 t/s
01:01.13 I slot- 0 print_timi | task 0 | n_gen = 259, tg =   85.4 t/s, tg_3s =   85.8 t/s
01:04.17 I slot- 0 print_timi | task 0 | n_gen = 399, tg =   65.8 t/s, tg_3s =   46.2 t/s
01:05.81 I slot- 0 print_timi | task 0 | prompt eval time =   12.2 s / 30025 tok (  0.4 ms/tok, 2455.5 t/s)
01:05.81 I slot- 0 print_timi | task 0 |        eval time =    7.7 s /  500 tok ( 15.4 ms/tok,  64.9 t/s)
01:05.81 I slot- 0 print_timi | task 0 |       total time =   19.9 s / 30525 tok
01:05.81 I slot- 0 print_timi | task 0 |    graphs reused =        148