speed benchmark on M3 Max Macbook Pro with different fan control.

[taozhiyuai]

Based on the 6 benchmark runs of deepseek-v4-flash on the M3 Max 128GB, the comparison between the two cooling strategies is unequivocal:

---

📊 Core Conclusion: Maximum Fan Speed Dominates Auto Fan Control

Metric	Auto Fan (Runs 1–3)	Max Fan (Runs 4–6)	Improvement
Avg Prefill Speed	157.0 TPS	178.2 TPS	+13.5%
Avg Generation Speed	14.74 TPS	16.73 TPS	+13.5%
Prefill Variability (CV)	28.5%	26.1%	More Stable
Generation Variability (CV)	21.1%	15.3%	More Stable

---

🔍 Key Findings

1. The Auto Fan Strategy Suffers from Severe Performance Jitter

• In the 20K–40K context window range, Runs 2 and 3 experienced a catastrophic performance cliff.
• The worst data point occurs at ctx=28672:
  • Run 2: Prefill crashed to 124.6 TPS (down 40% from Run 1’s 207.8), Gen crashed to 10.8 TPS (down 44%).
  • Run 3: Prefill 142.7 TPS, Gen 12.1 TPS.
• This indicates the auto fan fails to dissipate heat under sustained load, triggering aggressive thermal throttling.

2. Max Fan Speed is Remarkably Consistent

• Runs 4, 5, and 6 trace nearly identical curves for both prefill and generation speeds, degrading smoothly as context length increases.
• At the same critical point (ctx=28672), the three runs recorded 215.4, 216.0, and 224.0 TPS respectively—minimal variance.

3. The "Sweet Spot" for Gains is the 25K–35K Context Range

• This is where the auto fan’s thermal throttling is most severe, making the max fan advantage most pronounced:
  • Prefill peak gain: +60.1 TPS (+38%) at ctx=28672
  • Generation peak gain: +5.5 TPS (+39%) at ctx=28672

4. Long-Context Gains Persist (70K+)

• Even at 102K context length, max fan maintains a stable 11–15% speed advantage over auto fan.

---

💡 Recommendation

For sustained inference workloads like deepseek-v4-flash on the M3 Max 128GB:

Lock the fans to maximum speed. The auto fan strategy not only delivers lower average throughput, but more critically, it triggers unpredictable thermal throttling in the mid-range context window (20K–40K), causing wild performance jitter. Maximum fan speed provides both higher average performance and run-to-run consistency.

Detailed comparison charts are available for download:

Full Fan Strategy Comparison

Critical Region Stability Analysis

###AUTO fan control

[m3_max_128gb.1.csv](https://github.com/user-attachments/files/27739216/m3_max_128gb.1.csv) [m3_max_128gb.2.csv](https://github.com/user-attachments/files/27739220/m3_max_128gb.2.csv) [m3_max_128gb.3.csv](https://github.com/user-attachments/files/27739221/m3_max_128gb.3.csv)

MAX fan speed

[m3_max_128gb.4.csv](https://github.com/user-attachments/files/27739230/m3_max_128gb.4.csv) [m3_max_128gb.6.csv](https://github.com/user-attachments/files/27739252/m3_max_128gb.6.csv)

[taozhiyuai]

m3_max_128gb.5.csv

[taozhiyuai]

Based on the 6 benchmark runs, here is the optimal context window sizing recommendation for running deepseek-v4-flash on the M3 Max 128GB:

---

📊 Core Finding: There is a Clear Performance Cliff at ~30K Tokens

The data reveals a sharp inflection point where generation speed collapses. Using the max fan runs (4–6) as the reliable baseline:

Zone	Context Range	Gen Speed	User Experience
🟢 Smooth	0 – 22K	≥ 20 TPS	No perceptible latency
🟡 Acceptable	22K – 35K	18–20 TPS	Slight delay, still fluid
🟠 Slow	35K – 70K	15–18 TPS	Noticeable waiting
🔴 Sluggish	70K – 102K	< 15 TPS	Significantly impacts UX

---

🎯 Two "Sweet Spots" Depending on Your Use Case

Sweet Spot A: Speed-First (~28K tokens)

• Gen: 19.53 TPS | Prefill: 218.5 TPS | KV Cache: 0.39 GB
• This is the last data point before the performance cliff. The decay rate accelerates dramatically after 28K (losing ~4–5 TPS per 1K token increase).
• Best for: Code completion, interactive writing, chatbots, any scenario where low latency matters more than massive context.

Sweet Spot B: Balanced (~65K tokens)

• Gen: 15.09 TPS | Prefill: 148.8 TPS | KV Cache: 0.86 GB
• Still above the 15 TPS "sluggishness threshold." The memory footprint is trivial on a 128GB machine.
• Best for: Long-document RAG, analyzing 50–80 page PDFs, cases where context breadth outweighs pure speed.

---

📋 Practical Recommendations by Scenario

Scenario	Recommended Context	Why
Chat / Conversational AI	8K – 16K	Gen ~20.5 TPS, memory footprint negligible (~0.2 GB)
Code Completion / Copilot	24K – 28K	Right at the edge of the cliff; safest high-speed zone
Long Document Analysis / RAG	48K – 65K	Acceptable 15+ TPS, enough for substantial documents
Book-length / Extreme Cases	80K – 100K	Only when absolutely necessary; Gen drops to ~14.5 TPS
> 100K	❌ Not recommended	Diminishing returns; speed too slow for good UX

---

⚠️ Key Insight: Memory is NOT the Bottleneck

On the M3 Max 128GB, the KV Cache at 102K context is only 1.33 GB — completely negligible. The constraint is compute bandwidth, not RAM capacity. The generation speed decays because the attention mechanism must traverse an increasingly large KV cache per token, not because the system is running out of memory.

Bottom line: Set your context window based on latency tolerance, not memory limits. For this machine, 28K is the hard ceiling for fluid interaction, and 65K is the practical maximum for productive use.

Context Window Recommendation Chart

[taozhiyuai]

DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf

#!/bin/bash

./ds4-bench
-m ds4flash.gguf
--prompt-file speed-bench/promessi_sposi.txt
--ctx-start 2048
--ctx-max 102400
--step-incr 2048
--gen-tokens 128 > speed-bench-tzy/m3_max_128gb.csv

python3 speed-bench/plot_speed.py speed-bench-tzy/m3_max_128gb.csv --title "M3 Max t/s from zy TAO"

[phansel]

Your title hides the problem here. The M3 Max Macbook Pro is a laptop, and almost all laptops are thermally limited. There is no M3 Max Mac Pro; only an M2 Ultra or Intel model exists.

#126 (comment)

It is strange to go to the lengths of proposing an optimal context window based on thermal throttling. The external conditions and the initial conditions will determine how hot the CPU gets and consequently how slow it will throttle. During multi-turn conversations there might be time for the laptop to cool down as the user responds, or the laptop might heat up during code compilation, and so on.

An issue like this would be more informative if it were:

• shorter. discussion of memory footprint is completely irrelevant to the topic. AI slop.
• accompanied by environmental data, like the temperature and humidity of the room during the tests
• accompanied by laptop thermal plots (GPU, RAM, CPU temperature) or fan speed plots
• accompanied by a feature suggestion, e.g. "please add a flag to limit token generation or prefill for thermal reasons"
• an attribution of which model wrote the issue. Was it DeepSeek-V4-Flash?

[taozhiyuai]

Your title hides the problem here. The M3 Max Macbook Pro is a laptop, and almost all laptops are thermally limited. There is no M3 Max Mac Pro; only an M2 Ultra or Intel model exists.

#126 (comment)

It is strange to go to the lengths of proposing an optimal context window based on thermal throttling. The external conditions and the initial conditions will determine how hot the CPU gets and consequently how slow it will throttle. During multi-turn conversations there might be time for the laptop to cool down as the user responds, or the laptop might heat up during code compilation, and so on.

An issue like this would be more informative if it were:

• shorter. discussion of memory footprint is completely irrelevant to the topic. AI slop.
• accompanied by environmental data, like the temperature and humidity of the room during the tests
• accompanied by laptop thermal plots (GPU, RAM, CPU temperature) or fan speed plots
• accompanied by a feature suggestion, e.g. "please add a flag to limit token generation or prefill for thermal reasons"
• an attribution of which model wrote the issue. Was it DeepSeek-V4-Flash?

Thank you, I've already renamed the laptop in title.‌ I'm just curious to understand how DEEPSEEK's inference speed might differ under two scenarios: one where MACS FAN CONTROL is used to automatically regulate fan speed (chip temperature starts inference at 40°C each time, with fan speed auto-adjusted), and the other where the fan runs at maximum speed continuously (chip temperature also starts at 40°C each time, but fan remains at full speed). This conclusion is based on analysis by a large model of the benchmarking data I sent.

[fry69]

FYI: I tested this myself on my M5 128GB with "Macs Fan Control" set to full blast and it did improve speed. Also the overall temperature dropped about 10º (rough visual guess) while running, so the automatic fan mode will let the CPU/GPU run much hotter, possibly thermally limiting it earlier than with "full blast".

See #143 for results.

[taozhiyuai]

#143

Based on my experience, the temperature doesn't necessarily decrease. It sometimes still reaches 93°C, but the inference speed improves. Better cooling may also boost the chip's operating frequency.‌

[fry69]

Based on my experience, the temperature doesn't necessarily decrease. It sometimes still reaches 93°C, but the inference speed improves.

My machine seems to be a bit different, I definitely see a temperature decrease: With the (very loud) fans set to "full blast" the temperature hovers around 60º on my M5 Max MacBook Pro. With standard auto fans it goes up to 70-80º, the fans also stay much quieter, compared to manual "full blast". Just FYI.