modelopt

Reproducible recipe: serve abliterated Gemma-4-12B (gemma4_unified) at 50-118 tok/s on no-NVLink Blackwell (SM120) via vLLM nightly + ModelOpt FP8/NVFP4 + MTP spec-decode.

Lna-Lab production pipeline: GGUF -> modelopt-format NVFP4 + working MTP head for vLLM on RTX PRO 6000 Blackwell (SM120). Stages 2 (NVFP4) and 3 (MTP graft) are Lna-Lab originals; stage 1 (GGUF->bf16) reuses li-yifei/gguf-to-nvfp4.

NVFP4 AWQ Full quantization of SuperGemma4-26B-Abliterated-Multimodal for Blackwell GPUs — pre-built vLLM container + patches included

Guide for serving fine-tuned Qwen3.5-27B (dense, NVFP4) on DGX Spark via native vLLM. Includes critical config fixes for modelopt export_hf_checkpoint() that prevent silent FP32 dequantization.

EAGLE E4B speculative decoding drafter for Gemma 4 31B DECKARD HERETIC Uncensored NVFP4 — optimized for NVIDIA DGX Spark

Serve an abliterated Gemma-4-12B at high speeds on Blackwell GPUs without NVLink using vLLM, FP8 quantization, and MTP speculative decoding.

Compatibility matrix between NVIDIA TensorRT Model Optimizer (modelopt) PTQ and HuggingFace Transformers — static dependency scan + per-version probing + signature-drift detection, with styled HTML/notebook reports.

First usable Qwen3.8-Flash-Next abliterated NVFP4-PLE on a single DGX Spark. A multi-node assembly product that loaded fine but produced garbage, patched into single-node usability: healthy payloads, broken metadata (reciprocal scale_2, 73728 missing input_scale, PLE global off 3400x) fixed with 4-byte scalar patches. MTP 0 to 54.9%.