Field report: Gen2 A/B (RMPcieLinkSpeed 0x1 vs 0x2) on a Haswell-EP server board — both fail with keys verified delivered; BAR1 64 GB resize succeeds on the same 2015-era BIOS
Rig (2026-08-30/31, single board): ASUS Z10PE-D8 WS (C612, Xeon E5 v3/Haswell-EP), BIOS 4101 (2019-06-12, final release). 2× CMP 170HX on CPU2 root ports — 0x20C2 (8GB→64GB, Hynix) at 81:00.0 behind 80:01.0, 0x2082 (10GB→40GB, Samsung) at 82:00.0 behind 80:03.0. Direct root-port attachment, no switch. Slots bifurcated x8/x8; cards negotiate x4 (physical wiring). Driver stack: cmpunlocker v0.3 (tag f51da03), nvidia-open 610.43.03, kernel 7.0.0-27-generic, intel_iommu=on iommu=pt, pcie_aspm=off.
1. A config trap worth a troubleshooting entry: the initramfs key-delivery hole
/etc/modprobe.d/cmp-pcie-gen2.conf on the root filesystem does nothing if the driver loads from the initramfs. On this rig the module loads at T+1s (kernel log timestamps 1 s apart), so the conf must be inside the rebuilt initrd or RmForceEnableGen2/RMPcieLinkSpeed never reach the module. Symptom signature: grep -i 'ForceEnableGen2\|PcieLinkSpeed' /proc/driver/nvidia/params returns nothing while the conf file exists on disk; every retrain (0008 + gen2-hammer) clamps back to Gen1. This was our failure mode across four uninstall/reinstall cycles (Jul 30 → Aug 30) and produced a confident-but-wrong "Gen2 is silicon-fused on these cards" diagnosis before we found it.
Check: lsinitramfs /boot/initrd.img-$(uname -r) | grep cmp-pcie-gen2 after any update-initramfs.
2. The A/B: both spellings fail on this platform, with keys verified delivered
After fixing key delivery, across two separate boots:
| Boot |
RMPcieLinkSpeed |
Key in /proc/driver/nvidia/params |
Phases A+B |
Hammer (600/card, full coverage) |
LnkSta |
| 2026-08-31 00:25 |
0x1 |
yes |
LnkCap 0x00456102, LnkCap2 0x6, LnkCtl2=2, MISC1 0x20342d00, XP3G OVR0/VAL3/OVR3 take; OPT_GEN23 + VSEC_DEVICE fail (expected per wiki) |
0/1200 |
0x1041 (Gen1) |
| 2026-08-31 00:43 |
0x2 |
yes |
same |
0/1200 |
0x1041 (Gen1) |
So on this board the open 0x1 vs 0x2 question is answered for at least one platform class: no difference — the retrain is refused upstream of anything the key controls. The 3090s on CPU1 root ports train Gen3 normally, which points at the E5 v3-era root complexes (or their firmware) refusing the 5 GT/s target, not the cards. Single-board observation; posting because all three rigs in the wiki's confirmed list are modern consumer platforms — a server-vintage failure data point extends the known surface. Corollary: the wiki's "any card with the unlock installed runs Gen2" holds only on platforms whose root ports accept the retrain.
Two smaller notes:
- The installed
gen2.service shipped with TimeoutStartSec=45, which truncated card 2's hammer to ~7 s after card 1 consumed ~38 s (600 attempts × 50 ms ≈ 30 s per card). The wiki's own unit spec says 120 — worth aligning the shipped unit.
nvidia-smi --query-gpu=pcie.link.gen.max reported 1 even while the XVE-side LnkCap read Gen2; gen.current/LnkSta remained the only honest indicators, consistent with the wiki's "don't read dmesg" warning.
3. BAR1 64 GB resize works on this 2015-era BIOS with one setting flip
bar1-resize-unlock.patch (v0.3) + Advanced → PCI Subsystem Settings → Above 4G Decoding = Enabled (ships Disabled):
NVRM: CMP BAR1: before CYA=0x26 CFG=0x80000000 CAP=0x00000400
NVRM: CMP BAR1: after CYA=0x30 CFG=0x8000000a CAP=0x001ffc00
NVRM: CMP BAR1: resize enabled, will attempt up to 64 GB
NVRM: CMP BAR1: final size = 65536 MB ← both cards
nvidia-smi -q BAR1 Total = 65536 MiB on both cards — including the 40 GB (0x2082) card, i.e. the aperture exceeds VRAM without issue. Survives warm reboots; WPR2/late-PMA fix (PR #32) still live alongside it (0 Xid). With Above 4G Disabled: kernel caps at 256 MB (0x20C2) / 128 MB (0x2082), both BARs placed below 4 GB. Useful for other legacy-server-board owners: no consumer "ReBAR" toggle is needed — the plain Above-4G decode option is sufficient for the driver-side resize path, at least on C612/AMI 4101.
Happy to provide raw boot-log excerpts (journalctl -k filters) for both Gen2 boots and the BAR1 transitions if useful for adjudication.
Field report: Gen2 A/B (RMPcieLinkSpeed 0x1 vs 0x2) on a Haswell-EP server board — both fail with keys verified delivered; BAR1 64 GB resize succeeds on the same 2015-era BIOS
Rig (2026-08-30/31, single board): ASUS Z10PE-D8 WS (C612, Xeon E5 v3/Haswell-EP), BIOS 4101 (2019-06-12, final release). 2× CMP 170HX on CPU2 root ports — 0x20C2 (8GB→64GB, Hynix) at 81:00.0 behind 80:01.0, 0x2082 (10GB→40GB, Samsung) at 82:00.0 behind 80:03.0. Direct root-port attachment, no switch. Slots bifurcated x8/x8; cards negotiate x4 (physical wiring). Driver stack: cmpunlocker v0.3 (tag f51da03), nvidia-open 610.43.03, kernel 7.0.0-27-generic,
intel_iommu=on iommu=pt,pcie_aspm=off.1. A config trap worth a troubleshooting entry: the initramfs key-delivery hole
/etc/modprobe.d/cmp-pcie-gen2.confon the root filesystem does nothing if the driver loads from the initramfs. On this rig the module loads at T+1s (kernel log timestamps 1 s apart), so the conf must be inside the rebuilt initrd orRmForceEnableGen2/RMPcieLinkSpeednever reach the module. Symptom signature:grep -i 'ForceEnableGen2\|PcieLinkSpeed' /proc/driver/nvidia/paramsreturns nothing while the conf file exists on disk; every retrain (0008 + gen2-hammer) clamps back to Gen1. This was our failure mode across four uninstall/reinstall cycles (Jul 30 → Aug 30) and produced a confident-but-wrong "Gen2 is silicon-fused on these cards" diagnosis before we found it.Check:
lsinitramfs /boot/initrd.img-$(uname -r) | grep cmp-pcie-gen2after anyupdate-initramfs.2. The A/B: both spellings fail on this platform, with keys verified delivered
After fixing key delivery, across two separate boots:
RMPcieLinkSpeed/proc/driver/nvidia/params0x10x2So on this board the open
0x1vs0x2question is answered for at least one platform class: no difference — the retrain is refused upstream of anything the key controls. The 3090s on CPU1 root ports train Gen3 normally, which points at the E5 v3-era root complexes (or their firmware) refusing the 5 GT/s target, not the cards. Single-board observation; posting because all three rigs in the wiki's confirmed list are modern consumer platforms — a server-vintage failure data point extends the known surface. Corollary: the wiki's "any card with the unlock installed runs Gen2" holds only on platforms whose root ports accept the retrain.Two smaller notes:
gen2.serviceshipped withTimeoutStartSec=45, which truncated card 2's hammer to ~7 s after card 1 consumed ~38 s (600 attempts × 50 ms ≈ 30 s per card). The wiki's own unit spec says 120 — worth aligning the shipped unit.nvidia-smi --query-gpu=pcie.link.gen.maxreported 1 even while the XVE-side LnkCap read Gen2;gen.current/LnkSta remained the only honest indicators, consistent with the wiki's "don't read dmesg" warning.3. BAR1 64 GB resize works on this 2015-era BIOS with one setting flip
bar1-resize-unlock.patch(v0.3) + Advanced → PCI Subsystem Settings → Above 4G Decoding = Enabled (ships Disabled):nvidia-smi -qBAR1 Total = 65536 MiB on both cards — including the 40 GB (0x2082) card, i.e. the aperture exceeds VRAM without issue. Survives warm reboots; WPR2/late-PMA fix (PR #32) still live alongside it (0 Xid). With Above 4G Disabled: kernel caps at 256 MB (0x20C2) / 128 MB (0x2082), both BARs placed below 4 GB. Useful for other legacy-server-board owners: no consumer "ReBAR" toggle is needed — the plain Above-4G decode option is sufficient for the driver-side resize path, at least on C612/AMI 4101.Happy to provide raw boot-log excerpts (
journalctl -kfilters) for both Gen2 boots and the BAR1 transitions if useful for adjudication.