Fulcrum crashes with sigsegv every few minutes

[kami104]

I have the same issue (see here). Fulcrum crashes few minutes with status11/SEGV.

I'm running fulcrum 2.0.0 on a VM with Debian 13. I followed Minibolt guide with minor tweaks to run on Deb13 instead of Ubuntu 22.

My logs are follow:

Nov 09 21:47:48 minibolt-debian13 systemd[1]: fulcrum.service: Main process exited, code=killed, status=11/SEGV
Nov 09 21:47:48 minibolt-debian13 systemd[1]: fulcrum.service: Failed with result 'signal'.
Nov 09 21:47:48 minibolt-debian13 systemd[1]: fulcrum.service: Consumed 52.194s CPU time, 3.6G memory peak.
Nov 09 21:47:58 minibolt-debian13 systemd[1]: fulcrum.service: Scheduled restart job, restart counter is at 25.
Nov 09 21:47:58 minibolt-debian13 systemd[1]: Started fulcrum.service - Fulcrum.
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] Loaded SSL certificate: Fulcrum  expires: Wed November 7 2035 17:12:48
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] Loaded key type: private algorithm: RSA
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] Enabled JSON parser: simdjson
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] simdjson implementations:
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212]     haswell: Intel/AMD AVX2  [not supported]
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212]     westmere: Intel/AMD SSE4.2  [not supported]
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212]     fallback: Generic fallback implementation  [supported]
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] active implementation: fallback
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] jemalloc: version 5.3.0-0-g54eaed1
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] Qt: version 5.15.6
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] rocksdb: version 9.2.1-08f9322
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] simdjson: version 0.6.0
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] ssl: OpenSSL 1.1.1  11 Sep 2018
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] zmq: libzmq version: 4.3.5, cppzmq version: 4.10.0
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] UPnP: miniupnpc 2.3.0 (API version: 19)
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] Fulcrum 2.0 (Release df51b8a) - Sun Nov 9, 2025 21:47:58.212 CET - starting up ...
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.212] Max open files: 524288 (increased from default: 1024)
Nov 09 21:47:58 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:47:58.214] Loading database ...
Nov 09 21:48:09 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:09.436] DB memory: 4096.00 MiB
Nov 09 21:48:09 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:09.437] Coin: BTC
Nov 09 21:48:09 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:09.437] Chain: main
Nov 09 21:48:09 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:09.437] Verifying headers ...
Nov 09 21:48:10 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:10.368] Initializing header merkle cache ...
Nov 09 21:48:10 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:10.684] Checking tx counts ...
Nov 09 21:48:11 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:11.802] 134280232 total transactions
Nov 09 21:48:11 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:11.911] UTXO set: 38755422 utxos, 3255.455 MB
Nov 09 21:48:11 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:11.911] DB version: v4
Nov 09 21:48:11 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:11.911] The previous run of Fulcrum may have shut down uncleanly, however undo info is missing from the DB. It would normally be safest to undo the last 6 blocks, but since we cannot do that, we will proceed anyway and presume the DB is consistent. In the future, please use SIGTERM, SIGINT, or SIGQUIT to request a clean process exit, and wait for Fulcrum to exit cleanly.
Nov 09 21:48:11 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:11.911] BitcoinDMgr: starting 3 bitcoin RPC clients ...
Nov 09 21:48:11 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:11.911] BitcoinDMgr: started ok
Nov 09 21:48:12 minibolt-debian13 Fulcrum[8377]: [2025-11-09 21:48:12.012] <Controller> Block height 922917, downloading new blocks ...
Nov 09 21:48:24 minibolt-debian13 systemd[1]: fulcrum.service: Main process exited, code=killed, status=11/SEGV
Nov 09 21:48:24 minibolt-debian13 systemd[1]: fulcrum.service: Failed with result 'signal'.
Nov 09 21:48:24 minibolt-debian13 systemd[1]: fulcrum.service: Consumed 50.761s CPU time, 3.1G memory peak.

dmesg says this:

[11836.722862] Controller[7310]: segfault at 70092ffffc7 ip 000055993f1f4ac7 sp 00007f04a1ff9790 error 4 in Fulcrum[5f4ac7,55993ec00000+1154000] likely on CPU 1 (core 1, socket 0)
[11836.723162] Code: 08 4a 8b 4c 29 08 48 89 75 a8 4a 8d 34 28 8d 42 ff 48 63 d8 48 89 75 a0 48 c1 e3 03 48 89 4d c8 48 f7 db 89 45 b8 48 89 5d c0 <4b> 8b 1c 26 48 85 db 75 30 48 8b 45 a0 31 d2 49 83 ed 08 4c 89 30
[11954.270284] traps: Controller[7409] general protection fault ip:563b051f4ac7 sp:7f2a27bfb790 error:0 in Fulcrum[5f4ac7,563b04c00000+1154000]
[12165.608764] traps: Controller[7498] general protection fault ip:55b68f5f4ac7 sp:7ffaac3fc790 error:0 in Fulcrum[5f4ac7,55b68f000000+1154000]
[12781.614358] traps: Controller[7591] general protection fault ip:562fc27f4ac7 sp:7f953d7fc790 error:0 in Fulcrum[5f4ac7,562fc2200000+1154000]
[12985.726812] traps: Controller[7736] general protection fault ip:55efa9bf4ac7 sp:7f1878bfc790 error:0 in Fulcrum[5f4ac7,55efa9600000+1154000]
[13024.638971] Fulcrum[7763]: segfault at 7f17e44325a8 ip 000055b330bf4a99 sp 00007ffcd4a67f50 error 4 in Fulcrum[5f4a99,55b330600000+1154000] likely on CPU 4 (core 4, socket 0)
[13024.638984] Code: 49 c1 e4 03 85 d2 78 75 49 89 ff 4d 89 e5 0f 1f 00 48 8b 45 98 89 55 bc 4d 89 ec 49 f7 dc 48 8b 48 10 48 8b 40 08 4a 8d 34 29 <4e> 8b 74 28 08 4a 8b 4c 29 08 48 89 75 a8 4a 8d 34 28 8d 42 ff 48
[13272.744479] traps: Controller[7894] general protection fault ip:5640c9ff4ac7 sp:7fa666ffc790 error:0 in Fulcrum[5f4ac7,5640c9a00000+1154000]
[13345.880561] traps: Controller[7989] general protection fault ip:55e461df4ac7 sp:7fa98e3f9790 error:0 in Fulcrum[5f4ac7,55e461800000+1154000]
[13438.965283] traps: Controller[8072] general protection fault ip:558f87ff4ac7 sp:7fc0cbff9790 error:0 in Fulcrum[5f4ac7,558f87a00000+1154000]
[13704.401685] traps: Controller[8156] general protection fault ip:563ac5ff4ac7 sp:7fdcf35fc790 error:0 in Fulcrum[5f4ac7,563ac5a00000+1154000]
[13775.112611] Controller[8361]: segfault at 7f2ea1fc8bb0 ip 00005626aa3f4a99 sp 00007f2e78ffc790 error 4 in Fulcrum[5f4a99,5626a9e00000+1154000] likely on CPU 2 (core 2, socket 0)
[13775.113005] Code: 49 c1 e4 03 85 d2 78 75 49 89 ff 4d 89 e5 0f 1f 00 48 8b 45 98 89 55 bc 4d 89 ec 49 f7 dc 48 8b 48 10 48 8b 40 08 4a 8d 34 29 <4e> 8b 74 28 08 4a 8b 4c 29 08 48 89 75 a8 4a 8d 34 28 8d 42 ff 48
[13811.198156] Controller[8431]: segfault at 7f77c57c8ba8 ip 000056493c9f4a99 sp 00007f7796bfc790 error 4 in Fulcrum[5f4a99,56493c400000+1154000] likely on CPU 7 (core 7, socket 0)
[13811.198168] Code: 49 c1 e4 03 85 d2 78 75 49 89 ff 4d 89 e5 0f 1f 00 48 8b 45 98 89 55 bc 4d 89 ec 49 f7 dc 48 8b 48 10 48 8b 40 08 4a 8d 34 29 <4e> 8b 74 28 08 4a 8b 4c 29 08 48 89 75 a8 4a 8d 34 28 8d 42 ff 48

I hope this logs can help.

[cculianu]

Hmm since you installed the official binary -- it can't be some strange build error. I have never had it segfault like that.

The only possible explanation I can think of is bad RAM. Can you try running memtest86 on that box? Otherwise I can't think of any other explanation..

[cculianu]

PS: That issue you linked to is unrelated. That was due to a bug that was fixed. So it can't be that...

[kami104]

I booted the system with the linux ISO in memtest.org. It started automatically the test and after 6 passes it found no errors.