new lib crate: Sv2 ↔ Bitcoin Core IPC

[plebhash]

up until now, we had two options for deploying a Sv2 Template Provider:

• build a modified bitcoind from sv2 branch of @Sjors fork that listens on a dedicated Sv2 port
• build a bitcoin-mine sidecar from sv2-ipc branch of @Sjors fork, which connects via capnproto protocol over IPC to bitcoin-node that's built from early development iterations of Bitcoin Core

with bitcoin/bitcoin#31802 merged, Bitcoin Core v30.0 is about to add bitcoin-node to the official releases, meaning that essentially any software can leverage capnproto over IPC and benefit from Bitcoin Core mining interface.

some early work has already been done for this, namely:

• github.com/Sjors/sv2-tp is a dedicated repo for the C++ implementation of bitcoin-mine sidecar mentioned above (now renamed as sv2-tp)
• @rustaceanrob has done some experimental rust integration, described in detail in Implementation experience in Rust bitcoin-core/libmultiprocess#174
• @Shourya742 has done some experimental integration into SRI, where a new crate provides an executable that binds to Bitcoin Core over IPC and listens on a dedicated Sv2 port to perform the Sv2 Template Distribution Protocol

this issue is meant to track the work where, inspired by the references above, we build a crate that can be used as a library to be integrated into Rust apps.

this library shall act as a "translation" layer between Bitcoin Core's capnproto Mining Interface over IPC and Sv2 Template Distribution Protocol.

SRI apps using this library (e.g.: JDC, Pool) shall be able to connect to bitcoin-node (from official v30+ Bitcoin Core releases) over a UNIX socket and leverage APIs providing Sv2 Template Distribution Protocol messages over an async runtime.

[rustaceanrob]

I have had some out-of-band discussions and have a couple opinions on future direction of the IPC crates. IMO there should be a single crate that serves the auto-generated client code that is generated from the capnp files. I have an example here that simply builds and serves the Bitcoin Core types. It pulls from a git subtree to source the files, which itself can be versioned according to Bitcoin Core. Users could layer on top of this with more with tokio stack in a higher level crate. The idea would be to let people that make a wallet, template provider, or whatever else can pull the same auto-gen'd code.

[plebhash]

thanks for your input @rustaceanrob, it makes a lot of sense.

PR stratum-mining/stratum#1915 has a prototype bitcoin_core_sv2 crate that is quite aligned with your proposal above.

initially I was hardcoding the capnp files, which tbh felt quite dirty, so I replaced that with your bitcoin-capnp as a dependency (currently locked to commit a03a8408657938cf13803a0d582fa4edf604383e).

[plebhash]

that came with a few consequences worthy of note:

MSRV

the latest capnp releases v0.21.* have MSRV 1.81.0, while SRI MSRV is currently at 1.75.0, so bitcoin_core_sv2 is currently not compliant with current SRI MSRV policy.

rust version

SRI currently follows MSRV 1.75.0, and as a consequence all crates use Rust 2021

but bitcoin-capnp is using Rust 2024, so I was forced to use that on bitcoin_core_sv2 (which also makes it incompatible with SRI MSRV policy, since Rust 2024 was introduced with 1.85.0)

---

with that said, as far as I remember SRI MSRV policy was coming from a trait definition on network_helpers_sv2 that has been cleaned up, so we could potentially consider lifting it, along with Rust version to 2024?

that would require a deep CI cleanup

---

edit: actually, it shouldn't really matter whether the trait was cleaned up from network_helpers_sv2

it should simply boil down to us making that decision... even if that trait still existed in network_helpers_sv2, it wouldn't be violated by a MSRV 1.85

[Shourya742]

Now that async traits aren’t holding us back, we could even think about lowering the MSRV for protocol crates. We still need to be super careful, anything that might run on an ASIC controller could be stuck on really old toolchains.

[plebhash]

created stratum-mining/stratum#1918 so we can re-evaluate MSRV across the project, it's getting a bit off topic on this issue

also hiding comments above, I think MSRV won't be a blocker for bitcoin_core_sv2