Motivation
Mooncake provides a C++ DummyClient that communicates with a standalone
mooncake_client, but the Go bindings currently lack the APIs needed to create
and configure a DummyClient, access the shared-memory buffers prepared during
setup, and exercise the single-object PutFrom path.
This RFC proposes adding the required bindings and a multi-process Go benchmark
for validating multiple independent application processes sharing one
standalone mooncake_client.
An implementation is available in
kvcache-ai/Mooncake#3015.
It ports the relevant work from
zchuango/Mooncake#29
onto the current Mooncake main branch.
Runtime Architecture
The request path is:
Go worker -> DummyClient -> RPC -> standalone mooncake_client
-> RealClient -> Mooncake Store
The standalone mooncake_client owns the RealClient and exposes the required
RPC handlers. Each benchmark worker is an independent operating-system process
with its own DummyClient, address space, and registered shared-memory context.
Separate processes are required because goroutines would still share one address
space and would not reproduce the intended multi-client shared-memory setup.
Proposed Changes
1. Expose DummyClient through the Go and C bindings
Allow the Store bindings to explicitly create either a RealClient or a
DummyClient.
The proposed Go API uses:
New() to create the default RealClient;NewWithType(MOONCAKE_CLIENT_DUMMY) to create a DummyClient.
The public Go API uses the Go-defined ClientType type and does not expose CGo
types to callers.
2. Preserve the existing Go Setup API
Keep the existing seven-argument Go Setup method for RealClient and add:
DummySetup(memPoolSize, localBufferSize, serverAddress, ipcSocketPath)
Go does not support optional function parameters. Extending the existing
Setup signature would therefore break existing Go call sites.
3. Preserve C API compatibility
Keep the existing C creation and setup APIs unchanged for current direct C
callers, and add explicit entry points for dummy clients:
- keep
mooncake_store_create() as the default RealClient constructor;
- add
mooncake_store_create_with_type(client_type);
- keep the existing
mooncake_store_setup(...);
- add
mooncake_store_setup_dummy(...).
The Go wrapper will use the new entry points for DummyClient. Existing direct
C callers remain source-compatible, while the additional symbols provide the
configuration required by the Go binding.
Feedback on the exact function names and parameter layout is welcome, but this
RFC does not propose changing the signatures of the existing C APIs.
4. Provide access to DummyClient registered buffers
The benchmark must access the shared-memory buffers prepared during
DummyClient setup before calling PutFrom and GetInto.
The current implementation exposes operations to:
- obtain the number of registered buffers;
- query a buffer pointer and size by index;
- determine whether a buffer belongs to the hot cache;
- unregister the buffers.
The current implementation exposes these operations through the public Store
binding.
5. Complete the single-object PutFrom path
DummyClient already supports batch put_from, while the Go benchmark uses the
single-object PutFrom API.
The current implementation adds a dedicated RealClient::put_from_dummy_helper RPC. It:
- finds the shared-memory context using the client ID;
- translates the
DummyClient address into a pointer valid in the standalone
mooncake_client process;
- invokes the existing
RealClient single-object put_from implementation.
The handler is registered in both the production mooncake_client and the test
RPC server.
The current proposal retains a dedicated single-object RPC. This keeps the
single-object API explicit, preserves its direct forwarding path, and reuses
the existing RealClient single-object implementation. Reusing the batch RPC
with a one-object request was considered, but not selected because it would
make the single-object API depend on batch semantics internally.
6. Add a multi-process Go benchmark
The benchmark:
- starts independent worker processes with
os/exec;
- creates one
DummyClient per worker;
- distributes keys among workers;
- writes deterministic data with
PutFrom;
- reads data with
GetInto;
- verifies the returned contents;
- reports failures, verification errors, and throughput.
Using separate processes, rather than goroutines, provides independent address
spaces, DummyClient instances, shared-memory contexts, and CPU affinity.
Shared-Memory Safety
Pointers are process-local virtual addresses. RealClient must not directly
dereference an address received from DummyClient.
The RPC request includes the DummyClient address, size, device ID, and client
ID. RealClient uses the corresponding shared-memory context to translate the
dummy address into a valid local pointer.
Before accessing registered memory, the Go benchmark validates:
- conversion to the platform
int range;
- the required size against the registered buffer capacity.
Go slices are constructed using the actual accessible buffer length.
Scope
Included:
- C and Go
DummyClient bindings;
- compatibility-preserving C creation and setup APIs;
- a separate Go
DummySetup;
- access to registered
DummyClient buffers;
- single-object
PutFrom support;
- production and test RPC registration;
- a multi-process Go
DummyClient benchmark.
Explicitly excluded from this proposal:
- changes to
mooncake-store/benchmarks/stress_cluster_bench.cpp;
- CI and workflow changes;
- unrelated repository changes.
Validation
The current implementation has been validated by:
- building
mooncake_store, mooncake_master, and mooncake_client;
- building the existing Go
examples/basic program;
- validating Put, Exists, GetSize, and Get through the Go binding;
- completing a DummyClient write run with 2/2 successful operations and no
failures;
- completing a DummyClient read run with 2/2 successful operations, no
failures, and no verification errors;
- rebuilding both Go examples after automated review fixes;
- confirming that the pull request contains only the intended 11 files;
- confirming successful benchmark runs return exit status 0;
- confirming operation and worker failures return a non-zero exit status;
- verifying deliberately corrupted data produces
Verify Errors: 1 and
non-zero worker and master exit statuses;
- running Rust
cargo check --locked against the compatibility-preserving
C API;
- running
gofmt, targeted pre-commit hooks, and the project C/C++ format
check.
These checks provide small-scale functional validation, not full performance,
high-concurrency, or stress coverage.
Current Status
The implementation PR is ready for review. All 12 existing inline review threads
have been addressed and resolved.
References
Motivation
Mooncake provides a C++
DummyClientthat communicates with a standalonemooncake_client, but the Go bindings currently lack the APIs needed to createand configure a
DummyClient, access the shared-memory buffers prepared duringsetup, and exercise the single-object
PutFrompath.This RFC proposes adding the required bindings and a multi-process Go benchmark
for validating multiple independent application processes sharing one
standalone
mooncake_client.An implementation is available in
kvcache-ai/Mooncake#3015.
It ports the relevant work from
zchuango/Mooncake#29
onto the current Mooncake
mainbranch.Runtime Architecture
The request path is:
Go worker -> DummyClient -> RPC -> standalone
mooncake_client-> RealClient -> Mooncake Store
The standalone
mooncake_clientowns theRealClientand exposes the requiredRPC handlers. Each benchmark worker is an independent operating-system process
with its own
DummyClient, address space, and registered shared-memory context.Separate processes are required because goroutines would still share one address
space and would not reproduce the intended multi-client shared-memory setup.
Proposed Changes
1. Expose DummyClient through the Go and C bindings
Allow the Store bindings to explicitly create either a
RealClientor aDummyClient.The proposed Go API uses:
New()to create the defaultRealClient;NewWithType(MOONCAKE_CLIENT_DUMMY)to create aDummyClient.The public Go API uses the Go-defined
ClientTypetype and does not expose CGotypes to callers.
2. Preserve the existing Go Setup API
Keep the existing seven-argument Go
Setupmethod forRealClientand add:DummySetup(memPoolSize, localBufferSize, serverAddress, ipcSocketPath)Go does not support optional function parameters. Extending the existing
Setupsignature would therefore break existing Go call sites.3. Preserve C API compatibility
Keep the existing C creation and setup APIs unchanged for current direct C
callers, and add explicit entry points for dummy clients:
mooncake_store_create()as the defaultRealClientconstructor;mooncake_store_create_with_type(client_type);mooncake_store_setup(...);mooncake_store_setup_dummy(...).The Go wrapper will use the new entry points for
DummyClient. Existing directC callers remain source-compatible, while the additional symbols provide the
configuration required by the Go binding.
Feedback on the exact function names and parameter layout is welcome, but this
RFC does not propose changing the signatures of the existing C APIs.
4. Provide access to DummyClient registered buffers
The benchmark must access the shared-memory buffers prepared during
DummyClientsetup before callingPutFromandGetInto.The current implementation exposes operations to:
The current implementation exposes these operations through the public Store
binding.
5. Complete the single-object PutFrom path
DummyClientalready supports batchput_from, while the Go benchmark uses thesingle-object
PutFromAPI.The current implementation adds a dedicated
RealClient::put_from_dummy_helperRPC. It:DummyClientaddress into a pointer valid in the standalonemooncake_clientprocess;RealClientsingle-objectput_fromimplementation.The handler is registered in both the production
mooncake_clientand the testRPC server.
The current proposal retains a dedicated single-object RPC. This keeps the
single-object API explicit, preserves its direct forwarding path, and reuses
the existing
RealClientsingle-object implementation. Reusing the batch RPCwith a one-object request was considered, but not selected because it would
make the single-object API depend on batch semantics internally.
6. Add a multi-process Go benchmark
The benchmark:
os/exec;DummyClientper worker;PutFrom;GetInto;Using separate processes, rather than goroutines, provides independent address
spaces,
DummyClientinstances, shared-memory contexts, and CPU affinity.Shared-Memory Safety
Pointers are process-local virtual addresses.
RealClientmust not directlydereference an address received from
DummyClient.The RPC request includes the
DummyClientaddress, size, device ID, and clientID.
RealClientuses the corresponding shared-memory context to translate thedummy address into a valid local pointer.
Before accessing registered memory, the Go benchmark validates:
intrange;Go slices are constructed using the actual accessible buffer length.
Scope
Included:
DummyClientbindings;DummySetup;DummyClientbuffers;PutFromsupport;DummyClientbenchmark.Explicitly excluded from this proposal:
mooncake-store/benchmarks/stress_cluster_bench.cpp;Validation
The current implementation has been validated by:
mooncake_store,mooncake_master, andmooncake_client;examples/basicprogram;failures;
failures, and no verification errors;
Verify Errors: 1andnon-zero worker and master exit statuses;
cargo check --lockedagainst the compatibility-preservingC API;
gofmt, targeted pre-commit hooks, and the project C/C++ formatcheck.
These checks provide small-scale functional validation, not full performance,
high-concurrency, or stress coverage.
Current Status
The implementation PR is ready for review. All 12 existing inline review threads
have been addressed and resolved.
References
zchuango/Mooncake#29
kvcache-ai/Mooncake#3015