Conversation
Rebalance CI coverage so pull requests get comprehensive pre-merge unit and formatting feedback while resource-intensive end-to-end workloads run nightly.
…ai#3680) * [TENT] Add task failure reason counter and in-flight gauges Add three metrics for production diagnosis, built on the cached label cells: - tent_task_failures_total{transport, reason}: task-level failures with reason submit/poll/timeout/canceled. TaskInfo.failure_stage marks where the first failure originated (set at the submit-rejection and poll sites, never overwritten, so a poll failure followed by a rejected failover resubmit still counts as poll). Additive to the legacy read/write_failures_total counters, and the first failure metric that records TIMEOUT/CANCELED outcomes. - tent_inflight_attempts{transport}: gauge incremented on attempt submit and decremented on attempt finish — how much work is sitting in the engine right now. - tent_registered_buffer_bytes{transport}: gauge maintained on register/unregisterLocalMemory from desc.transports. * [TENT] Keep state gauges symmetric across setEnabled transitions Gauge updates are paired add/sub operations (attempt start/finish, register/unregister). Gating them on the runtime enable flag let a transition between the two halves of a pair permanently corrupt the gauge (stuck positive or negative). Gauges now update whenever initialized and ignore setEnabled: they track engine state, not samples. Counters and histograms keep the runtime gate. Add tests toggling setEnabled between paired updates for both gauges.
…vcache-ai#3619) Expose build version information over HTTP and Prometheus so operators can verify which binary a running process is using without shell access to the host. - Master admin server: new `GET /version` handler, registered alongside `/health` so it stays available in standby. - Real client HTTP server: same `/version` payload on the client HTTP port; it does not touch `client_`, so it is unaffected by `MC_STORE_CLIENT_METRIC` and by client initialization state. - `mooncake_client`: set the gflags version string so `--version` works, and log the display version at startup. - Master and client metrics: new `mooncake_build_info` gauge following the Prometheus "info" pattern, so dashboards and alerts can group or filter by the running build. Both surfaces share the metric name and are told apart by the scrape target's job/instance labels; on the client the version labels are added on top of the caller supplied labels. Both endpoints and the metric expose `version` (used for RPC handshake compatibility) and `display_version` (release plus short git hash). Docs: document the endpoint in the master admin endpoint table, the client endpoint table, the HTTP service reference, and the three client HTTP server switches that gate it. Tests: add `/version` coverage for the real client HTTP server and for the master admin server in both serving and standby states, plus build info metric serialization tests on the master and client sides. Co-authored-by: majingwei <majingwei@baidu.com>
…ths (kvcache-ai#3757) Tautological intent/failover tests and TCP tests that never constructed TcpTransport could not catch runtime regressions. Route default RDMA promotion through DecidePromotionHeadOnly and test TCP install plus batch worst-failure aggregation against the real engine. Signed-off-by: staryxchen <staryxchen@tencent.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…reads (kvcache-ai#3767) TCP bulk copies ran inline on the RPC io_context, serializing concurrent transfers and stalling Probe/Bootstrap. Offload the handlers and default rpc_server_threads higher when TCP is enabled so attachments can be read in parallel. Signed-off-by: staryxchen <staryxchen@tencent.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Register the wrapper module with CTest when unit tests are enabled so Go test failures propagate through the existing test path. Signed-off-by: Miguel Garcia <miguelgarciaroman8@gmail.com>
ci.yml listened for every labeled event, and auto-labeler already applies run-ci, so adding run-e2e-ci cancelled in-progress PR CI and reran every job. Keep Build & Test on open/push and move same-SHA retrigger to a workflow that only reacts to a human-applied run-ci label. Signed-off-by: staryxchen <staryxchen@tencent.com> Co-authored-by: Cursor <cursoragent@cursor.com>
* [Store] Introduce stateful region resource drivers * [Store] Refine region driver recovery boundaries * [Store] Validate CacheLib slab count * [Store] Clarify region driver ownership contracts
* [CI/Build] Keep non-CUDA wheels CUDA-free * [CI/Build] Run wheel smoke test under bash for CUDA scan The non-CUDA CUDA-dependency scan uses process substitution (< <(...)), which sh (dash) rejects at parse time, failing every wheel build variant regardless of VARIANT_FLAG. Run the smoke test step under bash.
* Fix data copy while not on same device * Format code * Format code * Fix code address comment by @staryxchen --------- Co-authored-by: shawnding <shawnding@tencent.com>
…che-ai#3790) * Map MOONCAKE_LOCAL_HOSTNAME to TENT rpc_server_hostname * code format --------- Co-authored-by: ruanzhao <ruanzhao@kingsoft.com>
…vcache-ai#3777) * [TENT] Prefer the LAG-effective port speed from ibv_query_port_speed * [TENT] Let tests inject verbs into RdmaContext and cover the effective-speed path * [TENT] Hold the last effective speed over transient query failures and count them --------- Co-authored-by: maxlisongsong <maxlisongsong@didiglobal.com>
* [Store] Add batch OpLog snapshot coordinator * [Bugfix][Store] Fix batch snapshot coordinator races --------- Co-authored-by: Yuchen Kou <kouyuchen@approaching.ai>
…ai#3785) Signed-off-by: Schatten <czhengt@qq.com>
* perf(store): remove redundant per-file deletion delay * test(store): make concurrent remove test deterministic
github-actions
Bot
added
documentation
Owner
Author
|
Closing this fork-local draft: the correct target is kvcache-ai/Mooncake:main, which is aligned with this branch's base. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Enable producer-view fencing for production batch OpLog writers in HA + etcd mode. The writer claims the acquired non-zero view before admission and uses that view for every batch transaction. Claim contention or runtime fencing failure closes writer admission without falling back to the legacy unfenced path.
Module
Type of Change
How Has This Been Tested?
Test commands:
Test results:
Checklist
AI Assistance Disclosure