Skip to content

Add feature-gated Rust heap usage tracking for Valkey modules - #238

Open
misaki112 wants to merge 3 commits into
valkey-io:mainfrom
misaki112:track_module_memory_usage
Open

misaki112 wants to merge 3 commits into
valkey-io:mainfrom
misaki112:track_module_memory_usage

Conversation

@misaki112

@misaki112 misaki112 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds optional per-module Rust heap usage tracking through the existing ValkeyAlloc allocator.

The new enable-usage-tracking feature exposes valkey_module::module_stats::snapshot() with:

  • Current and peak tracked bytes
  • Allocation and free counts
  • Derived live-allocation count
  • Saturating-accounting error count

Implementation

  • Tracks alloc, alloc_zeroed, and dealloc calls using aligned allocation sizes.
  • Uses saturating arithmetic to prevent counter underflow or overflow.
  • Adds the usage_tracking example module with commands to retain and release:
    • Vec<u8>
    • DashMap
    • boxed AtomicU64 storage
    • String
    • a custom heap-owning struct
  • Adds integration coverage confirming each allocation type increases tracked memory and returns near baseline after release.
  • Verifies Redis/Valkey keyspace memory is not attributed to module usage tracking.
  • Documents feature enablement, available metrics, and the realloc limitation in the README.

Limitation

GlobalAlloc::realloc is intentionally not tracked. Resize-heavy Vec and String paths can therefore temporarily overstate peak memory and make allocation counts noisier.

Validation

cargo test --lib module_stats
cargo build --example usage_tracking --features enable-usage-tracking
cargo test --features enable-usage-tracking --test integration test_usage_tracking_module_memory -- --exact
cargo check --features enable-usage-tracking --all-targets
cargo fmt --check

Potential Follow-up

This feature flag may be extended in the future to include additional per-module resource observability, such as CPU-usage metrics.

@misaki112
misaki112 force-pushed the track_module_memory_usage branch from d9c5f48 to 0f18887 Compare August 7, 2026 22:44
Signed-off-by: misaki112 <nies@cs.washington.edu>
Signed-off-by: misaki112 <nies@cs.washington.edu>
…, update README.md

Signed-off-by: misaki112 <nies@cs.washington.edu>
@misaki112
misaki112 force-pushed the track_module_memory_usage branch from 0f18887 to 445eda2 Compare August 7, 2026 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant