Skip to content

Fix node memory usage accuracy: use memoryUsage instead of workloadMe… - #2410

Open
si1enteee wants to merge 1 commit into
freelensapp:mainfrom
si1enteee:fix/node-memory-usage-accuracy
Open

si1enteee wants to merge 1 commit into
freelensapp:mainfrom
si1enteee:fix/node-memory-usage-accuracy

Conversation

@si1enteee

Copy link
Copy Markdown

…moryUsage

Problem:
The node list view uses workloadMemoryUsage (container_memory_working_set_bytes) as the primary memory metric, which includes reclaimable Page Cache. This causes:

  1. Inflated memory usage for nodes with large Page Cache (e.g. Elasticsearch, database nodes showing 72% when actual pressure is 48%)
  2. Usage exceeding 100% when working_set > allocatable
  3. Jump discontinuity when switching between Prometheus and Metrics Server data sources (working_set vs actual usage differ by 24%+)
  4. Inconsistency with the detail view which already shows both metrics separately

Additionally, the memoryUsage PromQL uses MemTotal - MemFree - Buffers - Cached which incorrectly treats tmpfs/shared memory in Cached as reclaimable, causing underestimation for nodes with large tmpfs or shared memory segments.

Fix:

  1. Switch renderMemoryUsage primary metric from workloadMemoryUsage to memoryUsage (actual node memory usage excluding reclaimable cache)
  2. Add workloadMemoryUsage as a tertiary line (LineProgress tertiaryValue) so working set / OOM risk information is preserved
  3. Add workload info to tooltip for context
  4. Update memoryUsage PromQL from: MemTotal - (MemFree + Buffers + Cached) to MemTotal - MemAvailable which correctly accounts for non-reclaimable cache (tmpfs, shm)

The detail view (node-charts.tsx) already correctly shows both metrics separately; this change brings the list view in line with that behavior.

Verified with simulated data for 6 scenarios:

  • ES data node: 72.4% to 48.3% (fixes inflation)
  • tmpfs node: 54.3% to 79.3% (fixes underestimation)
  • Shared memory node: 48.8% to 85.7% (fixes underestimation)
  • High load node: 108.5% to 95.0% (fixes >100%)
  • Normal/low load nodes: minimal change

…moryUsage

Problem:
The node list view uses workloadMemoryUsage (container_memory_working_set_bytes)
as the primary memory metric, which includes reclaimable Page Cache. This causes:

1. Inflated memory usage for nodes with large Page Cache (e.g. Elasticsearch,
   database nodes showing 72% when actual pressure is 48%)
2. Usage exceeding 100% when working_set > allocatable
3. Jump discontinuity when switching between Prometheus and Metrics Server
   data sources (working_set vs actual usage differ by 24%+)
4. Inconsistency with the detail view which already shows both metrics separately

Additionally, the memoryUsage PromQL uses MemTotal - MemFree - Buffers - Cached
which incorrectly treats tmpfs/shared memory in Cached as reclaimable, causing
underestimation for nodes with large tmpfs or shared memory segments.

Fix:
1. Switch renderMemoryUsage primary metric from workloadMemoryUsage to memoryUsage
   (actual node memory usage excluding reclaimable cache)
2. Add workloadMemoryUsage as a tertiary line (LineProgress tertiaryValue) so
   working set / OOM risk information is preserved
3. Add workload info to tooltip for context
4. Update memoryUsage PromQL from:
   MemTotal - (MemFree + Buffers + Cached) to MemTotal - MemAvailable
   which correctly accounts for non-reclaimable cache (tmpfs, shm)

The detail view (node-charts.tsx) already correctly shows both metrics
separately; this change brings the list view in line with that behavior.

Verified with simulated data for 6 scenarios:
- ES data node: 72.4% to 48.3% (fixes inflation)
- tmpfs node: 54.3% to 79.3% (fixes underestimation)
- Shared memory node: 48.8% to 85.7% (fixes underestimation)
- High load node: 108.5% to 95.0% (fixes >100%)
- Normal/low load nodes: minimal change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant