Skip to content

Fix metrics monitoring - #994

Merged
orunganiekan merged 3 commits into
SiLioLabs:masterfrom
Devadakene:fix-metrics-monitoring
Aug 31, 2026
Merged

Fix metrics monitoring#994
orunganiekan merged 3 commits into
SiLioLabs:masterfrom
Devadakene:fix-metrics-monitoring

Conversation

@Devadakene

Copy link
Copy Markdown
Contributor

Closes #877

Description:
This PR addresses incomplete operational monitoring by introducing granular metrics for charge results and RPC failovers, ensuring improved visibility and production readiness.

Changes:

  • scripts/metrics-server.ts: Added the keeper_charge_results_total counter to track detailed individual outcomes (Charged, Skipped, Paused, GracePeriodElapsed, etc.) via a new result label. Added the keeper_rpc_failovers_total counter to track RPC client failover events.
  • scripts/keeper.ts: Integrated the metrics-server directly into the keeper execution cycle. Both processPageLive and processPageDryRun now measure duration and report the granular charge outcomes. The startMetricsServer is safely initialized on startup.
  • scripts/rpc-client.ts: Instrumented the executeWithFailover loop to increment the failover metric whenever an RPC endpoint fails and the client falls back to an alternative.
  • scripts/grafana-dashboard.json: Appended two new panels to track the metrics:
    • Charge Results Breakdown: A timeseries chart tracking granular outcomes per cycle.
    • RPC Failover Rate: A stat panel monitoring RPC endpoint instability and failovers.

Acceptance Criteria Met:

  • Metrics endpoint now lists the new series (keeper_charge_results_total, keeper_rpc_failovers_total).
  • Dashboard JSON updated to reference the new series.
  • Uses low cardinality labels (result mapping exclusively to strict enum variants and string types).
  • Validation allows metrics to be gathered and viewed locally during dry keeper cycles.

@drips-wave

drips-wave Bot commented Aug 31, 2026

Copy link
Copy Markdown

@Devadakene Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@orunganiekan
orunganiekan merged commit 66ad13c into SiLioLabs:master Aug 31, 2026
0 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue 082: Add Prometheus metrics for keeper charge outcomes and RPC errors

2 participants