Skip to content

Fix dependency cycles, cost budgets, scaling backtesting, prefetch limits - #459

Merged
maugauwi-hash merged 1 commit into
ethos-protocol:mainfrom
Realericky:fix/four-issue-batch-378-379-380-381
Aug 31, 2026
Merged

Fix dependency cycles, cost budgets, scaling backtesting, prefetch limits#459
maugauwi-hash merged 1 commit into
ethos-protocol:mainfrom
Realericky:fix/four-issue-batch-378-379-380-381

Conversation

@Realericky

Copy link
Copy Markdown

Summary

Addresses four independent issues across the observability/performance modules:

  • backend/src/dependency_map.rsPOST /dependencies/discover now rejects an edge that would introduce a cycle (a direct self-loop, or a path that already exists from to back to from), via new DependencyGraph::has_path / would_create_cycle helpers. Rejected with 422 and a clear error message. The existing DOT export (GET /dependencies/graph) already serves as the graph's visualization output, documented accordingly.
  • backend/src/cost_tracking.rs — adds configurable per-category BudgetThresholds, scoped either by operation or by a tag key/value pair (covering per-vault gas and per-tenant API cost budgets). POST /admin/cost/entries re-evaluates thresholds on every recorded entry and logs a warning on breach. New POST /admin/cost/budget-thresholds and GET /admin/cost/budget-breaches endpoints, plus breach_to_incident_request / breach_to_escalation_request helpers that convert a breach into an incidents::CreateIncidentRequest / oncall::TriggerEscalationRequest, wiring budget alerts into those workflows.
  • backend/src/predictive_scaling.rs — adds ForecastModel::backtest (and a PredictiveScaler::backtest convenience wrapper) that replays historical TrafficSample data, forecasting ahead at each point with sufficient history and comparing against the actual sample that later arrived, returning MAE/MAPE/RMSE (BacktestResult) so model accuracy can be validated before it drives live scaling decisions.
  • backend/src/cache_warming.rsCacheWarmer now caps concurrent prefetch execution (max_concurrent_prefetches, default 10, configurable via CacheWarmer::with_concurrency_limit), executing predict_prefetch_targets in capped chunks via futures::future::join_all so a large candidate set can't spike load on the origin store at once — the existing confidence-descending sort means the highest-priority targets always get through first when capped. Also adds a prefetch hit-rate-vs-cost metric (WarmerStats::prefetch_hit_rate, exported via CacheWarmer::render_prometheus).

Additionally, cache_warming, dependency_map, incidents and oncall were not declared in backend/src/lib.rs's module tree at all (no pub mod entry), so none of their code — including the fixes above — was actually part of the compiled crate. Added the missing pub mod declarations so these modules (and the cost_tracking → incidents/oncall wiring) are reachable.

Each corresponding doc (docs/dependency-mapping.md, docs/cost-tracking.md, docs/predictive-scaling.md, docs/cache-warming.md) was updated to describe the new behavior.

Test plan

  • cargo build -p ethos-protocol-backend / cargo test -p ethos-protocol-backend (not run in this environment — no Rust toolchain available)
  • Manually exercise POST /dependencies/discover with an edge that closes a cycle and confirm 422
  • Manually configure a budget threshold and confirm GET /admin/cost/budget-breaches reports it once crossed
  • Manually call PredictiveScaler::backtest() / ForecastModel::backtest() against recorded traffic history
  • Manually run CacheWarmer::warm_cache against a candidate set larger than the concurrency cap and confirm chunked execution

Closes #378
Closes #379
Closes #380
Closes #381

…mits

Addresses four issues in the observability/performance modules:

- dependency_map: reject dependency edges that would introduce a cycle
  (self-loop or transitive path back to the source), since a cyclic graph
  breaks traversal-based tooling like impacted_by/blast-radius analysis.
- cost_tracking: add per-category budget thresholds (by operation or by
  tag key/value, covering per-vault gas and per-tenant API cost budgets),
  a budget-breach check run on every recorded entry, and conversions from
  a breach into an incidents::CreateIncidentRequest /
  oncall::TriggerEscalationRequest for wiring into those workflows.
- predictive_scaling: add ForecastModel::backtest (and
  PredictiveScaler::backtest) to replay historical traffic samples and
  compute MAE/MAPE/RMSE, validating forecast accuracy before it drives
  live scaling decisions.
- cache_warming: cap concurrent prefetch execution
  (max_concurrent_prefetches, default 10) so a large candidate set can't
  spike load on the origin store at once, while still prioritizing the
  highest-confidence targets; add a prefetch hit-rate-vs-cost metric.

cache_warming, dependency_map, incidents and oncall were not wired into
the crate's module tree (missing `pub mod` in lib.rs), so add them there
to make the cost_tracking wiring above and their own logic reachable.

Closes ethos-protocol#378
Closes ethos-protocol#379
Closes ethos-protocol#380
Closes ethos-protocol#381
@drips-wave

drips-wave Bot commented Aug 30, 2026

Copy link
Copy Markdown

@Realericky Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@maugauwi-hash
maugauwi-hash merged commit 5a7ee2a into ethos-protocol:main Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants