Rule Performance Profiler

[asulwer]

Rule Performance Profiler

Summary

Add per-invocation profiling that produces a call tree across rule dependency
chains and exports to a standard flame-graph format (speedscope / Chrome
trace), going beyond the aggregate counters we already collect.

What exists today

• RuleMetrics (RoslynRules/Models/RuleMetrics.cs) — thread-safe, per-rule
  aggregate counters: EvalCount, FailureCount, AverageExecutionTimeMs,
  FailureRatePercent, LastExecuted, TotalTicks. Exposed via Rule.Metrics.
• Per-rule lifecycle events (Rule.Events.cs): OnRuleExecuting
  (cancelable) and OnRuleExecuted (Result, Elapsed, Exception).
• Dependency structure: Rule.DependsOnRuleId / Rule.ParentRuleId, plus the
  topological sort and RuleGraphVisualizer.

Gap

RuleMetrics is aggregate-only — you can see that rule X averaged 0.4 ms over
1,000 evals, but not where a single evaluation spent its time across the
dependency chain, and there is no export a flame-graph tool can open.

Proposed design

A lightweight, opt-in profiling scope that records one node per rule
invocation (start ticks, duration, failed, rule Id, parent invocation), assembles
a call tree using the existing DependsOnRuleId / ParentRuleId links, and
serializes it.

• Opt-in so there is zero cost on the hot path when unused (aggregate
  RuleMetrics recording stays as-is).
• Leaf granularity is one rule — we compile each expression to a single IL
  delegate, so we cannot profile inside an expression. Call this out explicitly.
• Ship exporters, not a GUI: speedscope JSON (primary) and optionally
  Chrome chrome://tracing JSON. Users open the file in existing viewers.

Public API sketch

// Opt-in scope; disposing stops capture and yields the tree.
using (var session = ProfilerSession.Start(compiledWorkflow))
{
    compiledWorkflow.Execute(parameters);      // one or many runs
    ProfileReport report = session.Report();

    report.WriteSpeedscope("profile.speedscope.json");
    // report.WriteChromeTrace("profile.trace.json"); // optional

    foreach (RuleProfile p in report.Rules)        // aggregate roll-up
        Console.WriteLine($"{p.Description}: self={p.SelfMs}ms total={p.TotalMs}ms calls={p.Calls}");
}

public sealed class ProfileReport
{
    public IReadOnlyList<ProfileNode> Roots { get; }   // call tree
    public IReadOnlyList<RuleProfile> Rules { get; }   // per-rule roll-up
    public void WriteSpeedscope(string path);
    public string ToSpeedscopeJson();
}

public sealed class ProfileNode
{
    public Guid RuleId { get; }
    public string Description { get; }
    public double DurationMs { get; }
    public bool Failed { get; }
    public IReadOnlyList<ProfileNode> Children { get; }
}

Acceptance criteria

• ProfilerSession.Start(...) captures every rule invocation within its scope
  with duration and failed flag; no capture occurs when a session is not active.
• Call tree correctly nests children under the rule they depend on
  (DependsOnRuleId / ParentRuleId); independent rules appear as siblings/roots.
• WriteSpeedscope output opens cleanly in https://speedscope.app.
• Per-rule roll-up reports SelfMs, TotalMs, and Calls.
• Works for Execute and ExecuteParallel* (parallel invocations attributed
  to the correct rule; document any ordering caveats).
• Benchmark showing ~zero overhead when no session is active.
• Unit tests + a Demo sample that emits a .speedscope.json.

Non-goals

• No bundled/graphical flame-graph UI — export only.
• No sub-expression (intra-IL) profiling; one rule is the smallest unit.

Open questions

• Overhead model for ExecuteParallel* — per-thread buffers merged on Report()?
• Should aggregate RuleMetrics optionally feed a "no-tree" summary export too?