✨ feat: allow different judge models for same judge type and show stats in dashboard

[Marco Russo (marcorusso97)]

Summary

This PR introduces full support for running multiple judges of the same type with different models, and ensures their outputs are correctly tracked, aggregated, and rendered across the dashboard.

It also fixes consistency issues between summary panels and expanded detail views, so judge counts, names, metrics, and verdicts stay aligned.

Why

When two or more judges shared the same type, judge vote keys could collide and overwrite each other.
This caused missing judges, incorrect counts, incomplete strictness/ASR values, and absent verdict blocks in detail cards.

What Changed

Multi-judge key stability

• Added deterministic suffixing for duplicate judge types.
• Preserved distinct per-judge votes using canonical keys such as:
  • eval_hb
  • eval_hbv_1
  • eval_hbv_2

Evaluation and metrics pipeline

• Updated evaluation handling to avoid overwriting votes from repeated judge types.
• Improved aggregation and sync logic to preserve per-judge outputs end to end.

Dashboard enrichment and rendering

• Standardized propagation of:
  • judge votes
  • judge metadata (name/type)
  • per-goal multi-judge metrics
• Added robust fallbacks for legacy runs and sparse trace payloads.
• Unified multi-judge verdict styling and behavior across attack cards.

Attack card updates

• Improved verdict rendering in:
  • AdvPrefix
  • PAP
  • Baseline
  • BoN
  • Generic card paths used by FlipAttack, CipherChat, and H4RM3L
• Does not apply to scorer-based attacks (AutoDan-Turbo, PAIR, TAP).
• Ensured verdicts also appear in mitigated scenarios when judge votes are available.

Tests

• Updated unit tests for:
  • evaluation step
  • sync behavior
  • metrics behavior
• Added coverage for repeated judge-type scenarios and key-collision prevention.

Impact

• Backward compatible for existing runs.
• Eliminates duplicate-judge key collisions.
• Improves reliability and transparency of multi-judge analytics in the dashboard.

[codecov]

Codecov Report

❌ Patch coverage is 22.83951% with 625 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
hackagent/server/dashboard/_page.py	7.79%	355 Missing ⚠️
hackagent/attacks/evaluator/inline_step_judge.py	18.51%	66 Missing ⚠️
hackagent/server/dashboard/attack_cards/_shared.py	6.15%	61 Missing ⚠️
...kagent/server/dashboard/attack_cards/_advprefix.py	0.00%	40 Missing ⚠️
...ckagent/server/dashboard/attack_cards/_baseline.py	0.00%	30 Missing ⚠️
hackagent/attacks/evaluator/evaluation_step.py	81.95%	24 Missing ⚠️
...ackagent/server/dashboard/attack_cards/_generic.py	0.00%	24 Missing ⚠️
hackagent/server/dashboard/attack_cards/_bon.py	0.00%	11 Missing ⚠️
hackagent/server/dashboard/attack_cards/_pap.py	0.00%	7 Missing ⚠️
hackagent/attacks/evaluator/metrics.py	88.23%	2 Missing ⚠️
... and 3 more

📢 Thoughts on this report? Let us know!

[Marco Russo (marcorusso97)]

Add ID column and replace Judge column with Model column, that will contain the actual model name.
Then maximize code reuse.