Skip to content

[FEATURE] Add --compare flag for side-by-side agent comparison #16

@jwesleye

Description

@jwesleye

Feature Description

Send same query to multiple agents and see responses side-by-side for A/B testing.

Problem/Motivation

When evaluating different agents or models, I have to run queries multiple times and manually compare results.

Proposed Solution

chat_loop agent1 --compare agent2

# Shows both responses:
# [Agent 1]
# Response here...
# 
# [Agent 2] 
# Response here...
#
# Tokens: A1: 234, A2: 456
# Cost: A1: $0.01, A2: $0.02

Benefits

  • Easy A/B testing
  • Model comparison
  • Quality evaluation
  • Cost comparison

Priority

  • Critical
  • High
  • Medium
  • Low

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requeststatus: staleNo activity for extended period

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions