Multi-Agent Collaboration: Incentive Design for Quality × Speed

[landovsky]

Multi-Agent Collaboration: Incentive Design for Quality × Speed

The Core Tension

You're trying to optimize for quality × speed while preventing Goodhart's Law (when a measure becomes a target, it ceases to be a good measure).

Any metric you expose to agents (human or AI) will eventually be gamed:

• "Lines of code" → verbose code
• "Tasks completed" → cherry-picking easy tasks
• "Speed" → shallow work
• "Peer approval" → collusion, mutual back-scratching

---

Mechanism Options

1. Blind Parallel + Selection

Multiple agents work independently on the same problem. A separate evaluator (human or different AI) picks the best output. Agents don't see each other's work until after selection.

Agent A ──┐
Agent B ──┼──► Evaluator ──► Best output wins
Agent C ──┘

Pros: No coordination overhead, natural variation, hard to game
Cons: Expensive (N× compute), evaluator becomes bottleneck
Anti-gaming: Agents can't optimize for each other's behavior

2. Sequential Critique

Each agent improves the previous agent's work. Final output is what matters.

Agent A ──► Agent B (critique + improve) ──► Agent C (critique + improve) ──► Final

Pros: Builds on strengths, catches errors
Cons: First agent sets frame (anchoring), slow
Anti-gaming: Later agents are incentivized to find flaws, not praise

3. Adversarial Verification (Red Team)

Split agents into producers and verifiers. Verifiers are rewarded for finding flaws that survive to production.

Producers ──► Output ──► Verifiers ──► Flaws found?
                              │
                    ┌─────────┴─────────┐
                    ▼                   ▼
              Flaw confirmed      False positive
              (verifier wins)     (producer wins)

Pros: Strong error detection, hard to collude
Cons: Requires ground truth eventually, adversarial dynamics can be toxic for humans
Anti-gaming: Zero-sum between producer and verifier

4. Prediction Markets / Staking

Agents "stake" confidence on their outputs. Evaluated later against outcomes. High-confidence correct predictions build reputation; overconfidence is penalized.

Pros: Self-calibrating, rewards honesty about uncertainty
Cons: Needs delayed ground truth, complex to implement
Anti-gaming: Overconfidence is costly, hedging is obvious

---

Anti-Gaming Principles

Principle	Why It Works
Delayed evaluation	Can't optimize for what you can't see yet
Outcome over output	Measure "did the code work in prod" not "did it pass review"
Random spot-checks	Unpredictable scrutiny prevents consistent gaming
Multiple uncorrelated metrics	Hard to game several things at once
Evaluator rotation	Can't learn one evaluator's preferences
Hidden criteria weights	Don't know which metric matters most this time

---

For AI Agents Specifically

AI agents don't have intrinsic motivation — "incentives" means selection pressure and prompt design.

Selection Pressure

# Run N agents, select best
outputs = [agent.run(task) for _ in range(N)]
best = evaluator.select(outputs)  # Human or separate AI

Prompt-Based Incentives

You will be evaluated on:
- Correctness (verified by separate agent)
- Conciseness (shorter is better, all else equal)
- Your confidence calibration will be tracked over time

Constitutional Approach

Before submitting, ask yourself:
- Would I bet mass amounts of money that this is correct?
- What's the weakest part of this analysis?
- What would a critic say?

---

Practical Implementation for Agent Workflows

For Same-Role Parallelism (Multiple Analysts)

1. Run 2-3 Analysts in parallel on complex tasks
2. Have Planner receive all outputs and synthesize (not just pick one)
3. Track which Analyst's contributions survive to implementation
4. Use that signal to refine the weaker Analyst's prompt

Provenance Tracking (Add to Reviewer Agent)

## Provenance Tracking
When reviewing, note which upstream agent's work:
- Survived intact
- Required correction
- Caused downstream problems

Store as structured data, not prose.

Key Insight

Don't try to incentivize agents directly. Instead, measure outcomes and use selection pressure. Run variants, track which produces better results, update prompts accordingly.

---

For Human Teams

1. Evaluate outcomes, not effort — did the feature work, not how many hours
2. Peer review with blind attribution — remove names from code review
3. Rotate evaluators — no stable reviewer-author pairs
4. Delay bonuses — tie rewards to 90-day outcomes, not sprint completion
5. Spot-check with variable depth — sometimes deep audit, sometimes glance

The hardest part is patience. Good evaluation requires waiting for outcomes, and organizations hate waiting.