Skip to content

Implement agent reputation scoring with quality assessment and sybil resistance #497

Description

@devJaja

Summary

The qualityScorer.ts service exists, but needs a full reputation system that resists sybil attacks and rewards genuine quality. Reputation updates should be based on task outcomes, peer reviews, completion rate, and latency — not just task success.

Priority

P1

Files to work on

backend/src/services/qualityScorer.ts
backend/src/registry/registry.ts
backend/src/db/agents.ts

Requirements

  • Extend quality scoring to incorporate:
    • Task success/failure ratio (completed vs failed)
    • Output quality (via qualityScorer, e.g. report completeness, code compiles)
    • Response latency (faster agents score slightly higher, bounded)
    • Staking/bond amount (higher bond-backed agents get reputation weight multiplier)
    • No single wallet can create unlimited agents (agent-per-account cap)
  • Reputation scale: 0.0 – 5.0 with fractional precision
  • Decay: reputation of inactive agents decays 0.1/month of inactivity
  • Update flow: on task completed/failed, compute reputation delta and persist to DB
  • Expose GET /api/agents/:id returns current reputation and breakdown
  • Sybil resistance: enforce a max of agents per Stellar account; verify on-chain bond

Acceptance Criteria

  • Task success increases reputation; failure decreases it
  • Output quality score contributes to reputation delta
  • Higher-bond agents have a bounded reputation weight multiplier
  • Single account cannot register more than a configurable agent limit
  • Inactive agents' reputation decays over time
  • GET /api/agents/:id returns reputation breakdown (success, quality, latency, bond)
  • No agent can exceed 5.0 reputation

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions