Skip to content

Add Go (Golang) Polyglot Benchmark Suite (feat-go-ratelimiter) #4

Description

@Kaap10

Description

AgentBench currently supports Python (pytest) and Node.js (node --test). Adding Go (Golang) benchmark tasks evaluated with go test will expand polyglot evaluation capabilities for AI agents across systems languages.

Proposed Solution

  1. Create benchmarks/repos/go-rate-limiter/ containing a token-bucket rate limiter implementation (limiter.go) and test suite (limiter_test.go).
  2. Create benchmarks/tasks/13-feat-go-ratelimiter.yaml with evaluation command go test -v ..
  3. Enhance backend/app/evaluation/test_parser.py with parse_go_test() to extract PASS and FAIL counts from go test outputs.

Relevant Files

  • backend/app/evaluation/test_parser.py
  • benchmarks/tasks/13-feat-go-ratelimiter.yaml
  • benchmarks/repos/go-rate-limiter/

Acceptance Criteria

  • python agentbench.py run --benchmark feat-go-ratelimiter --provider mock --model mock-coder completes and computes score correctly.
  • Test output parser correctly captures Go assertion outputs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions