docs: add OpenCode + Qwen3.6-27B benchmark report

[KrishOberoi]

Summary

Adds a reproducible report for a complete OpenCode + Qwen3.6-27B pilot across all 14 Enterprise-Bench L1-L2 tasks.

• Public Harbor job: https://hub.harborframework.com/jobs/80a2855e-6706-4b12-8759-f32cbe3d0e0a
• Harbor version: 0.20.0
• Agent: OpenCode 1.18.15
• Model: Qwen3.6-27B via Groq
• Judge: GPT-5.6 Terra
• Attempts/concurrency: k=1, n=1
• Result: 6/14 passed, 42.86% mean reward, 2 execution errors

This is submitted as an honest full-suite community-challenge run report rather than a leaderboard row. A single attempt per task cannot produce the required pass@2 through pass@10 metrics, so none have been fabricated.

Type of contribution

• Task addition or update
• Dataset addition or update
• Agent result submission
• Documentation
• Bug fix / setup improvement
• Other

Validation

• make validate passes
• I ran at least one affected task, or explained why not below
• Dataset changes are synthetic and safe to publish
• New/changed task criteria avoid answer leakage
• Documentation is updated

All 14 benchmark tasks were run. The public Harbor job and trajectories are accessible without authentication.

Commands run

UV_PYTHON=3.12 make validate

Core benchmark invocation, with credentials omitted:

harbor run \
  -p tasks \
  -a opencode \
  -m 'groq/qwen/qwen3.6-27b' \
  --mcp-config mcp.json \
  -k 1 -n 1 --yes \
  --jobs-dir jobs/opencode-groq-qwen36-terra-full-v4

Notes for reviewers

• The selected job completed 14/14 trials and retains every unsuccessful result.
• eng-l1-c ended with NonZeroAgentExitCodeError after an invalid tool call.
• sales-l2-a ended with a provider ApiRateLimitError.
• This k=1 pilot is not claimed to be equivalent to the leaderboard's 140-trial methodology.
• The report contains no credentials or private data.

[nimit2801]

Thanks for the submission. First-touch impression: this is a documentation-only agent result report, not a leaderboard row.

Checks: no GitHub status checks are currently reported on the PR, though the author says UV_PYTHON=3.12 make validate passed. I will reassess if checks appear.

Result/source: Harbor job is listed as public at https://hub.harborframework.com/jobs/80a2855e-6706-4b12-8759-f32cbe3d0e0a with OpenCode 1.18.15 + Qwen3.6-27B via Groq, reporting 6/14 passed, 42.86% mean reward, and 2 execution errors.

Main risk: readers may compare a k=1/n=1 pilot with leaderboard methodology, so the non-leaderboard framing should stay clear.

Maintainer verification points:

1. Confirm the Harbor job and trajectories are publicly accessible without auth.
2. Spot-check the reported score/errors against the linked job.
3. Confirm the doc clearly avoids pass@2-pass@10 or leaderboard claims.

[KrishOberoi]

Thanks for the review. I independently rechecked each verification point using unauthenticated HTTP requests:

1. Public job and trajectories: The job opens publicly, and representative trial pages expose the trajectory, verifier, artifacts, config, lock, log, and exception tabs:

   • Passing trial: https://hub.harborframework.com/jobs/80a2855e-6706-4b12-8759-f32cbe3d0e0a/trials/7bbf3a51-c1b8-48f6-8eed-46cf57baa14e
   • NonZeroAgentExitCodeError: https://hub.harborframework.com/jobs/80a2855e-6706-4b12-8759-f32cbe3d0e0a/trials/6bf9444c-86ce-43f3-8074-d5d7fd80614a
   • ApiRateLimitError: https://hub.harborframework.com/jobs/80a2855e-6706-4b12-8759-f32cbe3d0e0a/trials/aee6d5cf-a43c-402b-ade5-2db36bda661b

2. Reported results: The public job reports 14/14 completed, 2 errors, and average reward 0.43. Its trial records contain six reward-1 outcomes, giving 6 / 14 = 42.857%. The two public error pages report the same exception types documented in the report.

3. Non-leaderboard framing: The report explicitly labels the run a k=1 pilot, states that pass@2 through pass@10 cannot be produced, and says it is not a 140-trial leaderboard-equivalent evaluation. No pass@k values or leaderboard row are claimed.

GitHub created CI run https://github.com/devrev/enterprise-bench/actions/runs/31281779123 with conclusion action_required, so the fork workflow has not executed yet and no status check is attached to the PR. Locally, UV_PYTHON=3.12 make validate completed successfully, including Ruff and the repository validator.