-
Notifications
You must be signed in to change notification settings - Fork 15
All issues
Issue creation is restricted in this repository
- #20 · MSKazemi opened
on Aug 8, 2026
Issues
is:issue state:open
is:issue state:open
Search results
Generate and send a real RBAC-hard-fail fixture to the EvalPort spec (discussion #51)
area: docsDocumentation and examplesDocumentation and examplesStatus: Open.#85 In MSKazemi/aobench;Ten tasks grant tool families that do not exist — three dev tasks get zero tools, and validate passes
area: benchmarkTasks, environments, snapshotsTasks, environments, snapshotsbugSomething isn't workingSomething isn't workingeffort: smallA few hours — good place to startA few hours — good place to starthelp wantedExtra attention is neededExtra attention is neededStatus: Open.#74 In MSKazemi/aobench;Flaky release gate: test_report_json_flag_emits_clean_json_on_stdout fails ~1 run in 8
area: infraBuild, CI, packaging, deploymentBuild, CI, packaging, deploymentbugSomething isn't workingSomething isn't workingeffort: mediumRoughly a dayRoughly a dayhelp wantedExtra attention is neededExtra attention is neededStatus: Open.#72 In MSKazemi/aobench;Two dev-split tasks are unpassable:
my own jobis owned by someone else, so a correct agent is RBAC hard-failedarea: benchmarkTasks, environments, snapshotsTasks, environments, snapshotsbugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is neededStatus: Open.#63 In MSKazemi/aobench;Verify every command in the docs actually does what the page says — from a clean checkout
area: docsDocumentation and examplesDocumentation and examplesdocumentationImprovements or additions to documentationImprovements or additions to documentationeffort: mediumRoughly a dayRoughly a daygood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededStatus: Open.#62 In MSKazemi/aobench;[Result] Claude Sonnet Benchmarking
area: reportingReports, leaderboard, scorecardsReports, leaderboard, scorecardsStatus: Open.#60 In MSKazemi/aobench;AWS Bedrock adapter
area: adaptersModel/agent adapters (OpenAI, LiteLLM, Gemini, Bedrock, MCP)Model/agent adapters (OpenAI, LiteLLM, Gemini, Bedrock, MCP)effort: mediumRoughly a dayRoughly a dayenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.Google Gemini adapter
area: adaptersModel/agent adapters (OpenAI, LiteLLM, Gemini, Bedrock, MCP)Model/agent adapters (OpenAI, LiteLLM, Gemini, Bedrock, MCP)effort: mediumRoughly a dayRoughly a dayenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.LiteLLM adapter — one file, ~100 providers
area: adaptersModel/agent adapters (OpenAI, LiteLLM, Gemini, Bedrock, MCP)Model/agent adapters (OpenAI, LiteLLM, Gemini, Bedrock, MCP)effort: mediumRoughly a dayRoughly a dayenhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is neededStatus: Open.Run a model and submit a leaderboard entry — negative results welcome
area: reportingReports, leaderboard, scorecardsReports, leaderboard, scorecardseffort: smallA few hours — good place to startA few hours — good place to startgood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededStatus: Open.Write a second task for a thin QCAT × role cell (32 of 50 have only one)
area: benchmarkTasks, environments, snapshotsTasks, environments, snapshotseffort: mediumRoughly a dayRoughly a daygood first issueGood for newcomersGood for newcomershelp wantedExtra attention is neededExtra attention is neededStatus: Open.Triage the 96 CodeQL findings — path injection on the server surfaces first
area: infraBuild, CI, packaging, deploymentBuild, CI, packaging, deploymenteffort: mediumRoughly a dayRoughly a dayhelp wantedExtra attention is neededExtra attention is neededtype: tech-debtCleanup, typing, lint, flaky testsCleanup, typing, lint, flaky testsStatus: Open.