Search results

[Feature]: Add vakra small training data and make it the default --m3-data source

enhancementNew feature or requestNew feature or request

evaluation

Status: Open.

#61 In cuga-project/cuga-eval;

· haroldship opened on Jun 9, 2026

[Bug]: Azure GPT-4.1 content filter false positives silently zero out AppWorld tasks

bugSomething isn't workingSomething isn't working

Status: Open.

#60 In cuga-project/cuga-eval;

· Sergey-Zeltyn opened on Jun 9, 2026

[Feature]: Add --dotenv flag to layer .env overrides on top of model profiles

enhancementNew feature or requestNew feature or request

Status: Open.

#58 In cuga-project/cuga-eval;

· haroldship opened on Jun 9, 2026

[Feature]: Add Vakra LangGraph agent as a comparison target in compare.sh

enhancementNew feature or requestNew feature or request

evaluation

Status: Open.

#57 In cuga-project/cuga-eval;

· haroldship opened on Jun 9, 2026

[Feature]: Evaluation bundle must include full Vakra judge scores in all artifacts (cuga + react)

enhancementNew feature or requestNew feature or request

evaluation

Status: Open.

#56 In cuga-project/cuga-eval;

· haroldship opened on Jun 8, 2026

fix(m3): make failure banner reachable by selectively suppressing ERR around evaluator invocations

Status: Open.

#55 In cuga-project/cuga-eval;

· coderabbitai opened on Jun 8, 2026

feat(m3): domain-level checkpoint callback for partial saves on interrupt

Status: Open.

#54 In cuga-project/cuga-eval;

· coderabbitai opened on Jun 8, 2026

[Feature]: Remove internal Oak Health Insurance data from GitHub history, replace with external dataset

enhancementNew feature or requestNew feature or request

Status: Open.

#53 In cuga-project/cuga-eval;

· haroldship opened on Jun 7, 2026

[Feature]: Standardize and enrich report.md (Pass@1, pass@k/pass^k/maj@k, token/cost stats, breakdowns) across all benchmarks

enhancementNew feature or requestNew feature or request

evaluation

Status: Open.

#51 In cuga-project/cuga-eval;

· haroldship opened on Jun 7, 2026

Re-evaluate whether GHSA-r7w7-9xr2-qq2r pip-audit ignore is still needed

Status: Open.

#50 In cuga-project/cuga-eval;

· haroldship opened on Jun 7, 2026

Track langchain-docling support for docling >=2.92 to reverse CVE-2026-47214 ignore

Status: Open.

#49 In cuga-project/cuga-eval;

· haroldship opened on Jun 7, 2026

[Docs]: Correct use of React and Codeact terminology

documentationImprovements or additions to documentationImprovements or additions to documentation

Status: Open.

#46 In cuga-project/cuga-eval;

· Sergey-Zeltyn opened on Jun 4, 2026