Skip to content

Improve output quality of execute workflow #253

Description

@JeremyDev87

Task Description

The quality of the code output from execute is poor, so I want to improve it. I would like to have an iteration that better understands the project context, verifies the quality, and makes corrections based on feedback.

Acceptance Criteria

  • Project Context Discovery

    • Automatically scan and summarize the project structure, tech stack, and key config files (e.g., package.json, pyproject.toml, pom.xml, etc.).
    • Identify and report the expected change scope (impacted modules/files) before applying changes.
  • Quality Verification

    • Run at least one verification pass after code generation/modification.
    • Execute at least two of the following when applicable: lint/format, tests, type-check/build, static checks.
    • Report results as pass/fail with a brief evidence/log summary and the commands executed.
  • Iterative Improvement (Fix Loop)

    • If verification fails or user feedback is provided, perform iterative corrections.
    • Each iteration logs: failure summary, suspected root cause, changes applied (per-file summary or diff summary), and re-verification results.
    • Support at least 2 iterations (e.g., max 3) with explicit stop conditions (all checks pass / max iterations reached / no further improvement possible).
  • Output Format and Change Quality

    • Final output uses a consistent structure:
      • Context Summary
      • Plan / Changes
      • Verification (commands + results)
      • Final Notes (risks / follow-ups)
    • Follow a “minimal necessary change” principle; avoid unnecessary refactoring.
  • Graceful Degradation

    • If lint/tests/build cannot run, explain why, provide best-effort alternative validation, and list exact commands for local/CI execution.
  • Regression Prevention

    • When feasible, add/extend a test or lightweight validation rule to prevent recurrence, or explicitly state why it’s not feasible.
  • Fail-Fast / Give-Up Capability

    • Provide a mechanism to stop and give up when the task is judged to be too difficult, too risky, or excessively resource-intensive.
    • When giving up, output: reason, what was attempted, and concrete next steps (e.g., what info is needed, how to reduce scope, suggested manual actions).

Additional Context

  • The current execute workflow often produces low-quality code because it doesn’t sufficiently incorporate project context (structure, conventions, dependencies, architectural patterns) and lacks reliable verification and iteration.
  • The desired improvement is a structured loop: understand context → apply changes → verify → iterate on failures/feedback → re-verify.
  • However, some tasks may be impractical in certain environments or project sizes (e.g., extremely large repos, expensive test suites, missing dependencies, strict time/memory limits).
    In those cases, the workflow needs an explicit “give up / fail-fast” capability: if the workflow determines the work would require excessive resources or is unlikely to succeed safely, it should stop rather than producing low-confidence output, and clearly explain why along with actionable next steps.

Metadata

Metadata

Assignees

Labels

leonidasTrigger Leonidas automation

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions