Skip to content

Prevent AI-Induced Version Downgrades (Knowledge Cutoff Regression) #3003

Description

@arii

UNDERSTAND THE ISSUE

AI code generators and reviewers frequently suggest outdated versions for pnpm packages, GitHub Actions, LLM models, and Node.js. This occurs because the model's training data cutoff causes it to perceive newer, valid versions as hallucinations, defaulting to the latest version it "remembers."

DETERMINE APPROACH

Implement a deterministic validation layer and context-injection step to prevent the AI from downgrading dependencies and locking core runtime versions.
To address this deterministically within an autonomous CI/CD loop, we should decouple the version validation from the LLM entirely using a three-pronged approach:

  1. Pre-Flight Context Enrichment: Inject current stack versions into the AI reviewer's system prompt prior to execution so it has factual grounding.
  2. Post-Generation Guardrails: Run a script using a semantic versioning library to compare the HEAD version against the AI's proposed version. Automatically strip downgrades.
  3. Hard File/Line Locks: Implement a regex or AST-based block that monitors the Node version specifically.

SPECIFY SCOPE

  • Version Change Detection in package.json, .github/workflows/, and config files.
  • Deterministic validation script checking against npm/GitHub.
  • Context injection updates for td_cli prompt builder.
  • Strict locks on .nvmrc and engines.node.

DEFINITION OF DONE

  • Pipeline intercepts and parses AI-proposed version diffs.
  • Downgrades are automatically rejected via semantic version comparison.
  • System prompt injects the latest supported versions.
  • Hard block on Node.js version modification unless explicitly overridden.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions