Skip to content

Skill Finder suggests Codex injected AGENTS.md context as a custom skill opportunity #117

Description

@samuelkahessay

Description

Skill Finder can surface Codex session-start injected context as a custom skill opportunity.

Example suggestion seen locally:

  • load-agents-instructions
  • Reason: "Repeatedly loads repo-specific AGENTS.md instructions before work, which is a consistent setup step across repositories."
  • Examples begin with # AGENTS.md instructions for ... <INSTRUCTIONS>

These are not user prompts or repeatable developer workflows. They are harness-injected instruction/context payloads stored in Codex JSONL as user-role records.

Steps to Reproduce

  1. Use the extension against Codex session logs that include session-start injected AGENTS.md context.
  2. Open Skill Finder.
  3. Run Analyze.
  4. Observe a custom skill opportunity based on loading AGENTS.md instructions rather than on an explicit user task.

Expected Behavior

Skill Finder should exclude injected instruction/context payloads from repeated workflow clustering and AI triage.

Actual Behavior

Injected Codex context can be clustered as repeated prompts and triaged into a custom skill opportunity.

Evidence

Local Codex logs contain records like:

{
  "type": "response_item",
  "payload": {
    "type": "message",
    "role": "user",
    "content": [
      {
        "type": "input_text",
        "text": "# AGENTS.md instructions for /path/to/repo\n\n<INSTRUCTIONS>..."
      },
      {
        "type": "input_text",
        "text": "<environment_context>...</environment_context>"
      }
    ]
  }
}

Because these are recorded as role: "user", the current Codex parser can expose them as messageText, and WorkflowAnalyzer does not currently filter this pattern as noise.

Proposed Fix

Filter these payloads at parse time in parser-codex.ts, and defensively filter them in analyzer-workflows.ts in case similar injected context reaches workflow clustering from another parser.

Local patch tested:

  • parser-codex.ts: ignore messages starting with # AGENTS.md instructions for ...\n\n<INSTRUCTIONS> and standalone <environment_context>...</environment_context> payloads.
  • analyzer-workflows.ts: treat the same patterns as noise.
  • Added regression tests in parser-codex.test.ts and analyzer-workflows.test.ts.

Targeted test run passed locally:

npm test -- src/core/parser-codex.test.ts src/core/analyzer-workflows.test.ts

Result: 2 test files passed, 21 tests passed.

Extension Version

0.1.0, locally built from source.

VS Code Version

1.123.2

Operating System

WSL/Linux remote extension host on Windows.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions