Skip to content

Mine session context for audit-relevant insights before auditing #13

@mickeylorenzini

Description

@mickeylorenzini

Problem

The harden skill currently sources audit criteria from two places:

  1. Web research (Step 1) — external best practices and canonical standards
  2. The deliverable itself (Step 2) — thematic and structural passes

It never systematically mines the session context (conversation history) for insights that should inform the audit. Whether conversational context gets caught depends entirely on whether the auditor happens to remember it from earlier in the context window. There is no explicit step that says "scan the session for insights, constraints, decisions, and feedback that should be reflected in or inform the audit of this deliverable."

Why This Matters

Session context is often the richest source of audit-relevant information that isn't in the deliverable yet:

  1. User corrections and feedback — "No, it should handle X case" gets discussed but may not land in the artifact
  2. Design rationale and rejected alternatives — Why approach B was chosen over A; constraints that motivated decisions
  3. Edge cases surfaced during brainstorming — Often identified verbally, then forgotten during implementation
  4. Domain constraints stated conversationally — "Our lab can't do X" or "The reviewer will care about Y"
  5. Acceptance criteria implied but not formalized — The user described what success looks like in conversation, but the deliverable doesn't reflect all of it

This is structurally analogous to the "audit without research" anti-pattern already documented in the skill — auditing against internal assumptions rather than importing external criteria. Session context is a second external source (alongside web research) that the skill should systematically mine before auditing.

Proposed Solution

Add a new sub-step to Step 1 (or a standalone Step 1.5) that systematically reviews session context before auditing:

  • Scan conversation for user feedback, corrections, constraints, and design decisions
  • Extract acceptance criteria implied but not formalized
  • Identify edge cases or requirements discussed but potentially not reflected in the deliverable
  • Feed these as additional audit criteria into Step 2

Consider also adding a corresponding anti-pattern: "Context-blind auditing" — auditing a deliverable produced during a rich session without reviewing that session for unincorporated insights. The deliverable is the output; the session contains the intent.

Likely Affected Files/Modules

  • skills/harden/SKILL.md — Steps 1 and 2, Anti-Patterns section

Acceptance Criteria

  • Harden skill includes an explicit step to mine session context for audit-relevant insights before or during Step 2
  • Session-derived insights are surfaced alongside web research findings as audit criteria
  • A corresponding anti-pattern is documented warning against context-blind auditing
  • The step includes guidance on what to look for (feedback, constraints, edge cases, implied acceptance criteria, design rationale)
  • The step is scoped to avoid redundant work when session context is minimal (e.g., user just said "harden this" with no prior discussion)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions