Skip to content

Improve SDD assessor audit status detection for historical FAIL text #54

@ryderstorm

Description

@ryderstorm

Background

During review of PR #52, we found that skill/scripts/assess-sdd-state.py can treat an SDD2 audit as still failing when the audit document contains historical remediation text such as FAIL → PASS in a re-audit delta.

Example from the PR discussion:

Re-Audit Delta (Run 2)

  • Changed gate statuses since previous run:
    • Requirement-to-test traceability: FAIL → PASS. Run 1 had three unmapped FRs

That text describes a resolved failure, but a broad scan for FAIL can make the assessor think the workflow is still blocked on Phase 2.

Desired Outcome

Update the assessor so it determines the current audit state from an explicit current-status signal instead of treating any historical FAIL text as active failure evidence.

Potential approaches:

  • Prefer the executive summary or latest/current gate table when present.
  • Ignore historical/re-audit delta sections when determining current pass/fail state.
  • Add or document a deterministic marker that Phase 2 audits should emit for machine-readable status.
  • Add regression coverage with an audit containing historical FAIL → PASS text and a current passing status.

Context

Raised during PR #52 review:
#52 (comment)

This is follow-up work and does not need to block PR #52.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions