Skip to content

Fail-closed runtime contract for workflow step outputs #313

@Christoffer91

Description

@Christoffer91

Problem

Workflow steps are still too trusting of agent output. In practice, step output can be treated as "good enough" even when the contract is loose, STATUS is missing, JSON payloads are malformed, or verifier output is not machine-safe enough.

The most concrete deep-research failure mode is that verifier output can be incomplete or malformed without the runtime reliably stopping the writer step.

This makes workflows fragile and allows bad output to propagate as if it were valid.

Goal

Make workflow step output fail-closed in the runtime, with particular focus on deep-research.

Scope

Update the workflow runtime so that:

  • STATUS is required
  • only known statuses are accepted:
    • done
    • retry
    • blocked
  • all *_JSON fields are validated as real JSON
  • verifier output that is missing or contains a malformed packet cannot proceed to writer
  • expects is actually enforced before the pipeline advances
  • blocked becomes a real runtime state, not just a word in output

Primary files

  • src/installer/step-ops.ts
  • workflows/deep-research/workflow.yml

Secondary files if needed

  • src/installer/types.ts
  • src/installer/status.ts
  • src/cli/cli.ts

Implementation notes

  • Start in src/installer/step-ops.ts, not with a broad schema redesign
  • Keep existing string-based expects initially if that keeps the change smaller
  • Make validation strict inside completeStep()
  • Do not allow malformed RESEARCH_PACKET_JSON or VERIFIED_PACKET_JSON to pass as best-effort output
  • STATUS: blocked should set step/run to blocked and stop the pipeline

Acceptance criteria

  • step output without STATUS cannot mark a step as done
  • unknown STATUS cannot mark a step as done
  • malformed *_JSON cannot advance the pipeline
  • a verifier step without valid VERIFIED_PACKET_JSON cannot allow writer to proceed
  • expects mismatch is not treated as success
  • STATUS: blocked sets step/run to blocked and stops forward progress

Test plan

Add tests for:

  • missing STATUS
  • unknown STATUS
  • malformed RESEARCH_PACKET_JSON
  • malformed VERIFIED_PACKET_JSON
  • STATUS: blocked
  • verifier output that looks plausible but does not satisfy the contract
  • valid verifier output that correctly allows writer to proceed

Non-goals

  • full workflow schema redesign
  • generic taint/trust engine
  • prompt polishing
  • larger branching engine work

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions