Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -230,3 +230,4 @@ await emitPagesChanged(pagesChanged)
- `docs/solutions/best-practices/diagnostic-patches-observability-discipline-2026-05-20.md` — stderr discipline; this applies the same discipline with the opposite operational goal (fail-soft, not fail-loud).
- `docs/solutions/best-practices/privacy-gate-promotion-leak-prevention-2026-06-04.md` — public-surface leak prevention; this adds the "don't render a placeholder public value" rule.
- `docs/solutions/workflow-issues/github-actions-step-output-interpolation-2026-04-21.md` — pass step outputs through `env:`, never interpolate into `run:`.
- `docs/solutions/best-practices/requirements-doc-survives-verification-2026-06-24.md` — reconcile against the deployed verifier's code, not the design doc; the "live source over prose" discipline generalized to requirements documents.
Original file line number Diff line number Diff line change
Expand Up @@ -97,3 +97,6 @@ Prose describes intent. Live data describes reality. Ground the review in realit
- [Agent and automation steps need their GitHub token wired explicitly](../workflow-issues/required-github-token-for-agent-steps-2026-06-22.md) — the companion pattern: restrict capability by token scope, not by token absence; keep privileged operations in a separate step with its own credential.
- [Survey workflow-side privacy gate](../security-issues/survey-workflow-side-privacy-gate-2026-05-16.md) — the same separate-step-with-its-own-credential pattern, where a privacy gate runs under a distinct token so the boundary is enforced by which step holds which credential.
- [Diagnostic patches observability discipline](../best-practices/diagnostic-patches-observability-discipline-2026-05-20.md) — verify behavior against live state rather than assumptions; the same discipline applied to observability patches.
- [Writing a requirements doc that survives verification](requirements-doc-survives-verification-2026-06-24.md) —
validate a plan's security invariants against live data, not its prose; the same "ground the
review in reality" discipline applied to requirements documents.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Privacy Gate Design for Data→Main Promotion Leak Prevention
date: 2026-06-04
last_updated: 2026-06-04
last_updated: 2026-06-24
verified: 2026-06-04
category: best-practices
module: github-workflows
Expand Down Expand Up @@ -194,3 +194,18 @@ return {ok: false, matchedFiles: redactedFiles}
fallback path must reuse the exact fail-closed predicates of the main gate.
- Issues: #3407 (wire the gate), #3408 (operator-actionable blocked output), #3429 (resolver PAT
hygiene), #3430 (redact node_ids in failure output), #3424 (accepted commit-history exposure).

## See also — privacy-gate correctness patterns

- [Structured-first attribution for public-allowlist privacy gates](wiki-page-structured-attribution-2026-06-04.md) —
three-state frontmatter read (absent / present-but-malformed / present-with-URLs) and the
substring/prefix/truthy leak vectors that structured attribution closes.
- [Survey workflow-side privacy gate](../security-issues/survey-workflow-side-privacy-gate-2026-05-16.md) —
defense-in-depth at the dispatch boundary: the workflow is its own privacy boundary, not a
downstream consumer of someone else's gate.
- [Pure-core privacy gates with a shared module and mutation-proof tests](pure-core-privacy-gates-shared-module-2026-06-22.md) —
gate in the pure core before sensitive data enters shared state; one shared module so
chokepoints cannot diverge; mutation-proof tests that fail when the gate is removed.
- [Verify the whole public perimeter](../security-issues/verify-whole-public-perimeter-2026-06-22.md) —
enumerate every public surface before claiming a privacy invariant holds; a gate that covers
only the surfaces you thought of is not a gate.
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ it('mutation proof: digest assembly drops private prose', () => {

## Related

- [Privacy-gate promotion leak prevention](../best-practices/privacy-gate-promotion-leak-prevention-2026-06-04.md) — the trusted-chokepoint and fail-closed-resolution patterns this gate extends into the pure core.
- [Wiki page structured attribution](../best-practices/wiki-page-structured-attribution-2026-06-04.md) — the present-but-empty vs absent distinction recurs here; encode it as a habit.
- [Privacy-gate promotion leak prevention](../best-practices/privacy-gate-promotion-leak-prevention-2026-06-04.md) — the trusted-chokepoint and fail-closed-resolution patterns this gate extends into the pure core. Its "See also — privacy-gate correctness patterns" section indexes the full cluster of gate docs, including this one.
- [Wiki page structured attribution](../best-practices/wiki-page-structured-attribution-2026-06-04.md) — the present-but-empty vs absent distinction recurs here; encode it as a habit. The three-state frontmatter read (absent / present-but-malformed / present-with-URLs) is the same discipline applied to structured provenance.
- [Survey workflow-side privacy gate](../security-issues/survey-workflow-side-privacy-gate-2026-05-16.md) — verify privacy inside the trusted workflow before any public side effect.
- [Private repo dispatch visibility gate](../security-issues/private-repo-dispatch-visibility-gate-2026-05-08.md) — the fail-closed predicate and opaque-identifier redaction this gate builds on.
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
---
title: Writing a requirements doc that survives verification
date: 2026-06-24
last_updated: 2026-06-24
problem_type: best_practice
category: best-practices
component: development_workflow
module: github-workflows
severity: high
verified: 2026-06-24
tags:
- requirements
- verification
- closed-schema
- dual-source
- sequencing
- planning
applies_when:
- writing or reviewing a requirements or brainstorm document
- the document touches a closed schema or a dual-source constraint
- a success criterion removes infrastructure while a replacement decision is still open
- a reviewer dissents and the dissent is at risk of being erased rather than recorded
---

# Writing a requirements doc that survives verification

## Context

Requirements docs that hand-wave get falsified in review. The strong ones make checkable claims:
every cited file, step, and contract can be verified against the live tree. The weak ones
describe intent in prose that sounds correct until someone opens the actual source.

Two failure modes are especially costly because they are unrecoverable if violated:

1. **Closed-schema dual-source misses.** A schema that is closed by construction — for example,
an event-gateway `EventType` union plus a `VALID_EVENT_TYPES` runtime set — requires updating
both sources in lockstep. A control-plane POST that sends an event type not present in both
sources becomes a hard `400`. The validator enforces completeness; it does not forgive a
partial update.

2. **Removal-before-replacement gaps.** A success criterion that removes infrastructure (e.g.,
a webhook poster) while an open question leaves the replacement's dormant-vs-live decision
unsettled can strand a daily run with no coverage in the window between removal and
replacement going live.

Neither failure announces itself during planning. Both surface in production.

## Guidance

### 1. Make checkable claims

Every cited file, step, and contract in the requirements doc must exist in the live tree. Verify
each one before the doc is considered complete:

- "The gateway validates `EventType` against `VALID_EVENT_TYPES`" → open the gateway source and
confirm the union and the runtime set are both present and in sync.
- "The webhook poster runs daily at 06:00 UTC" → open the workflow file and confirm the cron.
- "Redacted entries use `node_id` as the stable join key" → read the live metadata and confirm
the shape.

Prose describes intent. Live source describes reality. Ground the doc in reality.

**Before (hand-wavy):**
> The gateway will validate the event type before posting.

**After (checkable):**
> The gateway validates `event_type` against the `EventType` union in `src/types.ts` and the
> `VALID_EVENT_TYPES` set in `src/validate.ts`. Both must be updated when a new event type is
> added; a partial update produces a hard `400`.

### 2. Name closed-schema dual-source constraints explicitly

When a schema is closed by construction, the doc must name both sources and state the lockstep
requirement. "Update the schema" is not enough — it must say "update both `EventType` in
`src/types.ts` and `VALID_EVENT_TYPES` in `src/validate.ts`."

The dual-source update pair is the unit of correctness. A reviewer who sees only one source
updated should block the change.

```ts
// Both must be updated together — the validator enforces completeness
export type EventType = 'survey.completed' | 'invitation.accepted' | 'repo.archived'

export const VALID_EVENT_TYPES = new Set<EventType>([
'survey.completed',
'invitation.accepted',
'repo.archived',
])
```

### 3. Record dissent rather than erasing it

When a reviewer raises an objection that is heard but not adopted, record it as
advisory-but-not-adopted in the doc. The road-not-taken is part of the design record. A future
reader who encounters the same objection will know it was considered, not overlooked.

> **Reviewer note (advisory, not adopted):** Suggested deferring removal of the webhook poster
> until the gateway replacement is confirmed live. Accepted the risk given the short window;
> tracked as a sequencing dependency in the open questions.

Erasing the dissent makes the doc look more confident than it is and loses the context that
would help a future reader understand why the sequencing was chosen.

### 4. Surface sequencing gaps between removal and replacement

A success criterion that removes infrastructure while a replacement decision is still open is a
sequencing gap. Name it explicitly in the open questions section:

> **Open question:** Is the gateway replacement live before the webhook poster is removed, or
> does a window exist where neither fires? If a window exists, what is the fallback for daily
> runs in that window?

A gap that is named is a gap that can be closed. A gap that is implicit becomes a production
incident.

## Why This Matters

- **Closed-schema dual-source misses** produce hard `400`s that are unrecoverable without a
code change and redeploy. The validator enforces completeness; it does not forgive a partial
update.
- **Removal-before-replacement gaps** strand daily runs with no coverage. The window may be
short, but it is real, and it is invisible in the requirements doc unless named.
- **Erased dissent** removes the design record that would help a future reader understand why
a risky sequencing was chosen. The road-not-taken is part of the architecture.

## When to Apply

Apply this discipline when writing or reviewing any requirements or brainstorm document,
especially ones that:

- Touch a closed schema or a dual-source constraint.
- Include a success criterion that removes infrastructure.
- Have a reviewer dissent that is at risk of being resolved by deletion.

## Examples

### Dual-source update pair

A new event type `repo.archived` must appear in both sources:

```ts
// src/types.ts — add to the union
export type EventType = 'survey.completed' | 'invitation.accepted' | 'repo.archived'

// src/validate.ts — add to the runtime set
export const VALID_EVENT_TYPES = new Set<EventType>([
'survey.completed',
'invitation.accepted',
'repo.archived', // ← must be added here too; omitting produces a hard 400
])
```

### Sequencing gap made explicit

> **Success criterion:** Remove the legacy webhook poster from `owner/example-repo`.
>
> **Open question (sequencing):** The gateway replacement must be confirmed live and posting
> correctly before the webhook poster is removed. If the poster is removed first, the daily
> `survey.completed` event has no sender for the duration of the gap. Resolution: gate the
> removal PR on a confirmed gateway post in the prior 24 hours.

## See also

- [Byte-exact HMAC signing and fail-soft telemetry](byte-exact-gateway-signing-and-fail-soft-telemetry-2026-06-04.md) —
reconcile against the deployed verifier's code, not the design doc; the same "live source
over prose" discipline applied to a signing contract.
- [A second credential gives rotation isolation, not permission isolation](credential-mint-time-permission-scoping-2026-06-22.md) —
validate a plan's security invariants against live data, not its prose.
- [Verify the whole public perimeter](../security-issues/verify-whole-public-perimeter-2026-06-22.md) —
enumerate every public surface before claiming a privacy or security invariant holds.
- [Observability before structural change](observability-before-structural-change-2026-06-09.md) —
confirm the current behavior is observable before removing or replacing the path that produces it.
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,8 @@ if (structuredSources !== null) {

- `docs/solutions/best-practices/privacy-gate-promotion-leak-prevention-2026-06-04.md` — the
companion promotion-diff gate; same fail-closed-under-uncertainty principle, different surface
(content scan vs slug attribution).
(content scan vs slug attribution). See also its "See also — privacy-gate correctness patterns"
index for the full cluster of gate docs.
- `docs/solutions/security-issues/private-repo-dispatch-visibility-gate-2026-05-08.md` — the
fail-closed-on-unknown predicate this attribution model extends.
- `docs/solutions/security-issues/survey-workflow-side-privacy-gate-2026-05-16.md` — verifying
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -167,3 +167,4 @@ git log --oneline -15 # recent-change context

- `.agents/skills/generating-project-docs/SKILL.md` — the skill this pattern operationalizes
- `docs/solutions/runtime-errors/octokit-invitation-method-names-2026-04-17.md` — sibling discipline doc
- `docs/solutions/workflow-issues/safe-superseded-workflow-removal-2026-06-24.md` — the workflow-file analog: retiring a superseded workflow requires the same "no remaining callers, shared dependencies preserved" confirmation before deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -262,3 +262,4 @@ The breadth of the recheck gate matters as much as its existence. Routing the re
- Related (diagnostic discipline): `docs/solutions/best-practices/diagnostic-patches-observability-discipline-2026-05-20.md` — the three-PR investigation that surfaced the stderr-swallowing, printf-marker, and App-token issues fixed inline above
- Compliance: `docs/solutions/workflow-issues/github-actions-step-output-interpolation-2026-04-21.md` — env-var-only shell expansion pattern used in every new `run:` block
- Compliance: `docs/solutions/runtime-errors/autonomous-pipeline-silent-failures-2026-04-19.md` — aggregate status semantics in `Survey Repo`; same principle of "downstream steps must check upstream conclusions, not assume agent success"
- Related (promotion-boundary gate): `docs/solutions/best-practices/privacy-gate-promotion-leak-prevention-2026-06-04.md` — the trusted-chokepoint and fail-closed-resolution patterns that extend this workflow-side gate to the `data→main` promotion boundary.
Original file line number Diff line number Diff line change
Expand Up @@ -115,3 +115,13 @@ Bad split:
- reusable tracker calls inherit every repository secret
- tracker runs have no concurrency group
- daily-digest announce steps run during custom tracker prompts

## Related

- [Safe workflow consolidation: trace every invariant and caller](workflow-consolidation-invariant-trace-2026-06-24.md) —
when collapsing two scheduled workflows into one, trace each safety invariant and remaining
caller explicitly; the concurrency-group and dispatch-entry-point patterns here are the
invariants that consolidation must preserve.
- [Safely deleting a superseded workflow or signal path](safe-superseded-workflow-removal-2026-06-24.md) —
the inverse lesson: before retiring a duplicate execution path, confirm no remaining callers,
full replacement coverage, shared dependencies preserved, and no dangling declarations.
Loading
Loading