Skip to content

Release & rollback runbook (HITL) #34

Description

@tkowalczyk

Parent PRD

#22

What to build

This is the one HITL slice in the set. Its definition of done requires a human to walk the documented sequence against real infrastructure. Every other slice in this breakdown is verifiable by tests or CI.

Write the release and rollback runbook that makes manual deployment a repeatable procedure rather than a remembered one.

Deployment stays manual by decision, not by omission — see the deployment model decision in the PRD. A continuous deployment pipeline was considered at priority zero in the original audit and deliberately rejected: for a template it would encode account identifiers, secret names, and branch conventions that no cloner shares, and it could not be meaningfully exercised in this repository. The gap the audit identified is real, but it is a documentation gap, and this slice closes it.

The runbook covers, per environment, in order: the migration gate, the environment-scoped build, the deploy, and how to verify the result. It also documents the two operational procedures that currently exist nowhere — shipping a risky change to a fraction of traffic first, and recovering from a bad deploy without improvising under pressure.

The manual migration gate should be documented as a deliberate choice, so that its absence from automation reads as a decision rather than an oversight.

Note the interaction with #23: once required secrets are declared, deploy fails with a named list when one is unset. The runbook should say so, because that check is the safety net that partially compensates for having no pipeline.

Acceptance criteria

  • An ordered release sequence is documented per environment, covering migration, build, deploy, and verification
  • The manual migration gate is documented as a deliberate decision with its reasoning, not presented as a limitation
  • Rollback is documented with the command that performs it
  • Gradual rollout is documented with the command that performs it
  • Deploy-time secret validation from Secrets contract & typegen determinism #23 is described as part of the flow
  • Human verification: a maintainer follows the documented sequence for one environment end to end without consulting external documentation, and confirms it is complete and correct
  • Any step that turned out to need external documentation is folded back into the runbook
  • Existing verification passes: type-check, tests, lint

Blocked by

User stories addressed

Reference by number from the parent PRD:

  • User story 28
  • User story 29
  • User story 30
  • User story 31

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestworkers-best-practicesCloudflare Workers best-practices audit finding

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions