Skip to content

8.11 - Fix runbook's health-check endpoint, add a mainnet section - #180

Open
kelvinokwudili52-stack wants to merge 1 commit into
determined-001:mainfrom
kelvinokwudili52-stack:docs/runbook-ready-endpoint-and-mainnet
Open

8.11 - Fix runbook's health-check endpoint, add a mainnet section#180
kelvinokwudili52-stack wants to merge 1 commit into
determined-001:mainfrom
kelvinokwudili52-stack:docs/runbook-ready-endpoint-and-mainnet

Conversation

@kelvinokwudili52-stack

Copy link
Copy Markdown

closes #165

Summary

Two gaps in docs/RUNBOOK.md, both confirmed against the actual current code before writing anything:

1. Wrong health-check endpoint in the deploy step

Step 3 told the operator to watch /health go green. Checked against the actual gating logic:

  • render.yaml:33 and render.mainnet.yaml:44 both set healthCheckPath: /ready.
  • apps/api/Dockerfile:113's HEALTHCHECK also targets /ready.
  • apps/api/src/index.ts: /health returns ok: true unconditionally (liveness only); /ready calls container.ready() and returns 503 when the database isn't reachable.

So an operator following the old step could watch /health go green while /ready was still 503ing and no traffic was flowing. Corrected the step to /ready and added a line explaining the liveness/readiness distinction so it doesn't get reintroduced.

2. No mainnet section

Added "Mainnet: a second, separate service", placed right after the existing "Scaling past one instance" note (before the shared Deploy/Rollback/Restore/Key rotation procedures it references). It covers:

  • A table of what differs: service name (quay-api vs quay-api-mainnet), blueprint file, STELLAR_NETWORK, API URL, database, off-ramp mode.
  • An explicit "what's shared vs. per-environment" breakdown, per the issue's ask.
  • A ### ... — mainnet subsection under each of Deploy / Rollback / Restore / Key rotation, cross-referencing the shared procedure below it and calling out only what's actually different (which service's logs to watch, which backup belongs to which database, ANCHOR_URL/ANCHOR_HOME_DOMAIN/METRICS_TOKEN being mainnet-only secrets per render.mainnet.yaml's own header comment, etc.).
  • Cross-references to docs/MAINNET.md (the cutover procedure and Phase 5 verification checklist) rather than duplicating its content.

Verification

  • Confirmed the /ready vs /health behavior by reading apps/api/src/index.ts directly (lines 64-110), not just trusting the issue text.
  • Confirmed render.yaml, render.mainnet.yaml, and apps/api/Dockerfile's healthcheck all target /ready.
  • Cross-checked the new mainnet section against render.mainnet.yaml's secret list and docs/MAINNET.md's Phase 3/5/Rollback content for consistency.
  • Byte-scanned the changed file for stray control bytes before committing.
  • This is a docs-only change — no code, tests, or build affected. No pnpm typecheck/pnpm test/pnpm build run was needed, and none is available in this environment regardless (no pnpm/node_modules install here).

…ction

Step 3 of the deploy section told the operator to watch /health go green.
Both render.yaml and render.mainnet.yaml gate traffic on /ready
(healthCheckPath), and the Dockerfile HEALTHCHECK targets /ready too -
/health is liveness-only and returns ok: true unconditionally, so an
operator could watch the wrong endpoint go green while /ready was still
503ing on an unreachable database and no traffic was flowing. Corrected the
step to /ready and added a line explaining the distinction so it isn't
reintroduced.

Added a new 'Mainnet: a second, separate service' section covering which
Render service, which blueprint, which URL, and which secrets differ from
testnet, plus what's shared vs. per-environment for each of deploy,
rollback, restore and key rotation - cross-referencing docs/MAINNET.md for
the cutover procedure and Phase 5 verification checklist it must line up
with.
@drips-wave

drips-wave Bot commented Aug 29, 2026

Copy link
Copy Markdown

@kelvinokwudili52-stack Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@vercel

vercel Bot commented Aug 29, 2026

Copy link
Copy Markdown

@kelvinokwudili52-stack is attempting to deploy a commit to the determined's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8.11 - The runbook names the wrong health endpoint and has no mainnet section

1 participant