fix: health readiness/liveness split, service-auth key rotation, transport-aware CSRF Secure flag, and reqId in error bodies (#562, #555, #560, #558)

[TheCodingChef-eth]

Summary

This PR resolves four api-gateway issues in a single branch, with each file committed independently.
closes #562 — Health endpoint readiness vs liveness split
Problem: /api/health returned 503 for unhealthy status, but there was no dedicated liveness probe. Load balancers need a simple liveness check (always 200 when the process is up) separate from the readiness check.
Solution:

• Added GET /api/health/live — always returns 200 { status: "alive" } when the gateway process is running. Load balancers use this for liveness.
• GET /api/health remains the readiness probe — returns 503 when critical dependents (DB, upstream engines) are down.
• GET /api/health/all continues to aggregate all service health.
  Readiness 503 demo:
  GET /api/health → 503 { status: "unhealthy", service: "api-gateway", ... }
  GET /api/health/live → 200 { status: "alive", service: "api-gateway" }
  closes api-gateway service-auth plugin doesn't rotate internal service keys #555 — Service-auth key rotation
  Problem: Internal service credentials were static with no rotation mechanism. Compromise required full redeploy.
  Solution:
• createServiceAuth() and registerServiceAuth() now accept string | string[]. Pass an array of secrets to support overlapping keys during rotation.
• During rotation, both old and new keys are accepted. Once all consumers have switched, remove the old key.
• AppOptions.interServiceSecret added to buildApp() for testing rotation scenarios.
  Rotation demo:
  // Deploy with both keys during rotation window
  registerServiceAuth(fastify, [OLD_SECRET, NEW_SECRET]);

// After all consumers switch, deploy with only the new key
registerServiceAuth(fastify, NEW_SECRET);
closes #560 — CSRF cookie Secure flag transport-aware
Problem: buildCsrfCookieHeader set Secure only when NODE_ENV === 'production'. Staging HTTPS environments in NODE_ENV=development served cookies without Secure.
Solution:

• Added buildCsrfCookieHeader(name, value, request) utility that inspects the actual request protocol (request.protocol or x-forwarded-proto header) instead of NODE_ENV.
• HTTPS always gets Secure regardless of env. HTTP dev still works without Secure.
  Transport-aware demo:
  // HTTPS request → Secure flag set
  buildCsrfCookieHeader('csrf', 'token', { protocol: 'https' })
  // → "csrf=token; Path=/; SameSite=Strict; HttpOnly; Secure"

// HTTP request → no Secure flag
buildCsrfCookieHeader('csrf', 'token', { protocol: 'http' })
// → "csrf=token; Path=/; SameSite=Strict; HttpOnly"
closes #558 — reqId in error response bodies
Problem: Error responses didn't include the request id, so support couldn't correlate client complaints with server logs.
Solution:

• ErrorResponse interface now includes optional reqId field.
• createErrorResponse() accepts optional reqId parameter.
• Error handler passes request.id to all error responses (Zod 400, Fastify errors, generic 500).
• Header (x-request-id) and body (reqId) now match for correlation.
  Error body with reqId:
  {
  "error": {
  "code": "VALIDATION_ERROR",
  "message": "Invalid request data",
  "reqId": "550e8400-e29b-41d4-a716-446655440000"
  }
  }
  Files changed
  File Issue Description
  services/api-gateway/src/health.ts api-gateway health endpoint responds 200 even when some dependents fail #562 Add /api/health/live liveness endpoint
  services/api-gateway/src/health-all.test.ts api-gateway health endpoint responds 200 even when some dependents fail #562 Tests for liveness 200 and readiness 503
  shared/validation/plugins.ts api-gateway service-auth plugin doesn't rotate internal service keys #555,api-gateway CSRF cookie lacks Secure flag in non-prod HTTPS #560,api-gateway genReqId is generated but not used in error responses #558 Key rotation, CSRF Secure, reqId in error handler
  shared/validation/plugins.test.ts api-gateway service-auth plugin doesn't rotate internal service keys #555,api-gateway CSRF cookie lacks Secure flag in non-prod HTTPS #560,api-gateway genReqId is generated but not used in error responses #558 Tests for all three features
  services/api-gateway/src/index.ts api-gateway service-auth plugin doesn't rotate internal service keys #555 Add interServiceSecret to AppOptions
  services/api-gateway/src/service-auth.test.ts api-gateway service-auth plugin doesn't rotate internal service keys #555 Key rotation tests on gateway
  shared/validation/index.ts api-gateway genReqId is generated but not used in error responses #558 Add reqId to ErrorResponse
  shared/validation/errors.ts api-gateway genReqId is generated but not used in error responses #558 Add reqId to ErrorResponse
  services/api-gateway/src/error-response.test.ts api-gateway genReqId is generated but not used in error responses #558 Verify reqId in gateway errors
  services/api-gateway/src/genReqId-usage.test.ts api-gateway genReqId is generated but not used in error responses #558 Verify error handler passes reqId

[drips-wave]

@TheCodingChef-eth Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits