OBS-04] Delivery instability monitoring: deployment frequency vs stability trade-off

[paruff]

[OBS-04] Delivery instability monitoring: deployment frequency vs stability trade-off

Labels: observability, devops, p2-important, effort-small
Assignee: 🤖 Copilot
Phase: Phase 7 — Observability
DORA 2025 Source: “AI adoption is linked to higher throughput AND higher instability”; “Deployment frequency and change failure rate do not need to be in trade-off in well-engineered organisations”
Effort: Small (< 1 day)
Prerequisites: OBS-01 (Sentry), DEVOPS-01 (CI), METRICS-02 (rework rate)

---

The DORA 2025 Finding That Requires This

The 2025 report contains two findings that appear to contradict each other — but actually define the target:

1. “AI adoption is now linked to higher software delivery throughput” — deployments increase
2. “However, ongoing challenge remains of ensuring software works as intended before it’s delivered” — stability can decrease

The resolution: “Evidence that throughput and stability do not need to be in trade-off in well-engineered organisations.”

The Goald target is to be in that category — deploying frequently AND maintaining stability. This issue creates the monitoring infrastructure to know, at any point, whether you’re achieving both or trading one for the other.

---

The Four Classic DORA Metrics (Applied to Goald)

Metric	Definition	Goald Measurement Method	Target
Deployment frequency	How often code reaches users	Count of expo publish (OTA) + EAS builds per week	≥ 1/week once DEVOPS-03 is live
Lead time for changes	Issue created → code in production	GitHub issue created_at → OTA publish timestamp	< 3 days
Change failure rate	% of deploys causing user-visible failure	Sentry spike events ÷ total deploys	< 5%
Time to restore	When a failure occurs, how fast is it fixed	Time from Sentry alert to rollback/fix deploy	< 2 hours

---

Acceptance Criteria

Deployment Frequency Tracking

• Each Expo OTA publish logs a deployment record to a Firestore collection deployments:
  • timestamp, channel (staging/production), triggeredBy (CI/manual), commitSha, releaseId
• scripts/weekly-metrics.sh (from METRICS-02) updated to include deployment frequency

Lead Time Tracking

• GitHub Actions workflow updated: on merge to main, post a comment to the merged PR with:
  • The issue number(s) it closes
  • Timestamp of merge
  • OTA publish status and release ID
• This creates a paper trail for lead time calculation: issue created_at → PR merge timestamp

Change Failure Rate (extend METRICS-02)

• docs/METRICS.md updated with change failure rate target (< 5%) and calculation method
• Sentry project configured with a “Deployment” release marker on each OTA publish so Sentry can correlate error spikes to specific releases

Stability vs Throughput Dashboard

• scripts/weekly-metrics.sh updated to output all four DORA metrics:

  === Goald DORA Metrics (last 7 days) ===
  Deployment frequency:    N deploys   [target: ≥1/week]
  Lead time (avg):         N days      [target: <3 days]
  Change failure rate:     N%          [target: <5%]
  Time to restore (avg):   N hours     [target: <2 hours]
  ---
  Rework rate (14d):       N%          [target: <10%]
  PR revision rate:        N%          [target: <25%]
  Test coverage:           N%          [target: >80%]
  Friction self-rating:    N/5
  

• This single-screen view is the “weekly 3-minute health check” for the project

---

Files to Create / Change

• docs/METRICS.md — add four DORA metrics with targets and measurement methods
• scripts/weekly-metrics.sh — extend with DORA metrics
• .github/workflows/ci-quality.yml — add deployment record write to Firestore on merge to main
• src/services/deploymentService.ts (create — minimal, just writes deployment record)
• Update Sentry project: add release markers on each OTA publish