Skip to content

ci: test the Python services across a 3.10–3.14 matrix (closes #192)#195

Merged
cofade merged 4 commits into
mainfrom
claude/python-3-10-tests-h4hbi9
Jun 28, 2026
Merged

ci: test the Python services across a 3.10–3.14 matrix (closes #192)#195
cofade merged 4 commits into
mainfrom
claude/python-3-10-tests-h4hbi9

Conversation

@cofade

@cofade cofade commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Pull Request

What changed

  • Matrix both pytest CI lanes (duckdb-unit, image-unit) across Python ['3.10','3.11','3.12','3.13','3.14'] with fail-fast: false, in .github/workflows/tests.yml. Job count stays nine; each fans into five version cells.
  • Float only the native deps whose wheel windows can't span the range — exact == pins are impossible here (no single numpy ships both a cp310 and a cp314 wheel; onnxruntime==1.27.0 has no cp310 wheel at all). So: numpy>=2.0.0, onnxruntime>=1.23.2 (image-service) and pydantic>=2.12.5 (both services, for a cp314 pydantic-core on 3.14). Everything else stays exact-pinned (duckdb==1.4.4 already spans cp310–cp314; opencv-python-headless is abi3). pip resolves the newest interpreter-compatible wheel per cell.
  • New non-skipping guard image-service/tests/test_native_runtime.py asserts hole_detection._RUNTIME_AVAILABLE is True. The detection tests importorskip onnxruntime/cv2, so without this a broken floated-wheel resolution would skip them and pass the lane green — false confidence for exactly the deps the matrix de-risks.
  • Folds in the datetime.UTC → timezone.utc fix in both services/log_ring.py copies (identical to fix: Python 3.10 compat — timezone.utc + onnxruntime 1.23.2 (unbreaks prod) #191) so the new 3.10 cells pass instead of failing on the still-unmerged import fix.
  • ruff target-version → py310 in image-service/pyproject.toml so the UP (pyupgrade) rule can't rewrite timezone.utc back to the 3.11-only UTC on a dev box.
  • Docs: new ADR-028 (+ index row) recording the floated-pin pin-policy and the prod numpy 1.26→2.x consequence; updated ci-gates.md rows; new chapter-11 lessons-learned entry.

Why

References #192. CI tested the Python services on only 3.11, but the runtime is specified inconsistently across the repo (container path runs 3.12-slim; deploy-docs prose says 3.11; the bare-metal host that crashed ran 3.10). That single-version gap let a 3.11-only from datetime import UTC reach main in log_ring.py and crash both Python services at import time on deploy (#180) — an ImportError on datetime.UTC (added in 3.11) proves the runtime was < 3.11. The matrix spans a conservative 3.10 floor to a 3.14 forward ceiling so a version-specific API fails at PR time instead of on deploy. Full rationale in ADR-028.

Note on #191: this branch carries the identical UTC→timezone.utc swap that #191 makes to the same two files. Whichever merges second is a no-op / trivial conflict on those lines — recommend merging #191 first (my copy becomes a rebase no-op) or closing #191 in favour of this branch.

How tested

  • ESP32-CAM native (pio test -e native)
  • End-to-end (pytest tests/e2e)
  • Backend unit (Node 22 + TS)
  • image-service unit (Python 3.11)
  • duckdb-service unit (Python 3.11)
  • Homepage unit (React 19 + Vite)
  • Manual / cross-version verification (described below)

Ran both Python suites in clean venvs: image-service 98 passed (resolving numpy 2.4.6 / pydantic 2.13.4 — confirms the numpy 1.26→2.x bump is safe), duckdb-service 232 passed. Verified pip resolution on all five versions via pip install --dry-run --python-version <v> --only-binary=:all: (e.g. 3.10 → numpy 2.2.6 / onnxruntime 1.23.2; 3.14 → numpy 2.4.6 / onnxruntime 1.27.0). ruff check clean at py310 (no UTC rewrite). make check-citations clean. The matrix itself runs once this PR triggers CI.

Checklist

  • Tests added or updated to cover the change
  • Documentation updated where applicable (README, ARCHITECTURE, service-level docs)
  • No secrets, credentials, or large binaries committed
  • CI is green on this branch — the matrix only runs on PRs to main, so it executes for the first time on this PR
  • Breaking changes called out in the description (prod resolves numpy 2.x; floated pins loosen reproducibility for those deps — see ADR-028)

Screenshots / logs (optional)

N/A — CI-config and dependency change, no UI surface.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WTpzfM7M4q7gXzuwsxxhzp

Generated by Claude Code

claude and others added 4 commits June 27, 2026 11:04
@claude
ci: matrix duckdb-unit + image-unit across Python 3.10-3.14 (#192)
680a5ea 
Production runs Python 3.10 but CI only ran 3.11. That gap let a
3.11-only `from datetime import UTC` reach main in services/log_ring.py
and crash both Python services at import time on deploy (see #180);
the symptom was addressed separately by #191.

Run both pytest lanes across a ['3.10'..'3.14'] matrix (fail-fast:
false) so a version-specific API fails at PR time on the 3.10 floor
instead of on deploy.

Exact == pins can't span the range: no single numpy ships both a cp310
and a cp314 wheel, and onnxruntime 1.27.0 has no cp310 wheel. So float
only the blocking native deps to >= lower bounds and let pip resolve a
per-interpreter wheel:
- image-service: numpy>=2.0.0, onnxruntime>=1.23.2
- both services: pydantic>=2.12.5 (pinned pydantic-core lacks cp314)
duckdb 1.4.4 already spans cp310-cp314 and stays pinned. This moves a
fresh prod install onto numpy 2.x; rationale + trade-off in ADR-028.

Also carry the identical UTC -> timezone.utc change from #191 into both
log_ring.py copies, so the new 3.10 cells pass on this branch.

Verified: pip --dry-run resolves on 3.10-3.14; both suites pass locally
on numpy 2.4.6 (image-service 97, duckdb-service 232).

Docs: ADR-028 + index, ci-gates rows, chapter-11 lessons entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WTpzfM7M4q7gXzuwsxxhzp
@claude
test(ci): harden 3.10-3.14 matrix per senior review (#192)
f4e3267 
Addresses senior-review findings on the version-matrix change:

- P1: image-service detection tests `importorskip` onnxruntime/cv2, so a
  broken floated-wheel resolution on a cell would SKIP them and pass the
  lane green — false confidence for exactly the deps the matrix de-risks.
  Add tests/test_native_runtime.py, a non-skipping guard asserting
  hole_detection._RUNTIME_AVAILABLE is True, so a busted install turns the
  cell red.
- P1: correct the "prod runs 3.10" premise. The repo specifies its runtime
  three ways (Dockerfiles build 3.12-slim, deploy docs say 3.11, the #180
  host crashed on 3.10). Reframe 3.10 as a conservative floor proven by the
  incident, not a flat fact, across the workflow comment, ci-gates, ADR-028,
  and the chapter-11 lesson; flag the inconsistency as the deeper risk.
- P2: pin image-service ruff target-version py312 -> py310 so the UP
  (pyupgrade) rule can't rewrite timezone.utc back to the 3.11-only UTC.
- P2: note the #191 overlap in ADR-028 consequences.

Verified: image-service 98 passed on numpy 2.4.6; ruff check clean at
py310 (no UTC rewrite suggested); check-citations clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WTpzfM7M4q7gXzuwsxxhzp
@claude
docs(ci): tighten ADR-028 precision per re-review (#192)
2182822 
Doc-only precision fixes from the second senior-review pass; no behaviour
change:

- ruff comment no longer implies CI enforces the py310 floor (ruff is not
  run in any CI lane); reword to local `ruff --fix` / editor-on-save.
- Reframe the runtime drift accurately: the container path is consistently
  3.12-slim (dev + prod); the disagreement is deploy-docs prose (3.11) vs
  the bare-metal PM2 host that crashed (3.10) — not the Dockerfiles.
- onnxruntime cp314 ceiling 1.24 -> 1.24.1 (no 1.24.0 on PyPI).
- ADR-028: note duckdb-service is unexposed to the pyupgrade rewrite (no
  ruff config -> default rules exclude UP), and ground the numpy-2.x
  "low-risk" claim in the existing test_hole_detection.py inference tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WTpzfM7M4q7gXzuwsxxhzp
@cofade
Merge origin/main into claude/python-3-10-tests-h4hbi9
27ee7f3 
Resolve the ADR-028 numbering collision: main merged adr-028 (ML
inference is server-side only, f55a840) while this branch also added an
adr-028 (Python version matrix). Renumber this branch's ADR to 029:

- rename adr-028-python-version-matrix-floated-pins.md -> adr-029-...
- retitle the file heading ADR-028 -> ADR-029
- index README lists both 028 (ML inference, from main) and 029 (matrix)
- repoint every reference to the matrix ADR (tests.yml, ci-gates.md,
  chapter-11, both requirements.txt, pyproject.toml, test_native_runtime.py)
  from ADR-028 to ADR-029; leave main's ADR-028 (ML inference) refs in
  CLAUDE.md and the index untouched.

No code-path change; main's log_ring.py still imports datetime.UTC, so
this branch's timezone.utc fix remains required (references #191, #192).
@cofade cofade merged commit a0e7374 into main Jun 28, 2026
17 checks passed
This was referenced Jun 28, 2026
Closed
Closed
Open
cofade added a commit that referenced this pull request Jun 29, 2026
@cofade @claude
docs(deploy): align runbook Python-deps section with floated pins (AD…
1951ddc 
…R-029)

The PR's runbook rewrite asserted onnxruntime is "pinned to 1.23.2" under a
"Python 3.10 ceiling". After folding in main (#195 / ADR-029), the real
requirements float numpy>=2.0.0 / onnxruntime>=1.23.2 / pydantic>=2.12.5 for a
3.10-3.14 matrix — a floor, not a pin. Rewrote the step-2 comment and the
"Python 3.10 floor" paragraph to match, citing ADR-029, and noted that a pip
upgrade is not reverted on rollback.

Added a ch11 lessons-learned entry for the workspace-lockfile npm-ci miss this
PR corrects (per CLAUDE.md's mandatory doc gate). Addresses the senior-reviewer
P0/P2 findings on the PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cofade cofade mentioned this pull request Jun 29, 2026
Merged
cofade added a commit that referenced this pull request Jun 29, 2026
@cofade @claude
chore: reconcile the production Python runtime to a single 3.10 floor (…
4d503dc 
…closes #197) (#198)

The repo named its Python runtime three disagreeing ways — container
python:3.12-slim, deploy-docs prose 3.11, bare-metal PM2 host 3.10 — the
documented root cause behind incident #180 (a 3.11-only datetime.UTC that
crashed both Python services on deploy because CI/containers ran a different
version than the host). The 3.10-3.14 CI matrix from #195 (ADR-029) mitigated
the symptom but left the runtime-naming drift open; this closes it.

Adopt 3.10 as the authoritative floor and make it enforced, not hand-kept:

- Add /.python-version (=3.10) as the single source of truth.
- Add scripts/check-python-version.sh: asserts both Dockerfile.dev FROM lines,
  the image-service ruff floor (py310), and each CI matrix's version-aware
  minimum all match the anchor. Wired into make check-python-version, the
  husky pre-push hook, and a new python-version-consistency CI job.
- Pin both Python service images python:3.12-slim -> python:3.10-slim, so the
  container path mirrors the live 3.10 PM2 host (verified: both images build
  and import the native stack on 3.10 — onnxruntime resolves its 3.10 wheel,
  1.23.2, exactly as ADR-029 predicts).
- Reconcile every prose surface to 3.10 (constraints, building-block view,
  docker-compose doc, CLAUDE.md, CONTRIBUTING, runbook, production-deployment,
  PR/issue templates) and correct the now-ten CI job count across the docs.
- Record the reconciliation as done in ADR-029 and the chapter-11 lesson; the
  historical "named three ways" narrative is left intact (the guard scopes to
  machine-readable surfaces, so a literal-version grep can't self-trip on it).

The matrix stays 3.10-3.14: 3.10 is the floor we ship, not a cap on what the
code must tolerate. Requirements pins are untouched — the floated bounds
already resolve real wheels on 3.10. See ADR-029 for the floor rationale.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
cofade added a commit that referenced this pull request Jun 29, 2026
@cofade @claude
docs(deploy): align runbook Python-deps section with floated pins (AD…
135dc1f 
…R-029)

The PR's runbook rewrite asserted onnxruntime is "pinned to 1.23.2" under a
"Python 3.10 ceiling". After folding in main (#195 / ADR-029), the real
requirements float numpy>=2.0.0 / onnxruntime>=1.23.2 / pydantic>=2.12.5 for a
3.10-3.14 matrix — a floor, not a pin. Rewrote the step-2 comment and the
"Python 3.10 floor" paragraph to match, citing ADR-029, and noted that a pip
upgrade is not reverted on rollback.

Added a ch11 lessons-learned entry for the workspace-lockfile npm-ci miss this
PR corrects (per CLAUDE.md's mandatory doc gate). Addresses the senior-reviewer
P0/P2 findings on the PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@cofade @claude