Skip to content

fix: Python 3.10 compat — timezone.utc + onnxruntime 1.23.2 (unbreaks prod)#191

Closed
schutera wants to merge 2 commits into
mainfrom
fix/py310-datetime-utc-compat
Closed

fix: Python 3.10 compat — timezone.utc + onnxruntime 1.23.2 (unbreaks prod)#191
schutera wants to merge 2 commits into
mainfrom
fix/py310-datetime-utc-compat

Conversation

@schutera

@schutera schutera commented Jun 27, 2026

Copy link
Copy Markdown
Owner

What

from datetime import UTC / datetime.now(UTC) in both
duckdb-service/services/log_ring.py and image-service/services/log_ring.py
requires Python 3.11+ — the UTC alias was only added to datetime in 3.11.

Why it matters

The production host runs Python 3.10. Deploying aac9c78 (current main)
crashed both python services at import time: log_ring.install() runs at
import in app.py, so the ImportError takes the service down before it serves
any traffic. Prod has been held on the prior known-good revision until this lands.

Fix

Replace UTC with the equivalent timezone.utc (available since 3.2). Behavior
is identical on 3.11+ and now works on 3.10:

  • _now_iso() → millisecond ISO string with a Z suffix
  • log-rotation filename stamp %Y-%m-%d_%H-%M-%S

Both copies of log_ring.py are kept byte-identical.

Verification (run on Python 3.10.12, same minor as prod)

  • py_compile of both files
  • ✅ full compileall sweep of every python service/tooling dir — no other
    3.11-only syntax anywhere
  • ✅ grepped origin/main for other 3.11-isms (tomllib, StrEnum,
    ExceptionGroup, except*, asyncio.TaskGroup, typing.Self,
    datetime.UTC) — none outside these two files
  • ✅ stamp output confirmed byte-identical to the old UTC form
  • fromisoformat call sites (measurements.py, weather_worker.py) already
    guard for pre-3.11 input — left as-is

Also in this PR: onnxruntime pinned for Python 3.10

image-service/requirements.txt pinned onnxruntime==1.27.0, which has no
cp310 wheel
(pip offers max 1.23.2 on Python 3.10) — so installing
image-service's requirements would fail on the 3.10 prod host. This PR pins it to
onnxruntime==1.23.2 (the highest cp310 wheel), verified to load and run the
exported hole_detector.onnx on Python 3.10.12. Bump back up once the host moves
to Python 3.11+. Server-side inference only — the ESP runs no models
(ADR-028).

Introduced in 8938899 (part of #180, which closed #178).

🤖 Generated with Claude Code

`from datetime import UTC` / `datetime.now(UTC)` requires Python 3.11+, but the production host runs Python 3.10, so importing services/log_ring.py crashed both duckdb-service and image-service on deploy of aac9c78.

Replace with the equivalent `datetime.now(timezone.utc)` — identical on 3.11+ and supported on 3.10. Stamp output is byte-identical (verified on 3.10.12): millisecond ISO with a 'Z' suffix, and the %Y-%m-%d_%H-%M-%S rotation filename. Both copies of log_ring.py kept in sync.

Introduced in 8938899 (#178 phase 1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
onnxruntime==1.27.0 has no CPython 3.10 wheel (pip maxes at 1.23.2 on cp310), so installing image-service requirements fails on the 3.10 prod host. 1.23.2 is the highest cp310 wheel and loads+runs the exported hole_detector.onnx (verified on 3.10.12). Bump back up once the host moves to Python 3.11+.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@schutera schutera changed the title fix: Python 3.10 compat — use timezone.utc instead of datetime.UTC (unbreaks prod) fix: Python 3.10 compat — timezone.utc + onnxruntime 1.23.2 (unbreaks prod) Jun 27, 2026
cofade added a commit that referenced this pull request Jun 28, 2026
Resolve the ADR-028 numbering collision: main merged adr-028 (ML
inference is server-side only, f55a840) while this branch also added an
adr-028 (Python version matrix). Renumber this branch's ADR to 029:

- rename adr-028-python-version-matrix-floated-pins.md -> adr-029-...
- retitle the file heading ADR-028 -> ADR-029
- index README lists both 028 (ML inference, from main) and 029 (matrix)
- repoint every reference to the matrix ADR (tests.yml, ci-gates.md,
  chapter-11, both requirements.txt, pyproject.toml, test_native_runtime.py)
  from ADR-028 to ADR-029; leave main's ADR-028 (ML inference) refs in
  CLAUDE.md and the index untouched.

No code-path change; main's log_ring.py still imports datetime.UTC, so
this branch's timezone.utc fix remains required (references #191, #192).
cofade added a commit that referenced this pull request Jun 28, 2026
…195)

* ci: matrix duckdb-unit + image-unit across Python 3.10-3.14 (#192)

Production runs Python 3.10 but CI only ran 3.11. That gap let a
3.11-only `from datetime import UTC` reach main in services/log_ring.py
and crash both Python services at import time on deploy (see #180);
the symptom was addressed separately by #191.

Run both pytest lanes across a ['3.10'..'3.14'] matrix (fail-fast:
false) so a version-specific API fails at PR time on the 3.10 floor
instead of on deploy.

Exact == pins can't span the range: no single numpy ships both a cp310
and a cp314 wheel, and onnxruntime 1.27.0 has no cp310 wheel. So float
only the blocking native deps to >= lower bounds and let pip resolve a
per-interpreter wheel:
- image-service: numpy>=2.0.0, onnxruntime>=1.23.2
- both services: pydantic>=2.12.5 (pinned pydantic-core lacks cp314)
duckdb 1.4.4 already spans cp310-cp314 and stays pinned. This moves a
fresh prod install onto numpy 2.x; rationale + trade-off in ADR-028.

Also carry the identical UTC -> timezone.utc change from #191 into both
log_ring.py copies, so the new 3.10 cells pass on this branch.

Verified: pip --dry-run resolves on 3.10-3.14; both suites pass locally
on numpy 2.4.6 (image-service 97, duckdb-service 232).

Docs: ADR-028 + index, ci-gates rows, chapter-11 lessons entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WTpzfM7M4q7gXzuwsxxhzp

* test(ci): harden 3.10-3.14 matrix per senior review (#192)

Addresses senior-review findings on the version-matrix change:

- P1: image-service detection tests `importorskip` onnxruntime/cv2, so a
  broken floated-wheel resolution on a cell would SKIP them and pass the
  lane green — false confidence for exactly the deps the matrix de-risks.
  Add tests/test_native_runtime.py, a non-skipping guard asserting
  hole_detection._RUNTIME_AVAILABLE is True, so a busted install turns the
  cell red.
- P1: correct the "prod runs 3.10" premise. The repo specifies its runtime
  three ways (Dockerfiles build 3.12-slim, deploy docs say 3.11, the #180
  host crashed on 3.10). Reframe 3.10 as a conservative floor proven by the
  incident, not a flat fact, across the workflow comment, ci-gates, ADR-028,
  and the chapter-11 lesson; flag the inconsistency as the deeper risk.
- P2: pin image-service ruff target-version py312 -> py310 so the UP
  (pyupgrade) rule can't rewrite timezone.utc back to the 3.11-only UTC.
- P2: note the #191 overlap in ADR-028 consequences.

Verified: image-service 98 passed on numpy 2.4.6; ruff check clean at
py310 (no UTC rewrite suggested); check-citations clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WTpzfM7M4q7gXzuwsxxhzp

* docs(ci): tighten ADR-028 precision per re-review (#192)

Doc-only precision fixes from the second senior-review pass; no behaviour
change:

- ruff comment no longer implies CI enforces the py310 floor (ruff is not
  run in any CI lane); reword to local `ruff --fix` / editor-on-save.
- Reframe the runtime drift accurately: the container path is consistently
  3.12-slim (dev + prod); the disagreement is deploy-docs prose (3.11) vs
  the bare-metal PM2 host that crashed (3.10) — not the Dockerfiles.
- onnxruntime cp314 ceiling 1.24 -> 1.24.1 (no 1.24.0 on PyPI).
- ADR-028: note duckdb-service is unexposed to the pyupgrade rewrite (no
  ruff config -> default rules exclude UP), and ground the numpy-2.x
  "low-risk" claim in the existing test_hole_detection.py inference tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WTpzfM7M4q7gXzuwsxxhzp

---------

Co-authored-by: Claude <noreply@anthropic.com>
@cofade

cofade commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator

Closing as fully superseded by #195 (merged to main as a0e7374). Rebasing this branch onto current main leaves zero diff — both commits evaporate:

  • timezone.utc fix (both log_ring.py copies) — git drops it automatically on rebase: patch contents already upstream. ci: test the Python services across a 3.10–3.14 matrix (closes #192) #195 carried the byte-identical from datetime import UTCtimezone.utc change into both files.
  • onnxruntime pin — superseded by a more general fix. This PR hard-pinned onnxruntime==1.23.2 / numpy==1.26.4 to keep the Python-3.10 prod host installable. ci: test the Python services across a 3.10–3.14 matrix (closes #192) #195 instead floated them to onnxruntime>=1.23.2 / numpy>=2.0.0 (ADR-029) so each cell of the new 3.10–3.14 CI matrix resolves a per-interpreter wheel. A hard ==1.23.2 would now break the 3.14 cell (no cp314 wheel; cp314 needs ≥ 1.24.1), and numpy==1.26.4 has no cp314 wheel either. The 3.10 host still resolves onnxruntime 1.23.2 from the floated lower bound, so prod stays installable — this PR's intent is preserved and generalized.

Verified: git diff origin/main..HEAD after rebase is empty. Nothing left to merge.

Follow-up that outlives this PR: the host should still bump to Python 3.11+ eventually, and the repo's runtime version is specified three different ways (container 3.12-slim, deploy docs 3.11, the crashed host 3.10) — reconciling that to one authoritative version is tracked separately from #192/#195.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Turn the admin "Server Logs" panel into a real live log console

2 participants