Skip to content

test(ci): add a Python version matrix (3.10 → 3.14) to the pytest CI lanes #192

Description

@cofade

Problem

CI runs the Python services on only Python 3.11, but production runs Python 3.10. Every Python-touching job in .github/workflows/tests.yml hardcodes python-version: '3.11' via actions/setup-python@v5 — there is no version matrix:

  • duckdb-unit (pytest tests/ -q)
  • image-unit (pytest tests/ -q)
  • esp-native / esp-firmware (PlatformIO host Python)
  • e2e-pipeline and the ui-playwright seed step

This gap let a 3.11-only API reach main and crash both Python services on deploy: commit 8938899 (via #180) introduced from datetime import UTC / datetime.now(UTC) in services/log_ring.py (both the duckdb-service and image-service copies). datetime.UTC was added in 3.11; on the 3.10 prod host, log_ring.install() runs at import time in app.py, so the deploy of aac9c78 hit an ImportError before either service could serve traffic. Prod was held on the prior known-good revision until #191 (the UTCtimezone.utc swap) merged.

#191 fixes the symptom but does not add the CI lane that would have caught it at PR time. See the heads-up in #180 (comment).

Goal

Run the Python unit-test lanes across a version matrix so a version-specific API (in either direction — a 3.11-ism that breaks 3.10, or a future deprecation that breaks 3.14) fails at PR time instead of on deploy.

Target range: Python 3.10 → 3.14, where 3.10 is the current prod floor and 3.14 is the forward-compat ceiling:

3.10  →  current production host (the floor — must stay green)
3.11  →  current CI + Docker image target (python:3.11)
3.12
3.13
3.14  →  forward-compat ceiling

Proposed change

Add a matrix to at least duckdb-unit and image-unit (the two pure-pytest lanes — cheapest, highest signal, and the two services that actually broke):

duckdb-unit:
  strategy:
    fail-fast: false
    matrix:
      python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']
  steps:
    - uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}
        cache: pip
        cache-dependency-path: duckdb-service/requirements-dev.txt
    # ...

fail-fast: false so one version failing doesn't mask the others.

Open questions to settle while implementing

  • Dependency availability. Confirm the pinned requirements-dev.txt deps (notably duckdb) ship wheels for 3.13/3.14 on ubuntu-latest. If a newer interpreter has no wheel yet, either pin the matrix ceiling to what's installable or mark the bleeding-edge entry continue-on-error: true rather than blocking PRs on an upstream packaging gap — and log() / note which versions were dropped so it isn't a silent cap.
  • Scope of the matrix. e2e-pipeline / ui-playwright boot Docker images that already pin their own interpreter, so the matrix likely belongs on the unit lanes only; the host Python there just runs the seed/test driver. Decide whether to matrix those too or leave them single-version.
  • PlatformIO lanes. esp-native / esp-firmware use host Python only to run PlatformIO; a version matrix there is lower value. Probably leave as-is.

Docs to update (per CLAUDE.md)

Acceptance criteria

  • duckdb-unit and image-unit run green on Python 3.10 and 3.11 at minimum, with 3.12–3.14 added (or explicitly, visibly deferred with a logged reason if an upstream wheel is missing).
  • A reintroduced 3.11-only API (e.g. from datetime import UTC) fails the 3.10 lane.
  • ci-gates.md reflects the new matrix.
  • The incident is recorded in chapter 11.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions