Skip to content

feat: add named durable job queues and worker observability - #4

Open
joaovitoriasilva wants to merge 11 commits into
mainfrom
feat/durable-job-queues
Open

feat: add named durable job queues and worker observability#4
joaovitoriasilva wants to merge 11 commits into
mainfrom
feat/durable-job-queues

Conversation

@joaovitoriasilva

Copy link
Copy Markdown
Member

What does this PR do?

Adds named durable-job queues, selective and fair workers, and durable worker observability while preserving the existing default queue behavior.

Key changes:

  • Assign queues when durable subscribers register, then stamp each queue during outbox fan-out.
  • Support all-queue and explicit non-empty queue allowlists with fair rotation across selected queues.
  • Add the public blocking run_job_worker() entry point for standalone PostgreSQL workers.
  • Keep SQLite limited to one API process with one in-process consumer.
  • Add restart-unique worker identities, independent heartbeats, graceful-stop tracking, active-claim counts, status derivation, and bounded retention.
  • Expose queue and worker summaries through jasil.admin.
  • Add rolling-safe packaged migrations with a database-side default queue value.
  • Add real PostgreSQL competing-consumer and worker-observability conformance tests.

Related issue

No related issue was provided. This work was requested and implemented directly.

Type of change

Check all that apply

  • Bug fix
  • New feature (exception during freeze; link prior discussion/acknowledgment)
  • Documentation
  • Refactor / code quality
  • Translation
  • Other (describe below)

Checklist

  • This PR is focused on a single concern
  • An issue was opened and acknowledged before writing this code (if not a small fix)
  • If this PR introduces a new feature during freeze, it was discussed and explicitly acknowledged first
  • The PR is under ~300 lines changed, excluding lock files, migrations, and generated code
  • Tests have been added or updated where applicable
  • Documentation has been updated where applicable
  • Commit messages follow Conventional Commits format

Notes for the reviewer

Design decisions:

  • Queue ownership belongs to subscriber registration because one event can fan out to subscribers on different queues.
  • JASIL owns neutral worker identity and telemetry persistence. The host continues to own routes, authentication, authorization, UI, alerts, and health policy.
  • Durable jobs remain database-backed and do not require Redis.

Validation completed:

  • SQLite: 1,023 passed, 7 skipped, 98.21% coverage.
  • PostgreSQL 17: 1,028 passed, 2 skipped, 98.13% coverage.
  • MySQL 8.4 compatibility slice: 165 passed, 3 skipped.
  • Ruff, format, strict mypy, five import contracts, strict MkDocs, API reference validation, wheel smoke testing, and strict Twine checks passed.
  • The built wheel contains all three migration revisions and maps all four JASIL tables with base dependencies only.

@joaovitoriasilva joaovitoriasilva added the enhancement New feature or request label Aug 25, 2026
@joaovitoriasilva joaovitoriasilva changed the title Add named durable job queues and worker observability feat: add named durable job queues and worker observability Aug 25, 2026
# Conflicts:
#	tests/test_retention_and_storage.py
- Introduced properties in JobRunner for worker ID, clock, and queues.
- Improved handling of job completion and failure to log warnings on ownership loss.
- Added async shutdown functionality to lifecycle management.
- Enhanced worker telemetry to include metadata normalization and heartbeat management.
- Implemented cursor pagination for worker summaries.
- Updated tests to cover new features and ensure robustness against ownership loss scenarios.
- Refactored job CRUD operations to include worker ID and attempt tracking.
- Improved error handling and logging for job finalization processes.
# Conflicts:
#	CHANGELOG.md
#	pyproject.toml
#	tests/test_migrations.py
#	uv.lock
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants