Distributed Job Queue

A small multi-service job pipeline: clients submit work through an API gateway, jobs are recorded and queued, a scheduler persists authoritative state in PostgreSQL and drives priority queues, Node workers simulate execution and report outcomes, and a dashboard API serves listings and manual retries.

For background on the broker used here, see the RabbitMQ documentation.

Description

The system separates submission state (MongoDB, job-submission service) from scheduling and lifecycle state (PostgreSQL, shared by the job-scheduler and dashboard services). RabbitMQ carries commands and events between services. The web UI talks only to the API gateway, which enforces JWT authentication and proxies to the appropriate backend.

Features

• Priority-aware ready queues (low, medium, high) with different consumer prefetch limits on the worker.
• Per-message TTL on ready queues with dead-letter routing for expired or rejected messages.
• Retries with a configurable maximum per job, timeout checks, and a dead-letter path for exhausted or terminal failures.
• JWT-based access to job APIs through a single gateway.

Visuals

Stack

Layer	Technology
API gateway	Node.js, Express, http-proxy-middleware, JWT
Job submission	Node.js, Express, MongoDB, amqplib
Job scheduler	Java, Spring Boot, Spring AMQP, JPA, PostgreSQL
Dashboard API	Java, Spring Boot, Spring AMQP, JPA, PostgreSQL
Worker	Node.js, amqplib
Web UI	React, Vite
Infrastructure	Docker Compose: PostgreSQL 16, MongoDB 7, RabbitMQ 3 (management plugin)

Message queue

Broker: RabbitMQ.

Topology: A durable direct exchange jobs.exchange binds routing keys to queues. A separate durable direct exchange jobs.dlx is used as the dead-letter exchange for the ready queues.

Queues and routing keys (aligned across scheduler declarations and worker assertions):

Queue	Routing key	Role
jobs.submitted	jobs.submitted	New jobs from job-submission; consumed by the job-scheduler.
jobs.ready.low	jobs.ready.low	Low-priority work for workers.
jobs.ready.medium	jobs.ready.medium	Medium-priority work.
jobs.ready.high	jobs.ready.high	High-priority work.
jobs.running	jobs.running	Worker signals that execution has started; consumed by the job-scheduler.
jobs.results	jobs.results	Terminal success or failure from the worker; consumed by the job-scheduler.
jobs.retry	jobs.retry	Manual or API-initiated retry; consumed by the job-scheduler.
jobs.dead	jobs.dead (and default binding on jobs.dlx)	Dead-letter and dead-job payloads; consumed by the job-scheduler.

Ready queue policy: Each jobs.ready.* queue is durable and configured with a 60 second message TTL and dead-letter exchange jobs.dlx, so messages that are not consumed in time are dead-lettered to jobs.dead for handling by the scheduler.

Service responsibilities

API gateway (port 3000)
Listens on port 3000 in the default Compose setup. Validates JWT on /api/jobs routes. Proxies job submission and dashboard backends; the HTTP surface is summarized in the Usage section.

Job submission (port 3001)
Validates input, applies idempotency for duplicate submits, persists jobs in MongoDB, and publishes a JSON message to jobs.submitted on jobs.exchange. Exposes GET /api/v1/jobs/:id/status for submission-time status from MongoDB.

Job scheduler (port 8080)
Consumes jobs.submitted, jobs.running, jobs.results, jobs.retry, and jobs.dead. Persists jobs in PostgreSQL, assigns random maxRetries (1–5) and a random timeoutAt between 20 and 30 seconds after scheduling, publishes to the correct jobs.ready.* queue by priority, applies state transitions (pending, running, succeeded, failed, dead), scheduled timeout handling every 30 seconds, and republishes to ready or dead queues according to retry rules.

Dashboard (port 8081)
Read-only (plus retry) API over the same PostgreSQL job entities: list, detail, analytics, and POST retry which publishes to jobs.retry. Declares the exchange, jobs.retry queue, and bindings needed for retry publishing.

Worker
Asserts the exchange, ready queues (with matching DLX and TTL arguments), and consumes jobs.ready.low, jobs.ready.medium, and jobs.ready.high with prefetch 5, 10, and 20 respectively. Handlers run the shared simulation, publish to jobs.running and then to jobs.results.

Simulating logic

Worker execution is implemented in services/worker/handlers/simulateJob.js. For each message it:

1. Picks a random initial outcome: immediate failure, or continue as running.
2. If continuing, publishes a running notification, waits a random 1–4 seconds, then flips a coin for succeeded vs failed, and publishes the final result with an optional error string.

This is intentionally non-deterministic to exercise retries, dead-lettering, and UI states without real external workloads.

Roadmap and known follow-ups

• Transactional outbox for job-submission so MongoDB writes and RabbitMQ publishes stay consistent (called out in code; today a publish failure after a successful insert can leave a job in Mongo without a broker message).

Requirements

• Docker and Docker Compose for the backend and broker stack.
• Node.js compatible with the lockfiles in frontend/, services/api-gateway/, services/job-submission/, and services/worker/ (recent LTS recommended).
• JDK compatible with the Spring Boot services (see each service’s pom.xml for the configured Java release).

Installation and how to run

1. Backend and data stores

From the repository root:

docker compose up --build

This starts API gateway (3000), job-submission (3001), job-scheduler (8080), dashboard (8081), worker, PostgreSQL, MongoDB, and RabbitMQ. RabbitMQ management UI: http://localhost:15672 (default user and password are set in Compose to admin / admin).

2. Frontend

With the gateway reachable at http://localhost:3000:

cd frontend
npm install
npm run dev

Vite serves the app on port 5173 and proxies /api and /auth to the gateway.

Usage

Start the backend with Docker Compose and the frontend with npm run dev in frontend/ as described above. Open http://localhost:5173. Log in on the login screen (demo accounts are defined in the gateway, for example admin@test.com / admin123 and user@test.com / user123). After authentication, the app stores the JWT in sessionStorage and attaches it to job API requests. Use Submit Job to enqueue work and Dashboard for paginated listings and analytics. Open a job from the table to see details and trigger Retry when the server allows it. Job submission sends a fresh Idempotency-Key header on each create request to support safe retries from the UI.

Gateway API (as called from the frontend)

Paths are relative to the Vite dev server (http://localhost:5173); the dev proxy forwards /api and /auth to the gateway on port 3000. JSON request and response bodies unless noted. All /api/jobs routes require Authorization: Bearer.

Method	Path	Description
POST	/auth/login	Body: { "email", "password" }. Returns { "token" } on success.
GET	/health	Gateway liveness; not used by the SPA but available for ops.
POST	/api/jobs	Submit job. Optional header Idempotency-Key (UUID recommended). Body: type (image_resize | email | report_generation), priority (number 1–3), submittedBy (email), payload (object; shape depends on type, validated server-side). Proxied to job-submission.
GET	/api/jobs/:jobId/status	Submission record status from MongoDB.
GET	/api/jobs	Paginated list from the dashboard service. Query: pageNumber, pageSize, sortBy, sortDirection.
GET	/api/jobs/:jobId	Single job from PostgreSQL (scheduler/dashboard model).
GET	/api/jobs/analytics	Aggregated job metrics from the dashboard service.
POST	/api/jobs/:jobId/retry	Enqueues a retry via RabbitMQ when the job is eligible.

Project status

Active development. Core flows (submit, schedule, consume, result, retry, TTL dead-letter handling) are implemented. The transactional outbox item under Roadmap remains the main explicit consistency follow-up for submission.