Questiona about stopping an application while active jobs are still running

[arthurvanl]

So I added the graceful-shutdown so it wont quit jobs. But it can be that a flow-job is in the middle of it's step-job as I would like to call it.

So for example a flow-job would be like

• download-products
• parse-products
• handle-products

Lets say the server wants to stop when we are in parse-products. Applications doesn't stop yet because it still needs to process the current active job. But after parse-products is not active anymore, would it stop the application or does it continue to handle-products?

And if it doesn't continue how is the handle-products executed?

[egeominotti]

Hey @arthurvanl!

Great question. I investigated the flow job + graceful shutdown behavior in detail. Here's what happens:

Short answer: The next step (handle-products) will be promoted to the queue after parse-products completes and is ACKed. However, since the worker is shutting down, it won't pick up the new job.

What happens step by step:

1. parse-products completes -> worker sends ACK
2. ACK triggers dependency resolution via microtask (very fast, same event loop tick)
3. handle-products is promoted from waitingDeps to the active queue
4. But the worker is in closing state -> it stops polling for new jobs
5. handle-products sits in the queue with no worker to process it

With SQLite persistence (dataPath configured):

If you have persistence enabled, handle-products is safely recovered on restart:

// Server with persistence
bunqueue start --data-path ./data/queue.db

On restart, the recovery process (Phase 2 in backgroundTasks.ts) loads all pending jobs from SQLite, checks their dependency status, and promotes them to the queue. So handle-products will be picked up by the next worker that connects.

Without persistence (in-memory mode):

The job is lost when shutdown clears all in-memory state. This is expected behavior for in-memory mode.

Recommendation:

For production with flow jobs, always use persistence:

// .env or systemd Environment=
DATA_PATH=/opt/bunqueue/data/queue.db

This ensures no flow steps are lost during restarts or deployments. The graceful shutdown will:

1. Wait for active jobs to finish (up to SHUTDOWN_TIMEOUT_MS, default 30s)
2. ACK triggers dependency promotion (next step queued)
3. Persistence saves the state to SQLite
4. On restart, pending jobs are recovered and processing continues