Develop

[ShaneIsrael]

This pull request introduces a major refactor of the transcoding job queue to use a persistent database-backed queue (TranscodeJob model) instead of an in-memory Python list. This enables robust, multi-worker-safe transcoding job management, prevents job loss on process restarts, and allows for better status reporting and cancellation. Additionally, it introduces process locking for transcoding, improves worker startup and cleanup logic, and makes several operational improvements to concurrency and resource management.

Transcoding Queue Refactor and Improvements:

• Introduced the TranscodeJob database model to persistently track transcoding jobs, their status (pending, running, complete, failed), and timestamps, replacing the previous in-memory queue. [1] [2]
• Refactored the transcoding queue logic in api/transcoding.py to use the database for queueing, deduplication, claiming, and status reporting, with robust recovery of stale jobs and atomic job claiming for multi-worker safety. [1] [2] [3] [4]
• Updated cancellation logic to kill running jobs and clear all pending/running jobs from the database queue, ensuring no orphaned or stuck jobs remain.

Process Locking and Worker Coordination:

• Added process-level locking for transcoding using lockfiles, preventing concurrent transcoding runs and ensuring only one process performs cleanup or starts transcoding at a time. Lock helpers in util.py now support custom filenames. [1] [2] [3] [4]
• Improved worker startup logic to ensure only the first worker cleans up leftover chunk files, using a sentinel file in shared memory or tempdir.

Operational and Configuration Improvements:

• Gunicorn worker count is now capped by default (to 4), with environment variable overrides, to prevent excessive worker spawning on high-core machines and reduce resource exhaustion in containers.
• Added ENABLE_TRANSCODING to the .env.dev file for easier local configuration.

Bulk Import and Lock Handling:

• Released the scan lock before starting transcoding in bulk import, preventing unnecessary aborts of scheduled scans during long transcode operations. [1] [2]

These changes make transcoding more reliable, observable, and robust in multi-worker/container environments, and lay the groundwork for more advanced job management and status features.