Follow-up from #70 (addressed for PostgreSQL in #71). The Redis backend still has the original limitations:
-
process_scheduled() runs on every node — the Lua script moves at most 100 items per poll. Every WorkerEngine instance runs this loop independently, creating redundant Redis traffic that grows with the number of nodes.
-
dequeue() ignores activity_types filter — the signature accepts the parameter but the value is discarded. There is no workload isolation for the Redis backend yet.
-
No key partitioning — a single sorted set per queue_name for both the main queue and the scheduled set.
What needs to happen
Ref: #70, #71
Follow-up from #70 (addressed for PostgreSQL in #71). The Redis backend still has the original limitations:
process_scheduled()runs on every node — the Lua script moves at most 100 items per poll. EveryWorkerEngineinstance runs this loop independently, creating redundant Redis traffic that grows with the number of nodes.dequeue()ignoresactivity_typesfilter — the signature accepts the parameter but the value is discarded. There is no workload isolation for the Redis backend yet.No key partitioning — a single sorted set per
queue_namefor both the main queue and the scheduled set.What needs to happen
dequeue()(or coordinateprocess_scheduled()to run on a single node)activity_typesfiltering in the Redisdequeue()Lua scriptschedules_natively()returningtruefor Redis once scheduling is handled inlineRef: #70, #71