Rethink queue prefix concept

[lucasoares]

The queue prefix was created to consolidate multiple queue metrics (exposed on /metrics) under a single tag.

Since Deckard does not impose a limit on the number of queues users can create, there are use cases where thousands of queues are created for the same application.

For example:

• application_x:customers - a queue containing all customers created by the application_x
• application_x:customers:id::1232453223 - a specififc queue to the customer with the ID 1232453223, using the application_x:customers:id prefix

By using the separator ::, all metrics from these queues are grouped together under a single queue tag application_x:customers:id. This approach helps prevent the cardinality problem of timeseries databases.

I believe this concept is highly valuable, but it is currently somewhat hidden within Deckard's implementation. We should make it more accessible for everyone to use by documenting it and making it more easier to use.

Additionally, in some places within the documentation, the word "prefix" is used when referring to the : separator, which is not entirely accurate. The : separator is merely a suggestion to provide a symbolic representation of a queue domain. For instance, application_x:customers:facebook and application_x:customers:instagram indicate that these queues are created by the application_x application and contain information about customers from specific social media platforms, facilitating audits or observation using related tools.

So I propose:

• document properly the queue prefix concept
• review the implementation for the queue prefix (ex.: instead of setting the :: string in the queue name, add a queue_prefix attribute)
  • We should be careful for breaking changes of applications that already uses :: inside the queue name.