HI @beorn-!
Firstly, this is a great idea and a need when using prometheus at high scale, so congrats for your work.
Today, I have a 3 replica HA prometheus and we are struggling to send it to one of ours remote write endpoints. It's a Thanos receive and is not supporting our volumetry.
So, in order to help thanos support it, we are trying to have the dedup on prometheus side, just as highlander does.
With that said, we are a little difficult to setup highlander at our infrastructure.
It's a prometheus operator server, deployed at a k8s cluster. This is the setup that we have done:
Highlander is deployed standalone with these params:
highlander -r http://prometheus-simplehttp.prometheus-highlander.svc.cluster.local:9090
simplehttpbackend is also a deploy and is served behing a kubernetes service at port 9090
Here is a simple diagram of how it's working:
And this is our prometheus remote write config:
Could you provide information if we are missing something at our current config? Today it's not working properly.
Thank you in advance.
HI @beorn-!
Firstly, this is a great idea and a need when using prometheus at high scale, so congrats for your work.
Today, I have a 3 replica HA prometheus and we are struggling to send it to one of ours remote write endpoints. It's a Thanos receive and is not supporting our volumetry.
So, in order to help thanos support it, we are trying to have the dedup on prometheus side, just as highlander does.
With that said, we are a little difficult to setup highlander at our infrastructure.
It's a prometheus operator server, deployed at a k8s cluster. This is the setup that we have done:
Highlander is deployed standalone with these params:
highlander -r http://prometheus-simplehttp.prometheus-highlander.svc.cluster.local:9090
simplehttpbackend is also a deploy and is served behing a kubernetes service at port 9090
Here is a simple diagram of how it's working:
And this is our prometheus remote write config:
TENANT: xxxxxxxxxxx
name: thanos
proxyUrl: http://prometheus-highlander.prometheus-highlander.svc.cluster.local:9091/
url: http://thanos-receive-endpoint/api/v1/receive
Could you provide information if we are missing something at our current config? Today it's not working properly.
Thank you in advance.