msg="Error from scraper" scraper=workspaces err="context canceled, (organization=XXXXX, page=9)"

[ktella]

level=info TFBI=2025-04-25T17:50:32.492Z caller=main.go:61 msg="Starting tf_exporter" version=v0.3
level=info TFBI=2025-04-25T17:50:32.493Z caller=main.go:76 msg="Listening on address" address=0.0.0.0:9100
level=error TFBI=2025-04-25T17:53:36.805Z caller=collector.go:105 msg="Error from scraper" scraper=workspaces err="context canceled, (organization=XXXXXXXXX, page=9)"

I have tested this setup with our Dev Org and everything seems fine. But when i tested exactly the same with our Prod ORG it fails with above error.

Our PROD Org got is large one when compared to DEV.

[nicolaka]

@ktella thanks for raising an issue! you mind sharing how many workspaces does your prod env have ?

One suggestion I can think of is to increase the scrape interval and timeouts (https://github.com/nicolaka/TFBI/blob/main/prometheus/prometheus.yml ) . As this would give tfbi enough time to go through a large number of workspaces.

[ktella]

Our environment got 1000+ workspaces 50K RUM, 5000Applies. 150+ Projects.

its either timing out or hitting a API rate limit. Im not sure which one it is.

[nicolaka]

@ktella I just pushed a new version (v0.5) with some scale enhancements, can you give it a try and let me know if you're still having same issue ? Also is it TFE or TFC ?