Telemetry causing increased memory usage even when disabled

[rhamzeh]

Is there a proper way to disable the Telemetry? The memory of my service increases over time and the investigation lead to the telemetry. When setting Telemetry config to nil (aka using default telemetry)

Showing nodes accounting for 53.54MB, 100% of 53.54MB total
Showing top 10 nodes out of 61

      flat  flat%   sum%        cum   cum%
   23.50MB 43.89% 43.89%    23.50MB 43.89%  github.com/openfga/go-sdk/telemetry.DefaultTelemetryConfiguration
   14.50MB 27.09% 70.98%    14.50MB 27.09%  github.com/openfga/go-sdk/telemetry.(*Metrics).GetHistogram
   10.50MB 19.61% 90.59%    10.50MB 19.61%  github.com/openfga/go-sdk/telemetry.Configure
    2.04MB  3.80% 94.39%    12.54MB 23.41%  github.com/openfga/go-sdk/telemetry.Get</div>

After disabling telemetry, we notice that it is still causing memory increase:
After about an 2 hours of my process running, I am seeing this when I examine pprof. It was not there for the longest time, and now I see it increasing again.

fgaClient, err := client.NewSdkClient(&client.ClientConfiguration{
	ApiUrl: openfgaUrl,

	Telemetry: &telemetry.Configuration{
		Metrics: &telemetry.MetricsConfiguration{},
	},

	Credentials: &credentials.Credentials{
		Method: credentials.CredentialsMethodApiToken,
		Config: &credentials.Config{
			ApiToken: openfgaToken,
		},
	},
})

512.14kB 25.00% 25.00% 512.14kB 25.00% github.com/openfga/go-sdk/telemetry.(*Metrics).GetHistogram
1024.28kB 33.34% 33.34% 1024.28kB 33.34% github.com/openfga/go-sdk/telemetry.(*Metrics).GetHistogram

Showing top 10 nodes out of 54
flat flat% sum% cum cum%
2048.56kB 33.24% 33.24% 2048.56kB 33.24% github.com/openfga/go-sdk/telemetry.(*Metrics).GetHistogram
1042.10kB 16.91% 50.15% 2066.14kB 33.53% github.com/openfga/go-sdk/telemetry.Get
1024.04kB 16.62% 66.77% 1024.04kB 16.62% github.com/openfga/go-sdk/telemetry.Configure

References

Originally reported by @keitherw on September 15, 2025 and discussed in https://github.com/orgs/openfga/discussions/497

[keitherw]

Is there something I would also need to do with the running version of my openFGA? (v1.8.9)

[keitherw]

Also, one thing to note is that I only see the heap usage of the telemetry after about 40 minutes of execution time. I iterate an operation to listObjects every 2 seconds, monitoring the pprof heap.

[keitherw]

Something tells me that it 'does', work as expected, that it doesn't perform the telemetry when the configuration is empty, but not sure why I am seeing it come back eventually. Last test, it showed up again after running for about 2 hours. I am hitting it hard in that I am creating a new client every second. Note that I've since changed my code so that it doesn't create a new client every time I need it. I create a new client per 'customer' you could say, saving it in a cache that I clear after 30 minutes.

[aaguiarz]

In general we recommend to create a single instance of the client. That allows better usage of http connection pooling, and if you are using client credentials, reuse the access token.

Why do you need to create a client per customer?

[keitherw]

The way we need to use openFGA, we need a store per customer.

[aaguiarz]

You should be able to override the Store ID per request with request options parameters, e,g.

go-sdk/client/client.go

Line 1888 in 2e3106c

type ClientCheckOptions struct {

[keitherw]

Ok, I'll try that out. Thanks.

[keitherw]

Note that I did update my code to only create a single instance into openFGA and then set the store accordingly depending on the company I need to authorize. But the telemetry has come back.

  flat  flat%   sum%        cum   cum%

3073.48kB 39.99% 39.99% 3073.48kB 39.99% github.com/aws/aws-sdk-go/aws/endpoints.init
2048.11kB 26.65% 66.64% 2048.11kB 26.65% github.com/openfga/go-sdk/telemetry.Configure
514.63kB 6.70% 73.33% 514.63kB 6.70% bytes.growSlice
513kB 6.67% 80.01% 513kB 6.67% runtime.allocm

  flat  flat%   sum%        cum   cum%

3073.48kB 33.26% 33.26% 3073.48kB 33.26% github.com/aws/aws-sdk-go/aws/endpoints.init
2560.12kB 27.71% 60.97% 2560.12kB 27.71% github.com/openfga/go-sdk/telemetry.Configure
1042.10kB 11.28% 72.25% 3602.23kB 38.99% github.com/openfga/go-sdk/telemetry.Get
514.63kB 5.57% 77.82% 514.63kB 5.57% bytes.growSlice
513kB 5.55% 83.37% 513kB 5.55% runtime.allocm

[keitherw]

I noticed that my health checks were creating a new openFGA client instance every time so I've update it to re-use a specific client created for my health check.