Skip to content

Indexer metrics followups#655

Open
AndresJulia wants to merge 8 commits intomainfrom
CCIP-9200_indexer_metric_gaps
Open

Indexer metrics followups#655
AndresJulia wants to merge 8 commits intomainfrom
CCIP-9200_indexer_metric_gaps

Conversation

@AndresJulia
Copy link
Contributor

@AndresJulia AndresJulia requested review from a team and skudasov as code owners February 9, 2026 15:57
@AndresJulia AndresJulia requested a review from bukata-sa February 9, 2026 15:57
@github-actions
Copy link

github-actions bot commented Feb 9, 2026

👋 AndresJulia, thanks for creating this pull request!

To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team.

Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks!

@github-actions
Copy link

github-actions bot commented Feb 9, 2026

Code coverage report:

Package main CCIP-9200_indexer_metric_gaps diff
github.com/smartcontractkit/chainlink-ccv/aggregator 48.41% 48.41% +0.00%
github.com/smartcontractkit/chainlink-ccv/cmd 0.00% 0.00% +0.00%
github.com/smartcontractkit/chainlink-ccv/committee 100.00% 100.00% +0.00%
github.com/smartcontractkit/chainlink-ccv/common 0.00% 0.00% +0.00%
github.com/smartcontractkit/chainlink-ccv/executor 39.33% 39.33% +0.00%
github.com/smartcontractkit/chainlink-ccv/indexer 36.80% 37.13% +0.33%
github.com/smartcontractkit/chainlink-ccv/integration 37.03% 37.03% +0.00%
github.com/smartcontractkit/chainlink-ccv/pkg 100.00% 100.00% +0.00%
github.com/smartcontractkit/chainlink-ccv/pricer 15.70% 15.70% +0.00%
github.com/smartcontractkit/chainlink-ccv/protocol 62.97% 63.07% +0.10%
github.com/smartcontractkit/chainlink-ccv/verifier 55.59% 55.59% +0.00%

monitoring.Metrics().IncrementActiveRequestsCounter(c.Request.Context())

// Increment HTTP request counter
monitoring.Metrics().IncrementHTTPRequestCounter(c.Request.Context())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why removing this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because we have counter in RecordHTTPRequestDuration

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to record HttpRequestDuration on all endpoints? I think the duration should be more precisely set to particular endpoints since it has increased cardinality, while we keep httprequestcounter as a more generic one, for all endpoints

Copy link
Collaborator

@bukata-sa bukata-sa Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, yeah makes sense
@AndresJulia can you do this

  1. keep counter for all endpoints in middleware + record http status
  2. move http request duration to handler and instead of URI record path from predefined set of values: /verifierresults , /verifierresults/:messageID , /messages

sdkmetric.Instrument{Name: "indexer_storage_query_duration_seconds"},
sdkmetric.Stream{Aggregation: sdkmetric.AggregationExplicitBucketHistogram{
Boundaries: []float64{0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10},
Boundaries: []float64{0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i know it's kinda default, but is there any value in distinction between 1ms, 5ms, or 10ms calls?

}

i.monitoring.Metrics().RecordStorageQueryDuration(ctx, time.Since(startQueryMetric))
i.monitoring.Metrics().RecordStorageQueryDuration(ctx, time.Since(startQueryMetric), opName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to have full visibility, we should probably also include the query status (error=(true|false))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is another metric for errors

Copy link
Collaborator

@bukata-sa bukata-sa Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Marcin here, for latency we also might need a status here and in the storage query duration metric
I'm wondering if we can drop error metric in this case

@mateusz-sekara
Copy link
Collaborator

mateusz-sekara commented Feb 11, 2026

@AndresJulia, please make sure you merge main to your PR. I've moved some logic from indexer middleware to the shared packages, and now TokenVerifier relies on the same shared code (unified way of tracking API latencies)

Context: #676

@AndresJulia AndresJulia force-pushed the CCIP-9200_indexer_metric_gaps branch from 04bbc54 to a7969e1 Compare February 11, 2026 15:54
@@ -0,0 +1,48 @@
package middleware
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use integration/pkg/api/middleware/active_request.go instead ;)

@AndresJulia AndresJulia requested a review from emate February 13, 2026 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants