Document the single scrape target requirement for v2 metrics

[kaitlynmichael]

Documents that the v2 metrics endpoint is cluster-wide and must be scraped through a single Prometheus target.

Jira: DOC-6906 · Related: KBP-1033, observability#94, Zendesk #165134

Why

Every node runs an aggregator that reads the node list from CCS, scrapes every other node, and returns the same complete cluster-wide result. Envoy routes :8070/v2 to the local aggregator on whichever node is asked, so every node returns a complete and identical copy of the cluster's metrics.

That breaks the usual Prometheus convention of one exporter per host. Anyone applying the node_exporter pattern — or substituting node IPs because they have no cluster FQDN configured — lists every node as a target. The result:

• Every series duplicated once per target
• Every sum()-based Grafana panel inflated by the number of targets
• N² internal scrapes per interval (13 nodes → 169)
• No error anywhere. Prometheus shows all targets UP and Grafana renders normally

A customer spent several weeks unable to trust their CPU and connection dashboards because of this. The docs showed the correct single-target config but never said why it was required or what breaks otherwise.

What changed

Content	Where
Single-target note beside every v2 scrape_configs example	all 6 source files
Why the reason changed in v1→v2 — v1 forced one target via redirects from non-master nodes; v2 has no redirects, but still needs one target	all 3 v1-to-v2 pages, MSE _index
/v2/node alternative for conventional per-node topology, plus the caveat that aggregation adds the cluster/node labels so /v2/node omits them	all 3 v1-to-v2 pages, MSE _index
Verification step — count(up{...}) returns 1; node_metrics_up reports 0 for unreachable nodes	quickstart embed, get-started.md
<IP>:8070 → <cluster_name>:8070 — the old placeholder nudged readers toward the exact per-node config this PR warns against	4 current/shared files

9 files, +50/−4, reaching 8 rendered pages.

For reviewers — two things to know

1. Scope is much wider than the three pages on the ticket, because this content is duplicated. Two separate copy-paste clusters, neither using the embeds/ mechanism:

• The v1-to-v2 transition page exists three times — under /integrate/prometheus-with-redis-enterprise/, /operate/rs/monitoring/metrics_stream_engine/, and /operate/rs/references/metrics/. All 43 lines, same title and content, and already drifted: pairwise they differ by 2, 3, and 10 lines. Nobody diverged them on purpose; someone edited one copy and not the others.
• The Prometheus setup steps exist twice — the shared rs-prometheus-grafana-quickstart.md embed and a hand-maintained copy in monitoring/get-started.md.

Fixing only the pages named on the ticket would have left the same wrong guidance live on three others. The second commit here adds the third v1-to-v2 copy, which my first pass missed.

2. This intentionally reaches the frozen 7.8 and 7.22 pages. Two of the edited files are shared embeds (rs-prometheus-grafana-quickstart.md, rs-observability.md) that those versioned pages consume. Normally we don't propagate current content into frozen snapshots, but this is a factual correction and I verified it holds for both versions (see below). Say the word if you'd rather I fork the embed instead.

@markotrapani — the claims most worth your eye, since you reproduced this on a lab cluster: that the /v2/node label caveat is stated correctly, and that node_metrics_up is the right metric to point people at.

Verification

Checked against the product source at tags v8.2.1-2, v7.22.2-174, and v7.8.6-296 — all three agree:

• master_aggregator is autostart = true in supervisord_metrics_exporter.conf with no master-only guard, so it runs on every node
• listener-specs.go routes /v2 → local master_aggregator unconditionally; the IsCurrNodeMaster() branch above it only changes the listener name and adds Connection: close
• HandleMetricsRedirects feeds only the legacy / route, confirming redirects are not involved on v2
• node_metrics_up exists as far back as 7.8, so the verification tip is safe in shared content

Two small corrections to the ticket, applied here: the sub-endpoint list was missing /v2/db_controller, and node_metrics_up is a stronger check than count(up{...}) alone.

Hugo builds clean. Vale warnings are at the pre-existing baseline; the only remaining new errors are Grafana spellings, which this vocabulary already flags 30+ times in these same files.

Follow-up worth filing

Both duplication clusters above should collapse into shared embeds — one canonical v1-to-v2 page, one canonical setup-steps page. That drift is the root cause this ticket surfaced, it already caused a missed page inside this PR, and it will recur.

🤖 Generated with Claude Code

[github-actions]

DOC-6906

[github-actions]

Staging links:
https://redis.io/docs/staging/DOC-6906/integrate/prometheus-with-redis-enterprise/prometheus-metrics-v1-to-v2/
https://redis.io/docs/staging/DOC-6906/operate/rs/monitoring/
https://redis.io/docs/staging/DOC-6906/operate/rs/monitoring/get-started/
https://redis.io/docs/staging/DOC-6906/operate/rs/monitoring/metrics_stream_engine/
https://redis.io/docs/staging/DOC-6906/operate/rs/monitoring/metrics_stream_engine/prometheus-metrics-v1-to-v2/
https://redis.io/docs/staging/DOC-6906/operate/rs/references/metrics/
https://redis.io/docs/staging/DOC-6906/operate/rs/references/metrics/prometheus-metrics-v1-to-v2/

[github-actions]

🧠 Redis Memory

Found 5 related items from repository history:

• PR — RS: Remove outdated node_exporter version (1.8.1) from Prometheus metrics pages

• PR — RED-199772: Refine metrics documentation

  With this PR, we are refining the Metrics v2 documentation. It includes several improvements:

• PR — Update rs-prometheus-metrics-v2.md

  Remove redundant metrics

• PR — Update rs-prometheus-metrics-v2.md

  update metric

• PR — RS: Removed reverted v2 search metrics

  Staged preview: https://redis.io/docs/staging/DOC-6273/operate/rs/monitoring/metrics_stream_engine/prometheus-metrics-v2/#shard-metrics

Memory updated at fee6a0f

[dwdougherty]

Language LGTM.

[github-actions]

🧠 Redis Memory

Found 5 related items from repository history:

• Commit — Document the single scrape target requirement for v2 metrics

  Document the single scrape target requirement for v2 metrics

• PR — RS: Remove outdated node_exporter version (1.8.1) from Prometheus metrics pages

• PR — RED-199772: Refine metrics documentation

  With this PR, we are refining the Metrics v2 documentation. It includes several improvements:

• PR — RS: Removed reverted v2 search metrics

  Staged preview: https://redis.io/docs/staging/DOC-6273/operate/rs/monitoring/metrics_stream_engine/prometheus-metrics-v2/#shard-metrics

• PR — DOC-6833: Fix four RDI Cloud doc issues from #rdi-cloud

  • Remove Active-Active from the 'multiple targets' use case sentence; add it as an explicit unsupported target topology under 'When not to u

Memory updated at 0a846ac

[markotrapani]

Looks good to me!