feat(postgres): emit source replication log space metrics for Postgres - #4640
feat(postgres): emit source replication log space metrics for Postgres#4640dtunikov wants to merge 4 commits into
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
❌ Test FailureAnalysis: Not a flaky test: the flow-worker container crashed with exit code 2 (Go unrecovered panic) ~13 minutes into the run while all databases stayed healthy, stalling every mirror and cascading into a 20-minute package timeout with ~600 downstream STATUS_SETUP/STATUS_SNAPSHOT failures — the worker crash needs to be diagnosed from the artifact's docker-flow-worker.log, with the PR's new worker-side disk-space metrics collection as the prime suspect. |
dtunikov
changed the title
❌ Test FailureAnalysis: All 11 subtests of TestParseReplicationOffsetText fail deterministically in 0.00s across both matrix jobs because the PR changed parsedReplicationOffset.mechanism from string to protos.MySqlReplicationMechanism while the test at replication_mechanism_test.go:44 still compares it against wantMechanism().String(), causing a testify type mismatch. |
5c86331 to
4d5c28e
Compare
dtunikov
changed the title
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
2 participants
Summary
Adds PostgreSQL replication-slot WAL-capacity metrics so we can alert before a source invalidates a slot because retained WAL has reached its configured limit.
source_log_space_used: WAL bytes retained for the slot.source_log_space_limit: effective retention cap, derived frommax_slot_wal_keep_sizeandwal_keep_size; omitted when retention is unbounded.source_log_space_safe_ratio: remaining safe WAL headroom divided by the effective retention cap.safe_wal_sizeis already emitted as a raw byte metric. The normalized safe ratio makes alerting straightforward: alert whensource_log_space_safe_ratio < 0.1. A value at or below zero means the configured limit has been reached or exceeded.The safe ratio is emitted only when PostgreSQL provides
safe_wal_sizeand a finite retention cap is configured (PostgreSQL 13+). Its absence means the source has no finite slot-WAL cap or does not supportsafe_wal_size, not that the slot is healthy.Validation
go test ./otel_metricsgo test ./connectors/postgres -run '^TestWalRetentionSettingsLimitBytes$' -count=1