Locate a deployment's loss to a path segment, measured from both ends

[bojieli]

The question this answers

A slow deployment raises one question no instrument here answered: not how bad the path is, but whose fault it is. pathprobe finds an erasure floor, pathmeasure says what a stack achieves, doctor checks preconditions and placement — all three describe a path end to end. An operator losing a seventh of everything that crosses it learns only that they are losing a seventh.

Three candidates, three different responses, and only one is ours:

Segment	What it is	What fixes it
client-to-internet	the client's access link and first mile	their ISP; no transport repairs a first mile
client-to-gateway	the long haul this transport carries	coding, pacing, lane recovery
gateway-to-internet	the gateway's own transit onward	its provider, or moving the gateway

How the legs are separated

A shared-element argument over legs measured in the same minutes. A client reaching an unfiltered anchor near itself crosses its own access link and nothing else; the same client reaching the gateway crosses that link and the long haul; the gateway reaching an anchor near itself shares neither. So a lossy gateway leg with a clean local anchor is the long haul, and the same loss with a lossy local anchor is a first mile the gateway leg merely inherited — the report says which, in those words.

Both ends run concurrently, not in sequence: the characterised path moves between roughly zero and seventeen percent within minutes, so sequential legs can differ by more than any fault would move them.

Anchor choice is load-bearing, not a detail

A probe from a filtered network to a filtered destination measures the filter. Read as loss it would convict a healthy access link of the worst fault this report can report and send the operator to their ISP — a client in China probing a blocked host sees near-total loss on a first mile in perfect health.

Rather than geolocate either end, one Chinese and one global anchor are probed from both, and each end is judged by whichever answered cleanly. One clean anchor proves the link carries traffic; the disagreement between the two becomes a finding of its own. Every anchor gets both instruments, so:

• an address that answers echo but refuses a handshake is reported as filtering, charged to no segment;
• a leg that returns nothing is reported as unanswered, never as 100% loss — from the client those are indistinguishable, and only one is a finding.

What it reads rather than probes

The client-to-gateway segment comes from the running transport's own metrics where --metrics-listen is available. That is the only measurement separating upstream from downstream — the directions this project has measured differing by fourteen points, which a round-trip probe would average into nothing. Ratios drawn from an idle session are not quoted as measurements.

--ssh runs this same code on the gateway over the operator's existing ssh configuration, so no key material passes through the process, and stdin carries the request (hence BatchMode=yes: a password prompt would hang rather than fail). An older remote binary falls back to parsing ping(8); every leg measured that way is labelled, because the fallback gives up the property that made the two ends comparable. Without a far vantage point the gateway's transit is reported as not measured rather than passing silently.

Two failure modes worth calling out

• No echo socket — an ordinary unprivileged Linux host. The verdict falls back to the share of handshakes that paid a retransmission timeout, not the connection failure rate: TCP retransmits, so nearly every connection completes on a path erasing a fifth of everything, and the failure rate would call it clean.
• A blocked anchor made --json unencodable. lossmodel.Analyze returns BurstFactor = Inf × 0 = NaN for a run with no arrivals, and JSON has no NaN — so one filtered anchor killed the whole report with nothing on stdout. Invisible in text mode, where NaN > 0 renders as -. Guarded at both doors, with regression tests.

Overlap removed

doctor's decomposition of the gateway's hop to a destination is a subtraction; this command measures that same leg. Two commands answering one question by two methods will disagree, so the derived figure now says which it is and names where the measured one lives. docs/CHOOSING-A-GATEWAY.md gains a fourth instrument, marked as the one belonging after deployment rather than before it.

Verified

Ran against a live China↔US tunnel: correctly attributed 10.6% downstream / 2.5% upstream erasure to the long haul with a clean local anchor, corroborated by an independent ICMP leg at burst factor 1.0 — erasure, not congestion.

go test -short ./..., go vet, gofmt, staticcheck -checks=all,-U1000, the Python suite, and changelog.py check all pass. gosec reports zero findings in the new code. The metrics parser is tested against a page produced by the real metrics.Registry rather than a fixture — which caught that erasure is only aggregated for an observation naming its controller.

🤖 Generated with Claude Code

https://claude.ai/code/session_011HCTFdNai8pNgSyJwV8vGW