Skip to content

Validate measurement accuracy with controlled real network paths #43

Description

@korya

The simulation suite passes all 33 scenarios, but that does not establish the same accuracy or convergence on real transports. Add a repeatable validation layer between the fluid simulator and public-server smoke tests.

Current evidence and limitations:

  • internal/linksim explicitly does not model TCP. It approximates rates, queueing, send-buffer credit, and capacity changes.
  • The simulator creates a foreign and a self sample together at each probe step, while production interleaves probes and records actual asynchronous completion and failure.
  • TestLive checks positive idle latency, throughput, and RPM; it does not assert accuracy against known link conditions or default-budget convergence.
  • Public live runs use Linux; six-target cross-compilation is not measurement validation on every supported OS/architecture.

Relevant code: simulator, simulated probes, live smoke test.

Acceptance criteria:

  • Provide a reproducible harness running the real client and reference server over controlled bandwidth, delay, queue, and loss conditions, with explicit accuracy and budget tolerances.
  • Cover upload send-buffer accounting, flow ramp/convergence, bufferbloat, capacity changes, and sparse/failed probes.
  • Compare client estimates with independent observations of delivered bytes and path timing; do not use the simulator itself as ground truth.
  • Record sanitized observations sufficient to calibrate the simulator and reproduce relevant engine decisions.
  • Match simulator probe scheduling/completion more closely where evidence shows a material difference.
  • Document a staged runtime-validation matrix across supported platforms, with unsupported environments reported as blocked.
  • Keep ordinary tests offline and deterministic; separate public smoke tests from controlled accuracy checks.

Related: #20, #24, INV-3, LOAD-3 through LOAD-14, LAT-10, docs/guidelines.md algorithm-change requirements. This is a validation gap, not a claim that every current real-network measurement is inaccurate.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions