Skip to content

Repository files navigation

NaN Poison Lab

One non-finite sample. Four aggregation policies. Four different versions of reality.

This interactive lab shows how NaN, +Inf, and -Inf travel through a metric pipeline. The default incident has one shard running at 96% utilization while all of its observed samples are NaN. A strict aggregate becomes non-finite; a drop policy reports a healthy-looking 68%; a clamp policy fabricates 62%; and a dual-signal policy keeps the finite view while paging on its lost coverage.

The classifier is an executable Flex program written in Lex. A dependency-free Node.js service generates OpenMetrics text, runs it through the compiled scanner, applies the four policies, renders the results, and exposes correlated OTLP-shaped metrics, logs, and traces.

The failure

OpenMetrics requires NaN, +Inf, and -Inf to be supported. Its specification is unusually explicit: NaN is not a missing-value marker, and producers must not use it for missing or otherwise bad data. It can legitimately arise from calculations such as division by zero. See the OpenMetrics specification.

OpenTelemetry similarly preserves IEEE 754 floating-point semantics while representing “no recorded value” separately. See the OpenTelemetry metrics data model and its special floating-point JSON mapping.

That leaves every metric pipeline with a policy decision:

Policy Default result Failure mode
Strict propagation NaN The aggregate cannot be rendered or compared numerically
Drop non-finite 68.0% The incident shard disappears from the population
Clamp to bounds 62.4% A finite but invented number replaces the measurement
Quarantine + signal 68.0% + alert The finite view survives and its 91.7% coverage is explicit

The exact display behavior of a non-finite result depends on the query engine and visualization layer. This lab models the policy boundary directly instead of assuming one vendor’s renderer.

Why telemetry.sh helps

The useful evidence spans all three signals:

  • worker.utilization.displayed exposes the value produced by each policy;
  • telemetry.input.coverage measures how much of the population remains finite;
  • telemetry.metric.non_finite.count identifies the integrity failure directly;
  • a warning log records the special-value class and rejected sample count;
  • an error log identifies the shard whose finite observation disappeared;
  • four query spans preserve policy, coverage, invalid count, and panel state.

The strongest operational pattern is a dual signal:

  1. quarantine non-finite values from the numeric aggregate;
  2. alert on their count and the finite population’s coverage;
  3. retain the raw special value and correlated resource identity for diagnosis.

That avoids both a blank dashboard and a healthy-looking lie.

Run locally

Requirements:

  • Flex 2.x
  • a C compiler
  • GNU Make
  • Node.js 20 or newer
npm ci
make check
make run

Open http://127.0.0.1:3000.

Useful endpoints:

GET /api/simulate
GET /api/telemetry
GET /healthz

Every UI control is also a query parameter:

/api/simulate?poison=%2BInf&shards=16&points=30&poisonShard=11&poisonRate=70&healthy=64&incident=99

Run with Docker

docker compose up --build

Then open http://127.0.0.1:3000.

Executable model

model/nonfinite.l contains the complete lexical classifier. The service generates records in the Prometheus/OpenMetrics text exposition format:

worker_utilization_percent{shard="07",sample="01"} NaN

The build compiles the scanner:

flex --outfile=build/nonfinite.c model/nonfinite.l
cc -std=c99 -O2 -o build/nonfinite-scanner build/nonfinite.c

The scanner emits typed JSON Lines:

{"line":151,"metric":"worker_utilization_percent","shard":"07","kind":"nan","value":"NaN"}

The service aggregates only those parsed records. The browser receives the model’s output; it does not reimplement the failure.

Architecture

browser controls
      │
      ▼
Node.js service ── OpenMetrics text ──▶ compiled Lex scanner
      │                                      │
      ├── four aggregation policies ◀────────┤
      ├── shard population view ◀────────────┤
      └── OTLP-shaped evidence ◀─────────────┘

No database, framework, credentials, CDN, or third-party JavaScript dependency is required.

Safety

This is a deterministic educational simulation. It sends no external traffic and exports no telemetry.

License

MIT

About

Interactive Lex lab exposing incidents hidden by NaN and infinity metric poisoning.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages