PID Reuse Lab

Same PID. Different process.

This interactive lab shows how telemetry grouped only by process.pid can merge two unrelated process lifetimes. An erroring worker exits, its numeric PID returns to the operating system, and a healthy replacement later receives the same PID. A PID-only dashboard now blames the current process for the retired process's errors.

The result is generated by a real GNU Octave numerical model written in MATLAB-compatible syntax. A dependency-free Python service invokes the model and renders correlated OTLP-shaped metrics, logs, and traces.

The default paradox

The default 15-minute scenario uses PID 4242 twice:

Lifetime	Instance	Version	Requests	Error rate
Retired	worker-retired-a7f3	1.8.4	7,200	35%
Current	worker-current-f21c	1.9.0	10,200	1%

A query grouped only by PID returns one series with a 15.1% combined error rate. It attributes 2,520 historical errors to the process that owns PID 4242 now, even though that current process has a 1% error rate.

The same merged series also makes a monotonic process.cpu.time counter fall and makes process.uptime jump backward. These are not impossible process behaviors. They are evidence that the resource identity changed.

Why telemetry.sh helps

The lab correlates the identity-bearing attributes across all three signals:

• process.pid, which is the same for both lifetimes;
• process.creation.time, which changes when the PID is reassigned;
• service.instance.id, which separates the retired and current workers;
• service.version, which confirms that the two owners ran different builds;
• process.uptime, process.cpu.time, and process.memory.usage;
• structured exit, allocation, and health logs;
• failed and successful request spans tied to the correct lifetime.

The PID-only view gives a plausible but wrong diagnosis. The identity-aware view shows that the retired worker failed and the current worker is healthy.

Run locally

Requirements:

• GNU Octave 11.3
• Python 3.11 or newer
• GNU Make

make check
make run

Open http://127.0.0.1:3000.

Useful endpoints:

GET /api/simulate
GET /api/telemetry
GET /healthz

Every UI control is also a query parameter:

/api/simulate?firstLifetimeSec=360&reuseGapSec=30&scrapeIntervalSec=15&requestsPerSec=20&oldErrorRatePct=35&newErrorRatePct=1

Run with Docker

docker compose up --build

The image uses the GNU Octave 11.3 toolchain, executes the model self-test during the build, and starts the lab on port 3000.

Architecture

browser controls
      │
      ▼
Python standard-library HTTP service
      │
      ├──▶ GNU Octave / MATLAB-compatible model
      │       ├── process lifetime A
      │       ├── PID release + reuse gap
      │       ├── process lifetime B
      │       └── scrape + request accounting
      │
      └──▶ correlated OTLP-shaped evidence
              ├── PID-only metric view
              ├── lifetime-aware metric view
              ├── process exit/start logs
              └── request traces by instance

The browser does not calculate the collision, rates, reset, or sample points. It renders the model and telemetry API results.

What to try

• Shorten the reuse gap. A sparse scrape schedule can make the owner change look like a single discontinuity instead of a period with no process.
• Raise the retired error rate. The current worker looks progressively worse in the PID-only view while its real error rate does not change.
• Increase the retired lifetime. More historical traffic is attached to the current PID owner.
• Widen the scrape interval. The negative apparent CPU rate becomes more likely to be clamped or discarded by a dashboard.
• Switch the signal inspector between metrics, logs, and traces. The lifetime identity is consistent across every signal.

Standards behind the lab

• The OpenTelemetry process resource conventions define both process.pid and process.creation.time as identifying attributes for an operating-system process.
• The OpenTelemetry service conventions define service.instance.id as the unique identity for a service instance.
• The OpenTelemetry OS process metrics define process.cpu.time, process.memory.usage, and process.uptime.

Safety

This is a deterministic educational model. It does not spawn PID churn, kill processes, inspect the host process table, export telemetry, or contact upstream systems.

License

MIT