test(ublk): chaos backend with partial failure rates and latency

Makefile

@@ -1,4 +1,4 @@
-.PHONY: test test-unit test-integration test-rapid fuzz-uring fuzz-uring-submit fuzz-uring-cancel cover cover-html chain flushbench flushbench-race stress fault sigkill build lint lint-fmt lint-tidy lint-vet fmt hooks
+.PHONY: test test-unit test-integration test-chaos test-rapid fuzz-uring fuzz-uring-submit fuzz-uring-cancel cover cover-html chain flushbench flushbench-race stress fault sigkill build lint lint-fmt lint-tidy lint-vet fmt hooks
 
 test: test-unit test-integration
 
@@ -10,6 +10,12 @@ test-integration:
 	go test -c -race -tags=integration -o /tmp/ublk.test ./ublk/
 	sudo /tmp/ublk.test -test.v -test.timeout=300s
 
+# Run just the chaos backend integration tests. Useful for iterating
+# on the chaos wrapper without waiting for the full integration suite.
+test-chaos:
+	go test -c -race -tags=integration -o /tmp/ublk.test ./ublk/
+	sudo /tmp/ublk.test -test.v -test.timeout=120s -test.run=TestChaos
+
 # Run the io_uring fuzz targets back-to-back. Default budget is 30s
 # per target, override with FUZZTIME=2m make fuzz-uring (or longer).
 # These targets need no kernel module and no root.

TODO.md

@@ -499,34 +499,6 @@ utilisation, backend panic count.
 
 ## Testing
 
-### Probabilistic chaos backend
-
-Add a `chaosBackend` wrapper (in `ublk/` test helpers or a new
-`ublk/testutil/` package) that wraps any `Backend` and randomly injects
-failures at a configurable rate:
-
-```go
-type ChaosConfig struct {
-    WriteErrorRate float64       // fraction of WriteAt calls that return EIO
-    ReadErrorRate  float64       // fraction of ReadAt calls that return EIO
-    MaxDelay       time.Duration // uniform random delay [0, MaxDelay] per call
-}
-```
-
-Write integration tests that drive `TestTortureRandomIO` with a chaos backend
-and verify:
-- Errors surface as `EIO` to the caller at the block device level (no hangs,
-  no panics, no silent data corruption on the successful path).
-- `Close()` terminates in bounded time even when the backend is injecting
-  errors and/or latency into in-flight IOs.
-- After the chaos backend is swapped out for a healthy one, subsequent reads
-  return correct data (no residual corruption state in the worker or ring).
-
-**Why this is distinct from `fault_integration_test.go`:** the existing fault
-tests use fully-on or fully-off failure modes with a static config. The chaos
-backend exercises partial failure rates and latency injection, which is the
-realistic failure mode for remote or unreliable storage backends.
-
 ### Linearizability checking (extension of the `rapid` state machine test)
 
 Once the `rapid` state machine test (above) is in place, instrument it to