feat: QEMU ESP32-S3 testing platform + swarm configurator (ADR-061/062)#260
Merged
feat: QEMU ESP32-S3 testing platform + swarm configurator (ADR-061/062)#260
Conversation
Implement full QEMU emulation framework for firmware testing without physical hardware: Mock CSI Generator (mock_csi.c): - 10 test scenarios: empty room, static/walking person, fall, multi-person, channel sweep, MAC filter, ring overflow, boundary RSSI, zero-length - Physics-based signal model with breathing modulation and Doppler - LFSR pseudo-random noise, CONFIG_CSI_MOCK_ENABLED Kconfig guard - Scenario 255 runs all sequentially QEMU Runner & CI: - qemu-esp32s3-test.sh: build, merge flash image, run QEMU, validate - validate_qemu_output.py: 14 automated checks (boot, NVS, edge, vitals, crash detection) with colored output and severity-based exit codes - generate_nvs_matrix.py: 14 NVS provisioning configs for matrix testing - firmware-qemu.yml: GitHub Actions CI with 4-scenario matrix Fuzz Testing: - 3 libFuzzer targets: CSI serialize, NVS config validation, ring buffer - Host-compilable ESP-IDF stubs (no ESP-IDF dependency for fuzzing) - 6 seed corpus files for guided fuzzing - Makefile with ASAN + UBSAN sanitizers Documentation: - firmware/esp32-csi-node/README.md: comprehensive QEMU testing guide - Root README.md: collapsed QEMU testing section Build verified: normal firmware build (RC=0) with mock_csi excluded. Closes #259 Co-Authored-By: claude-flow <ruv@ruv.net>
Fix 9 bugs (LFSR bias, MAC filter init, scenario loop, NVS boundary values), add 7 new files completing Layers 3 (mesh), 4 (GDB), 5 (coverage), 8 (snapshots), 9 (chaos testing), expand CI with fuzz and NVS validation jobs, update README with full platform overview. Co-Authored-By: claude-flow <ruv@ruv.net>
- Add --help/-h flags to all 4 shell scripts with usage, env vars, examples - Add prerequisite checks with install hints (apt/brew/pip) for missing tools - Standardize exit codes (0=PASS, 1=WARN, 2=FAIL, 3=FATAL) across all scripts - Standardize MESH_TIMEOUT to QEMU_TIMEOUT with backward compatibility - Add SKIP_BUILD precheck for missing flash image in qemu-esp32s3-test.sh - Add argparse to validate_qemu_output.py (was using raw sys.argv) - Improve error messages in generate_nvs_matrix.py with NVS tool install hints - Add socket connection warnings in inject_fault.py connect_monitor() - Add example output epilog to check_health.py --help - Add glossary (14 terms) and quick-start section to ADR-061 - Add GDB debugging walkthrough to ADR-061 Layer 4 - Fix stat portability in CI workflow (stat -c%s -> portable file_size()) - Add -type f to find commands in CI workflow Co-Authored-By: claude-flow <ruv@ruv.net>
CRITICAL: - inject_fault.py: make nvs_corrupt write actual bytes via --flash arg; heap_exhaust and corrupt_frame now pause VM with honest WARNING about GDB stub requirement for real memory writes - firmware-qemu.yml: remove github.run_id from cache key (was causing 100% cache miss rate, rebuilding QEMU every run) - mock_csi.c: change scenario_elapsed_ms() to int64_t (uint32 wrapped at ~49 days) HIGH: - qemu-mesh-test.sh: pass --results flag to validate_mesh_test.py (was passing positional arg to named-only parameter) - test/Makefile: separate corpus directories per fuzz target (corpus_serialize/, corpus_edge/, corpus_nvs/) - qemu-snapshot-test.sh: replace log truncation with tail-based extraction (truncation created sparse file while QEMU held fd) MEDIUM: - mock_csi.c: reset s_mac_filter_initialized in mock_csi_init() - mock_csi.c: fix LFSR polynomial comment (32,31,29,1 not 32,22,2,1) - sdkconfig.coverage: add FreeRTOS timer stack 4096 and WDT tuning - firmware-qemu.yml: replace continue-on-error with FUZZER_CRASH env - qemu-chaos-test.sh: rename heap_pressure to heap_exhaust for consistency - validate_qemu_output.py: fix docstring "14 checks" -> "16 checks" - generate_nvs_matrix.py: deduplicate temp file cleanup paths LOW: - mock_csi.c: remove M_PI float suffix, fix overflow burst flag - qemu-snapshot-test.sh: fix now_ms() for macOS date +%s%N - ADR-061: fix scenario 8 RSSI range to -90...-10 dBm - launch.json: remove contradictory compound debug config Co-Authored-By: claude-flow <ruv@ruv.net>
YAML-driven orchestrator for testing multiple ESP32-S3 QEMU instances with configurable topologies (star/mesh/line/ring), role-based nodes (sensor/coordinator/gateway), and swarm-level health assertions. New files: - ADR-062: architecture decision record - qemu_swarm.py: main orchestrator (1097 lines) - YAML config parsing with schema validation - 4 topology implementations with TAP/SLIRP fallback - Per-node NVS provisioning via provision.py --dry-run - Signal-safe cleanup, dry-run mode, JSON results output - swarm_health.py: 9-assertion health oracle (653 lines) - 7 preset configs: smoke (2n/15s), standard (3n/60s), large-mesh (6n/90s), line-relay (4n/60s), ring-fault (4n/75s), heterogeneous (5n/90s), ci-matrix (3n/30s) - CI: swarm-test job in firmware-qemu.yml Co-Authored-By: claude-flow <ruv@ruv.net>
CRITICAL:
- Delete stale nvs_provision.bin before provisioning each node
- Fix log filename mismatch: swarm_health.py now finds qemu_node{i}.log
with node_{i}.log fallback
- CI swarm-test job builds firmware instead of downloading missing artifact
- Accept both qemu_flash.bin and qemu_flash_base.bin as base image
HIGH:
- Replace broad "heap" substring match with precise regex patterns
(HEAP_ERROR, heap_caps_alloc.*failed, etc.) to avoid false positives
- Guard os.geteuid() with hasattr for Windows compatibility
- Offset SLIRP ports by +100 to avoid collision with aggregator on 5005
- Assertions now WARN (not vacuous PASS) when no parseable data found
MEDIUM:
- Mark network_partitioned_recovery as "(future)" in ADR-062
- Fix node_id prefix dedup bug (node_1 no longer matches node_10)
- Add duplication note in qemu_swarm.py pointing to swarm_health.py
- Document implicit TDM auto-assignment in ADR YAML schema
- swarm_health.py only checks sensor nodes for frame production
- Fix channel 0 treated as falsy
Co-Authored-By: claude-flow <ruv@ruv.net>
Plain-language guide for testing firmware without hardware: - What QEMU does and when to use it - Prerequisites and install steps - First test run walkthrough - Understanding test output (exit codes, check names) - Multi-node swarm testing with presets - Writing custom YAML swarm configs - Scenario table (10 mock CSI types) - Topology options (star/mesh/line/ring) - GDB debugging walkthrough - Full test suite commands - 6 QEMU troubleshooting entries Co-Authored-By: claude-flow <ruv@ruv.net>
- Add collapsed QEMU Swarm Configurator section (ADR-062) with usage examples, topology/role/preset/assertion summaries - Update ADR count from 49 to 62 in docs table - Add user guide and ADR-061/062 links to Documentation Links section Co-Authored-By: claude-flow <ruv@ruv.net>
install-qemu.sh (328 lines): - Auto-detects OS (Ubuntu, Fedora, Arch, macOS, WSL) - Installs build deps, clones Espressif QEMU fork, builds with SLIRP - Symlinks to ~/.local/bin, verifies esp32s3 machine support - Installs Python deps (esptool, pyyaml, esp-idf-nvs-partition-gen) - Flags: --check, --uninstall, --install-dir, --branch, --skip-deps qemu-cli.sh (362 lines): - Single entry point for all QEMU operations - 11 commands: install, test, mesh, swarm, snapshot, chaos, fuzz, nvs, health, status, help - Auto-detects QEMU in PATH / ~/.espressif/qemu/ / QEMU_PATH env - Status command shows install state of all tools - Delegates to existing scripts with args passthrough User guide updated to reference installer and CLI. Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: claude-flow <ruv@ruv.net>
Tested on Windows with Espressif QEMU 9.0.0 — firmware boots, generates mock CSI frames, runs edge processing. 14/16 validation checks pass. Fixes: - Guard wifi_init_sta() with CONFIG_CSI_MOCK_SKIP_WIFI_CONNECT (QEMU has no RF PHY, WiFi init stalled at calibration) - Guard stream_sender_init_with() (UDP needs network stack) - Guard ota_update_init_ex() (HTTP server needs network) - Guard display_task_start() with CONFIG_DISPLAY_ENABLE (no I2C hardware in QEMU) - Add mock_csi_init() call in main.c when CONFIG_CSI_MOCK_ENABLED - Add #include "sdkconfig.h" to mock_csi.c (ESP-IDF not auto-including) - Suppress unused s_bad_mac warning Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: claude-flow <ruv@ruv.net>
ruvnet
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #259
What's included
ADR-061: 9-Layer QEMU Testing Platform
ADR-062: Swarm Configurator
Developer Experience
bash scripts/install-qemu.shbash scripts/qemu-cli.sh testQuality
Stats
Test plan
🤖 Generated with claude-flow