Skip to content

Add WiFi/HTTP transport variant (alongside USB CDC)#40

Open
dsk2k wants to merge 5 commits into
HermannBjorgvin:mainfrom
dsk2k:wifi-transport-poc
Open

Add WiFi/HTTP transport variant (alongside USB CDC)#40
dsk2k wants to merge 5 commits into
HermannBjorgvin:mainfrom
dsk2k:wifi-transport-poc

Conversation

@dsk2k

@dsk2k dsk2k commented May 27, 2026

Copy link
Copy Markdown

Adds a third host↔board transport option — WiFi/HTTP — alongside main's BLE and @dilbery's PR #26 USB CDC. Daemon publishes an HTTP bridge on the host LAN, board polls it over WiFi. Compile-time selectable via build flag so existing setups don't change unless they opt in.

Tested on real hardware

Board: Waveshare ESP32-S3-Touch-AMOLED-2.16 (480×480).
Host: Windows 10 (Dutch locale), Python 3.14, PlatformIO 6.1.19.
Network: standard home WiFi (2.4GHz), board → AP → host on the same /24.

Verified end-to-end:

  • Daemon publishes payload at http://<host>:8787/payload with bearer-token auth (token persists in ~/.clawdmeter/wifi_token, mode 0600 on POSIX). /health is open for diagnostics.
  • Firmware connects to WiFi STA, polls the bridge every 5s (configurable WIFI_POLL_MS), parses the same JSON schema as USB CDC, renders Usage + Activity exactly as on the other transports.
  • netstat audit on host: 24 concurrent TimeWait entries from the board's IP on port 8787, ~5 new unique source ports every 25s. Matches WIFI_POLL_MS = 5000 exactly.
  • Physical USB-unplug test: USB-C cable pulled from the host, board moved to a wall-charger. Daemon log shows the USB CDC writer recycling its port and falling through to run_session_no_port() (HTTP-only mode). Board continues receiving Usage + Activity live, polling over WiFi only.

Scope of this PR

Built on top of @dilbery's PR #26 (USB CDC transport) plus my Windows-port and Activity backport from those discussions, so this branch carries some commits that aren't WiFi-specific. The WiFi-only delta is the single commit 36d97d0 — 9 files, 423 insertions:

  • firmware/src/wifi_link.{h,cpp} (new) — public API mirrors serial_link.h, body wrapped in #ifdef CLAWDMETER_TRANSPORT_WIFI so USB-only builds don't need wifi_config.h.
  • firmware/src/wifi_config.h.example — template, real wifi_config.h is .gitignored.
  • firmware/src/main.cpp — macro layer (link_initserial_link_init or wifi_link_init) so the rest of main.cpp doesn't change between transports.
  • firmware/src/ui.h — conditional include for the active transport's link_state_t.
  • firmware/platformio.ini — new env waveshare_amoled_216_wifi extends the USB env + -DCLAWDMETER_TRANSPORT_WIFI.
  • daemon/claude_usage_daemon.py — stdlib http.server.ThreadingHTTPServer thread; _http_payload_lock-guarded cache; helper run_session_no_port() keeps API + state.json polling alive when there's no USB port.
  • daemon/run-daemon.ps1 — sets CLAWDMETER_HTTP=1 for Windows scheduled-task runs.
  • .gitignore — excludes wifi_config.h.

Design notes

  • Decoupling: update_http_payload() runs unconditionally during payload assembly, so a USB CDC write failure doesn't take HTTP down (and vice versa). Both transports serve the same data without doubling the upstream API hit rate.
  • Token handling: stdlib secrets.token_urlsafe(32) at first daemon start, secrets.compare_digest() for constant-time validation. POSIX chmod 0o600 on the token file; on Windows the default user-profile ACL is sufficient.
  • No HTTPS — bridge is LAN-only, same threat model as @dilbery's USB-only path. Don't expose to the Internet without a reverse proxy.
  • Architecture precedent: same shape as rootedlab-code/claude-code-usage-monitor (different host-side data source, same transport pattern). Their bearer-token + LAN HTTP + 5s poll approach worked, ours follows.

Known follow-ups

  • First-time WiFi setup is compile-time only. rootedlab-code uses a captive portal on softAP (board boots as ClaudeMonitor-XXYY until configured). Doable here too — Arduino WiFi has the primitives — but kept out of this PR to keep the diff focused. Hardcoded wifi_config.h works for personal-use scenarios.
  • Daemon serves both transports simultaneously, which produces benign Write failed; recycling port log spam when the board is on WiFi-only. A future cleanup is a CLAWDMETER_TRANSPORT={USB,WIFI,BOTH} env-var split; for now both modes coexist.

How to use

# Host side
export CLAWDMETER_HTTP=1               # (or set in run-daemon.ps1 on Windows)
./install-mac.sh                       # or your platform's installer
# Daemon prints bearer token at startup:
#   "WiFi-bridge token: <copy this into wifi_config.h>"

# Firmware side
cp firmware/src/wifi_config.h.example firmware/src/wifi_config.h
# Edit wifi_config.h with SSID / password / host IP / token
pio run -e waveshare_amoled_216_wifi -t upload

Happy to split this into a smaller cleaner PR off main (or a stacked PR against #26) if that's easier to review — let me know what works.

Drew and others added 5 commits May 20, 2026 18:54
Replaces the BLE GATT data link with a wired USB serial channel.
Designed for desk-side use where the device is already cabled for power
and BLE pairing/bonding flakiness is unwelcome.

Firmware:
- New serial_link.{h,cpp} owns all Serial I/O including the legacy
  `screenshot` debug command. Line-delimited protocol:
    host -> dev : {"s":..,...} usage payload | "screenshot"
    dev -> host : ACK | NACK | REQ | READY
- main.cpp: ble_* -> serial_link_*; BLE HID button bindings stubbed
  with TODO for future USB HID composite.
- ui.{h,cpp}: SCREEN_BLUETOOTH -> SCREEN_LINK, "USB Link" status screen.
- platformio.ini: NimBLE dependency and build flags removed.
- ble.{cpp,h} deleted (-310 lines).

Daemon:
- claude-usage-daemon.sh full rewrite: stty + printf over /dev/ttyACM0,
  -hupcl to suppress the DTR-toggle reset that fires on port open with
  default Arduino USB CDC config.
- Background reader tails device output and flags REQ refresh requests.
- Two consecutive write failures recycle the port to recover from USB
  re-enumeration.
- Service unit no longer depends on bluetooth.target.

Docs:
- CLAUDE.md describes the wire protocol, -hupcl gotcha, and port
  resilience. install.sh checks for stty instead of bluetoothctl.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reflect the wire change made in the parent commit: USB-only Linux install
flow, /dev/ttyACM0 setup, dialout group, wire-protocol table, and the
-hupcl gotcha. macOS install section dropped since this branch hasn't
ported the macOS Python daemon yet (see main for that).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors the bash daemon's USB rewrite on the macOS side:
- daemon/claude_usage_daemon.py: drop bleak, use pyserial. Port discovery
  globs /dev/cu.usbmodem* (the callout node — does NOT assert DTR on open,
  unlike /dev/tty.usbmodem*). HUPCL cleared explicitly via termios as
  belt-and-braces. Background reader_task tails the port and sets a
  refresh asyncio.Event when the firmware emits REQ.
- install-mac.sh: swap bleak/httpx for pyserial/httpx in the venv setup,
  drop the Bluetooth permission-priming step (no longer needed).
- README: restore macOS install section, document the cu.* vs tty.*
  gotcha alongside the existing Linux -hupcl note.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Branches three open PRs into one working stack on a Windows host with the
480x480 Waveshare AMOLED-2.16:

 - Base:           PR HermannBjorgvin#26 (dilbery's USB CDC transport)
 - Plus daemon:    cross-platform port + Windows hook-compat
 - Plus feature:   PR HermannBjorgvin#22 (tobby168's Activity screen / Claude Code hooks)
 - Plus 3 fixes:   firmware buffers + parse for multi-session payloads

Windows daemon port (vs PR HermannBjorgvin#26)
-------------------------------
daemon/claude_usage_daemon.py:
- find_port() on win32 uses pyserial list_ports filtered by USB VID
  (303A Espressif, 10C4 SiLabs, 1A86 WCH, 0403 FTDI). Locale-independent —
  works on non-English Windows where caption strings are translated.
- termios import + HUPCL clear gated to sys.platform != "win32".
- write_payload uses ensure_ascii=False so UTF-8 stays raw (em-dashes etc
  are 3 bytes instead of 6-byte \uXXXX escapes; smaller payloads, less
  pressure on ArduinoJson on the firmware side).

daemon/clawdmeter_hook.py (from PR HermannBjorgvin#22):
- fcntl is Unix-only; replace with msvcrt.locking on win32.

daemon/install_hooks_win.py (new):
- Windows analogue of install-mac.sh's inline-Python hook registration.
  Backs up settings.json before writing, idempotent.

Windows host scripts (from PR HermannBjorgvin#23): flash-win.ps1, install-win.ps1,
setup-win.ps1, uninstall-win.ps1, daemon/run-daemon.ps1. Brought along so
this branch is buildable + installable on Windows without cherry-picking.

Activity backport on USB CDC (vs PR HermannBjorgvin#22)
----------------------------------------
firmware/src/data.h:  PR HermannBjorgvin#22 ActivityData / SessionData / TodoItem structs.
firmware/src/ui.h:    +SCREEN_ACTIVITY enum + ui_update_activity() API.
firmware/src/ui.cpp:  PR HermannBjorgvin#22 init_activity_screen / render_activity /
                      activity_gesture_cb / ui_update_activity grafted onto
                      PR HermannBjorgvin#26's ui.cpp; cycle order Splash > Usage > Link >
                      Activity > Usage.
firmware/src/main.cpp: parse_json gets optional ActivityData* out-param;
                       sessions / todos extracted into the struct;
                       ui_update_activity() called after each successful parse.

daemon/claude_usage_daemon.py:
- load_activity_sessions() + state_file_mtime() from PR HermannBjorgvin#22.
- Main loop tracks last_state_mtime; payload includes "sessions" whenever
  the file changes OR a fresh API poll happens.

Three firmware fixes needed for multi-session payloads
------------------------------------------------------
Real-world payloads with 2 sessions x 5-10 todos each routinely hit
700-1000 bytes — well past anything PR HermannBjorgvin#26's defaults handled.

1. firmware/src/serial_link.cpp: LINE_BUF_SIZE / DATA_BUF_SIZE 192 -> 4096.
   192 was tuned for the ~120-byte Usage-only payload; longer lines were
   silently dropped at the line-accumulator level.

2. firmware/src/main.cpp: Serial.setRxBufferSize(4096) before Serial.begin.
   USB-CDC RX defaults to ~256 bytes; a 798-byte payload arrived as
   strlen=317 on the firmware side (truncated at receive). Bumping the
   buffer is the actual fix (CFG_TUD_CDC_RX_BUFSIZE redefine is ignored
   because Arduino-ESP32 overrides it via CONFIG_TINYUSB_CDC_RX_BUFSIZE).

3. firmware/src/main.cpp: parse_json is two-pass.
   - Pass 1 is a Filter()'d deserialization that only extracts the Usage
     scalars (s/sr/w/wr/st/ok). A tiny doc, always succeeds.
   - Pass 2 is a best-effort full parse for the "sessions" array. If it
     fails (oversized, mid-line truncation, unicode oddity) we leave
     ActivityData empty but still ACK the payload upstream — Usage stays
     intact, Activity falls back to its empty render state.

Tested
------
Waveshare ESP32-S3-Touch-AMOLED-2.16 on Windows 10 (Dutch locale, Realtek
BT 5.1 adapter unused, USB-C only). Two parallel Claude Code sessions on
the same host:
  - 11+ todos across two sessions, ~1KB payload, ACK'd reliably.
  - Activity screen renders project name, in-progress todo activeForm,
    counter coloured by phase, todo list windowed around the in-progress
    item. PWR cycles Usage <-> Link <-> Activity.
  - state.json updates on every PreToolUse/PostToolUse/UserPromptSubmit/
    Stop/SessionStart; daemon picks it up on its 5s tick.
Third transport option for the daemon-to-firmware link, complementing
PR HermannBjorgvin#22 (BLE GATT, main) and PR HermannBjorgvin#26 (USB CDC, dilbery's branch). Built on
top of usb-transport-win so both transports stay available; pick at build
time via the new platformio.ini env.

Why
---
USB CDC works great when the board sits on the desk next to the PC. WiFi
makes the board fully portable — wall-charger or Li-Po battery, anywhere
in range of the home network. The PC's daemon publishes the same JSON
payload over both transports concurrently, so a user can hot-swap
between USB and WiFi without restarting either side.

Followed the architecture of github.com/rootedlab-code/claude-code-usage-monitor
which has the exact same pattern (ESP32 over WiFi -> local HTTP bridge
on the host LAN). Their precedent showed the captive-portal + NVS-storage
+ bearer-token-auth path is solid; we use the same shape.

Daemon
------
daemon/claude_usage_daemon.py:
- threading.Lock-guarded `_http_payload` cache holds the latest assembled
  wire payload. Updated in lockstep with the existing USB CDC writer, so
  both transports serve the same data without duplicate API calls.
- stdlib `http.server.ThreadingHTTPServer` on port 8787 with bearer-token
  auth. /payload (auth required), /health (open, for diagnostics).
- Token persists in ~/.clawdmeter/wifi_token (mode 0600 on POSIX).
- New helper `run_session_no_port()` keeps the API poll + state.json
  watcher running when there's no USB port — the board may be on WiFi
  power-only at that point.
- Toggle via CLAWDMETER_HTTP=1 environment variable, off by default.

daemon/run-daemon.ps1:
- Sets CLAWDMETER_HTTP=1 so the scheduled-task daemon starts the HTTP
  bridge alongside USB CDC. Harmless on USB-only setups.

Firmware
--------
firmware/src/wifi_link.{h,cpp} (new):
- Public API mirrors serial_link.h so main.cpp swaps transports with
  only a #define choice — link_init / link_tick / link_get_state /
  link_get_port_name / link_has_data / link_get_data / link_send_ack.
- Arduino WiFi (STA mode) + HTTPClient, 5s poll cadence, 4s timeout.
- Logs WiFi connect/reconnect, HTTP error codes, RX byte counts. Bearer
  token + bridge URL come from compile-time wifi_config.h.

firmware/src/wifi_config.h.example:
- Template with WIFI_SSID / WIFI_PASSWORD / BRIDGE_HOST / BRIDGE_PORT /
  BRIDGE_TOKEN placeholders. wifi_config.h is in .gitignore.

firmware/src/main.cpp:
- Macro layer maps link_* calls to either serial_link_* or wifi_link_*
  based on CLAWDMETER_TRANSPORT_WIFI. Zero changes elsewhere in the file
  — parse_json, ui_update, ACK send all go through the same call sites.

firmware/src/ui.h:
- Conditional include of either transport's header so link_state_t
  resolves to the one in scope.

firmware/platformio.ini:
- New env `waveshare_amoled_216_wifi` extends the USB env and adds
  -DCLAWDMETER_TRANSPORT_WIFI. Pick via `pio run -e ... -t upload`.

Wrapping
--------
firmware/src/wifi_link.cpp body is itself wrapped in
`#ifdef CLAWDMETER_TRANSPORT_WIFI` so the USB CDC env doesn't need to
have wifi_config.h or pull in the HTTP/TLS libraries. Both envs build
clean from a fresh checkout.

Tested on real hardware
-----------------------
- Waveshare ESP32-S3-Touch-AMOLED-2.16, Windows 10 (Dutch locale)
- Bord polled the daemon at 5s cadence over LAN (Realtek BT not involved)
- Verified via netstat: 24 TCP connections from board-IP visible at any
  time, ~5 unique source ports appearing every 25s (matches WIFI_POLL_MS).
- USB-C cable physically removed, board on a USB wall-charger, daemon's
  USB CDC writer logged "Write failed; recycling port" then fell through
  to HTTP-only mode via run_session_no_port(). Board kept receiving live
  Usage + Activity data over WiFi the entire time.
@alasdaircs

alasdaircs commented Jun 3, 2026

Copy link
Copy Markdown

Related: I built a variant that takes a different architectural position — the ESP32 calls the Anthropic API directly over Wi-Fi using a long-lived token, so there's no host daemon at all (not even an HTTP bridge). The BLE HID keyboard is kept for the physical buttons; the GATT data service is removed.

It also implements the captive-portal provisioning you flagged as a known follow-up: on first boot the device starts a softAP named "ClawdMeter", serves a setup form at 192.168.4.1, and stores credentials to NVS. Re-provisioning is triggered from a Wi-Fi screen on the device.

Fork is here if it's useful context: https://github.com/alasdaircs/clawdmeter-wifi

@HermannBjorgvin I'm happy to leave it as a standalone fork — just wanted to surface it given the overlap with this issue. If a PR that adds this as an opt-in build env would be welcome, say the word and I'll write one up properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants