Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

----> UDP Supported - ESP8266 :)

image image

web_serial

ci

v1.0.0 -- field-hardened. Both platforms bench-tested at length: ESP8266 (80 and 160 MHz) through a six-crash forensic campaign that ended with the umm/lwip SYS-context allocator collision identified from decoded HWDT stacks and closed (ALLOCQUIET + a cohabitation valve); ESP32 (Arduino and IDF, 16 ports, 4096 B buffers, M=15 loopback storms at 90+ %% engine). Killer-config uptimes went from 27-65 s to storms outliving the bench session. Full war journal: docs/BUILD_STATUS.md; every trap harvested: docs/PIEGES_MATERIEL_ESP8266_ESP32.md. It remains what it is: a LAN bench tool, no auth, one operator.

An in-browser managed SERIAL SWITCH for ESP8266 and ESP32. Like an Ethernet switch, but for serial: ports (physical UART, web console, TCP, UDP, PC-serial bridges) are grouped into VLANs -- a byte in on one port reaches every UP port in the same VLAN, each at its own speed. Ports are created at runtime from the web UI, with a memory guard that refuses any allocation that would drop the largest free block below the platform safety floor. Fourth of the family (web_i2c, web_spi, web_onewire) -- same transport, same log/decoder discipline -- and the FIRST with no build-time graft: the uart debug callback is a public runtime seam.

The hub's ports

port nature class
physical UART the ESP's real serial pins QoS
firmware (host) other components on the same uart: bus native QoS bypass (zero code, zero loss by construction)
web frontend text+hex console, tap log, decoders best-effort, lag-tolerant
TCP232 raw network serial -- PuTTY, telnet, Eltima Splitter direct write=QoS path, read=best-effort
PC serial bridge the browser opens a LOCAL COM port (Web Serial API) and joins the hub best-effort

QoS doctrine (the sketch's yellow marker): the physical<->firmware path loses NOTHING -- it is ESPHome's own native path, merely observed. Web/TCP/PC are observers-and-talkers that may drop under load (flooded badge, honest banners), never at the main path's expense.

Protocols and physical layers

  • Modbus RTU -- silence framing (Modbus preset: 4 ms), decoder with slave address, function names, exception frames, and the CRC16 verified in-browser (poly A001, wire order LSB first).
  • DMX512 -- 250000 8N2 preset; RX frames decoded (start code, channel count, first 32 channels as live bars). TX is experimental: the BREAK is generated by the baud trick (one 0x00 at 90000 baud = ~100 us low), then the frame at 250000. Good for bench fixtures, not for show-critical timing.
  • NMEA 0183 -- sentences named, XOR checksum verified. AT dialogs spotted. Everything else: honest hex + ascii dual view.
  • RS485 -- half-duplex: give de_pin:; the hub asserts it around every write and releases after flush() (MAX485-style DE+/RE- tied together).
  • RS422 / plain TTL -- electrically transparent, nothing to configure.

Install

external_components:
  - source: github://kaboom748/web_serial
    components: [web_serial]
    refresh: 1d

uart:
  id: ser_bus
  tx_pin: GPIO17
  rx_pin: GPIO16
  baud_rate: 115200

web_serial:
  port: 8891
  uart_id: ser_bus
  tcp_port: 2323     # optional raw TCP232
  # de_pin: GPIO4    # RS485 driver-enable
  owner: true        # false if ANOTHER component reads this uart (tap mode)
  gap_ms: 10         # silence closing a frame
  # heap_floor: 6144 # override the safety floor (bytes); lower it, at your own
                     # risk, on a RAM-starved ESP8266 sharing many components

Several hubs? Just instantiate several web_serial: blocks (MULTI_CONF), one per uart. Open http://<ip>:8891/; point PuTTY at <ip>:2323.

The managed switch

  • VLANs per port -- assign each port a VLAN (1-8); ports only forward within their VLAN. Isolate two VCOM sessions, or wire a monitor-only group.
  • Dynamic ports -- + TCP / + UDP / + Bridge in the UI (or PORT ADD TCP 2324 512) create ports live; PORT DEL frees them. Up to 8 ports on ESP8266, 16 on ESP32.
  • Port up/down -- admin shutdown per port (blocks both directions).
  • Storm control -- PORT RATE <id> <bytes/s> caps a port's ingress; the excess is dropped and counted (a red dot flags a tripping port). Blind to slow gain-1 loops by design: that is loop detect's job (below).
  • Loop detect -- LOOPDETECT ON|OFF|KILL (default ON), fully PASSIVE: zero bytes are ever injected in the stream, on any port. The hub fingerprints its own egress per network port (FNV-1a + length, 4-slot ring, ~40 B/port); an ingress chunk matching a <=3s-old fingerprint from a port of the SAME vlan, three in a row, means the frame came back where it started: recirculation, a loop. Cross-port loops (tcp<->bridge piping) included; badge and KILL land on the ingress port. The UART is exempt (the TX->RX jumper is a deliberate echo). PORT UP / PORT RESET / a vlan change clears the badge.
  • Inter-VLAN bridge detect -- XVLANDETECT ON|OFF|KILL (default ON), same passive engine, other verdict: an ingress matching recent hub egress from ANOTHER vlan crossed once and died there -- not a loop, a bridge between vlans, i.e. broken isolation. OFF tolerates deliberate bridging in total silence (the disabled pass neither counts nor consumes fingerprints, so a live loop elsewhere still gets caught); ON badges XVLAN + logs; KILL downs the bridged-into port. Both detectors are independent -- all four combinations work. A true circular loop through two vlans is attributed to LOOP on the re-entry ports, not XVLAN.
  • Detection limit, accepted and field-verified -- both detectors match EXACT chunks (hash + length). Large blocks that the transport re-chunks in flight -- the bridge WS framing (64 B), the hub's own TCP reads (64 B), OS/driver COM coalescing, or egress-buffer eviction under pressure -- come home in different pieces and are deliberately NOT flagged: a miss, never a false alarm. Short chunks (a lone CR+LF is enough) traverse atomically and are caught within three passes, and a re-fragmenting loop still screams in the counters (symmetric tx/rx across ports, climbing drops). A sliding-window (Rabin-Karp) mode was considered and rejected to keep the zero-false-positive contract.
  • Hot vlan changes -- PORT VLAN <id> <v> (or the VLAN pill) applies live; it also purges that port's detection state (ring, streaks, badges): a vlan change is a topology change, detection restarts from zero and the 3 s stale-fingerprint window is closed.
  • Multi-port PC bridge -- up to 4 local COM ports via the Web Serial API, each bound to its own hub bridge port. Both WS directions carry the bridge slot id (0x01 slot data up, 0x02 slot data down); the page demuxes per slot, and disconnecting (or unplugging) a port sends PORT DEL -- no orphaned bridge ports, no duplicated egress. The + Bridge button now opens the same connect flow (a bridge without a COM port behind it is a dead end).
  • Per-port counters -- TX / RX / drop, live; the Reset counters button (or PORT RESET) zeroes them and clears LOOP/XVLAN badges. Each row also shows a live rate pill (storm-control cap, click to edit) and the egress buffer size on hover; Line settings, owner/tap mode and the wire-tap visibility state all round-trip from the hub, so a page reload always shows the LIVE configuration.
  • Console VLAN -- the web console talks on the VLAN you pick (CVLAN).
  • UDP for VCOM -- a UDP port learns its peer from the first datagram (matches VCOM's UDP mode). Works on BOTH platforms: ESP32 through the BSD socket layer, ESP8266 through the bundled wser module (raw udp_pcb, lwip-only dependency, gauntlet-tested standalone: 8 suites, byte-exact 100k-op fuzz, ASan/UBSan, 6/6 mutations caught -- see wser_*.{h,cpp} and the PIEGES-compliance notes inside). Same behavior on both: peer learned from the first datagram, txerr counter, self-recreating socket after 5 consecutive send failures, 5s/3-strikes bind backoff. DESIGN DECISION (learning-only, no configured target): the hub never initiates -- the UDP client MUST speak first, the hub learns the peer from that first datagram and replies there. Consequences, accepted: a fresh UDP port in a busy VLAN drops egress until the first datagram arrives; a hub-to-hub UDP trunk is impossible (two pure learners never start -- ESP-NOW is the answer for that); and last-talker-wins -- any LAN sender can steal the peer, fine for a bench tool. Field bug fixed on the way: peer storage is now a real (aligned) sockaddr_storage -- lwip's sendto rejects misaligned sockaddr pointers with EINVAL, which made every reply fail forever while ingress worked (the climbing-txerr signature).
  • Persistence -- dynamic ports, VLANs, the console VLAN, per-port shapers (rate/out), buffer sizes, loop/xvlan detect modes AND trunk/hairpin wiring (wire) are saved to ESPHome preferences and restored at boot (your startup-config). NOT persisted: the live LINE settings -- the YAML uart config wins at every boot (runtime baud changes are session-only; making them sticky is an open design decision).

Memory guard (heap / largest block / fragmentation)

The defense onion (who fires first, who sleeps -- health = outer layers work, deep layers sleep): largest < 7 680 -> governor blue tick (quota /2); < 6 144 -> cohabitation valve (one lwip submission per pass, info at 3.5 s); < 5 120 -> radio brake (hard); < 1 536 -> tap guard (frames dropped, cause heap). Field reference: blue ticks with drop causes heap = 0 = the outermost layer absorbing everything. YAML-side constraints and their observable signatures: see CONFIG_ESP8266_TERRAIN.md.

The side panel graphs free heap, largest contiguous block, and fragmentation %% live. The largest block is what actually matters on ESP8266: the heap can read 30 KB while the biggest allocatable chunk is 4 KB. Before allocating a port buffer, the guard checks that the largest block would stay above the floor (8 KB on ESP8266, 20 KB on ESP32) -- if not, the allocation is refused with a message, never a reboot. You choose the buffer size per port (256 B default / 1 KB max on 8266; 512 B / 4 KB on ESP32); an inline preview shows whether it fits before you commit. The floor is adjustable live (the Floor field, or FLOOR <bytes>) and can be set in YAML (heap_floor:) -- on a heavily loaded ESP8266 where free heap is already near 8 KB, lower it deliberately or move web_serial to its own chip.

owner vs tap (the destructive-read rule)

Reading a UART CONSUMES bytes. In owner mode the hub reads RX itself and is fully interactive. If another component (modbus controller, a PZEM...) owns the reads, set owner: false: the hub then only OBSERVES through the debug callback (their reads and writes appear in the log), and hub writes are still possible but may interleave with the owner's protocol -- your call.

Live line settings

Baud / data bits / parity / stop bits change from the page through ESPHome's official load_settings() -- no reflash, with DMX (250000 8N2) and Modbus (4 ms gap) one-click presets. Framing gap and optional delimiter (0A for line protocols) too.

The PC serial bridge (Web Serial API)

Chrome's navigator.serial lets the page open a COM port ON YOUR PC and bridge it into the hub: PC-port RX goes onto the ESP's wire; the wire's RX is written to the PC port. Binary WS frames, 0x01/0x02 tags. Secure-context caveat: on plain http://<ip> Chrome hides the API -- enable chrome://flags/#unsafely-treat-insecure-origin-as-secure for the ESP's origin (documented, honest limit).

The web UI -- three tabs

General: the switch table (VLAN pills, up/down, rate/out shapers, wire pills, LOOP/XVLAN badges, live per-port meters), send-on-the-wire, the log, SERIAL DECODE, and the vlan terminal (below). UART: line settings, presets, the ESP8266 pin-constraints map. Views: eight live cards -- TOPOLOGY ring (click a node for its blast radius), the GOVERNOR AIMD chronogram with cause-colored collapse ticks (red backlog / amber duty / blue heap, all three field-captured), LIVE MODEL (a continuous regression test: measured vs bottleneck-law prediction, the exact multi-vlan Sigma-I law, a cycles/byte cost line and the service ratio that names duty-broken regimes), FLOW & CONSERVATION (byte-exact at every regime the bench produced), INTER-ARRIVALS, MEMORY MAP with the three floors, TAP FUNNEL, and ENDURANCE (uptime, valve firings PER HOUR, the retained black-box line, largest/frag vigil).

The LOGGING knobs (Detail Full/Summary, Delivery Live/Batch/Off, Filter) govern the LOG stream only -- never the wire, never the counters, never the terminal. Full+Live feeds the decoder and the histogram; Summary/Batch are the operator's own valve under storms. Details: docs/GUIDE_COMMANDES_web_serial.md.

The vlan terminal -- the console that cannot lie

The RAW panel is a byte-true mirror of everything the vlan delivers to the console member, fed by its own ring + a dedicated binary stream under the same egress valves. Its contract: loss is possible, lying is not. Any ring overflow (counted on the console port's own drop) travels IN the stream as a confession record and prints inline as a red [lost N B] marker you cannot copy over; the header banner states vlan N raw -- in-sync or LOST N B at all times; every send and file/clipboard transfer closes with a delivery receipt from the members' real counters (delivered -> u0 +12512 (drop +32) | t2 +12512). Party-line doctrine: one neutral tint for ALL data (a byte is a byte -- com, bridge, uart, tcp); color is reserved for instruments (lost=red, receipt=blue, local echo=gray). Scope = the console's vlan, exactly like the wire tap: move the console to see another vlan.

Documentation

  • docs/GUIDE_COMMANDES_web_serial.md -- every command, every instrument, reading the colored ticks, the terminal contract, the LOGGING knobs.
  • docs/CONFIG_ESP8266_TERRAIN.md -- each YAML constraint with its OBSERVABLE signature (power_save blue ticks, f_cpu, safe-mode economics, the defense onion table).
  • docs/PIEGES_MATERIEL_ESP8266_ESP32.md -- 56 field traps, each paid for.
  • docs/LESSONS.md -- the distilled engineering lessons.
  • docs/BUILD_STATUS.md -- the verbatim campaign journal (the war log).

Known limits (assumed, not bugs)

  • LAN tool, no auth, one web client + one TCP232 client (a new one replaces).
  • Simultaneous talkers interleave on the wire (it is a hub, not a switch).
  • DMX TX timing is approximate (baud-trick break); DMX RX depends on the UART swallowing the break as a framing artifact -- gap framing still splits frames correctly at 4 ms.
  • ESP8266: UART0 belongs to the logger by default (set baud_rate: 0 on the logger or use swapped pins); UART1 is TX-only.
  • Web Serial API needs Chrome + the insecure-origin flag on plain http.

Host tests (8 suites, one command)

bash tests/host/run_all.sh

Six C mirrors (crc16, framing, switch incl. the cons-ring truth contracts, guard incl. the valve arithmetic, loopdet, shaper), the decoder JS mirrors, and a 177-assertion UI suite that extracts the real page out of PROGMEM and drives it in jsdom -- badges, pagination merge, the Views model, the terminal's loss markers. CI runs the same script plus a three-target compile matrix (esp8266 / esp32-arduino / esp32-idf).

Hardware sign-off and the full campaign journal: docs/BUILD_STATUS.md.

UART on ESP8266 -- the constraints map (mirrored in the in-app guide)

Three hardware combos ONLY -- UART0 (GPIO1/3, USB-shared), UART0-swap (GPIO15/13, the full-duplex choice; move the logger to UART1), UART1 (GPIO2, TX-only by silicon). Anything else = silent SOFTWARE bit-bang: ~57600 practical, sustained TX can wedge the chip under WiFi load (this hub shields its egress to ~2 ms/pass there). Strapping rules: GPIO15 low at reset, GPIO0 low = flash mode, GPIO2 high; never GPIO6-11/16 for serial. DMX (250000 8N2, exact divider on the 8266): frames are 513 B -- rx_buffer_size: 1024. A build-time audit prints the verdict in the config dump and the UI (UART row: green hardware / amber software), the egress quota law follows it (wire-saturating on hardware -- measured 1918/1920 B/s at 19200, 14.7 kB/s at 250000 -- shielded on software), and the info panel shows rx_buffer + TX FIFO. ESP32: pin matrix, any pins, 5 Mbaud; GPIO34-39 are input-only.

Traffic management (per-port shaping + live instruments)

Every port has TWO independent rate pills: rate (ingress storm control, unchanged) and out (NEW egress shaper -- a lazy 1 s token bucket at the drain; excess overflows the port's buffer, evicts oldest, and counts in drop, exactly a real switch on a speed mismatch: cap the uart at out 9600 under a 19200 flow and watch half the bytes die AT that port). PORT ORATE <id> <bps> (0 = unlimited), persisted (config magic v4 -- saved ports reset ONCE on first flash of this build). Live bars per port (4 Hz bs telemetry, snprintf-on-stack, gated by client/backlog/heap): OUT fill for every buffered port, IN fill where a real buffer exists (uart ring, wser udp ring), an honest flow-ctl label for TCP (its backpressure lives in the TCP window, not in a local buffer). SYSTEM gains three instruments: Peak (session high-water of tx+rx), Engine load (duty-pool utilisation %), and Headroom est. (throughput/load -- labeled as the estimate it is). The uart bar row also states the wire's exact ceiling (baud/10).

TRUNK -- the virtual wire (declared inter-VLAN bridging)

Two slot-less bridge ports can be WIRED together into a RAM null-modem inside the hub: PORT WIRE <a> <b> (or the wire pill), PORT WIRE <a> -1 to unplug. Egress into a wired bridge ENQUEUES; a governed drain (same fairness cursor, same duty budget, <=256 B/pass) injects into the peer's ingress. The out pill IS the wire's baud -- shape the trunk like any port. Put the two ends in DIFFERENT vlans and you have a legitimate, counted, shaped inter-VLAN trunk: the crossing is DECLARED, so XVLAN detection exempts it (green XVLAN ALLOWED badge on both rows) while still hunting clandestine paths; LOOP detection stays fully armed around it. The origin rule traverses the wire (no ping-pong inside the pair). Persisted (config magic v5 -- saved ports reset once); load-time validation unplugs any asymmetric/invalid pair. Related instruments: hbrake per-port counter (bytes dropped by the radio heap brake -- every extinguisher signs its catches) and RADIOFLOOR <n> (tune the brake's floor; 0 disables it with a red warning -- documented FIQ-wedge risk; not persisted, re-arms at reboot).

Session additions: hairpin, meters, operator handles

HAIRPIN: PORT WIRE n n (or the wire pill with the port's own id) turns a slot-less bridge into a declared LOOPBACK -- its egress re-enters its own ingress through the governed drain: the integrated loopback plug, a one-port storm generator (the out pill is the oscillator's period), an echo responder, and the pure-hub benchmark target. Cyan \u21a9 LOOPBACK badge; the LOOP detector will light on it and that is the witness telling the truth about a mirror. WIRE now also RETROFITS a 256 B buffer onto legacy buffer-less bridges (refused under the heap floor) -- the silent black-hole class is gone. METERS: every port row shows its own \u2195 B/s (1 s window, internal traffic included) and THROUGHPUT gains Switched -- everything the fanout carries, boundary AND internal; TX/RX only ever saw the boundary. OPERATOR: a Reboot hub button (App.safe_reboot, deferred ack, confirm guard), a raw cmd > mini-console in SYSTEM (every command now has a handle; replies land in the log), a t:"pdel" notice on port deletion (the browser closes the matching COM link -- zombie links are extinct), and PORT ADOPT <id> + a Re-link button per link (re-attach an open COM to a slot-less bridge after a hub reboot instead of delete-and-recreate).

ESP8266: wifi: ssid: !secret wifi_ssid password: !secret wifi_password min_auth_mode: WPA2 power_save_mode: none -----------------------------------> More reliable (no packet drop)

esphome: name: wemo-test friendly_name: WEMO-TEST platformio_options: board_build.f_cpu: 160000000L -----------------------> More Speed

The de_pin is required when using an RS-485 transceiver for DMX output. It controls the transceiver's Driver Enable (DE) pin, allowing it to drive the DMX bus only during transmission. Without configuring de_pin, the transceiver may remain disabled, preventing any DMX data from being transmitted. [BUG]

Standalone use: if the device isn't paired with Home Assistant, set api: reboot_timeout: 0s. ESPHome's default 15-minute API watchdog reboots the node when no native-API client connects — web_serial traffic on the HTTP/TCP ports does not count — which silently kills your serial session every 900 s.

About

In-browser managed SERIAL SWITCH for ESP8266/ESP32 -- VLANs, TCP/UDP/Web-Serial bridge ports, trunks and hairpins, passive loop detect, Modbus/DMX/NMEA decoders, and a truth-contract vlan terminal

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages