v5.3.0 transport detection has no fallback when the port cannot be enumerated (virtualised serial devices) — stub upload then fails (ESPTOOL-1370)

[keithshawlab]

Operating System

Container: Ubuntu 24.04 (Docker, espressif/idf:v6.0.2). Host: macOS 15 (Apple Silicon), OrbStack 2.2.3.

Esptool Version

v5.3.0 and v5.3.1 fail. v5.2.0 and v4.8.1 succeed.

Python Version

3.12 (IDF's bundled venv, pyserial from the same env)

Chip Description

ESP32-S3 (QFN56, revision v0.2), 16MB flash, 8MB embedded PSRAM (AP_3v3), native USB (USB-OTG, 303a:0009)

Device Description

LilyGo T-Display S3. No USB-serial bridge chip — USB-C goes directly to the ESP32-S3's USB peripheral.

Hardware Configuration

Nothing attached that touches the USB or strapping pins.

How is Esptool Run

From inside a Docker container. The host's serial device is exposed to the container by OrbStack, which forwards it under its macOS name (/dev/cu.usbmodem2101) as a character device with no sysfs USB backing and no ttyACM* equivalent.

Full Esptool Command Line that Was Run

esptool --chip esp32s3 --port /dev/cu.usbmodem2101 flash-id

Esptool Output

esptool v5.3.0
Serial port /dev/cu.usbmodem2101:
Connecting...
Failed to get VID/PID of a device on /dev/cu.usbmodem2101, using standard reset sequence.
Connected to ESP32-S3 on /dev/cu.usbmodem2101:
Chip type:          ESP32-S3 (QFN56) (revision v0.2)
Features:           Wi-Fi, BT 5 (LE), Dual Core + LP Core, 240MHz, Embedded PSRAM 8MB (AP_3v3)
Crystal frequency:  40MHz
MAC:                e0:72:a1:a2:d1:e4
Uploading stub flasher...

A fatal error occurred: Serial data stream stopped: Possible serial noise or corruption.

More Information

Summary: v5.3.0's "Add transport mode detection based on USB VID:PID instead of hard-coded ROM symbols" has no fallback when the port cannot be enumerated. _get_vid_pid() matches serial.tools.list_ports.comports() entries by device path; where the OS exposes a serial device without USB descriptors, that list is empty, so the lookup returns (None, None) and esptool silently treats a native-USB part as a UART bridge. The stub upload is the first operation that fails.

This is not specific to OrbStack — it should reproduce anywhere a serial port is presented without USB descriptors (VM passthrough, socat/RFC2217-style bridges, some WSL setups, udev symlink arrangements).

Enumeration, same board, same moment:

container:  comports found: 0
macOS host: comports found: 3
            /dev/cu.usbmodem2101  vid=0x303a pid=0x0009  ESP32-S3

Controlled matrix — same board, same silicon revision, same cable:

esptool	Transport detection	Where	Stub upload
4.8.1	none (ROM symbols)	container	works
5.2.0	none (ROM symbols)	container	works
5.3.0	succeeds → USB mode: USB-OTG	host, real USB	works
5.3.0	fails → standard reset sequence	container	fails
5.3.1	fails → standard reset sequence	container	fails

The third row is the positive control: the same 5.3.0 with the same esp-flasher-stub succeeds on this exact board as soon as VID/PID resolves. So the stub binary is not incompatible with this chip — the detection is what breaks it. 5.2.0 has no transport concept to get wrong, which is why it is unaffected despite also having no VID/PID.

Workarounds: --no-stub works (≈23 s vs ≈13 s for a 1.2 MB image, since the ROM loader cannot do compressed writes). Pinning esptool==5.2.0 restores full behaviour including idf.py flash.

--before usb-reset does not help — possibly a second bug. It is OR'd into the reset choice (loader.py:815), but the device-type predicates at loader.py:1242 / 1248 call get_usb_vid_pid() directly, so an explicit flag cannot override a failed lookup:

$ esptool --chip esp32s3 --port /dev/cu.usbmodem2101 --before usb-reset flash-id
Failed to get VID/PID of a device on /dev/cu.usbmodem2101, using standard reset sequence.
Uploading stub flasher...
A fatal error occurred: Serial data stream stopped: Possible serial noise or corruption.

Suggested fix: an explicit override for when enumeration is impossible — e.g. --usb-mode {auto,usb-otg,usb-jtag-serial,uart}, or honouring --before usb-reset in the device-type predicates. Descriptors the hypervisor does not pass through cannot be recovered by probing, so detection alone can never cover this case. Falling back to 5.2.0's ROM-symbol detection when the lookup fails would also work.

Not a duplicate of #1178 / esp-flasher-stub#94. Those share the symptom (ESP32-S3 rev v0.2, failure at Uploading stub flasher..., 5.3.0 vs 4.8.1) but were intermittent — a second identical board never failed, the original recovered days later unchanged, and the report was closed as not reproducible. This failure is 100% deterministic and tracks the environment exactly: it fails whenever comports() is empty and succeeds whenever it is not.

I Have Read the Troubleshooting Guide

Yes. The applicable remedies were tested and none change the outcome:

Remedy	Result
Lower baud rate (--baud 9600)	No change — identical failure
timeout = 60, mem_end_rom_timeout = 20 via esptool.cfg	No change (Loaded custom configuration confirmed)
--chip esp32s3 to skip autodetection	Already always specified
Cable / hub / shorter path	Same cable succeeds with 5.2.0, and with 5.3.0 on the host
Power, brownout	Same board and supply; outcome depends only on esptool version
SPI pins disconnected	Devkit, nothing attached to SPI or strapping pins
USB-to-Serial drivers	N/A — no bridge chip, native USB-OTG
Port permissions / in use	Port opens; chip ID, MAC and flash-id reads all succeed

The version bisect on identical hardware is what rules out the serial-quality class of causes:
the failure tracks the esptool version and the enumeration environment, nothing physical.

Other Steps to Reproduce

Reproduces on any serial device presented without USB descriptors, not just OrbStack — VM
passthrough, socat/RFC2217 bridges, some WSL setups. The precondition is testable without
hardware:

python -c "import serial.tools.list_ports as l; print(len(list(l.comports())))"   # 0 => affected

I Have Read the Troubleshooting Guide

• I confirm I have read the troubleshooting guide.

[github-actions]

Hi @keithshawlab! Please be aware that (1) the following suggestions are generated by a bot and haven't been fact-checked by Espressif Systems, (2) burning eFuses and enabling security features are irreversible operations and can damage your ESP32 device. We hope that this message will help you until an Espressif Engineer looks at your issue.

---

I'm sorry, but the knowledge sources provided do not contain enough information to fully address this specific bug report. The knowledge sources cover general information about the esptool flasher stub and the v5.3.0/v5.3.1 release notes, but do not include details about the internal transport detection logic, the _get_vid_pid() fallback behavior, or any planned fixes for this issue.

That said, based on what is available in the knowledge sources, I can confirm a few relevant facts:

• v5.3.0 introduced "Add transport mode detection based on USB VID:PID instead of hard-coded ROM symbols", which is precisely the change you've identified as the root cause. [v5.3.0 release]
• v5.3.1 fixed "Fix operations over USB-Serial/JTAG transport in Secure Download Mode" and "Preserve VID/PID lookup for symlinked ports", but does not appear to address the case where comports() returns an empty list (no USB descriptors available). [v5.3.1 release]
• The --no-stub workaround you found is a documented option — it bypasses the stub entirely and falls back to the ROM bootloader, at the cost of performance. [Flasher Stub docs]

Your analysis appears well-reasoned and your suggested fixes are actionable. The issue is clearly a regression introduced in v5.3.0 that affects any environment where the serial port cannot be enumerated via serial.tools.list_ports.comports().

Recommended next steps:

1. File this as a bug in the esptool issue tracker if you haven't already (your write-up is thorough and well-evidenced).
2. Workarounds in the meantime:
   • Use --no-stub (slower but functional).
   • Pin to esptool==5.2.0 if stub performance is required.
3. For stub-related behavior questions, the esp-flasher-stub issue tracker may also be relevant, though the root cause here is in esptool's transport detection, not the stub binary itself.

The Espressif esptool team would need to weigh in on the preferred fix approach (e.g., an explicit --usb-mode override flag, or falling back to ROM-symbol detection when VID/PID lookup fails).

[peterdragun]

Hi @keithshawlab thank you for the detailed report! I was able to reproduce the same issue. I will take a look and create a fallback solution that will fix the issue. But I don't think we will reintroduce the old method of detection, as that was quite fragile and hard to maintain - with each revision, this address could change, and we could only support one of them.

On the --before usb-reset note: your board is ESP32-S3 USB-OTG (303a:0009), not USB-Serial/JTAG (303a:1001). --before usb-reset only selects the USB Serial/JTAG reset sequence. In your case (USB-OTG), the default reset should be used. See: https://docs.espressif.com/projects/esptool/en/latest/esp32s3/esptool/advanced-options.html#reset-before-operation-before
The issue in this case is not with reset sequences, but with the transferred block size. It is lower without the stub, and that is the reason it works with --no-stub.

I would consider passing the device to the container in a way that it could detect the VID and PID. I know it might not be possible in all cases, like remote serial ports (RFC 2217), but this is documented as not supported.