DMX input compatibility issues with the mark-time-between-packets limitation causing dropped packets

[SpComb]

The current dmx-input implementation suffers from compatibility issues with DMX transmitters driving the DMX bus at 100% utilization with minimum idle periods between packets, including our own dmx-output implementation.

This is a known issue with the current implementation, and documented in the [dmx-input] mtbp_min config:

Minimum mark-time-between-packets (us, approximately).

Values ~128us or above are recommended. Default 0 -> 128us

Values below 32us will drastically increase DMX input -> UART RX interrupt overhead, and may cause WiFi connections to fail.

DMX input may fail with UART RX break desynchronization errors if this is too high.

This works reasonably well with some DMX transmitters (ChamSys MagicDMX, Enttec DMX USB), but not with others (e.g. our own dmx-output implementation). It does not meet the DMX512A specification, which specifies a minimum of 0 and maximum of 1s "MARK" Before BREAK (MBB) value: https://tsp.esta.org/tsp/documents/docs/ANSI-ESTA_E1-11_2008R2018.pdf

For example, MagicQ (default Mixed + Changes Only mode) sends bursts of two Art-Net packets once a second when idle, and with our artnet -> dmx-output implementation, this results in a pair of DMX packets with a <100us mark-time-before-break every second:

This is a perfectly valid DMX signal, but it leads to the following symptoms on the DMX input:

• The UART_INTR_BRK_DET interrupt fires before either any RX ISR has emptied out the TX queue
• uart_read returns -ESPIPE
• console log dmx_input_process_error: UART RX break desynchronized
• dmx stats input rx_desync counter
• The LEDs stay on some previous value, and only update once some DMX packet jitters enough to hit the mtbp fifo timing

This is particularly noticable when Clearing the MagicQ programmer - the LEDs on a DMX input controller will stay stuck on the last value for several seconds, until the UART RX break happens to align with an empty RX FIFO and the DMX packet gets received correctly.

Fixes

Reliably detecting the end of a DMX packet isn't trivial given how the DMX protocol and ESP-32 UART interrupts work.

The normal approach of processing the DMX data incrementally to match the specific DMX addresses isn't applicable, we need to handle complete packets because of how we implement the DMX -> Art-NET input.

Fixes for the following cases:

Full DMX packets

dmx_input_read() should stop reading after 513 frames and immediately return the full packet for artnet input processing without further delay.

This would require uart_read() needs to set the rxfifo_full_thrhd dynamically based on the remaining read buffer size, such that the UART_INTR_RXFIFO_FULL ISR reliably fires immediately on the last frame of a normal full-length packet.

Partial DMX packets

The UART_INTR_BRK_DET interrupt needs to be processed during the 88us + 8us minimum break / mark-after-break period in order to handle partial DMX packets correctly.

The root cause of this issue is that we currently have a very pessimistic implementation that only accepts the break detect interrupt with an otherwise empty UART RX FIFO:

esp/components/uart/esp32/intr.c

Lines 75 to 87 in b8fee49

	// The BREAK condition decodes as a 0x00 byte, with a framing error
	if (uart_ll_get_rxfifo_len(uart->dev) == 1 && uart_rx_read_rxfifo_byte(uart) == 0x00) {
	// clean break, mark for uart_read() return
	uart->rx_break = true;
	} else {
	// break triggered with old data remaining in the FIFO, or was delayed until new data in the FIFO
	// impossible to delinate where the break happened
	uart->rx_break = true;
	uart->rx_overflow = true;
	// reset RX fifo to avoid coalescing RX buffer data across breaks
	uart_ll_rxfifo_rst(uart->dev);
	}

Maybe we can safely assume that the ISR will be handled before the next frame gets pushed to the RX FIFO, and we can just empty it out into the RX buffer and set the rx_break flag - discarding the last 0x00 byte?

Then the dmx-input will still have some time to process the previous packet and the break before the RX FIFO overflows with the start of the next packet?

The risk is that if the UART_INTR_BRK_DET ISR is delayed enough for the DMX START code frame to make its way into the RX FIFO, then it will be indistinguishable from the BREAK's 0x00 byte, and the DMX packet will be corrupted.