out_aio_blocks_per_endpoint of 2 causes dropped packets

[defparam]

Hi There,

Thanks for this library and all your USB work. Quick thing I just wanted to mention in case anyone else was having issues using ffs bulk at high speed rate. I modified the USBCat example to create a bidirectional pipe to a consumer/producer to push full bandwidth on both the device model and host model (in place of the stdout stuff). When the design was pushing bandwidth I was noticed dropped packets. I debugged the stack and figured out that the low default out_aio_blocks_per_endpoint (of 2) was causing incoming packets at the device to be overwritten before processed via the onComplete handler. When I increased out_aio_blocks_per_endpoint to 8 I no longer had dropped packet issues.

It might be worth changing the default in the __init__.py to from 2 to 8. For anyone else with this issue you can just pass out_aio_blocks_per_endpoint=8 to the super().__init__ call inside your USBCat class.

Thanks

[vpelletier]

Hello, sorry for reacting so late.

I am surprised having too few AIO blocks causes dropped packets.

At least, if this still uses bulk endpoints: I would expect the UDC to NAK/NYET traffic for as long as it lacks any buffer to store the data in. It should not have any buffer available when the AIO blocks are full and handed back to python. This should limit the bandwidth rather than cause packet loss.

If it uses isochronous then it would be expected, but then I think you would have mentioned it.

If it is indeed using bulk endpoints, then there is a bug somewhere which is causing packet loss. I would recommend capturing USB traces with wireshark/tcpdump on the host (at the cost of extra load, which can disturb the measure), or even better with a hardware USB analyzer (with the possible limit of how much buffer space there is in the analyzer, and if you do not have one already it can be quite expensive). If you see packets being ACK but the contained data does not appear at the python level on the device then the issue is on the device side. If it is being NAK/NYET and does not get retried then it would be on the host side.

In both modes, increasing out_aio_blocks_per_endpoint and/or out_aio_blocks_max_packet_count gives the UDC more buffer space to receive data while python is busy with transfer completion, and hence at least improves the tolerance to bursts, if not improve the bandwidth. By how much depends on how large the bandwidth bursts are (in a performance benchmark, they would probably be functionally infinite), the completion latency, and whether there is a wakeup cost.

For example, if it takes the transmission time of 1 packet to process 10 but the CPU takes the equivalent of 20 packets worth of time to wake up on transfer completion, then the UDC must have at least 4 AIO blocks of 10 packets each available initially:

• the first one will wake the CPU up once filled
• the second and third gets filled while the CPU wakes up
• the CPU starts processing data at full speed, while the 4th block is absorbing the traffic
• 3 packets into the 4th block, the CPU has re-submitted the 3 completed blocks to the UDC, and it starts waiting for the next completion (in 10-3=7 packets worth of time), possibly going back to sleep

Those parameters are a compromise between memory usage and burst traffic tolerance, and are very much hardware-dependent. This is why they are exposed as parameters on the Function class, in order to be tailored to the needs & resources of the hardware.