Skip to content

Binary Transfer Protocol

Jump to bottom
HlexNC edited this page Mar 14, 2026 · 1 revision

Binary Transfer Protocol

Used for sending large binary payloads (images, future CV frames) between Mobile and Desktop over the existing WebSocket/TLS connection. Introduced in v0.5.0-test for the image inversion test in the Testing Module.

Why Binary Transfer

The previous approach encoded images as base64 inside JSON messages (~33% size overhead) and sent them as a single large WebSocket text frame. This blocked the event loop during serialization, causing counter updates to freeze during image processing.

The binary transfer splits payloads into 64 KB chunks, keeping the event loop responsive. Processing happens on a blocking thread pool (tokio::task::spawn_blocking) so the WebSocket loop is never stalled.

Protocol Version

Both sides advertise protocolVersion during the handshake. The current version is 1.

  • Mobile sends protocolVersion: 1 in the handshake payload.
  • Desktop echoes protocolVersion: 1 in the handshake response.
  • A mismatch logs a warning on both sides but does not abort the connection (for now).

When making a breaking change to this protocol, increment PROTOCOL_VERSION in both:

  • apps/desktop/src-tauri/src/device_linking.rs — PROTOCOL_VERSION constant
  • apps/mobile/src/services/DeviceLinkingService.ts — PROTOCOL_VERSION static constant

Upload Flow (Mobile → Desktop)

Mobile                                  Desktop
  |                                        |
  |-- Text: img_start ------------------>  |  { type: "img_start", taskId, totalChunks, mime }
  |-- Binary: chunk 0 ----------------->  |  [4 bytes BE u32 index][raw bytes]
  |-- Binary: chunk 1 ----------------->  |  [4 bytes BE u32 index][raw bytes]
  |-- Binary: chunk N ----------------->  |
  |-- Text: img_end ------------------->  |  { type: "img_end", taskId }
  |                                        |
  |                          [invert on blocking thread]
  |                                        |
  |<-- Text: img_result_start ----------  |  { type: "img_result_start", taskId, totalChunks }
  |<-- Binary: chunk 0 ----------------  |  [4 bytes BE u32 index][raw PNG bytes]
  |<-- Binary: chunk N ----------------  |
  |<-- Text: img_result_end -----------  |  { type: "img_result_end", taskId, success: true }

On failure (inversion error or timeout):

  |<-- Text: img_result_end -----------  |  { type: "img_result_end", taskId, success: false, error: "..." }

Frame Format

Control Frames (Text / JSON)

Message Direction Required Fields
img_start Mobile → Desktop type, taskId, totalChunks, mime
img_end Mobile → Desktop type, taskId
img_result_start Desktop → Mobile type, taskId, totalChunks
img_result_end (success) Desktop → Mobile type, taskId, success: true
img_result_end (failure) Desktop → Mobile type, taskId, success: false, error

Binary Frames

[Byte 0-3]  uint32 big-endian  chunk index (0-based)
[Byte 4+]   raw bytes           chunk payload

Chunk size: BINARY_CHUNK_SIZE = 65536 (64 KB). Last chunk may be smaller.

Chunks may theoretically arrive out of order (WebSocket does not guarantee order in burst conditions). Both sides store chunks in a HashMap<index, bytes> and assemble in index order after all chunks arrive.

Timeouts

Side Timeout Behaviour on expiry
Mobile (sender) 60 s inboundTransfer is cleared, promise rejects with "Image inversion timeout"
Desktop (receiver) 30 s inbound_transfer is discarded on next chunk arrival or img_end

Platform Notes

Android

Binary send/receive goes through ZelaraPinnedWebSocket → ZelaraWebSocketModule (OkHttp native layer). Binary frames are base64-encoded at the native bridge boundary:

  • Receive: OkHttp onMessage(ByteString) → base64 encode → emit onBinaryMessage JS event → JS decodes back to ArrayBuffer.
  • Send: JS encodes ArrayBuffer to base64 → ZelaraPinnedWebSocket.sendBinary() → ZelaraWebSocketModule.sendBinary() decodes and sends as OkHttp ByteString.

The double base64 conversion is a platform constraint and does not affect correctness.

iOS

Uses React Native's native WebSocket with binaryType = 'arraybuffer'. Binary frames arrive as ArrayBuffer in the standard onmessage event.

⚠️ Untested on real iOS hardware. RN 0.76+ should support binaryType='arraybuffer', but this has not been verified end-to-end on a physical device. See TODO in DeviceLinkingService.ts.

Key Files

File Role
apps/desktop/src-tauri/src/device_linking.rs InboundImageTransfer, send_binary_result(), PROTOCOL_VERSION, BINARY_CHUNK_SIZE
apps/mobile/src/services/DeviceLinkingService.ts sendImageInversionTest(), handleBinaryMessage(), assembleAndResolveInbound()
apps/mobile/src/services/ZelaraPinnedWebSocket.ts Android binary bridge (sendBinary, onbinarymessage)
apps/mobile/android/.../ZelaraWebSocketModule.kt OkHttp onMessage(ByteString) handler, sendBinary() method

Extending to Other Use Cases

When recycling CV validation needs binary transfer (sending camera frames to Desktop for ML processing), the binary transfer logic in DeviceLinkingService.ts is currently tightly coupled to the image inversion test. Before adding a second binary transfer use case, refactor into a general sendBinaryPayload(bytes, taskType) abstraction so the chunk/reassemble lifecycle is reused rather than duplicated.

Clone this wiki locally