From 73f5d296a1406065510194d096488552334fe23e Mon Sep 17 00:00:00 2001 From: wipesides Date: Thu, 16 Jul 2026 16:33:40 +0300 Subject: [PATCH 01/10] docs(bitmap): specify the bitmap surface protocol --- protocols/bitmap.md | 316 ++++++++++++++++++++++++++++++++++++++++++ protocols/graphics.md | 3 + 2 files changed, 319 insertions(+) create mode 100644 protocols/bitmap.md diff --git a/protocols/bitmap.md b/protocols/bitmap.md new file mode 100644 index 0000000..36aba64 --- /dev/null +++ b/protocols/bitmap.md @@ -0,0 +1,316 @@ +# Ratty Bitmap Surface Protocol + +Ratty Bitmap Surface is a terminal protocol for registering 2D bitmap assets, +placing them in terminal cell space, changing placement and crop properties +without re-uploading pixels, and replacing live pixels without changing the +bitmap identity. + +Version 1 uses the `ratty;i` APC namespace. It supports PNG registration and +full-frame RGBA8 replacement only. + +## Transport and framing + +Commands use APC (Application Program Command) framing: + +```text +ESC _ ratty;i;[;...][;] ESC \ +``` + +Both the two-byte `ESC \` string terminator and the single-byte C1 ST +terminator are accepted. Header fields are semicolon-separated. A command with +a payload places its base64 data after the header fields as the final +semicolon-separated item. + +Each individual `r` or `f` APC chunk may decode to at most 64 MiB. Ratty +preflights the encoded payload length before base64 decoding. The v1 encoded +APC bound is derived as +`len(ESC _ ratty;i;) + 4096 + 4 * ceil(64 MiB / 3) + len(ESC \)`: 4096 bytes +are reserved for the verb, fields, and separators, and the two-byte terminator +is the larger accepted terminator. The verb, fields, and separators may occupy +at most those 4096 bytes. For `r` and `f`, the header extends through the final +semicolon that separates the payload, so semicolon runs in an alleged payload +cannot bypass the header limit. A client must split a transfer before any +individual command reaches the complete APC bound. + +If an unterminated bitmap APC reaches the encoded bound, Ratty discards bytes +through the next `ESC \` or C1 ST without retaining or displaying them, then +resumes normal terminal parsing after the terminator. This bound applies only +to the `ratty;i` namespace; it does not change RGP or Kitty limits. + +Bitmap IDs, placement IDs, sequence numbers, source coordinates, and dimensions +are unsigned decimal `u32` values on the wire. Destination `row` and `col` are +unsigned decimal values limited to the `u16` range `0..=65535`. A bitmap ID is +written as `id`; a placement ID is written as `pid`. Bitmap and placement IDs +each belong to a single global namespace of their kind. Destination `w` and +`h`, source `src_w` and `src_h`, and frame `w` and `h` must be nonzero; IDs, +coordinates, and sequence numbers have no additional nonzero constraint. +Opacity is a finite decimal floating-point value whose effective value is in +`[0,1]`; finite input outside that range is clamped to the nearest endpoint. + +The verbs are: + +- `s`: query support +- `r`: register a bitmap +- `p`: create a placement +- `u`: update a placement +- `f`: replace a bitmap frame +- `d`: delete a placement or bitmap + +## Support discovery + +A client queries support with: + +```text +ESC _ ratty;i;s ESC \ +``` + +Ratty replies with exactly: + +```text +ESC _ ratty;i;s;v=1;fmt=png;frame=rgba8;payload=1;chunk=1;placement=1;crop=1;fit=contain|cover|fill;filter=nearest|linear;opacity=1 ESC \ +``` + +The reply advertises protocol version 1, PNG payload registration, RGBA8 frame +replacement, chunked transfers, independently addressable placements, source +cropping, the three fit modes, the two filter modes, and opacity. If no reply +arrives, the client must assume that Ratty Bitmap Surface is unsupported. +Support queries are the only version 1 commands that produce a reply. + +## Coordinate systems and placement model + +A registered bitmap owns one pixel image and one stable bitmap identity. It may +have multiple placements, and each placement has a globally unique `pid`. +Registration and frame replacement operate on the shared bitmap; placement and +update commands operate on one placement. + +Destination `row`, `col`, `w`, and `h` are measured in terminal cells. +`row,col` is the top-left placement anchor, `w` is the number of columns, and +`h` is the number of rows. Source `src_x`, `src_y`, `src_w`, and `src_h` are +measured in source pixels from the bitmap's top-left origin. + +When no source rectangle is specified, the full bitmap is used. A supplied +source rectangle is clamped to the bitmap bounds. The command is rejected if +the clamped intersection is empty. Source and destination widths and heights +must be nonzero. + +## Register bitmap (`r`) + +Registration carries a base64-encoded PNG payload. A one-chunk registration is: + +```text +ESC _ ratty;i;r;id=42;fmt=png;source=payload;more=0; ESC \ +``` + +The first chunk requires: + +- `id`: the bitmap ID +- `fmt=png`: the version 1 registration format +- `source=payload`: the version 1 registration source +- `more`: `1` when more chunks follow or `0` on the final chunk +- a base64 payload item + +`name` is optional diagnostic metadata. For a multi-chunk registration, later +chunks use the same `id`, include `more`, and carry the next base64 payload +item. `fmt`, `source`, and `name` may be repeated after the first chunk only +when their values exactly match the first chunk. + +```text +ESC _ ratty;i;r;id=42;fmt=png;source=payload;more=1;name=photo.png; ESC \ +ESC _ ratty;i;r;id=42;more=1; ESC \ +ESC _ ratty;i;r;id=42;more=0; ESC \ +``` + +After the per-chunk size preflight, Ratty base64-decodes each chunk and retains +decoded bytes while `more=1`. +`more=0` finalizes the transfer: Ratty decodes the accumulated PNG exactly +once, creates the bitmap, and clears the pending transfer. The bitmap becomes +visible to placement commands only after successful finalization. + +A pending registration may contain at most 64 MiB of decoded payload bytes. If +a chunk would exceed that limit, Ratty rejects the chunk and discards the whole +pending registration. An invalid PNG on finalization also clears the pending +transfer and does not register a bitmap. Other malformed chunks make no state +change. + +Registering an `id` that is already registered is rejected and never replaces +the existing bitmap or its placements. + +## Place bitmap (`p`) + +A placement refers to an already registered bitmap and receives its own +globally unique placement ID: + +```text +ESC _ ratty;i;p;id=42;pid=7;row=4;col=2;w=80;h=30;fit=contain;filter=linear;opacity=1 ESC \ +``` + +Required fields are `id`, `pid`, `row`, `col`, `w`, and `h`. The destination +dimensions must be nonzero. The optional placement fields are: + +- the complete `src_x`, `src_y`, `src_w`, `src_h` source rectangle +- `fit=contain|cover|fill`, default `contain` +- `filter=nearest|linear`, default `linear` +- `opacity`, default `1` + +A source rectangle, when present, must contain all four source fields. Opacity +must be finite and is clamped to the inclusive range `[0,1]`. + +Placement fails without mutation when the bitmap does not exist or `pid` is +already in use. A bitmap can have any number of distinct placements. + +## Update placement (`u`) + +An update changes an existing placement without re-registering the bitmap or +changing its bitmap ID: + +```text +ESC _ ratty;i;u;pid=7;src_x=300;src_y=120;src_w=900;src_h=600 ESC \ +``` + +`pid` and at least one mutable field are required. Mutable fields are: + +- `row` or `col`, independently +- `w` and `h`, as a complete pair +- `src_x`, `src_y`, `src_w`, and `src_h`, as a complete quartet +- `fit` +- `filter` +- `opacity` + +Updated destination dimensions must be nonzero. Updated source rectangles use +the same clamping and nonempty-intersection rules as placement. Updated opacity +must be finite and is clamped to `[0,1]`. + +Updates are transactional. A partial `w/h` pair, partial source quartet, +unknown placement, invalid value, or failed validation rejects the whole +command and leaves the placement unchanged. + +## Replace frame (`f`) + +Frame replacement changes the pixels of a registered bitmap while retaining +its identity and every placement: + +```text +ESC _ ratty;i;f;id=42;seq=123;fmt=rgba8;w=1280;h=720;more=0; ESC \ +``` + +The first chunk requires `id`, `seq`, `fmt=rgba8`, `w`, `h`, `more`, and a +base64 payload item. The dimensions must exactly match the registered bitmap's +dimensions. Continuation chunks require `id`, `seq`, `more`, and the next +payload item. If `fmt`, `w`, or `h` is repeated on a continuation, it must +exactly match the first chunk. + +Chunks are assembled by `(id, seq)`. `seq` is mandatory and must increase for +each bitmap. When a newer sequence begins, Ratty discards any incomplete older +sequence for that bitmap. A stale sequence never replaces displayed pixels. + +Ratty calculates the expected byte length as `w * h * 4` using checked +arithmetic. Arithmetic overflow rejects the frame. If accumulated decoded data +ever exceeds that expected length, Ratty immediately rejects the chunk and +discards the affected pending `(id, seq)` frame instead of retaining excess +data. + +On the final `more=0` chunk, the decoded payload length must equal the expected +byte length. Pixels are tightly packed RGBA8 in row-major order from the +top-left. After all validation succeeds, Ratty atomically replaces the pixels +of the existing bitmap. Its bitmap ID, underlying image handle, dimensions, +and all placements remain unchanged. + +Invalid base64, inconsistent continuation metadata, an oversized accumulated +payload, or a final length mismatch discards the affected pending `(id, seq)` +frame. Missing metadata, zero or mismatched dimensions, arithmetic overflow, +stale sequencing, or any other malformed frame is rejected. Every frame error +preserves the last valid displayed pixels and all placements. + +## Delete (`d`) + +Delete one placement with: + +```text +ESC _ ratty;i;d;pid=7 ESC \ +``` + +Delete one bitmap with: + +```text +ESC _ ratty;i;d;id=42 ESC \ +``` + +Exactly one of `pid` or `id` is required. A command with neither ID or both IDs +is malformed and does not delete anything. Deleting an unknown placement or +bitmap is an idempotent no-op. In particular, deleting an unknown bitmap ID +does not cancel a pending registration for that ID. Bitmap deletion cascades +only when the bitmap is already registered. + +Deleting a placement does not affect its bitmap or sibling placements. +Deleting a bitmap atomically deletes all placements that refer to it. + +## Fit and filtering rules + +Fit is resolved from the selected source rectangle into the destination: + +- `fill`: map the full source rectangle to the full destination; aspect ratio + may change. +- `contain`: preserve aspect ratio, center the image, and leave transparent + letterboxing in the unused destination area. +- `cover`: preserve aspect ratio and fill the destination by applying a + symmetric crop to the source. + +`nearest` selects nearest-neighbor sampling. `linear` selects linear sampling. +Filtering belongs to the placement, so placements that share a bitmap may use +different filter modes. + +## Errors and mutation rules + +Ratty consumes commands in the `ratty;i` namespace even when they are +malformed or unsupported. It logs a warning and sends no error reply. A +malformed or unsupported command makes no state change, except that a failed or +overflowed pending transfer is discarded as described above. + +Unknown verbs do nothing. Duplicate keys, unsupported values, missing required +fields, invalid base64, and invalid numeric values are malformed. Only a valid +support query generates output. + +Registration, placement, and frame state are separate. Placement changes never +create or replace bitmap pixels. Frame changes never alter placement records. +No version 1 operation accepts filesystem paths, compressed image formats other +than registration PNG, dirty rectangles, codecs, or network sources. + +## Complete example + +Query support: + +```text +ESC _ ratty;i;s ESC \ +ESC _ ratty;i;s;v=1;fmt=png;frame=rgba8;payload=1;chunk=1;placement=1;crop=1;fit=contain|cover|fill;filter=nearest|linear;opacity=1 ESC \ +``` + +Register a PNG bitmap, potentially using repeated chunks with the same ID: + +```text +ESC _ ratty;i;r;id=42;fmt=png;source=payload;more=0; ESC \ +``` + +Place it in terminal cell space: + +```text +ESC _ ratty;i;p;id=42;pid=7;row=4;col=2;w=80;h=30;fit=contain;filter=linear;opacity=1 ESC \ +``` + +Change the placement's source crop without uploading the PNG again: + +```text +ESC _ ratty;i;u;pid=7;src_x=300;src_y=120;src_w=900;src_h=600 ESC \ +``` + +Replace the bitmap's pixels with a sequenced, fixed-dimension RGBA8 frame: + +```text +ESC _ ratty;i;f;id=42;seq=123;fmt=rgba8;w=1280;h=720;more=0; ESC \ +``` + +Delete the placement and then the bitmap: + +```text +ESC _ ratty;i;d;pid=7 ESC \ +ESC _ ratty;i;d;id=42 ESC \ +``` diff --git a/protocols/graphics.md b/protocols/graphics.md index fffbcb9..10a7143 100644 --- a/protocols/graphics.md +++ b/protocols/graphics.md @@ -3,6 +3,9 @@ Ratty Graphics Protocol (RGP) is a custom terminal protocol for inserting 3D objects into the terminal as first-class inline objects. +The `ratty;g` namespace is 3D and object-oriented. The separate `ratty;i` +namespace is 2D and bitmap/texture-oriented. + The goal is to attach a semantic graphics object to terminal cells, so it becomes part of the terminal surface rather than an external overlay. From da76f3527f0a43501da14bcb4b039b63d44dd005 Mon Sep 17 00:00:00 2001 From: wipesides Date: Thu, 16 Jul 2026 16:34:40 +0300 Subject: [PATCH 02/10] feat(bitmap): parse and manage bitmap surfaces --- src/bitmap.rs | 2337 +++++++++++++++++++++++++++++++++++++++++++++++++ src/lib.rs | 1 + 2 files changed, 2338 insertions(+) create mode 100644 src/bitmap.rs diff --git a/src/bitmap.rs b/src/bitmap.rs new file mode 100644 index 0000000..7e75c52 --- /dev/null +++ b/src/bitmap.rs @@ -0,0 +1,2337 @@ +//! Ratty Bitmap Surface protocol parsing. + +use std::{collections::HashMap, fmt}; + +use base64::Engine as _; +use bevy::prelude::{Handle, Image}; + +/// Ratty Bitmap Surface APC prefix. +pub const BITMAP_APC_START: &[u8] = b"\x1b_ratty;i;"; +const ST: &[u8] = b"\x1b\\"; +const C1_ST: u8 = 0x9c; +const SUPPORT_REPLY: &[u8] = b"\x1b_ratty;i;s;v=1;fmt=png;frame=rgba8;payload=1;chunk=1;placement=1;crop=1;fit=contain|cover|fill;filter=nearest|linear;opacity=1\x1b\\"; +pub(crate) const MAX_BITMAP_CHUNK_DECODED_BYTES: usize = 64 * 1024 * 1024; +pub(crate) const BITMAP_APC_HEADER_ALLOWANCE: usize = 4 * 1024; +const MAX_BITMAP_CHUNK_BASE64_BYTES: usize = MAX_BITMAP_CHUNK_DECODED_BYTES.div_ceil(3) * 4; +pub(crate) const MAX_BITMAP_APC_BYTES: usize = + BITMAP_APC_START.len() + BITMAP_APC_HEADER_ALLOWANCE + MAX_BITMAP_CHUNK_BASE64_BYTES + ST.len(); +const MAX_REGISTRATION_BYTES: usize = 64 * 1024 * 1024; +const CHUNK_PAYLOAD_TOO_LARGE: &str = "bitmap APC chunk payload exceeds 64 MiB"; +const HEADER_TOO_LARGE: &str = "bitmap APC header exceeds 4 KiB"; + +/// How source pixels are fitted into a placement's destination rectangle. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum BitmapFit { + /// Preserve aspect ratio and letterbox the unused destination area. + Contain, + /// Preserve aspect ratio and crop symmetrically to fill the destination. + Cover, + /// Stretch the source to fill the destination. + Fill, +} + +/// Texture filtering for a bitmap placement. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum BitmapFilter { + /// Select the nearest source pixel. + Nearest, + /// Interpolate between neighboring source pixels. + Linear, +} + +/// A rectangle in source-pixel coordinates. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct SourceRect { + /// Horizontal offset from the bitmap's left edge. + pub x: u32, + /// Vertical offset from the bitmap's top edge. + pub y: u32, + /// Source width in pixels. + pub width: u32, + /// Source height in pixels. + pub height: u32, +} + +/// One decoded chunk of a bitmap registration transfer. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct BitmapRegisterChunk { + /// Bitmap identifier. + pub bitmap_id: u32, + /// Registration format metadata, present on the first chunk. + pub format: Option, + /// Registration source metadata, present on the first chunk. + pub source: Option, + /// Optional diagnostic payload name. + pub name: Option, + /// Whether additional chunks follow. + pub more: bool, + /// Decoded payload bytes for this chunk. + pub data: Vec, +} + +/// A complete bitmap placement request. +#[derive(Clone, Debug, PartialEq)] +pub struct BitmapPlacement { + /// Bitmap identifier. + pub bitmap_id: u32, + /// Globally unique placement identifier. + pub placement_id: u32, + /// Destination row in terminal cells. + pub row: u16, + /// Destination column in terminal cells. + pub col: u16, + /// Destination width in terminal cells. + pub columns: u32, + /// Destination height in terminal cells. + pub rows: u32, + /// Optional source-pixel crop. + pub source: Option, + /// Fit mode. + pub fit: BitmapFit, + /// Filtering mode. + pub filter: BitmapFilter, + /// Clamped placement opacity. + pub opacity: f32, +} + +/// Transactional changes to an existing bitmap placement. +#[derive(Clone, Debug, Default, PartialEq)] +pub struct BitmapPlacementUpdate { + /// Optional destination row. + pub row: Option, + /// Optional destination column. + pub col: Option, + /// Optional destination width, paired with `rows`. + pub columns: Option, + /// Optional destination height, paired with `columns`. + pub rows: Option, + /// Optional complete source-pixel crop. + pub source: Option, + /// Optional fit mode. + pub fit: Option, + /// Optional filtering mode. + pub filter: Option, + /// Optional clamped opacity. + pub opacity: Option, +} + +/// One decoded chunk of a sequenced RGBA8 frame transfer. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct BitmapFrameChunk { + /// Bitmap identifier. + pub bitmap_id: u32, + /// Per-bitmap frame sequence number. + pub sequence: u32, + /// Frame format metadata, present on the first chunk. + pub format: Option, + /// Frame width metadata, present on the first chunk. + pub width: Option, + /// Frame height metadata, present on the first chunk. + pub height: Option, + /// Whether additional chunks follow. + pub more: bool, + /// Decoded RGBA8 payload bytes for this chunk. + pub data: Vec, +} + +/// A parsed Ratty Bitmap Surface operation. +#[derive(Clone, Debug, PartialEq)] +pub enum BitmapOperation { + /// Query protocol support. + SupportQuery, + /// Register a PNG bitmap transfer chunk. + Register(BitmapRegisterChunk), + /// Create a placement. + Place(BitmapPlacement), + /// Update an existing placement. + Update { + /// Placement identifier. + placement_id: u32, + /// Fields to update transactionally. + update: BitmapPlacementUpdate, + }, + /// Replace bitmap pixels with a frame transfer chunk. + Frame(BitmapFrameChunk), + /// Delete one placement. + DeletePlacement(u32), + /// Delete one bitmap and its placements. + DeleteBitmap(u32), + /// Consume an unknown protocol verb without mutation. + Ignored, +} + +/// An error in a command within the Ratty Bitmap Surface namespace. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct BitmapProtocolError { + message: &'static str, + cleanup: Option, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +enum BitmapErrorCleanup { + DiscardPendingRegistration { + bitmap_id: u32, + }, + DiscardPendingFrame { + bitmap_id: u32, + sequence: u32, + format: Option, + width: Option, + height: Option, + }, +} + +impl BitmapProtocolError { + fn new(message: &'static str) -> Self { + Self { + message, + cleanup: None, + } + } + + fn frame_payload( + message: &'static str, + bitmap_id: u32, + sequence: u32, + format: Option, + width: Option, + height: Option, + ) -> Self { + Self { + message, + cleanup: Some(BitmapErrorCleanup::DiscardPendingFrame { + bitmap_id, + sequence, + format, + width, + height, + }), + } + } + + fn registration_payload(message: &'static str, bitmap_id: u32) -> Self { + Self { + message, + cleanup: Some(BitmapErrorCleanup::DiscardPendingRegistration { bitmap_id }), + } + } +} + +impl fmt::Display for BitmapProtocolError { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + formatter.write_str(self.message) + } +} + +impl std::error::Error for BitmapProtocolError {} + +type Fields<'a> = HashMap<&'a str, &'a str>; + +/// Consumes a complete Ratty Bitmap Surface APC sequence. +pub fn consume_sequence(sequence: &[u8]) -> Option> { + if !sequence.starts_with(BITMAP_APC_START) { + return None; + } + + Some(parse_sequence(sequence)) +} + +/// Returns the exact v1 support-discovery response. +pub fn support_reply() -> Vec { + SUPPORT_REPLY.to_vec() +} + +fn parse_sequence(sequence: &[u8]) -> Result { + parse_sequence_with_payload_limit(sequence, MAX_BITMAP_CHUNK_DECODED_BYTES) +} + +fn parse_sequence_with_payload_limit( + sequence: &[u8], + payload_limit: usize, +) -> Result { + parse_sequence_with_limits(sequence, payload_limit, BITMAP_APC_HEADER_ALLOWANCE) +} + +fn parse_sequence_with_limits( + sequence: &[u8], + payload_limit: usize, + header_limit: usize, +) -> Result { + let content_end = if sequence.ends_with(&[C1_ST]) { + sequence.len() - 1 + } else if sequence.ends_with(ST) { + sequence.len() - ST.len() + } else { + return Err(BitmapProtocolError::new("invalid bitmap APC terminator")); + }; + let content = std::str::from_utf8(&sequence[BITMAP_APC_START.len()..content_end]) + .map_err(|_| BitmapProtocolError::new("bitmap APC is not valid UTF-8"))?; + if bitmap_header_extent(content) > header_limit { + return Err(BitmapProtocolError::new(HEADER_TOO_LARGE)); + } + let mut parts: Vec<_> = content.split(';').collect(); + let verb = parts + .first() + .copied() + .ok_or_else(|| BitmapProtocolError::new("missing bitmap verb"))?; + if verb.is_empty() { + return Err(BitmapProtocolError::new("missing bitmap verb")); + } + parts.remove(0); + + match verb { + "s" => parse_support(&parts), + "r" => parse_register(&parts, payload_limit), + "p" => parse_place(&parts), + "u" => parse_update(&parts), + "f" => parse_frame(&parts, payload_limit), + "d" => parse_delete(&parts), + _ => Ok(BitmapOperation::Ignored), + } +} + +fn bitmap_header_extent(content: &str) -> usize { + if content.starts_with("r;") || content.starts_with("f;") { + content + .rfind(';') + .map_or(content.len(), |separator| separator + 1) + } else { + content.len() + } +} + +fn parse_support(parts: &[&str]) -> Result { + if parts.is_empty() { + Ok(BitmapOperation::SupportQuery) + } else { + Err(BitmapProtocolError::new( + "support query does not accept fields", + )) + } +} + +fn parse_register( + parts: &[&str], + payload_limit: usize, +) -> Result { + let (payload, header) = split_payload(parts)?; + let fields = parse_fields(header, &["id", "fmt", "source", "more", "name"])?; + let format = optional_string(&fields, "fmt"); + let source = optional_string(&fields, "source"); + if format.is_some() != source.is_some() { + return Err(BitmapProtocolError::new( + "registration format and source must be provided together", + )); + } + if format.as_deref().is_some_and(|value| value != "png") { + return Err(BitmapProtocolError::new("unsupported bitmap format")); + } + if source.as_deref().is_some_and(|value| value != "payload") { + return Err(BitmapProtocolError::new( + "unsupported bitmap registration source", + )); + } + + let bitmap_id = required_u32(&fields, "id")?; + let data = decode_payload(payload, payload_limit).map_err(|error| { + if error.message == CHUNK_PAYLOAD_TOO_LARGE { + BitmapProtocolError::registration_payload(error.message, bitmap_id) + } else { + error + } + })?; + + Ok(BitmapOperation::Register(BitmapRegisterChunk { + bitmap_id, + format, + source, + name: optional_string(&fields, "name"), + more: required_bool(&fields, "more")?, + data, + })) +} + +fn parse_place(parts: &[&str]) -> Result { + let fields = parse_fields( + parts, + &[ + "id", "pid", "row", "col", "w", "h", "src_x", "src_y", "src_w", "src_h", "fit", + "filter", "opacity", + ], + )?; + let columns = required_nonzero_u32(&fields, "w")?; + let rows = required_nonzero_u32(&fields, "h")?; + + Ok(BitmapOperation::Place(BitmapPlacement { + bitmap_id: required_u32(&fields, "id")?, + placement_id: required_u32(&fields, "pid")?, + row: required_u16(&fields, "row")?, + col: required_u16(&fields, "col")?, + columns, + rows, + source: parse_source(&fields)?, + fit: parse_fit(fields.get("fit").copied())?.unwrap_or(BitmapFit::Contain), + filter: parse_filter(fields.get("filter").copied())?.unwrap_or(BitmapFilter::Linear), + opacity: parse_opacity(fields.get("opacity").copied())?.unwrap_or(1.0), + })) +} + +fn parse_update(parts: &[&str]) -> Result { + let fields = parse_fields( + parts, + &[ + "pid", "row", "col", "w", "h", "src_x", "src_y", "src_w", "src_h", "fit", "filter", + "opacity", + ], + )?; + let placement_id = required_u32(&fields, "pid")?; + let columns = optional_nonzero_u32(&fields, "w")?; + let rows = optional_nonzero_u32(&fields, "h")?; + if columns.is_some() != rows.is_some() { + return Err(BitmapProtocolError::new( + "update width and height must be provided together", + )); + } + let update = BitmapPlacementUpdate { + row: optional_u16(&fields, "row")?, + col: optional_u16(&fields, "col")?, + columns, + rows, + source: parse_source(&fields)?, + fit: parse_fit(fields.get("fit").copied())?, + filter: parse_filter(fields.get("filter").copied())?, + opacity: parse_opacity(fields.get("opacity").copied())?, + }; + if update == BitmapPlacementUpdate::default() { + return Err(BitmapProtocolError::new( + "placement update contains no mutable fields", + )); + } + + Ok(BitmapOperation::Update { + placement_id, + update, + }) +} + +fn parse_frame( + parts: &[&str], + payload_limit: usize, +) -> Result { + let (payload, header) = split_payload(parts)?; + let fields = parse_fields(header, &["id", "seq", "fmt", "w", "h", "more"])?; + let format = optional_string(&fields, "fmt"); + let width = optional_nonzero_u32(&fields, "w")?; + let height = optional_nonzero_u32(&fields, "h")?; + let metadata_count = usize::from(format.is_some()) + + usize::from(width.is_some()) + + usize::from(height.is_some()); + if metadata_count != 0 && metadata_count != 3 { + return Err(BitmapProtocolError::new( + "frame format and dimensions must be provided together", + )); + } + if format.as_deref().is_some_and(|value| value != "rgba8") { + return Err(BitmapProtocolError::new("unsupported bitmap frame format")); + } + let bitmap_id = required_u32(&fields, "id")?; + let sequence = required_u32(&fields, "seq")?; + let more = required_bool(&fields, "more")?; + let data = decode_payload(payload, payload_limit).map_err(|error| { + BitmapProtocolError::frame_payload( + error.message, + bitmap_id, + sequence, + format.clone(), + width, + height, + ) + })?; + + Ok(BitmapOperation::Frame(BitmapFrameChunk { + bitmap_id, + sequence, + format, + width, + height, + more, + data, + })) +} + +fn parse_delete(parts: &[&str]) -> Result { + let fields = parse_fields(parts, &["id", "pid"])?; + match (fields.get("id"), fields.get("pid")) { + (Some(id), None) => Ok(BitmapOperation::DeleteBitmap(parse_u32(id)?)), + (None, Some(placement_id)) => { + Ok(BitmapOperation::DeletePlacement(parse_u32(placement_id)?)) + } + _ => Err(BitmapProtocolError::new( + "delete requires exactly one bitmap or placement ID", + )), + } +} + +fn split_payload<'a>( + parts: &'a [&'a str], +) -> Result<(&'a str, &'a [&'a str]), BitmapProtocolError> { + let (payload, header) = parts + .split_last() + .ok_or_else(|| BitmapProtocolError::new("missing bitmap payload"))?; + if payload.is_empty() { + return Err(BitmapProtocolError::new("missing bitmap payload")); + } + Ok((payload, header)) +} + +fn parse_fields<'a>( + parts: &'a [&'a str], + allowed: &[&str], +) -> Result, BitmapProtocolError> { + let mut fields = HashMap::new(); + for part in parts { + let (key, value) = part + .split_once('=') + .ok_or_else(|| BitmapProtocolError::new("malformed bitmap field"))?; + if !allowed.contains(&key) { + return Err(BitmapProtocolError::new("unknown bitmap field")); + } + if fields.insert(key, value).is_some() { + return Err(BitmapProtocolError::new("duplicate bitmap field")); + } + } + Ok(fields) +} + +fn required_u32(fields: &Fields<'_>, key: &str) -> Result { + fields + .get(key) + .ok_or_else(|| BitmapProtocolError::new("missing required bitmap field")) + .and_then(|value| parse_u32(value)) +} + +fn parse_u32(value: &str) -> Result { + value + .parse() + .map_err(|_| BitmapProtocolError::new("invalid unsigned bitmap integer")) +} + +fn required_nonzero_u32(fields: &Fields<'_>, key: &str) -> Result { + let value = required_u32(fields, key)?; + if value == 0 { + Err(BitmapProtocolError::new( + "bitmap dimensions must be nonzero", + )) + } else { + Ok(value) + } +} + +fn optional_nonzero_u32( + fields: &Fields<'_>, + key: &str, +) -> Result, BitmapProtocolError> { + fields + .get(key) + .map(|value| { + let value = parse_u32(value)?; + if value == 0 { + Err(BitmapProtocolError::new( + "bitmap dimensions must be nonzero", + )) + } else { + Ok(value) + } + }) + .transpose() +} + +fn required_u16(fields: &Fields<'_>, key: &str) -> Result { + fields + .get(key) + .ok_or_else(|| BitmapProtocolError::new("missing required bitmap field"))? + .parse() + .map_err(|_| BitmapProtocolError::new("invalid terminal-cell coordinate")) +} + +fn optional_u16(fields: &Fields<'_>, key: &str) -> Result, BitmapProtocolError> { + fields + .get(key) + .map(|value| { + value + .parse() + .map_err(|_| BitmapProtocolError::new("invalid terminal-cell coordinate")) + }) + .transpose() +} + +fn required_bool(fields: &Fields<'_>, key: &str) -> Result { + match fields.get(key).copied() { + Some("0") => Ok(false), + Some("1") => Ok(true), + Some(_) => Err(BitmapProtocolError::new("invalid bitmap boolean")), + None => Err(BitmapProtocolError::new("missing required bitmap field")), + } +} + +fn optional_string(fields: &Fields<'_>, key: &str) -> Option { + fields.get(key).map(|value| (*value).to_owned()) +} + +fn parse_source(fields: &Fields<'_>) -> Result, BitmapProtocolError> { + let values = ["src_x", "src_y", "src_w", "src_h"].map(|key| fields.get(key).copied()); + let [x, y, width, height] = match values { + [None, None, None, None] => return Ok(None), + [Some(x), Some(y), Some(width), Some(height)] => [x, y, width, height], + _ => { + return Err(BitmapProtocolError::new( + "source rectangle requires all four fields", + )); + } + }; + let width = parse_u32(width)?; + let height = parse_u32(height)?; + if width == 0 || height == 0 { + return Err(BitmapProtocolError::new( + "source dimensions must be nonzero", + )); + } + Ok(Some(SourceRect { + x: parse_u32(x)?, + y: parse_u32(y)?, + width, + height, + })) +} + +fn parse_fit(value: Option<&str>) -> Result, BitmapProtocolError> { + match value { + None => Ok(None), + Some("contain") => Ok(Some(BitmapFit::Contain)), + Some("cover") => Ok(Some(BitmapFit::Cover)), + Some("fill") => Ok(Some(BitmapFit::Fill)), + Some(_) => Err(BitmapProtocolError::new("unsupported bitmap fit mode")), + } +} + +fn parse_filter(value: Option<&str>) -> Result, BitmapProtocolError> { + match value { + None => Ok(None), + Some("nearest") => Ok(Some(BitmapFilter::Nearest)), + Some("linear") => Ok(Some(BitmapFilter::Linear)), + Some(_) => Err(BitmapProtocolError::new("unsupported bitmap filter mode")), + } +} + +fn parse_opacity(value: Option<&str>) -> Result, BitmapProtocolError> { + value + .map(|value| { + let opacity: f32 = value + .parse() + .map_err(|_| BitmapProtocolError::new("invalid bitmap opacity"))?; + if !opacity.is_finite() { + return Err(BitmapProtocolError::new("bitmap opacity must be finite")); + } + Ok(opacity.clamp(0.0, 1.0)) + }) + .transpose() +} + +fn decode_payload(payload: &str, payload_limit: usize) -> Result, BitmapProtocolError> { + let estimated_len = estimated_decoded_payload_len(payload) + .ok_or_else(|| BitmapProtocolError::new("bitmap payload length overflow"))?; + if estimated_len > payload_limit { + return Err(BitmapProtocolError::new(CHUNK_PAYLOAD_TOO_LARGE)); + } + base64::engine::general_purpose::STANDARD + .decode(payload) + .map_err(|_| BitmapProtocolError::new("invalid bitmap payload base64")) +} + +fn estimated_decoded_payload_len(payload: &str) -> Option { + let len = payload.len(); + let remainder_bytes = match len % 4 { + 0 => 0, + 2 => 1, + 3 => 2, + _ => 3, + }; + let padding = if len.is_multiple_of(4) { + payload + .as_bytes() + .iter() + .rev() + .take(2) + .take_while(|byte| **byte == b'=') + .count() + } else { + 0 + }; + (len / 4) + .checked_mul(3)? + .checked_add(remainder_bytes)? + .checked_sub(padding) +} + +/// A decoded bitmap and its eventual stable Bevy image handle. +pub struct RegisteredBitmap { + width: u32, + height: u32, + rgba: Vec, + handle: Option>, +} + +impl RegisteredBitmap { + /// Returns the bitmap width in pixels. + pub(crate) fn width(&self) -> u32 { + self.width + } + + /// Returns the bitmap height in pixels. + pub(crate) fn height(&self) -> u32 { + self.height + } + + /// Returns the stable Bevy image handle after the renderer uploads the bitmap. + pub(crate) fn handle(&self) -> Option<&Handle> { + self.handle.as_ref() + } + + /// Moves pixels that have not yet been synchronized into the Bevy image asset. + pub(crate) fn take_pending_rgba(&mut self) -> Option> { + (!self.rgba.is_empty()).then(|| std::mem::take(&mut self.rgba)) + } + + /// Records the stable Bevy image handle created by the renderer. + pub(crate) fn set_handle(&mut self, handle: Handle) { + debug_assert!( + self.handle + .as_ref() + .is_none_or(|current| current == &handle) + ); + self.handle = Some(handle); + } +} + +/// The validated state of one independently addressable bitmap placement. +#[derive(Clone, Debug, PartialEq)] +pub struct BitmapPlacementState { + generation: u64, + bitmap_id: u32, + row: u16, + col: u16, + columns: u32, + rows: u32, + source: Option, + fit: BitmapFit, + filter: BitmapFilter, + opacity: f32, +} + +impl BitmapPlacementState { + /// Returns this placement lifetime's monotonically increasing generation. + pub(crate) fn generation(&self) -> u64 { + self.generation + } + + /// Returns the registered bitmap used by this placement. + pub(crate) fn bitmap_id(&self) -> u32 { + self.bitmap_id + } + + /// Returns the placement's terminal row. + pub(crate) fn row(&self) -> u16 { + self.row + } + + /// Returns the placement's terminal column. + pub(crate) fn col(&self) -> u16 { + self.col + } + + /// Returns the placement width in terminal columns. + pub(crate) fn columns(&self) -> u32 { + self.columns + } + + /// Returns the placement height in terminal rows. + pub(crate) fn rows(&self) -> u32 { + self.rows + } + + /// Returns the validated source crop, if present. + pub(crate) fn source(&self) -> Option { + self.source + } + + /// Returns the placement fit mode. + pub(crate) fn fit(&self) -> BitmapFit { + self.fit + } + + /// Returns the placement filtering mode. + pub(crate) fn filter(&self) -> BitmapFilter { + self.filter + } + + /// Returns the clamped placement opacity. + pub(crate) fn opacity(&self) -> f32 { + self.opacity + } +} + +struct PendingBitmapTransfer { + format: String, + source: String, + name: Option, + data: Vec, +} + +struct PendingBitmapFrame { + sequence: u32, + format: String, + width: u32, + height: u32, + expected_len: usize, + data: Vec, +} + +/// In-memory lifecycle state for registered bitmaps, frames, and placements. +#[derive(Default)] +pub struct BitmapSurfaceState { + pending_registrations: HashMap, + pending_frames: HashMap, + bitmaps: HashMap, + placements: HashMap, + latest_frame_sequences: HashMap, + next_placement_generation: u64, + dirty: bool, +} + +impl BitmapSurfaceState { + /// Parses and applies one complete bitmap APC sequence, including parser-directed cleanup. + pub(crate) fn consume_and_apply( + &mut self, + sequence: &[u8], + ) -> Option>, BitmapProtocolError>> { + let parsed = consume_sequence(sequence)?; + Some(match parsed { + Ok(operation) => self.apply(operation), + Err(error) => { + self.apply_error_cleanup(&error); + Err(error) + } + }) + } + + /// Applies one parsed bitmap protocol operation transactionally. + pub fn apply( + &mut self, + operation: BitmapOperation, + ) -> Result>, BitmapProtocolError> { + match operation { + BitmapOperation::SupportQuery => Ok(Some(support_reply())), + BitmapOperation::Register(chunk) => { + self.apply_registration(chunk)?; + Ok(None) + } + BitmapOperation::Place(placement) => { + self.apply_placement(placement)?; + Ok(None) + } + BitmapOperation::Update { + placement_id, + update, + } => { + self.apply_placement_update(placement_id, update)?; + Ok(None) + } + BitmapOperation::Frame(chunk) => { + self.apply_frame(chunk)?; + Ok(None) + } + BitmapOperation::DeletePlacement(placement_id) => { + if self.placements.remove(&placement_id).is_some() { + self.dirty = true; + } + Ok(None) + } + BitmapOperation::DeleteBitmap(bitmap_id) => { + self.delete_bitmap(bitmap_id); + Ok(None) + } + BitmapOperation::Ignored => Ok(None), + } + } + + /// Returns a registered bitmap without allowing map mutation. + pub(crate) fn bitmap(&self, bitmap_id: u32) -> Option<&RegisteredBitmap> { + self.bitmaps.get(&bitmap_id) + } + + /// Returns a registered bitmap for renderer-owned upload bookkeeping. + pub(crate) fn bitmap_mut(&mut self, bitmap_id: u32) -> Option<&mut RegisteredBitmap> { + self.bitmaps.get_mut(&bitmap_id) + } + + /// Iterates registered bitmaps without exposing mutable map access. + pub(crate) fn bitmaps(&self) -> impl Iterator { + self.bitmaps.iter() + } + + /// Returns a placement without allowing map mutation. + #[cfg(test)] + pub(crate) fn placement(&self, placement_id: u32) -> Option<&BitmapPlacementState> { + self.placements.get(&placement_id) + } + + /// Iterates placements without exposing mutable map access. + pub(crate) fn placements(&self) -> impl Iterator { + self.placements.iter() + } + + /// Reports whether visible bitmap state changed since the last dirty reset. + pub(crate) fn is_dirty(&self) -> bool { + self.dirty + } + + /// Returns and clears the visible-state dirty flag. + pub(crate) fn take_dirty(&mut self) -> bool { + std::mem::take(&mut self.dirty) + } + + /// Applies terminal upward scrolling to cell-anchored placements. + pub(crate) fn apply_scroll(&mut self, rows_scrolled: u16) { + if rows_scrolled == 0 || self.placements.is_empty() { + return; + } + + let mut changed = false; + self.placements.retain(|_, placement| { + let new_row = placement.row as i64 - rows_scrolled as i64; + if new_row + placement.rows as i64 <= 0 { + changed = true; + return false; + } + let row = new_row.max(0) as u16; + changed |= row != placement.row; + placement.row = row; + true + }); + self.dirty |= changed; + } + + fn apply_registration( + &mut self, + chunk: BitmapRegisterChunk, + ) -> Result<(), BitmapProtocolError> { + if self.bitmaps.contains_key(&chunk.bitmap_id) { + return Err(BitmapProtocolError::new("bitmap ID is already registered")); + } + + let bitmap_id = chunk.bitmap_id; + let mut pending = match self.pending_registrations.remove(&bitmap_id) { + Some(pending) => { + if chunk + .format + .as_deref() + .is_some_and(|value| value != pending.format) + || chunk + .source + .as_deref() + .is_some_and(|value| value != pending.source) + || chunk + .name + .as_ref() + .is_some_and(|value| Some(value) != pending.name.as_ref()) + { + self.pending_registrations.insert(bitmap_id, pending); + return Err(BitmapProtocolError::new( + "registration continuation metadata does not match", + )); + } + pending + } + None => PendingBitmapTransfer { + format: chunk + .format + .clone() + .filter(|value| value == "png") + .ok_or_else(|| { + BitmapProtocolError::new("first registration chunk requires PNG format") + })?, + source: chunk + .source + .clone() + .filter(|value| value == "payload") + .ok_or_else(|| { + BitmapProtocolError::new("first registration chunk requires payload source") + })?, + name: chunk.name.clone(), + data: Vec::new(), + }, + }; + + pending + .data + .len() + .checked_add(chunk.data.len()) + .filter(|length| *length <= MAX_REGISTRATION_BYTES) + .ok_or_else(|| { + BitmapProtocolError::new("bitmap registration payload exceeds 64 MiB") + })?; + pending.data.extend_from_slice(&chunk.data); + + if chunk.more { + self.pending_registrations.insert(bitmap_id, pending); + return Ok(()); + } + + let decoded = image::load_from_memory_with_format(&pending.data, image::ImageFormat::Png) + .map_err(|_| BitmapProtocolError::new("invalid PNG bitmap payload"))? + .to_rgba8(); + let (width, height) = decoded.dimensions(); + self.bitmaps.insert( + bitmap_id, + RegisteredBitmap { + width, + height, + rgba: decoded.into_raw(), + handle: None, + }, + ); + self.dirty = true; + Ok(()) + } + + fn apply_placement(&mut self, placement: BitmapPlacement) -> Result<(), BitmapProtocolError> { + if self.placements.contains_key(&placement.placement_id) { + return Err(BitmapProtocolError::new("placement ID is already in use")); + } + let bitmap = self + .bitmaps + .get(&placement.bitmap_id) + .ok_or_else(|| BitmapProtocolError::new("placement bitmap is not registered"))?; + if placement.columns == 0 || placement.rows == 0 { + return Err(BitmapProtocolError::new( + "placement dimensions must be nonzero", + )); + } + let opacity = validate_opacity(placement.opacity)?; + let source = clamp_source(placement.source, bitmap.width, bitmap.height)?; + let generation = self + .next_placement_generation + .checked_add(1) + .ok_or_else(|| BitmapProtocolError::new("bitmap placement generation overflow"))?; + self.next_placement_generation = generation; + self.placements.insert( + placement.placement_id, + BitmapPlacementState { + generation, + bitmap_id: placement.bitmap_id, + row: placement.row, + col: placement.col, + columns: placement.columns, + rows: placement.rows, + source, + fit: placement.fit, + filter: placement.filter, + opacity, + }, + ); + self.dirty = true; + Ok(()) + } + + fn apply_placement_update( + &mut self, + placement_id: u32, + update: BitmapPlacementUpdate, + ) -> Result<(), BitmapProtocolError> { + if update == BitmapPlacementUpdate::default() { + return Err(BitmapProtocolError::new( + "placement update contains no fields", + )); + } + if update.columns.is_some() != update.rows.is_some() { + return Err(BitmapProtocolError::new( + "placement width and height must be updated together", + )); + } + let current = self + .placements + .get(&placement_id) + .ok_or_else(|| BitmapProtocolError::new("placement does not exist"))?; + let bitmap = self + .bitmaps + .get(¤t.bitmap_id) + .ok_or_else(|| BitmapProtocolError::new("placement bitmap is not registered"))?; + let mut next = current.clone(); + if let Some(row) = update.row { + next.row = row; + } + if let Some(col) = update.col { + next.col = col; + } + if let (Some(columns), Some(rows)) = (update.columns, update.rows) { + if columns == 0 || rows == 0 { + return Err(BitmapProtocolError::new( + "placement dimensions must be nonzero", + )); + } + next.columns = columns; + next.rows = rows; + } + if let Some(source) = update.source { + next.source = clamp_source(Some(source), bitmap.width, bitmap.height)?; + } + if let Some(fit) = update.fit { + next.fit = fit; + } + if let Some(filter) = update.filter { + next.filter = filter; + } + if let Some(opacity) = update.opacity { + next.opacity = validate_opacity(opacity)?; + } + self.placements.insert(placement_id, next); + self.dirty = true; + Ok(()) + } + + fn apply_frame(&mut self, chunk: BitmapFrameChunk) -> Result<(), BitmapProtocolError> { + let bitmap = self + .bitmaps + .get(&chunk.bitmap_id) + .ok_or_else(|| BitmapProtocolError::new("frame bitmap is not registered"))?; + let bitmap_dimensions = (bitmap.width, bitmap.height); + let latest = self.latest_frame_sequences.get(&chunk.bitmap_id).copied(); + if latest.is_some_and(|latest| chunk.sequence <= latest) { + return Err(BitmapProtocolError::new("stale bitmap frame sequence")); + } + + let bitmap_id = chunk.bitmap_id; + let mut pending = match self.pending_frames.remove(&bitmap_id) { + Some(pending) if chunk.sequence < pending.sequence => { + self.pending_frames.insert(bitmap_id, pending); + return Err(BitmapProtocolError::new("stale bitmap frame sequence")); + } + Some(pending) if chunk.sequence == pending.sequence => { + if chunk + .format + .as_deref() + .is_some_and(|value| value != pending.format) + || chunk.width.is_some_and(|value| value != pending.width) + || chunk.height.is_some_and(|value| value != pending.height) + { + return Err(BitmapProtocolError::new( + "frame continuation metadata does not match", + )); + } + pending + } + Some(previous) => match new_pending_frame(&chunk, bitmap_dimensions) { + Ok(next) => next, + Err(error) => { + self.pending_frames.insert(bitmap_id, previous); + return Err(error); + } + }, + None => new_pending_frame(&chunk, bitmap_dimensions)?, + }; + + let new_len = match pending.data.len().checked_add(chunk.data.len()) { + Some(length) if length <= pending.expected_len => length, + _ => { + return Err(BitmapProtocolError::new( + "bitmap frame payload is too large", + )); + } + }; + pending.data.reserve(new_len - pending.data.len()); + pending.data.extend_from_slice(&chunk.data); + if chunk.more { + self.pending_frames.insert(bitmap_id, pending); + return Ok(()); + } + if pending.data.len() != pending.expected_len { + return Err(BitmapProtocolError::new( + "bitmap frame payload length does not match", + )); + } + + self.bitmaps + .get_mut(&bitmap_id) + .expect("bitmap existence checked above") + .rgba = pending.data; + self.latest_frame_sequences + .insert(bitmap_id, chunk.sequence); + self.dirty = true; + Ok(()) + } + + fn delete_bitmap(&mut self, bitmap_id: u32) { + if !self.bitmaps.contains_key(&bitmap_id) { + return; + } + self.pending_registrations.remove(&bitmap_id); + self.pending_frames.remove(&bitmap_id); + self.latest_frame_sequences.remove(&bitmap_id); + if self.bitmaps.remove(&bitmap_id).is_some() { + self.placements + .retain(|_, placement| placement.bitmap_id != bitmap_id); + self.dirty = true; + } + } + + fn apply_error_cleanup(&mut self, error: &BitmapProtocolError) { + if let Some(BitmapErrorCleanup::DiscardPendingRegistration { bitmap_id }) = + error.cleanup.as_ref() + { + self.pending_registrations.remove(bitmap_id); + return; + } + let Some(BitmapErrorCleanup::DiscardPendingFrame { + bitmap_id, + sequence, + format, + width, + height, + }) = error.cleanup.as_ref() + else { + return; + }; + let should_discard = self.pending_frames.get(bitmap_id).is_some_and(|pending| { + if *sequence < pending.sequence { + return false; + } + if *sequence == pending.sequence { + return true; + } + self.bitmaps.get(bitmap_id).is_some_and(|bitmap| { + format.as_deref() == Some("rgba8") + && *width == Some(bitmap.width) + && *height == Some(bitmap.height) + }) + }); + if should_discard { + self.pending_frames.remove(bitmap_id); + } + } +} + +fn new_pending_frame( + chunk: &BitmapFrameChunk, + bitmap_dimensions: (u32, u32), +) -> Result { + let format = chunk + .format + .clone() + .filter(|value| value == "rgba8") + .ok_or_else(|| BitmapProtocolError::new("first frame chunk requires RGBA8 format"))?; + let width = chunk + .width + .ok_or_else(|| BitmapProtocolError::new("first frame chunk requires width"))?; + let height = chunk + .height + .ok_or_else(|| BitmapProtocolError::new("first frame chunk requires height"))?; + let expected_len = width + .checked_mul(height) + .and_then(|pixels| pixels.checked_mul(4)) + .and_then(|bytes| usize::try_from(bytes).ok()) + .ok_or_else(|| BitmapProtocolError::new("bitmap frame dimensions overflow"))?; + if (width, height) != bitmap_dimensions { + return Err(BitmapProtocolError::new( + "bitmap frame dimensions must match registered bitmap", + )); + } + Ok(PendingBitmapFrame { + sequence: chunk.sequence, + format, + width, + height, + expected_len, + data: Vec::new(), + }) +} + +fn clamp_source( + source: Option, + bitmap_width: u32, + bitmap_height: u32, +) -> Result, BitmapProtocolError> { + let Some(source) = source else { + return Ok(None); + }; + if source.width == 0 || source.height == 0 { + return Err(BitmapProtocolError::new( + "source dimensions must be nonzero", + )); + } + let end_x = source.x.saturating_add(source.width).min(bitmap_width); + let end_y = source.y.saturating_add(source.height).min(bitmap_height); + let start_x = source.x.min(bitmap_width); + let start_y = source.y.min(bitmap_height); + if end_x <= start_x || end_y <= start_y { + return Err(BitmapProtocolError::new( + "source rectangle is outside bitmap bounds", + )); + } + Ok(Some(SourceRect { + x: start_x, + y: start_y, + width: end_x - start_x, + height: end_y - start_y, + })) +} + +fn validate_opacity(opacity: f32) -> Result { + if !opacity.is_finite() { + return Err(BitmapProtocolError::new("bitmap opacity must be finite")); + } + Ok(opacity.clamp(0.0, 1.0)) +} + +#[cfg(test)] +mod tests { + use super::*; + + const SUPPORT_REPLY: &[u8] = b"\x1b_ratty;i;s;v=1;fmt=png;frame=rgba8;payload=1;chunk=1;placement=1;crop=1;fit=contain|cover|fill;filter=nearest|linear;opacity=1\x1b\\"; + const PNG_2X2: &[u8] = &[ + 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, + 0x52, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x02, 0x08, 0x06, 0x00, 0x00, 0x00, 0x72, + 0xb6, 0x0d, 0x24, 0x00, 0x00, 0x00, 0x12, 0x49, 0x44, 0x41, 0x54, 0x78, 0x9c, 0x63, 0xf8, + 0xcf, 0xc0, 0xf0, 0x1f, 0x0c, 0x81, 0x34, 0x18, 0x00, 0x00, 0x49, 0xc8, 0x09, 0xf7, 0xf9, + 0xab, 0xb6, 0x0d, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82, + ]; + const RGBA_2X2: &[u8] = &[ + 255, 0, 0, 255, 0, 255, 0, 255, 0, 0, 255, 255, 255, 255, 255, 255, + ]; + + fn parse(command: &[u8]) -> Result { + consume_sequence(command).expect("bitmap namespace should be consumed") + } + + fn register_chunk(bitmap_id: u32, data: &[u8], more: bool) -> BitmapOperation { + BitmapOperation::Register(BitmapRegisterChunk { + bitmap_id, + format: Some("png".into()), + source: Some("payload".into()), + name: None, + more, + data: data.to_vec(), + }) + } + + fn registration_continuation(bitmap_id: u32, data: &[u8], more: bool) -> BitmapOperation { + BitmapOperation::Register(BitmapRegisterChunk { + bitmap_id, + format: None, + source: None, + name: None, + more, + data: data.to_vec(), + }) + } + + fn placement(bitmap_id: u32, placement_id: u32) -> BitmapOperation { + BitmapOperation::Place(BitmapPlacement { + bitmap_id, + placement_id, + row: 1, + col: 2, + columns: 8, + rows: 4, + source: None, + fit: BitmapFit::Contain, + filter: BitmapFilter::Linear, + opacity: 1.0, + }) + } + + fn frame_chunk(bitmap_id: u32, sequence: u32, data: &[u8], more: bool) -> BitmapOperation { + BitmapOperation::Frame(BitmapFrameChunk { + bitmap_id, + sequence, + format: Some("rgba8".into()), + width: Some(2), + height: Some(2), + more, + data: data.to_vec(), + }) + } + + fn frame_continuation( + bitmap_id: u32, + sequence: u32, + data: &[u8], + more: bool, + ) -> BitmapOperation { + BitmapOperation::Frame(BitmapFrameChunk { + bitmap_id, + sequence, + format: None, + width: None, + height: None, + more, + data: data.to_vec(), + }) + } + + fn registered_state() -> BitmapSurfaceState { + let mut state = BitmapSurfaceState::default(); + state + .apply(register_chunk(1, PNG_2X2, false)) + .expect("valid bitmap test fixture should succeed"); + state.take_dirty(); + state + } + + #[test] + fn accepts_both_string_terminators() { + assert_eq!( + parse(b"\x1b_ratty;i;s\x1b\\").expect("valid bitmap test fixture should succeed"), + BitmapOperation::SupportQuery + ); + assert_eq!( + parse(b"\x1b_ratty;i;s\x9c").expect("valid bitmap test fixture should succeed"), + BitmapOperation::SupportQuery + ); + } + + #[test] + fn returns_exact_support_reply() { + assert_eq!(support_reply(), SUPPORT_REPLY); + } + + #[test] + fn parses_one_shot_registration() { + let operation = parse( + b"\x1b_ratty;i;r;id=42;fmt=png;source=payload;more=0;name=photo.png;aGVsbG8=\x1b\\", + ) + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + operation, + BitmapOperation::Register(BitmapRegisterChunk { + bitmap_id: 42, + format: Some("png".into()), + source: Some("payload".into()), + name: Some("photo.png".into()), + more: false, + data: b"hello".to_vec(), + }) + ); + } + + #[test] + fn parses_registration_continuation_chunk() { + let operation = parse(b"\x1b_ratty;i;r;id=42;more=1;AQID\x9c") + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + operation, + BitmapOperation::Register(BitmapRegisterChunk { + bitmap_id: 42, + format: None, + source: None, + name: None, + more: true, + data: vec![1, 2, 3], + }) + ); + } + + #[test] + fn parses_placement_defaults() { + let operation = parse(b"\x1b_ratty;i;p;id=42;pid=7;row=4;col=2;w=80;h=30\x1b\\") + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + operation, + BitmapOperation::Place(BitmapPlacement { + bitmap_id: 42, + placement_id: 7, + row: 4, + col: 2, + columns: 80, + rows: 30, + source: None, + fit: BitmapFit::Contain, + filter: BitmapFilter::Linear, + opacity: 1.0, + }) + ); + } + + #[test] + fn parses_explicit_placement_fields_and_clamps_opacity() { + let operation = parse(b"\x1b_ratty;i;p;id=42;pid=7;row=4;col=2;w=80;h=30;src_x=3;src_y=5;src_w=20;src_h=10;fit=cover;filter=nearest;opacity=2.5\x1b\\") + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + operation, + BitmapOperation::Place(BitmapPlacement { + bitmap_id: 42, + placement_id: 7, + row: 4, + col: 2, + columns: 80, + rows: 30, + source: Some(SourceRect { + x: 3, + y: 5, + width: 20, + height: 10, + }), + fit: BitmapFit::Cover, + filter: BitmapFilter::Nearest, + opacity: 1.0, + }) + ); + } + + #[test] + fn parses_full_placement_update() { + let operation = parse(b"\x1b_ratty;i;u;pid=7;row=8;col=9;w=40;h=20;src_x=3;src_y=5;src_w=20;src_h=10;fit=fill;filter=nearest;opacity=-0.2\x1b\\") + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + operation, + BitmapOperation::Update { + placement_id: 7, + update: BitmapPlacementUpdate { + row: Some(8), + col: Some(9), + columns: Some(40), + rows: Some(20), + source: Some(SourceRect { + x: 3, + y: 5, + width: 20, + height: 10, + }), + fit: Some(BitmapFit::Fill), + filter: Some(BitmapFilter::Nearest), + opacity: Some(0.0), + }, + } + ); + } + + #[test] + fn parses_one_shot_frame() { + let operation = + parse(b"\x1b_ratty;i;f;id=42;seq=123;fmt=rgba8;w=1;h=1;more=0;AQIDBA==\x1b\\") + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + operation, + BitmapOperation::Frame(BitmapFrameChunk { + bitmap_id: 42, + sequence: 123, + format: Some("rgba8".into()), + width: Some(1), + height: Some(1), + more: false, + data: vec![1, 2, 3, 4], + }) + ); + } + + #[test] + fn parses_frame_continuation_chunk() { + let operation = parse(b"\x1b_ratty;i;f;id=42;seq=123;more=1;AQID\x9c") + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + operation, + BitmapOperation::Frame(BitmapFrameChunk { + bitmap_id: 42, + sequence: 123, + format: None, + width: None, + height: None, + more: true, + data: vec![1, 2, 3], + }) + ); + } + + #[test] + fn parses_both_delete_targets() { + assert_eq!( + parse(b"\x1b_ratty;i;d;pid=7\x1b\\").expect("valid bitmap test fixture should succeed"), + BitmapOperation::DeletePlacement(7) + ); + assert_eq!( + parse(b"\x1b_ratty;i;d;id=42\x1b\\").expect("valid bitmap test fixture should succeed"), + BitmapOperation::DeleteBitmap(42) + ); + } + + #[test] + fn leaves_other_namespaces_unconsumed() { + assert_eq!(consume_sequence(b"\x1b_ratty;g;s\x1b\\"), None); + assert_eq!(consume_sequence(b"plain text"), None); + } + + #[test] + fn consumes_unknown_verbs_as_ignored() { + assert_eq!( + parse(b"\x1b_ratty;i;x;id=1\x1b\\").expect("valid bitmap test fixture should succeed"), + BitmapOperation::Ignored + ); + } + + #[test] + fn rejects_empty_bitmap_verb() { + assert!(parse(b"\x1b_ratty;i;\x1b\\").is_err()); + } + + #[test] + fn rejects_invalid_base64_and_duplicate_keys() { + assert!(parse(b"\x1b_ratty;i;r;id=1;fmt=png;source=payload;more=0;%%%\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;p;id=1;id=2;pid=3;row=0;col=0;w=1;h=1\x1b\\").is_err()); + } + + #[test] + fn preflights_registration_payload_size_before_base64_decode() { + let error = parse_sequence_with_payload_limit( + b"\x1b_ratty;i;r;id=1;fmt=png;source=payload;more=0;%%%%\x1b\\", + 2, + ) + .expect_err("three decoded bytes must exceed a two-byte chunk limit"); + + assert_eq!(error.to_string(), CHUNK_PAYLOAD_TOO_LARGE); + assert_eq!( + error.cleanup, + Some(BitmapErrorCleanup::DiscardPendingRegistration { bitmap_id: 1 }) + ); + } + + #[test] + fn rejects_overlong_non_payload_header_before_field_collection() { + let error = parse_sequence_with_limits( + b"\x1b_ratty;i;p;id=1;pid=2;row=0;col=0;w=1;h=1\x1b\\", + MAX_BITMAP_CHUNK_DECODED_BYTES, + 8, + ) + .expect_err("the complete placement content exceeds the test header limit"); + + assert_eq!(error.to_string(), "bitmap APC header exceeds 4 KiB"); + } + + #[test] + fn rejects_register_and_frame_semicolon_amplification_as_header() { + for command in [ + b"\x1b_ratty;i;r;;;;;;;;;;;;payload\x1b\\".as_slice(), + b"\x1b_ratty;i;f;;;;;;;;;;;;payload\x1b\\".as_slice(), + ] { + let error = parse_sequence_with_limits(command, MAX_BITMAP_CHUNK_DECODED_BYTES, 8) + .expect_err("the final payload separator places semicolon runs in the header"); + + assert_eq!(error.to_string(), "bitmap APC header exceeds 4 KiB"); + } + } + + #[test] + fn accepts_header_at_injected_boundary() { + assert_eq!( + parse_sequence_with_limits(b"\x1b_ratty;i;s\x1b\\", MAX_BITMAP_CHUNK_DECODED_BYTES, 1,) + .expect("one-byte support header equals the test limit"), + BitmapOperation::SupportQuery + ); + } + + #[test] + fn preflights_frame_payload_size_with_sequence_cleanup_metadata() { + let error = parse_sequence_with_payload_limit( + b"\x1b_ratty;i;f;id=7;seq=9;fmt=rgba8;w=1;h=1;more=0;%%%%\x1b\\", + 2, + ) + .expect_err("three decoded bytes must exceed a two-byte chunk limit"); + + assert_eq!(error.to_string(), CHUNK_PAYLOAD_TOO_LARGE); + assert_eq!( + error.cleanup, + Some(BitmapErrorCleanup::DiscardPendingFrame { + bitmap_id: 7, + sequence: 9, + format: Some("rgba8".into()), + width: Some(1), + height: Some(1), + }) + ); + } + + #[test] + fn oversized_frame_chunk_preflight_discards_matching_pending_sequence() { + let mut state = registered_state(); + state + .apply(frame_chunk(1, 9, &[1, 2], true)) + .expect("first frame chunk should remain pending"); + let error = + parse_sequence_with_payload_limit(b"\x1b_ratty;i;f;id=1;seq=9;more=0;%%%%\x1b\\", 2) + .expect_err("estimated decoded payload exceeds the test chunk limit"); + + state.apply_error_cleanup(&error); + + assert!(!state.pending_frames.contains_key(&1)); + } + + #[test] + fn rejects_missing_ids() { + assert!(parse(b"\x1b_ratty;i;r;fmt=png;source=payload;more=0;AQ==\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;p;id=1;row=0;col=0;w=1;h=1\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;u;row=1\x1b\\").is_err()); + } + + #[test] + fn rejects_zero_dimensions_and_non_finite_opacity() { + assert!(parse(b"\x1b_ratty;i;p;id=1;pid=2;row=0;col=0;w=0;h=1\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;u;pid=2;w=1;h=0\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;p;id=1;pid=2;row=0;col=0;w=1;h=1;opacity=NaN\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;u;pid=2;opacity=inf\x1b\\").is_err()); + } + + #[test] + fn rejects_partial_source_and_destination_groups() { + assert!(parse(b"\x1b_ratty;i;p;id=1;pid=2;row=0;col=0;w=1;h=1;src_x=0\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;u;pid=2;src_x=0;src_y=0;src_w=1\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;u;pid=2;w=1\x1b\\").is_err()); + } + + #[test] + fn rejects_missing_first_frame_metadata() { + assert!(parse(b"\x1b_ratty;i;f;id=1;seq=2;fmt=rgba8;w=1;more=0;AQIDBA==\x1b\\").is_err()); + assert!( + parse(b"\x1b_ratty;i;f;id=1;seq=2;fmt=rgba8;w=0;h=1;more=0;AQIDBA==\x1b\\").is_err() + ); + } + + #[test] + fn rejects_empty_update_and_ambiguous_delete() { + assert!(parse(b"\x1b_ratty;i;u;pid=2\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;d\x1b\\").is_err()); + assert!(parse(b"\x1b_ratty;i;d;id=1;pid=2\x1b\\").is_err()); + } + + #[test] + fn rejects_bad_terminator_inside_bitmap_namespace() { + assert!(parse(b"\x1b_ratty;i;s").is_err()); + assert!(parse(b"\x1b_ratty;i;s\x1bX").is_err()); + } + + #[test] + fn decodes_one_shot_png_registration_and_replies_only_to_support() { + let mut state = BitmapSurfaceState::default(); + + assert_eq!( + state + .apply(BitmapOperation::SupportQuery) + .expect("valid bitmap test fixture should succeed"), + Some(support_reply()) + ); + assert!(!state.is_dirty()); + assert_eq!( + state + .apply(register_chunk(1, PNG_2X2, false)) + .expect("valid bitmap test fixture should succeed"), + None + ); + + let bitmap = state + .bitmap(1) + .expect("valid bitmap test fixture should succeed"); + assert_eq!((bitmap.width, bitmap.height), (2, 2)); + assert_eq!(bitmap.rgba, RGBA_2X2); + assert!(bitmap.handle.is_none()); + assert!(state.is_dirty()); + } + + #[test] + fn assembles_chunked_png_and_requires_matching_metadata() { + let mut state = BitmapSurfaceState::default(); + let split = 31; + state + .apply(register_chunk(1, &PNG_2X2[..split], true)) + .expect("valid bitmap test fixture should succeed"); + let mismatched = BitmapOperation::Register(BitmapRegisterChunk { + bitmap_id: 1, + format: Some("png".into()), + source: Some("payload".into()), + name: Some("different.png".into()), + more: true, + data: vec![9], + }); + assert!(state.apply(mismatched).is_err()); + state + .apply(registration_continuation(1, &PNG_2X2[split..], false)) + .expect("valid bitmap test fixture should succeed"); + + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + RGBA_2X2 + ); + assert!(state.pending_registrations.is_empty()); + } + + #[test] + fn invalid_png_and_registration_overflow_discard_pending_transfer() { + let mut state = BitmapSurfaceState::default(); + state + .apply(register_chunk(1, b"not ", true)) + .expect("valid bitmap test fixture should succeed"); + assert!( + state + .apply(registration_continuation(1, b"png", false)) + .is_err() + ); + assert!(!state.pending_registrations.contains_key(&1)); + assert!(state.bitmap(1).is_none()); + + let oversized = vec![0; MAX_REGISTRATION_BYTES + 1]; + assert!(state.apply(register_chunk(2, &oversized, true)).is_err()); + assert!(!state.pending_registrations.contains_key(&2)); + assert!(state.bitmap(2).is_none()); + } + + #[test] + fn duplicate_bitmap_id_does_not_mutate_existing_state() { + let mut state = registered_state(); + let before = state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba + .clone(); + + assert!( + state + .apply(register_chunk(1, b"replacement", false)) + .is_err() + ); + + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + before + ); + assert!(!state.is_dirty()); + } + + #[test] + fn placement_requires_bitmap_supports_siblings_and_rejects_duplicate_id() { + let mut state = registered_state(); + assert!(state.apply(placement(99, 10)).is_err()); + state + .apply(placement(1, 10)) + .expect("valid bitmap test fixture should succeed"); + state + .apply(placement(1, 11)) + .expect("valid bitmap test fixture should succeed"); + let before = state + .placement(10) + .expect("valid bitmap test fixture should succeed") + .clone(); + + assert!(state.apply(placement(1, 10)).is_err()); + + assert_eq!(state.placements().count(), 2); + assert_eq!(state.placement(10), Some(&before)); + } + + #[test] + fn placement_generations_advance_only_for_successful_new_lifetimes() { + let mut state = registered_state(); + state + .apply(placement(1, 10)) + .expect("valid bitmap test fixture should succeed"); + let first_generation = state + .placement(10) + .expect("valid bitmap test fixture should succeed") + .generation(); + + state + .apply(BitmapOperation::Update { + placement_id: 10, + update: BitmapPlacementUpdate { + opacity: Some(0.5), + ..BitmapPlacementUpdate::default() + }, + }) + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + state + .placement(10) + .expect("valid bitmap test fixture should succeed") + .generation(), + first_generation + ); + assert!(state.apply(placement(1, 10)).is_err()); + + state + .apply(BitmapOperation::DeletePlacement(10)) + .expect("valid bitmap test fixture should succeed"); + state + .apply(placement(1, 10)) + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + state + .placement(10) + .expect("valid bitmap test fixture should succeed") + .generation(), + first_generation + 1 + ); + } + + #[test] + fn placement_clamps_source_rect_and_rejects_empty_intersection() { + let mut state = registered_state(); + let mut clamped = match placement(1, 10) { + BitmapOperation::Place(value) => value, + _ => unreachable!(), + }; + clamped.source = Some(SourceRect { + x: 1, + y: 1, + width: u32::MAX, + height: 20, + }); + state + .apply(BitmapOperation::Place(clamped)) + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + state + .placement(10) + .expect("valid bitmap test fixture should succeed") + .source, + Some(SourceRect { + x: 1, + y: 1, + width: 1, + height: 1 + }) + ); + + let mut empty = match placement(1, 11) { + BitmapOperation::Place(value) => value, + _ => unreachable!(), + }; + empty.source = Some(SourceRect { + x: 2, + y: 0, + width: 1, + height: 1, + }); + assert!(state.apply(BitmapOperation::Place(empty)).is_err()); + assert!(state.placement(11).is_none()); + } + + #[test] + fn placement_update_is_transactional_and_clamps_opacity() { + let mut state = registered_state(); + state + .apply(placement(1, 10)) + .expect("valid bitmap test fixture should succeed"); + state.take_dirty(); + let before = state + .placement(10) + .expect("valid bitmap test fixture should succeed") + .clone(); + + let invalid = BitmapPlacementUpdate { + row: Some(9), + columns: Some(0), + rows: Some(5), + ..Default::default() + }; + assert!( + state + .apply(BitmapOperation::Update { + placement_id: 10, + update: invalid + }) + .is_err() + ); + assert_eq!(state.placement(10), Some(&before)); + assert!(!state.is_dirty()); + + let valid = BitmapPlacementUpdate { + row: Some(9), + opacity: Some(4.0), + ..Default::default() + }; + state + .apply(BitmapOperation::Update { + placement_id: 10, + update: valid, + }) + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + state + .placement(10) + .expect("valid bitmap test fixture should succeed") + .row, + 9 + ); + assert_eq!( + state + .placement(10) + .expect("valid bitmap test fixture should succeed") + .opacity, + 1.0 + ); + assert!( + state + .apply(BitmapOperation::Update { + placement_id: 404, + update: Default::default() + }) + .is_err() + ); + } + + #[test] + fn applies_one_shot_and_chunked_rgba8_frames() { + let mut state = registered_state(); + let first = [7; 16]; + state + .apply(frame_chunk(1, 1, &first, false)) + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + first + ); + + let second = [8; 16]; + state + .apply(frame_chunk(1, 2, &second[..7], true)) + .expect("valid bitmap test fixture should succeed"); + state + .apply(frame_continuation(1, 2, &second[7..], false)) + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + second + ); + assert!(state.pending_frames.is_empty()); + } + + #[test] + fn frame_validates_exact_length_dimensions_and_checked_expected_size() { + let mut state = registered_state(); + let original = state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba + .clone(); + assert!(state.apply(frame_chunk(1, 1, &[1; 15], false)).is_err()); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + original + ); + + let mut wrong_dimensions = match frame_chunk(1, 2, &[2; 16], false) { + BitmapOperation::Frame(value) => value, + _ => unreachable!(), + }; + wrong_dimensions.width = Some(3); + assert!( + state + .apply(BitmapOperation::Frame(wrong_dimensions)) + .is_err() + ); + + let overflow = BitmapFrameChunk { + bitmap_id: 1, + sequence: 3, + format: Some("rgba8".into()), + width: Some(u32::MAX), + height: Some(u32::MAX), + more: true, + data: vec![], + }; + assert!(state.apply(BitmapOperation::Frame(overflow)).is_err()); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + original + ); + } + + #[test] + fn newer_frame_cancels_incomplete_older_and_rejects_stale_sequences() { + let mut state = registered_state(); + state + .apply(frame_chunk(1, 10, &[1; 4], true)) + .expect("valid bitmap test fixture should succeed"); + state + .apply(frame_chunk(1, 11, &[2; 16], false)) + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + [2; 16] + ); + assert!( + state + .apply(frame_continuation(1, 10, &[1; 12], false)) + .is_err() + ); + assert!(state.apply(frame_chunk(1, 11, &[3; 16], false)).is_err()); + assert!(state.apply(frame_chunk(1, 9, &[4; 16], false)).is_err()); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + [2; 16] + ); + } + + #[test] + fn malformed_newer_frame_does_not_cancel_an_incomplete_valid_frame() { + let mut state = registered_state(); + state + .apply(frame_chunk(1, 10, &[1; 4], true)) + .expect("valid bitmap test fixture should succeed"); + + assert!( + state + .apply(frame_continuation(1, 11, &[2; 4], true)) + .is_err() + ); + state + .apply(frame_continuation(1, 10, &[1; 12], false)) + .expect("valid bitmap test fixture should succeed"); + + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + [1; 16] + ); + } + + #[test] + fn invalid_base64_continuation_discards_matching_pending_frame() { + let mut state = registered_state(); + let original = state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba + .clone(); + state + .apply(frame_chunk(1, 10, &[1; 4], true)) + .expect("valid bitmap test fixture should succeed"); + + let result = state + .consume_and_apply(b"\x1b_ratty;i;f;id=1;seq=10;more=0;%%%\x1b\\") + .expect("valid bitmap test fixture should succeed"); + + assert!(result.is_err()); + assert!(!state.pending_frames.contains_key(&1)); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + original + ); + } + + #[test] + fn invalid_base64_newer_first_chunk_cancels_older_pending_frame() { + let mut state = registered_state(); + let original = state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba + .clone(); + state + .apply(frame_chunk(1, 10, &[1; 4], true)) + .expect("valid bitmap test fixture should succeed"); + + let result = state + .consume_and_apply(b"\x1b_ratty;i;f;id=1;seq=11;fmt=rgba8;w=2;h=2;more=0;%%%\x1b\\") + .expect("valid bitmap test fixture should succeed"); + + assert!(result.is_err()); + assert!(!state.pending_frames.contains_key(&1)); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + original + ); + } + + #[test] + fn invalid_base64_newer_chunk_without_metadata_preserves_older_pending_frame() { + let mut state = registered_state(); + state + .apply(frame_chunk(1, 10, &[1; 4], true)) + .expect("valid bitmap test fixture should succeed"); + + let result = state + .consume_and_apply(b"\x1b_ratty;i;f;id=1;seq=11;more=0;%%%\x1b\\") + .expect("valid bitmap test fixture should succeed"); + + assert!(result.is_err()); + assert_eq!( + state + .pending_frames + .get(&1) + .expect("valid bitmap test fixture should succeed") + .sequence, + 10 + ); + } + + #[test] + fn invalid_base64_newer_chunk_with_wrong_dimensions_preserves_older_pending_frame() { + let mut state = registered_state(); + state + .apply(frame_chunk(1, 10, &[1; 4], true)) + .expect("valid bitmap test fixture should succeed"); + + let result = state + .consume_and_apply(b"\x1b_ratty;i;f;id=1;seq=11;fmt=rgba8;w=3;h=2;more=0;%%%\x1b\\") + .expect("valid bitmap test fixture should succeed"); + + assert!(result.is_err()); + assert_eq!( + state + .pending_frames + .get(&1) + .expect("valid bitmap test fixture should succeed") + .sequence, + 10 + ); + } + + #[test] + fn invalid_base64_stale_frame_does_not_cancel_newer_pending_frame() { + let mut state = registered_state(); + state + .apply(frame_chunk(1, 11, &[1; 4], true)) + .expect("valid bitmap test fixture should succeed"); + + let result = state + .consume_and_apply(b"\x1b_ratty;i;f;id=1;seq=10;more=0;%%%\x1b\\") + .expect("valid bitmap test fixture should succeed"); + + assert!(result.is_err()); + assert_eq!( + state + .pending_frames + .get(&1) + .expect("valid bitmap test fixture should succeed") + .sequence, + 11 + ); + } + + #[test] + fn corrupt_frame_discards_transfer_and_preserves_last_valid_pixels() { + let mut state = registered_state(); + let original = state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba + .clone(); + state + .apply(frame_chunk(1, 1, &[1; 12], true)) + .expect("valid bitmap test fixture should succeed"); + assert!( + state + .apply(frame_continuation(1, 1, &[2; 8], true)) + .is_err() + ); + + assert!(!state.pending_frames.contains_key(&1)); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + original + ); + state + .apply(frame_chunk(1, 2, &[3; 16], false)) + .expect("valid bitmap test fixture should succeed"); + assert_eq!( + state + .bitmap(1) + .expect("valid bitmap test fixture should succeed") + .rgba, + [3; 16] + ); + } + + #[test] + fn deletes_are_idempotent_and_bitmap_delete_cascades() { + let mut state = registered_state(); + state + .apply(placement(1, 10)) + .expect("valid bitmap test fixture should succeed"); + state + .apply(placement(1, 11)) + .expect("valid bitmap test fixture should succeed"); + state + .apply(BitmapOperation::DeletePlacement(10)) + .expect("valid bitmap test fixture should succeed"); + assert!(state.placement(10).is_none()); + assert!(state.placement(11).is_some()); + assert!(state.bitmap(1).is_some()); + + state + .apply(BitmapOperation::DeletePlacement(404)) + .expect("valid bitmap test fixture should succeed"); + state + .apply(BitmapOperation::DeleteBitmap(404)) + .expect("valid bitmap test fixture should succeed"); + state + .apply(BitmapOperation::Ignored) + .expect("valid bitmap test fixture should succeed"); + state + .apply(BitmapOperation::DeleteBitmap(1)) + .expect("valid bitmap test fixture should succeed"); + assert!(state.bitmap(1).is_none()); + assert!(state.placement(11).is_none()); + assert_eq!(state.bitmaps().count(), 0); + assert_eq!(state.placements().count(), 0); + } + + #[test] + fn deleting_unknown_bitmap_preserves_its_pending_registration() { + let mut state = BitmapSurfaceState::default(); + state + .apply(register_chunk(42, &PNG_2X2[..20], true)) + .expect("first registration chunk should remain pending"); + + state + .apply(BitmapOperation::DeleteBitmap(42)) + .expect("deleting an unknown bitmap is a no-op"); + + assert!(state.pending_registrations.contains_key(&42)); + state + .apply(registration_continuation(42, &PNG_2X2[20..], false)) + .expect("pending registration should still be completable"); + assert!(state.bitmap(42).is_some()); + } +} diff --git a/src/lib.rs b/src/lib.rs index dec3810..69381b1 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -7,6 +7,7 @@ #![warn(clippy::unwrap_used)] pub mod camera; +pub mod bitmap; pub mod cli; pub mod config; mod direct_render; From fdbb44efca3c77f3444b96b05c976b64787b561f Mon Sep 17 00:00:00 2001 From: wipesides Date: Thu, 16 Jul 2026 16:35:41 +0300 Subject: [PATCH 03/10] feat(bitmap): render and synchronize bitmap surfaces --- src/bitmap_material.rs | 260 +++++++++ src/inline.rs | 404 +++++++++++++- src/lib.rs | 3 +- src/plugin.rs | 22 +- src/shaders/bitmap_surface.wgsl | 59 ++ src/systems.rs | 919 +++++++++++++++++++++++++++++++- 6 files changed, 1654 insertions(+), 13 deletions(-) create mode 100644 src/bitmap_material.rs create mode 100644 src/shaders/bitmap_surface.wgsl diff --git a/src/bitmap_material.rs b/src/bitmap_material.rs new file mode 100644 index 0000000..11c9e3d --- /dev/null +++ b/src/bitmap_material.rs @@ -0,0 +1,260 @@ +//! Rendering material and layout math for bitmap-surface placements. + +use bevy::asset::uuid_handle; +use bevy::math::{UVec2, Vec2}; +use bevy::prelude::{Asset, Handle, Image, TypePath}; +use bevy::render::render_resource::{AsBindGroup, ShaderType}; +use bevy::shader::{Shader, ShaderRef}; +use bevy::sprite_render::{AlphaMode2d, Material2d}; + +use crate::bitmap::{BitmapFit, SourceRect}; + +/// Handle for the embedded bitmap-surface shader. +pub(crate) const BITMAP_SURFACE_SHADER: Handle = + uuid_handle!("226f825e-49b6-4cae-bfc4-a03efadfb255"); + +/// A material for one bitmap placement. +/// +/// Placements keep independent parameters while sharing the same `Image` +/// handle, including when their filtering modes differ. +#[derive(Asset, TypePath, AsBindGroup, Clone)] +pub struct BitmapSurfaceMaterial { + /// Shared bitmap image. + #[texture(0)] + pub image: Handle, + /// Crop, fit, filtering, and opacity parameters for this placement. + #[uniform(1)] + pub params: BitmapSurfaceUniform, +} + +/// Shader parameters resolved for one bitmap placement. +#[derive(Clone, Copy, Debug, PartialEq, ShaderType)] +pub struct BitmapSurfaceUniform { + /// Top-left normalized source coordinate. + pub uv_min: Vec2, + /// Bottom-right normalized source coordinate. + pub uv_max: Vec2, + /// Placement opacity in the inclusive range `[0, 1]`. + pub opacity: f32, + /// `0` for nearest-neighbor filtering or `1` for linear filtering. + pub filter_mode: u32, + /// Top-left normalized destination content bound. + pub content_min: Vec2, + /// Bottom-right normalized destination content bound. + pub content_max: Vec2, +} + +impl Material2d for BitmapSurfaceMaterial { + fn fragment_shader() -> ShaderRef { + BITMAP_SURFACE_SHADER.into() + } + + fn alpha_mode(&self) -> AlphaMode2d { + AlphaMode2d::Blend + } +} + +/// Resolved normalized crop and destination-content bounds. +#[derive(Clone, Copy, Debug, Default, PartialEq)] +pub struct ResolvedBitmapLayout { + /// Top-left normalized source coordinate. + pub uv_min: Vec2, + /// Bottom-right normalized source coordinate. + pub uv_max: Vec2, + /// Top-left normalized destination content bound. + pub content_min: Vec2, + /// Bottom-right normalized destination content bound. + pub content_max: Vec2, +} + +/// Resolves source cropping and fit into normalized shader coordinates. +/// +/// Source rectangles are clamped to the bitmap bounds. Protocol state rejects +/// empty sources and destinations before calling this function; zero-sized +/// input defensively resolves to an empty layout. +pub fn resolve_bitmap_layout( + bitmap_size: UVec2, + source: Option, + destination_pixels: Vec2, + fit: BitmapFit, +) -> ResolvedBitmapLayout { + if bitmap_size.x == 0 + || bitmap_size.y == 0 + || destination_pixels.x <= 0.0 + || destination_pixels.y <= 0.0 + { + return ResolvedBitmapLayout::default(); + } + + let (source_min, source_max) = clamped_source(bitmap_size, source); + let source_size = source_max - source_min; + if source_size.x <= 0.0 || source_size.y <= 0.0 { + return ResolvedBitmapLayout::default(); + } + + let bitmap_size = bitmap_size.as_vec2(); + let mut layout = ResolvedBitmapLayout { + uv_min: source_min / bitmap_size, + uv_max: source_max / bitmap_size, + content_min: Vec2::ZERO, + content_max: Vec2::ONE, + }; + + match fit { + BitmapFit::Fill => {} + BitmapFit::Contain => { + let scale = + (destination_pixels.x / source_size.x).min(destination_pixels.y / source_size.y); + let content_size = source_size * scale / destination_pixels; + layout.content_min = (Vec2::ONE - content_size) * 0.5; + layout.content_max = layout.content_min + content_size; + } + BitmapFit::Cover => { + let scale = + (destination_pixels.x / source_size.x).max(destination_pixels.y / source_size.y); + let visible_source_size = destination_pixels / scale; + let crop = (source_size - visible_source_size) * 0.5; + layout.uv_min = (source_min + crop) / bitmap_size; + layout.uv_max = (source_max - crop) / bitmap_size; + } + } + + layout +} + +fn clamped_source(bitmap_size: UVec2, source: Option) -> (Vec2, Vec2) { + let Some(source) = source else { + return (Vec2::ZERO, bitmap_size.as_vec2()); + }; + + let min_x = source.x.min(bitmap_size.x); + let min_y = source.y.min(bitmap_size.y); + let max_x = source.x.saturating_add(source.width).min(bitmap_size.x); + let max_y = source.y.saturating_add(source.height).min(bitmap_size.y); + ( + Vec2::new(min_x as f32, min_y as f32), + Vec2::new(max_x as f32, max_y as f32), + ) +} + +#[cfg(test)] +mod tests { + use bevy::math::{UVec2, Vec2}; + + use super::*; + use crate::bitmap::{BitmapFit, SourceRect}; + + fn assert_vec2(actual: Vec2, expected: Vec2) { + assert!( + actual.abs_diff_eq(expected, 1.0e-6), + "expected {expected:?}, got {actual:?}" + ); + } + + #[test] + fn fill_maps_the_selected_source_to_the_entire_destination() { + let layout = resolve_bitmap_layout( + UVec2::new(400, 200), + None, + Vec2::new(300.0, 300.0), + BitmapFit::Fill, + ); + + assert_vec2(layout.uv_min, Vec2::ZERO); + assert_vec2(layout.uv_max, Vec2::ONE); + assert_vec2(layout.content_min, Vec2::ZERO); + assert_vec2(layout.content_max, Vec2::ONE); + } + + #[test] + fn contain_letterboxes_landscape_content_vertically() { + let layout = resolve_bitmap_layout( + UVec2::new(400, 200), + None, + Vec2::new(300.0, 300.0), + BitmapFit::Contain, + ); + + assert_vec2(layout.uv_min, Vec2::ZERO); + assert_vec2(layout.uv_max, Vec2::ONE); + assert_vec2(layout.content_min, Vec2::new(0.0, 0.25)); + assert_vec2(layout.content_max, Vec2::new(1.0, 0.75)); + } + + #[test] + fn contain_letterboxes_portrait_content_horizontally() { + let layout = resolve_bitmap_layout( + UVec2::new(200, 400), + None, + Vec2::new(300.0, 300.0), + BitmapFit::Contain, + ); + + assert_vec2(layout.content_min, Vec2::new(0.25, 0.0)); + assert_vec2(layout.content_max, Vec2::new(0.75, 1.0)); + } + + #[test] + fn cover_crops_landscape_content_symmetrically() { + let layout = resolve_bitmap_layout( + UVec2::new(400, 200), + None, + Vec2::new(300.0, 300.0), + BitmapFit::Cover, + ); + + assert_vec2(layout.uv_min, Vec2::new(0.25, 0.0)); + assert_vec2(layout.uv_max, Vec2::new(0.75, 1.0)); + assert_vec2(layout.content_min, Vec2::ZERO); + assert_vec2(layout.content_max, Vec2::ONE); + } + + #[test] + fn cover_crops_portrait_content_symmetrically() { + let layout = resolve_bitmap_layout( + UVec2::new(200, 400), + None, + Vec2::new(300.0, 300.0), + BitmapFit::Cover, + ); + + assert_vec2(layout.uv_min, Vec2::new(0.0, 0.25)); + assert_vec2(layout.uv_max, Vec2::new(1.0, 0.75)); + } + + #[test] + fn explicit_crop_is_normalized_against_the_full_bitmap() { + let layout = resolve_bitmap_layout( + UVec2::new(400, 200), + Some(SourceRect { + x: 100, + y: 50, + width: 200, + height: 100, + }), + Vec2::new(200.0, 100.0), + BitmapFit::Fill, + ); + + assert_vec2(layout.uv_min, Vec2::new(0.25, 0.25)); + assert_vec2(layout.uv_max, Vec2::new(0.75, 0.75)); + } + + #[test] + fn source_crop_is_clamped_to_bitmap_bounds() { + let layout = resolve_bitmap_layout( + UVec2::new(400, 200), + Some(SourceRect { + x: 300, + y: 100, + width: 500, + height: 500, + }), + Vec2::new(100.0, 100.0), + BitmapFit::Fill, + ); + + assert_vec2(layout.uv_min, Vec2::new(0.75, 0.5)); + assert_vec2(layout.uv_max, Vec2::ONE); + } +} diff --git a/src/inline.rs b/src/inline.rs index a6d6e45..a1440dd 100644 --- a/src/inline.rs +++ b/src/inline.rs @@ -5,6 +5,8 @@ use std::path::Path; use bevy::prelude::*; +use crate::bitmap::{BitmapSurfaceState, MAX_BITMAP_APC_BYTES}; +use crate::bitmap_material::{BitmapSurfaceMaterial, BitmapSurfaceUniform}; use crate::camera::{OptionalVec3, TerminalCameraUpdate}; use crate::kitty::{KittyOperation, KittyParserState, refresh_kitty_placeholder_anchors}; use crate::model::{ @@ -65,12 +67,59 @@ pub struct TerminalRgpObject { pub object_id: u32, } +/// Marker identifying one rendered bitmap-surface placement. +#[derive(Component, Clone, Copy, Debug, PartialEq, Eq)] +pub struct TerminalBitmapPlacement { + /// Globally unique placement identifier. + pub placement_id: u32, + /// Registered bitmap identifier shared by this placement. + pub bitmap_id: u32, +} + +/// Stable Bevy assets owned by one bitmap placement. +pub(crate) struct BitmapPlacementRenderCache { + /// Placement lifetime rendered by this cache entry. + pub(crate) generation: u64, + /// Registered bitmap used by the placement. + pub(crate) bitmap_id: u32, + /// Stable render entity. + pub(crate) entity: Entity, + /// Stable destination quad mesh. + pub(crate) mesh: Handle, + /// Stable per-placement material. + pub(crate) material: Handle, + /// Last values synchronized into the stable render objects. + pub(crate) state: BitmapPlacementRenderState, +} + +/// Render-facing placement values used to avoid dirtying unchanged Bevy assets. +#[derive(Clone)] +pub(crate) struct BitmapPlacementRenderState { + pub(crate) image: Handle, + pub(crate) destination: Vec2, + pub(crate) transform: Transform, + pub(crate) uniform: BitmapSurfaceUniform, +} + +/// Renderer-side identities retained across protocol updates. +#[derive(Default)] +pub(crate) struct BitmapRenderCache { + /// Stable image handles keyed by bitmap ID. + pub(crate) images: HashMap>, + /// Stable entity and material handles keyed by placement ID. + pub(crate) placements: HashMap, +} + /// Inline object registry and anchor state. #[derive(Resource, Default)] pub struct TerminalInlineObjects { pending_bytes: Vec, + discarding_oversized_bitmap_apc: bool, + bitmap_discard_saw_escape: bool, pending_rgp_payloads: HashMap, kitty: KittyParserState, + pub(crate) bitmap: BitmapSurfaceState, + pub(crate) bitmap_render: BitmapRenderCache, dirty: bool, last_viewport_size: Vec2, last_cols: u16, @@ -88,7 +137,89 @@ impl TerminalInlineObjects { camera_updates: &mut Vec, terminal_output: &mut bool, ) -> Vec> { - self.pending_bytes.extend_from_slice(chunk); + self.consume_pty_output_with_limit( + chunk, + parser, + camera_updates, + terminal_output, + MAX_BITMAP_APC_BYTES, + ) + } + + #[cfg(test)] + fn consume_pty_output_with_bitmap_limit( + &mut self, + chunk: &[u8], + parser: &mut vt100::Parser, + bitmap_apc_limit: usize, + ) -> Vec> { + let mut camera_updates = Vec::new(); + let mut terminal_output = false; + self.consume_pty_output_with_limit( + chunk, + parser, + &mut camera_updates, + &mut terminal_output, + bitmap_apc_limit, + ) + } + + fn consume_pty_output_with_limit( + &mut self, + mut chunk: &[u8], + parser: &mut vt100::Parser, + camera_updates: &mut Vec, + terminal_output: &mut bool, + bitmap_apc_limit: usize, + ) -> Vec> { + const INGEST_BLOCK_BYTES: usize = 64 * 1024; + + let mut replies = Vec::new(); + while !chunk.is_empty() { + if self.discarding_oversized_bitmap_apc { + let Some(consumed) = self.discard_oversized_bitmap_bytes(chunk) else { + return replies; + }; + self.discarding_oversized_bitmap_apc = false; + self.bitmap_discard_saw_escape = false; + chunk = &chunk[consumed..]; + continue; + } + + let block_limit = if self + .pending_bytes + .starts_with(crate::bitmap::BITMAP_APC_START) + { + bitmap_apc_limit.saturating_sub(self.pending_bytes.len()) + } else { + INGEST_BLOCK_BYTES.min(bitmap_apc_limit.max(1)) + }; + if block_limit == 0 { + self.begin_oversized_bitmap_discard(); + continue; + } + let take = chunk.len().min(INGEST_BLOCK_BYTES).min(block_limit); + self.pending_bytes.extend_from_slice(&chunk[..take]); + chunk = &chunk[take..]; + replies.extend(self.process_pending_bytes(parser, camera_updates, terminal_output)); + + if self + .pending_bytes + .starts_with(crate::bitmap::BITMAP_APC_START) + && self.pending_bytes.len() >= bitmap_apc_limit + { + self.begin_oversized_bitmap_discard(); + } + } + replies + } + + fn process_pending_bytes( + &mut self, + parser: &mut vt100::Parser, + camera_updates: &mut Vec, + terminal_output: &mut bool, + ) -> Vec> { let mut replies = Vec::new(); let mut cursor = 0; @@ -138,9 +269,30 @@ impl TerminalInlineObjects { } } + fn begin_oversized_bitmap_discard(&mut self) { + warn!("discarding oversized Ratty Bitmap Surface APC sequence"); + self.bitmap_discard_saw_escape = self.pending_bytes.last() == Some(&ST[0]); + self.pending_bytes.clear(); + self.discarding_oversized_bitmap_apc = true; + } + + fn discard_oversized_bitmap_bytes(&mut self, bytes: &[u8]) -> Option { + for (index, byte) in bytes.iter().copied().enumerate() { + if self.bitmap_discard_saw_escape && byte == ST[1] { + return Some(index + 1); + } + if byte == C1_ST { + return Some(index + 1); + } + self.bitmap_discard_saw_escape = byte == ST[0]; + } + None + } + /// Returns whether inline objects need synchronization. pub fn needs_sync(&self, viewport_size: Vec2, cols: u16, rows: u16) -> bool { self.dirty + || self.bitmap.is_dirty() || self.last_viewport_size != viewport_size || self.last_cols != cols || self.last_rows != rows @@ -149,14 +301,22 @@ impl TerminalInlineObjects { /// Marks synchronization as complete. pub fn finish_sync(&mut self, viewport_size: Vec2, cols: u16, rows: u16) { self.dirty = false; + self.bitmap.take_dirty(); self.last_viewport_size = viewport_size; self.last_cols = cols; self.last_rows = rows; } + /// Marks only bitmap protocol changes as synchronized. + pub(crate) fn finish_bitmap_sync(&mut self) { + self.bitmap.take_dirty(); + } + /// Applies upward scroll to anchored objects. pub fn apply_scroll(&mut self, rows_scrolled: u16) { - if rows_scrolled == 0 || self.anchors.is_empty() { + if rows_scrolled == 0 + || (self.anchors.is_empty() && self.bitmap.placements().next().is_none()) + { return; } @@ -175,16 +335,18 @@ impl TerminalInlineObjects { anchor.row = new_row.max(0) as u16; true }); + self.bitmap.apply_scroll(rows_scrolled); self.dirty = true; } /// Returns whether any anchors need scroll tracking. pub fn has_scroll_tracked_anchors(&self) -> bool { - self.anchors.keys().any(|object_id| { - self.objects - .get(object_id) - .is_some_and(InlineObject::scrolls_with_text) - }) + self.bitmap.placements().next().is_some() + || self.anchors.keys().any(|object_id| { + self.objects + .get(object_id) + .is_some_and(InlineObject::scrolls_with_text) + }) } /// Refreshes placeholder-derived Kitty anchors. @@ -219,6 +381,21 @@ impl TerminalInlineObjects { cursor_position: (u16, u16), camera_updates: &mut Vec, ) -> (bool, Option>) { + if let Some(result) = self.bitmap.consume_and_apply(sequence) { + debug!(bytes = sequence.len(), "received bitmap surface command"); + return match result { + Ok(Some(reply)) => { + info!("bitmap support query answered: v1"); + (true, Some(reply)) + } + Ok(None) => (true, None), + Err(error) => { + warn!("failed to apply bitmap surface command: {error}"); + (true, None) + } + }; + } + if let Some(reply) = self.handle_rgp_sequence(sequence, camera_updates) { return (true, reply); } @@ -672,3 +849,216 @@ fn apply_vec3_update(target: &mut Vec3, update: [Option; 3]) { target.z = z; } } + +#[cfg(test)] +mod tests { + use base64::Engine as _; + + use super::*; + + const BITMAP_SUPPORT_REPLY: &[u8] = b"\x1b_ratty;i;s;v=1;fmt=png;frame=rgba8;payload=1;chunk=1;placement=1;crop=1;fit=contain|cover|fill;filter=nearest|linear;opacity=1\x1b\\"; + const PNG_2X2: &[u8] = &[ + 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, + 0x52, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x02, 0x08, 0x06, 0x00, 0x00, 0x00, 0x72, + 0xb6, 0x0d, 0x24, 0x00, 0x00, 0x00, 0x12, 0x49, 0x44, 0x41, 0x54, 0x78, 0x9c, 0x63, 0xf8, + 0xcf, 0xc0, 0xf0, 0x1f, 0x0c, 0x81, 0x34, 0x18, 0x00, 0x00, 0x49, 0xc8, 0x09, 0xf7, 0xf9, + 0xab, 0xb6, 0x0d, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82, + ]; + + fn parser() -> vt100::Parser { + vt100::Parser::new(24, 80, 0) + } + + fn consume( + objects: &mut TerminalInlineObjects, + chunk: &[u8], + parser: &mut vt100::Parser, + ) -> Vec> { + let mut camera_updates = Vec::new(); + let mut terminal_output = false; + objects.consume_pty_output(chunk, parser, &mut camera_updates, &mut terminal_output) + } + + fn register_bitmap(objects: &mut TerminalInlineObjects, parser: &mut vt100::Parser) { + let payload = base64::engine::general_purpose::STANDARD.encode(PNG_2X2); + let command = format!("\x1b_ratty;i;r;id=7;fmt=png;source=payload;more=0;{payload}\x1b\\"); + assert!(consume(objects, command.as_bytes(), parser).is_empty()); + } + + #[test] + fn consumes_bitmap_apc_without_leaking_it_into_mixed_terminal_text() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + + let replies = consume(&mut objects, b"left\x1b_ratty;i;s\x1b\\right", &mut parser); + + assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); + assert_eq!(parser.screen().contents(), "leftright"); + } + + #[test] + fn buffers_fragmented_bitmap_apc_until_its_terminator_arrives() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + + assert!(consume(&mut objects, b"before\x1b_ratty;i;", &mut parser).is_empty()); + assert_eq!(parser.screen().contents(), "before"); + let replies = consume(&mut objects, b"s\x1b\\after", &mut parser); + + assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); + assert_eq!(parser.screen().contents(), "beforeafter"); + } + + #[test] + fn bounds_and_discards_oversized_fragmented_bitmap_apc_then_recovers_after_split_st() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + let limit = 32; + + objects.consume_pty_output_with_bitmap_limit( + b"before\x1b_ratty;i;r;id=1;", + &mut parser, + limit, + ); + objects.consume_pty_output_with_bitmap_limit(b"AAAAAAAAAAAAAAAA", &mut parser, limit); + assert!(objects.pending_bytes.len() <= limit); + assert!(objects.discarding_oversized_bitmap_apc); + + objects.consume_pty_output_with_bitmap_limit(b"discarded\x1b", &mut parser, limit); + let replies = objects.consume_pty_output_with_bitmap_limit( + b"\\after\x1b_ratty;i;s\x1b\\", + &mut parser, + limit, + ); + + assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); + assert_eq!(parser.screen().contents(), "beforeafter"); + assert!(!objects.discarding_oversized_bitmap_apc); + assert!(objects.pending_bytes.len() <= limit); + } + + #[test] + fn discards_oversized_bitmap_apc_until_c1_st_then_recovers() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + let limit = 24; + + objects.consume_pty_output_with_bitmap_limit( + b"\x1b_ratty;i;r;id=1;AAAAAAAA", + &mut parser, + limit, + ); + let replies = objects.consume_pty_output_with_bitmap_limit( + b"discarded\x9ctail\x1b_ratty;i;s\x9c", + &mut parser, + limit, + ); + + assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); + assert_eq!(parser.screen().contents(), "tail"); + assert!(!objects.discarding_oversized_bitmap_apc); + } + + #[test] + fn bitmap_apc_limit_does_not_apply_to_fragmented_rgp_sequences() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + let limit = 8; + + objects.consume_pty_output_with_bitmap_limit(b"\x1b_ratty;g;", &mut parser, limit); + let replies = objects.consume_pty_output_with_bitmap_limit(b"s\x1b\\", &mut parser, limit); + + assert_eq!(replies, vec![crate::rgp::support_reply()]); + assert!(parser.screen().contents().is_empty()); + } + + #[test] + fn accepts_c1_st_for_bitmap_support_query() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + + let replies = consume(&mut objects, b"\x1b_ratty;i;s\x9c", &mut parser); + + assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); + assert!(parser.screen().contents().is_empty()); + } + + #[test] + fn dispatches_adjacent_bitmap_and_rgp_sequences_in_wire_order() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + + let replies = consume( + &mut objects, + b"\x1b_ratty;i;s\x1b\\\x1b_ratty;g;s\x1b\\", + &mut parser, + ); + + assert_eq!( + replies, + vec![BITMAP_SUPPORT_REPLY.to_vec(), crate::rgp::support_reply()] + ); + assert!(parser.screen().contents().is_empty()); + } + + #[test] + fn dispatches_bitmap_before_adjacent_kitty_and_keeps_bitmap_state_isolated() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + register_bitmap(&mut objects, &mut parser); + + let replies = consume( + &mut objects, + b"\x1b_ratty;i;s\x1b\\\x1b_Ga=d;\x1b\\\x1b_ratty;g;d\x1b\\", + &mut parser, + ); + + assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); + assert!(objects.bitmap.bitmap(7).is_some()); + assert!(parser.screen().contents().is_empty()); + } + + #[test] + fn malformed_bitmap_sequences_are_consumed_without_terminal_output() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + + let replies = consume( + &mut objects, + b"before\x1b_ratty;i;p;id=broken\x1b\\after", + &mut parser, + ); + + assert!(replies.is_empty()); + assert_eq!(parser.screen().contents(), "beforeafter"); + } + + #[test] + fn bitmap_placements_participate_in_dirty_and_scroll_tracking() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + register_bitmap(&mut objects, &mut parser); + consume( + &mut objects, + b"\x1b_ratty;i;p;id=7;pid=9;row=5;col=2;w=8;h=3\x1b\\", + &mut parser, + ); + + assert!(objects.needs_sync(Vec2::ZERO, 0, 0)); + assert!(objects.has_scroll_tracked_anchors()); + objects.finish_sync(Vec2::ZERO, 0, 0); + assert!(!objects.needs_sync(Vec2::ZERO, 0, 0)); + + objects.apply_scroll(2); + + assert_eq!( + objects + .bitmap + .placement(9) + .expect("bitmap placement should exist") + .row(), + 3 + ); + assert!(objects.needs_sync(Vec2::ZERO, 0, 0)); + } +} diff --git a/src/lib.rs b/src/lib.rs index 69381b1..b3b3821 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -6,8 +6,9 @@ #![warn(missing_docs)] #![warn(clippy::unwrap_used)] -pub mod camera; pub mod bitmap; +pub mod bitmap_material; +pub mod camera; pub mod cli; pub mod config; mod direct_render; diff --git a/src/plugin.rs b/src/plugin.rs index bd3a1a2..79cfbb1 100644 --- a/src/plugin.rs +++ b/src/plugin.rs @@ -1,7 +1,11 @@ //! Bevy plugin wiring for the terminal application. +use bevy::asset::load_internal_asset; use bevy::prelude::*; +use bevy::shader::Shader; +use bevy::sprite_render::Material2dPlugin; +use crate::bitmap_material::{BITMAP_SURFACE_SHADER, BitmapSurfaceMaterial}; use crate::camera::{ ActivateTerminalCameraPreset, TerminalCameraSlots, TerminalCameraSystemSet, TerminalCameraUpdate, activate_terminal_camera_presets, apply_terminal_camera_updates, @@ -22,7 +26,8 @@ use crate::systems::{ animate_terminal_plane_warp, apply_inline_objects, apply_instance_brightness, finish_terminal_model_load, handle_window_resize, pump_pty_output, render_terminal_widget, request_exit_on_primary_window_close, shutdown_terminal_runtime_on_exit, - sync_asset_to_terminal_cursor, sync_inline_objects, sync_rgp_objects, sync_terminal_materials, + sync_asset_to_terminal_cursor, sync_bitmap_placements, sync_inline_objects, sync_rgp_objects, + sync_terminal_materials, }; use crate::terminal::TerminalRedrawState; @@ -42,6 +47,12 @@ pub struct TerminalPlugin; impl Plugin for TerminalPlugin { fn build(&self, app: &mut App) { + load_internal_asset!( + app, + BITMAP_SURFACE_SHADER, + "shaders/bitmap_surface.wgsl", + Shader::from_wgsl + ); app.init_resource::() .init_resource::() .init_resource::() @@ -136,8 +147,14 @@ impl Plugin for TerminalPlugin { ) .add_systems( Update, - sync_inline_objects + sync_bitmap_placements .after(TerminalRedrawSet) + .after(TerminalCameraSystemSet::Transition), + ) + .add_systems( + Update, + sync_inline_objects + .after(sync_bitmap_placements) // Deterministic vs the Transition set: on the frame a // Mobius exit finishes, spawned inline entities must see // the restored mode, not race it. @@ -173,6 +190,7 @@ impl Plugin for TerminalPlugin { .run_if(|config: Res| config.cursor.model.visible), ) .add_systems(Last, shutdown_terminal_runtime_on_exit) + .add_plugins(Material2dPlugin::::default()) .add_plugins(DirectTerminalRenderPlugin) .add_plugins(TerminalPresentPlugin); } diff --git a/src/shaders/bitmap_surface.wgsl b/src/shaders/bitmap_surface.wgsl new file mode 100644 index 0000000..36038eb --- /dev/null +++ b/src/shaders/bitmap_surface.wgsl @@ -0,0 +1,59 @@ +// Renders one bitmap placement with independent crop, fit, filtering, and opacity. +#import bevy_sprite::mesh2d_vertex_output::VertexOutput + +struct BitmapSurfaceUniform { + uv_min: vec2, + uv_max: vec2, + opacity: f32, + filter_mode: u32, + content_min: vec2, + content_max: vec2, +}; + +@group(#{MATERIAL_BIND_GROUP}) @binding(0) var bitmap_image: texture_2d; +@group(#{MATERIAL_BIND_GROUP}) @binding(1) var params: BitmapSurfaceUniform; + +fn clamped_texel(position: vec2, minimum: vec2, maximum: vec2) -> vec4 { + return textureLoad(bitmap_image, clamp(position, minimum, maximum), 0); +} + +fn sample_nearest(uv: vec2, minimum: vec2, maximum: vec2) -> vec4 { + let size = vec2(textureDimensions(bitmap_image)); + let texel = vec2(floor(uv * size)); + return clamped_texel(texel, minimum, maximum); +} + +fn sample_linear(uv: vec2, minimum: vec2, maximum: vec2) -> vec4 { + let size = vec2(textureDimensions(bitmap_image)); + let position = uv * size - vec2(0.5); + let base = vec2(floor(position)); + let weight = fract(position); + let top_left = clamped_texel(base, minimum, maximum); + let top_right = clamped_texel(base + vec2(1, 0), minimum, maximum); + let bottom_left = clamped_texel(base + vec2(0, 1), minimum, maximum); + let bottom_right = clamped_texel(base + vec2(1, 1), minimum, maximum); + return mix(mix(top_left, top_right, weight.x), mix(bottom_left, bottom_right, weight.x), weight.y); +} + +@fragment +fn fragment(mesh: VertexOutput) -> @location(0) vec4 { + if (any(mesh.uv < params.content_min) || any(mesh.uv > params.content_max)) { + return vec4(0.0); + } + + let content_size = params.content_max - params.content_min; + let content_uv = clamp((mesh.uv - params.content_min) / content_size, vec2(0.0), vec2(1.0)); + let source_uv = mix(params.uv_min, params.uv_max, content_uv); + let image_size = vec2(textureDimensions(bitmap_image)); + let minimum = vec2(floor(params.uv_min * image_size)); + let maximum = vec2(ceil(params.uv_max * image_size)) - vec2(1); + + var color: vec4; + if (params.filter_mode == 0u) { + color = sample_nearest(source_uv, minimum, maximum); + } else { + color = sample_linear(source_uv, minimum, maximum); + } + color.a *= params.opacity; + return color; +} diff --git a/src/systems.rs b/src/systems.rs index 3ab55b8..e15f133 100644 --- a/src/systems.rs +++ b/src/systems.rs @@ -19,16 +19,19 @@ //! The redraw path updates the terminal texture and presentation state first, then the inline //! object systems rebuild or reposition scene entities that depend on the terminal grid. -use std::collections::HashMap; +use std::collections::{HashMap, HashSet}; use std::sync::mpsc::TryRecvError; +use crate::bitmap::{BitmapFilter, BitmapPlacementState}; +use crate::bitmap_material::{BitmapSurfaceMaterial, BitmapSurfaceUniform, resolve_bitmap_layout}; use crate::camera::{ MIN_ORTHOGRAPHIC_SCALE, TerminalCameraSlots, TerminalCameraUpdate, TerminalMobiusSource, }; use crate::config::{AppConfig, CURSOR_DEPTH}; use crate::direct_render::DirectTerminalSceneExchange; use crate::inline::{ - InlineKittyPlaneLayout, InlineObject, TerminalInlineObjectPlane, TerminalInlineObjectSprite, + BitmapPlacementRenderCache, BitmapPlacementRenderState, InlineKittyPlaneLayout, InlineObject, + TerminalBitmapPlacement, TerminalInlineObjectPlane, TerminalInlineObjectSprite, TerminalInlineObjects, TerminalRgpObject, }; use crate::model::CursorModel; @@ -538,7 +541,15 @@ pub(crate) struct SyncInlineParams<'w, 's> { plane_warp: Res<'w, TerminalPlaneWarp>, time: Res<'w, Time>, plane_query: Query<'w, 's, (Entity, &'static Transform), With>, - sprite_query: Query<'w, 's, Entity, With>, + sprite_query: Query< + 'w, + 's, + Entity, + ( + With, + Without, + ), + >, plane_image_query: Query<'w, 's, Entity, With>, rgp_query: Query<'w, 's, Entity, With>, asset_server: Res<'w, AssetServer>, @@ -655,6 +666,324 @@ pub(crate) fn sync_inline_objects(mut params: SyncInlineParams) { inline_objects.finish_sync(viewport.size, terminal.cols, terminal.rows); } +/// Bitmap-surface synchronization parameters. +#[derive(SystemParam)] +pub(crate) struct SyncBitmapParams<'w, 's> { + commands: Commands<'w, 's>, + inline_objects: ResMut<'w, TerminalInlineObjects>, + terminal: Res<'w, TerminalSurface>, + viewport: Res<'w, TerminalViewport>, + camera_slots: Res<'w, TerminalCameraSlots>, + images: ResMut<'w, Assets>, + meshes: ResMut<'w, Assets>, + materials: ResMut<'w, Assets>, +} + +#[derive(Debug, PartialEq, Eq)] +struct BitmapSyncChanges { + transform: bool, + mesh: bool, + material: bool, +} + +fn bitmap_sync_changes( + previous: &BitmapPlacementRenderState, + next: &BitmapPlacementRenderState, +) -> BitmapSyncChanges { + BitmapSyncChanges { + transform: previous.transform != next.transform, + mesh: previous.destination != next.destination, + material: previous.image != next.image || previous.uniform != next.uniform, + } +} + +/// Uploads registered bitmaps once and synchronizes placement entities in place. +pub(crate) fn sync_bitmap_placements(mut params: SyncBitmapParams) { + let SyncBitmapParams { + commands, + inline_objects, + terminal, + viewport, + camera_slots, + images, + meshes, + materials, + } = &mut params; + if !inline_objects.needs_sync(viewport.size, terminal.cols, terminal.rows) { + return; + } + + let bitmap_ids = inline_objects + .bitmap + .bitmaps() + .map(|(bitmap_id, _)| *bitmap_id) + .collect::>(); + let restarted_bitmap_ids = bitmap_ids + .iter() + .filter(|bitmap_id| { + inline_objects.bitmap_render.images.contains_key(bitmap_id) + && inline_objects + .bitmap + .bitmap(**bitmap_id) + .is_some_and(|bitmap| bitmap.handle().is_none()) + }) + .copied() + .collect::>(); + for bitmap_id in &restarted_bitmap_ids { + if let Some(handle) = inline_objects.bitmap_render.images.remove(bitmap_id) { + images.remove(&handle); + } + } + let restarted_placement_ids = inline_objects + .bitmap_render + .placements + .iter() + .filter_map(|(placement_id, cache)| { + restarted_bitmap_ids + .contains(&cache.bitmap_id) + .then_some(*placement_id) + }) + .collect::>(); + for placement_id in restarted_placement_ids { + if let Some(cache) = inline_objects + .bitmap_render + .placements + .remove(&placement_id) + { + commands.entity(cache.entity).despawn(); + meshes.remove(&cache.mesh); + materials.remove(&cache.material); + } + } + let removed_bitmap_ids = inline_objects + .bitmap_render + .images + .keys() + .filter(|bitmap_id| !bitmap_ids.contains(bitmap_id)) + .copied() + .collect::>(); + for bitmap_id in removed_bitmap_ids { + if let Some(handle) = inline_objects.bitmap_render.images.remove(&bitmap_id) { + images.remove(&handle); + } + } + + for bitmap_id in bitmap_ids { + sync_bitmap_image(bitmap_id, inline_objects, images); + } + + let placements = inline_objects + .bitmap + .placements() + .map(|(placement_id, placement)| (*placement_id, placement.clone())) + .collect::>(); + let active_placement_generations = placements + .iter() + .map(|(placement_id, placement)| (*placement_id, placement.generation())) + .collect::>(); + let removed_placement_ids = inline_objects + .bitmap_render + .placements + .iter() + .filter_map(|(placement_id, cache)| { + active_placement_generations + .get(placement_id) + .is_none_or(|generation| *generation != cache.generation) + .then_some(*placement_id) + }) + .collect::>(); + for placement_id in removed_placement_ids { + if let Some(cache) = inline_objects + .bitmap_render + .placements + .remove(&placement_id) + { + commands.entity(cache.entity).despawn(); + meshes.remove(&cache.mesh); + materials.remove(&cache.material); + } + } + + let cell_width = viewport.size.x / terminal.cols.max(1) as f32; + let cell_height = viewport.size.y / terminal.rows.max(1) as f32; + for (placement_id, placement) in placements { + let Some(image) = inline_objects + .bitmap_render + .images + .get(&placement.bitmap_id()) + .cloned() + else { + continue; + }; + let destination = Vec2::new( + placement.columns() as f32 * cell_width, + placement.rows() as f32 * cell_height, + ); + let center = Vec2::new( + viewport.center.x - viewport.size.x * 0.5 + + (placement.col() as f32 + placement.columns() as f32 * 0.5) * cell_width, + viewport.center.y + viewport.size.y * 0.5 + - (placement.row() as f32 + placement.rows() as f32 * 0.5) * cell_height, + ); + let Some(bitmap) = inline_objects.bitmap.bitmap(placement.bitmap_id()) else { + continue; + }; + let layout = resolve_bitmap_layout( + UVec2::new(bitmap.width(), bitmap.height()), + placement.source(), + destination, + placement.fit(), + ); + let uniform = bitmap_uniform(&placement, layout); + let transform = Transform::from_xyz(center.x, center.y, 5.0); + let render_state = BitmapPlacementRenderState { + image: image.clone(), + destination, + transform, + uniform, + }; + + if let Some(cache) = inline_objects + .bitmap_render + .placements + .get_mut(&placement_id) + { + debug_assert_eq!(cache.bitmap_id, placement.bitmap_id()); + let changes = bitmap_sync_changes(&cache.state, &render_state); + if changes.mesh + && let Some(mut mesh) = meshes.get_mut(&cache.mesh) + { + *mesh = + Rectangle::new(render_state.destination.x, render_state.destination.y).into(); + cache.state.destination = render_state.destination; + } + if changes.material + && let Some(mut material) = materials.get_mut(&cache.material) + { + material.image = render_state.image.clone(); + material.params = render_state.uniform; + cache.state.image = render_state.image.clone(); + cache.state.uniform = render_state.uniform; + } + if changes.transform { + commands.entity(cache.entity).insert(render_state.transform); + cache.state.transform = render_state.transform; + } + debug!( + placement_id, + transform = changes.transform, + mesh = changes.mesh, + material = changes.material, + "synchronized bitmap placement update" + ); + continue; + } + + let mesh = meshes.add(Rectangle::new(destination.x, destination.y)); + let material = materials.add(BitmapSurfaceMaterial { + image, + params: uniform, + }); + let visibility = match camera_slots.active().mode { + TerminalPresentationMode::Flat2d => Visibility::Visible, + TerminalPresentationMode::Plane3d + | TerminalPresentationMode::Perspective3d + | TerminalPresentationMode::Mobius3d => Visibility::Hidden, + }; + let entity = commands + .spawn(( + TerminalBitmapPlacement { + placement_id, + bitmap_id: placement.bitmap_id(), + }, + TerminalInlineObjectSprite, + Mesh2d(mesh.clone()), + MeshMaterial2d(material.clone()), + transform, + visibility, + )) + .id(); + inline_objects.bitmap_render.placements.insert( + placement_id, + BitmapPlacementRenderCache { + generation: placement.generation(), + bitmap_id: placement.bitmap_id(), + entity, + mesh, + material, + state: render_state, + }, + ); + } + inline_objects.finish_bitmap_sync(); +} + +fn sync_bitmap_image( + bitmap_id: u32, + inline_objects: &mut TerminalInlineObjects, + images: &mut Assets, +) { + let cached_handle = inline_objects.bitmap_render.images.get(&bitmap_id).cloned(); + let Some(bitmap) = inline_objects.bitmap.bitmap_mut(bitmap_id) else { + return; + }; + let pending_pixels = bitmap.take_pending_rgba(); + + if let Some(handle) = cached_handle.or_else(|| bitmap.handle().cloned()) { + if let Some(pixels) = pending_pixels + && let Some(mut image) = images.get_mut(&handle) + { + image.data = Some(pixels); + } + bitmap.set_handle(handle.clone()); + inline_objects + .bitmap_render + .images + .insert(bitmap_id, handle); + return; + } + + let Some(pixels) = pending_pixels else { + return; + }; + let mut image = Image::new_fill( + Extent3d { + width: bitmap.width(), + height: bitmap.height(), + depth_or_array_layers: 1, + }, + TextureDimension::D2, + &[0, 0, 0, 0], + TextureFormat::Rgba8UnormSrgb, + bevy::asset::RenderAssetUsages::default(), + ); + image.sampler = ImageSampler::nearest(); + image.data = Some(pixels); + let handle = images.add(image); + bitmap.set_handle(handle.clone()); + inline_objects + .bitmap_render + .images + .insert(bitmap_id, handle); +} + +fn bitmap_uniform( + placement: &BitmapPlacementState, + layout: crate::bitmap_material::ResolvedBitmapLayout, +) -> BitmapSurfaceUniform { + BitmapSurfaceUniform { + uv_min: layout.uv_min, + uv_max: layout.uv_max, + opacity: placement.opacity(), + filter_mode: match placement.filter() { + BitmapFilter::Nearest => 0, + BitmapFilter::Linear => 1, + }, + content_min: layout.content_min, + content_max: layout.content_max, + } +} + fn inline_layout( anchor: &crate::inline::InlineAnchor, terminal: &TerminalSurface, @@ -2240,3 +2569,587 @@ mod tests { assert_eq!(preset.mobius_source, None); } } + +#[cfg(test)] +mod bitmap_sync_tests { + use super::*; + use crate::bitmap::{ + BitmapFilter, BitmapFit, BitmapFrameChunk, BitmapOperation, BitmapPlacement, + BitmapPlacementUpdate, BitmapRegisterChunk, SourceRect, + }; + use crate::bitmap_material::BitmapSurfaceMaterial; + + type RenderedPlacement = ( + Entity, + Handle, + Handle, + Handle, + Transform, + ); + type RenderedPlacements = HashMap; + + const PNG_2X2: &[u8] = &[ + 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, + 0x52, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x02, 0x08, 0x06, 0x00, 0x00, 0x00, 0x72, + 0xb6, 0x0d, 0x24, 0x00, 0x00, 0x00, 0x12, 0x49, 0x44, 0x41, 0x54, 0x78, 0x9c, 0x63, 0xf8, + 0xcf, 0xc0, 0xf0, 0x1f, 0x0c, 0x81, 0x34, 0x18, 0x00, 0x00, 0x49, 0xc8, 0x09, 0xf7, 0xf9, + 0xab, 0xb6, 0x0d, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82, + ]; + + fn render_state() -> BitmapPlacementRenderState { + BitmapPlacementRenderState { + image: Handle::default(), + destination: Vec2::new(320.0, 180.0), + transform: Transform::from_xyz(10.0, 20.0, 5.0), + uniform: BitmapSurfaceUniform { + uv_min: Vec2::ZERO, + uv_max: Vec2::ONE, + opacity: 1.0, + filter_mode: 1, + content_min: Vec2::ZERO, + content_max: Vec2::ONE, + }, + } + } + + #[test] + fn source_only_bitmap_update_changes_only_material() { + let previous = render_state(); + let mut next = previous.clone(); + next.uniform.uv_min = Vec2::new(0.25, 0.0); + + assert_eq!( + bitmap_sync_changes(&previous, &next), + BitmapSyncChanges { + transform: false, + mesh: false, + material: true, + } + ); + } + + #[test] + fn position_only_bitmap_update_changes_only_transform() { + let previous = render_state(); + let mut next = previous.clone(); + next.transform.translation.x += 40.0; + + assert_eq!( + bitmap_sync_changes(&previous, &next), + BitmapSyncChanges { + transform: true, + mesh: false, + material: false, + } + ); + } + + fn test_app() -> App { + let mut app = App::new(); + app.init_resource::() + .init_resource::>() + .init_resource::>() + .init_resource::>() + .insert_resource( + TerminalSurface::new(&AppConfig::default()) + .expect("bitmap sync test fixture should satisfy this invariant"), + ) + .insert_resource(TerminalViewport { + size: Vec2::new(800.0, 480.0), + center: Vec2::ZERO, + }) + .init_resource::() + .add_systems(Update, sync_bitmap_placements); + app + } + + fn register_and_place(app: &mut App, placement_ids: &[u32]) { + let mut objects = app.world_mut().resource_mut::(); + objects + .bitmap + .apply(BitmapOperation::Register(BitmapRegisterChunk { + bitmap_id: 42, + format: Some("png".into()), + source: Some("payload".into()), + name: None, + more: false, + data: PNG_2X2.to_vec(), + })) + .expect("bitmap sync test fixture should satisfy this invariant"); + for &placement_id in placement_ids { + objects + .bitmap + .apply(BitmapOperation::Place(BitmapPlacement { + bitmap_id: 42, + placement_id, + row: placement_id as u16, + col: 2, + columns: 8, + rows: 4, + source: None, + fit: BitmapFit::Contain, + filter: BitmapFilter::Linear, + opacity: 1.0, + })) + .expect("bitmap sync test fixture should satisfy this invariant"); + } + } + + fn rendered_placements(app: &mut App) -> RenderedPlacements { + let world = app.world_mut(); + let mut query = world.query::<( + Entity, + &TerminalBitmapPlacement, + &Mesh2d, + &MeshMaterial2d, + &Transform, + )>(); + let placements = query + .iter(world) + .map(|(entity, placement, mesh, material, transform)| { + ( + placement.placement_id, + (entity, mesh.0.clone(), material.0.clone(), *transform), + ) + }) + .collect::>(); + let materials = world.resource::>(); + placements + .into_iter() + .map(|(placement_id, (entity, mesh, material, transform))| { + let image = materials + .get(&material) + .expect("bitmap sync test fixture should satisfy this invariant") + .image + .clone(); + (placement_id, (entity, image, mesh, material, transform)) + }) + .collect() + } + + fn rendered_marker(app: &mut App, placement_id: u32) -> TerminalBitmapPlacement { + let world = app.world_mut(); + let mut query = world.query::<&TerminalBitmapPlacement>(); + *query + .iter(world) + .find(|placement| placement.placement_id == placement_id) + .expect("bitmap sync test fixture should satisfy this invariant") + } + + #[test] + fn one_bitmap_with_two_placements_uploads_once_and_spawns_two_entities() { + let mut app = test_app(); + register_and_place(&mut app, &[7, 8]); + + app.update(); + + let rendered = rendered_placements(&mut app); + assert_eq!(rendered.len(), 2); + assert_eq!(app.world().resource::>().len(), 1); + assert_eq!(rendered[&7].1, rendered[&8].1); + } + + #[test] + fn placement_updates_preserve_entity_image_mesh_and_material_handles() { + let mut app = test_app(); + register_and_place(&mut app, &[7]); + app.update(); + let before = rendered_placements(&mut app) + .remove(&7) + .expect("bitmap sync test fixture should satisfy this invariant"); + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::Update { + placement_id: 7, + update: BitmapPlacementUpdate { + row: Some(5), + col: Some(6), + columns: Some(10), + rows: Some(6), + source: Some(SourceRect { + x: 1, + y: 0, + width: 1, + height: 2, + }), + fit: Some(BitmapFit::Fill), + filter: Some(BitmapFilter::Nearest), + opacity: Some(0.5), + }, + }) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + + let after = rendered_placements(&mut app) + .remove(&7) + .expect("bitmap sync test fixture should satisfy this invariant"); + assert_eq!(before.0, after.0); + assert_eq!(before.1, after.1); + assert_eq!(before.2, after.2); + assert_eq!(before.3, after.3); + assert_ne!(before.4, after.4); + let material = app + .world() + .resource::>() + .get(&after.3) + .expect("bitmap sync test fixture should satisfy this invariant"); + assert_eq!(material.params.uv_min, Vec2::new(0.5, 0.0)); + assert_eq!(material.params.uv_max, Vec2::ONE); + assert_eq!(material.params.filter_mode, 0); + assert_eq!(material.params.opacity, 0.5); + } + + #[test] + fn valid_frame_mutates_pixels_in_place_without_replacing_render_objects() { + let mut app = test_app(); + register_and_place(&mut app, &[7, 8]); + app.update(); + let before = rendered_placements(&mut app); + let replacement = vec![17; 16]; + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::Frame(BitmapFrameChunk { + bitmap_id: 42, + sequence: 1, + format: Some("rgba8".into()), + width: Some(2), + height: Some(2), + more: false, + data: replacement.clone(), + })) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + + let after = rendered_placements(&mut app); + assert_eq!(before, after); + let image = app + .world() + .resource::>() + .get(&after[&7].1) + .expect("bitmap sync test fixture should satisfy this invariant"); + assert_eq!(image.data.as_deref(), Some(replacement.as_slice())); + } + + #[test] + fn malformed_and_stale_frames_leave_pixels_and_render_objects_unchanged() { + let mut app = test_app(); + register_and_place(&mut app, &[7]); + app.update(); + let before_render = rendered_placements(&mut app); + let image_handle = before_render[&7].1.clone(); + let before_pixels = app + .world() + .resource::>() + .get(&image_handle) + .expect("bitmap sync test fixture should satisfy this invariant") + .data + .clone(); + + assert!( + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::Frame(BitmapFrameChunk { + bitmap_id: 42, + sequence: 1, + format: Some("rgba8".into()), + width: Some(2), + height: Some(2), + more: false, + data: vec![1; 15], + })) + .is_err() + ); + app.update(); + assert_eq!(before_render, rendered_placements(&mut app)); + assert_eq!( + app.world() + .resource::>() + .get(&image_handle) + .expect("bitmap sync test fixture should satisfy this invariant") + .data, + before_pixels + ); + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::Frame(BitmapFrameChunk { + bitmap_id: 42, + sequence: 2, + format: Some("rgba8".into()), + width: Some(2), + height: Some(2), + more: false, + data: vec![2; 16], + })) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + let valid_render = rendered_placements(&mut app); + let valid_pixels = app + .world() + .resource::>() + .get(&image_handle) + .expect("bitmap sync test fixture should satisfy this invariant") + .data + .clone(); + assert_eq!(valid_render, before_render); + assert_eq!(valid_pixels.as_deref(), Some(&[2_u8; 16][..])); + + assert!( + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::Frame(BitmapFrameChunk { + bitmap_id: 42, + sequence: 1, + format: Some("rgba8".into()), + width: Some(2), + height: Some(2), + more: false, + data: vec![3; 16], + })) + .is_err() + ); + app.update(); + + let after_render = rendered_placements(&mut app); + assert_eq!(valid_render, after_render); + let pixels = &app + .world() + .resource::>() + .get(&image_handle) + .expect("bitmap sync test fixture should satisfy this invariant") + .data; + assert_eq!(pixels, &valid_pixels); + } + + #[test] + fn placement_and_bitmap_deletion_clean_exact_render_assets() { + let mut app = test_app(); + register_and_place(&mut app, &[7, 8]); + app.update(); + let before = rendered_placements(&mut app); + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::DeletePlacement(7)) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + let after_placement_delete = rendered_placements(&mut app); + assert!(!after_placement_delete.contains_key(&7)); + assert_eq!(after_placement_delete[&8].0, before[&8].0); + assert_eq!(app.world().resource::>().len(), 1); + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::DeletePlacement(8)) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + assert!(rendered_placements(&mut app).is_empty()); + assert_eq!(app.world().resource::>().len(), 1); + assert!( + app.world() + .resource::>() + .is_empty() + ); + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::DeleteBitmap(42)) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + assert!(rendered_placements(&mut app).is_empty()); + assert!(app.world().resource::>().is_empty()); + assert!( + app.world() + .resource::>() + .is_empty() + ); + } + + #[test] + fn delete_and_reregister_before_sync_starts_a_new_render_lifetime() { + let mut app = test_app(); + register_and_place(&mut app, &[7]); + app.update(); + let before = rendered_placements(&mut app) + .remove(&7) + .expect("bitmap sync test fixture should satisfy this invariant"); + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::DeleteBitmap(42)) + .expect("bitmap sync test fixture should satisfy this invariant"); + register_and_place(&mut app, &[7]); + app.update(); + + let after = rendered_placements(&mut app) + .remove(&7) + .expect("bitmap sync test fixture should satisfy this invariant"); + assert_ne!(before.0, after.0); + assert_ne!(before.1, after.1); + assert_eq!(app.world().resource::>().len(), 1); + } + + #[test] + fn deleting_and_recreating_same_placement_id_on_same_bitmap_uses_fresh_render_assets() { + let mut app = test_app(); + register_and_place(&mut app, &[7]); + app.update(); + let before = rendered_placements(&mut app) + .remove(&7) + .expect("bitmap sync test fixture should satisfy this invariant"); + + { + let mut objects = app.world_mut().resource_mut::(); + objects + .bitmap + .apply(BitmapOperation::DeletePlacement(7)) + .expect("bitmap sync test fixture should satisfy this invariant"); + objects + .bitmap + .apply(BitmapOperation::Place(BitmapPlacement { + bitmap_id: 42, + placement_id: 7, + row: 9, + col: 4, + columns: 5, + rows: 3, + source: None, + fit: BitmapFit::Fill, + filter: BitmapFilter::Nearest, + opacity: 0.75, + })) + .expect("bitmap sync test fixture should satisfy this invariant"); + } + app.update(); + + let after = rendered_placements(&mut app) + .remove(&7) + .expect("bitmap sync test fixture should satisfy this invariant"); + assert_ne!(before.0, after.0); + assert_eq!(before.1, after.1); + assert_ne!(before.2, after.2); + assert_ne!(before.3, after.3); + let marker = rendered_marker(&mut app, 7); + assert_eq!(marker.bitmap_id, 42); + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::DeletePlacement(7)) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + assert!(rendered_placements(&mut app).is_empty()); + assert!(app.world().resource::>().is_empty()); + assert!( + app.world() + .resource::>() + .is_empty() + ); + assert_eq!(app.world().resource::>().len(), 1); + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::DeleteBitmap(42)) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + assert!(app.world().resource::>().is_empty()); + } + + #[test] + fn deleting_and_recreating_same_placement_id_on_different_bitmap_rebinds_everything() { + let mut app = test_app(); + register_and_place(&mut app, &[7]); + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::Register(BitmapRegisterChunk { + bitmap_id: 43, + format: Some("png".into()), + source: Some("payload".into()), + name: None, + more: false, + data: PNG_2X2.to_vec(), + })) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + let before = rendered_placements(&mut app) + .remove(&7) + .expect("bitmap sync test fixture should satisfy this invariant"); + + { + let mut objects = app.world_mut().resource_mut::(); + objects + .bitmap + .apply(BitmapOperation::DeletePlacement(7)) + .expect("bitmap sync test fixture should satisfy this invariant"); + objects + .bitmap + .apply(BitmapOperation::Place(BitmapPlacement { + bitmap_id: 43, + placement_id: 7, + row: 2, + col: 6, + columns: 4, + rows: 2, + source: None, + fit: BitmapFit::Contain, + filter: BitmapFilter::Linear, + opacity: 1.0, + })) + .expect("bitmap sync test fixture should satisfy this invariant"); + } + app.update(); + + let after = rendered_placements(&mut app) + .remove(&7) + .expect("bitmap sync test fixture should satisfy this invariant"); + assert_ne!(before.0, after.0); + assert_ne!(before.1, after.1); + assert_ne!(before.2, after.2); + assert_ne!(before.3, after.3); + let marker = rendered_marker(&mut app, 7); + assert_eq!(marker.bitmap_id, 43); + assert_eq!(app.world().resource::>().len(), 2); + + app.world_mut() + .resource_mut::() + .bitmap + .apply(BitmapOperation::DeletePlacement(7)) + .expect("bitmap sync test fixture should satisfy this invariant"); + app.update(); + assert!(rendered_placements(&mut app).is_empty()); + assert!(app.world().resource::>().is_empty()); + assert!( + app.world() + .resource::>() + .is_empty() + ); + assert_eq!(app.world().resource::>().len(), 2); + + { + let mut objects = app.world_mut().resource_mut::(); + objects + .bitmap + .apply(BitmapOperation::DeleteBitmap(42)) + .expect("bitmap sync test fixture should satisfy this invariant"); + objects + .bitmap + .apply(BitmapOperation::DeleteBitmap(43)) + .expect("bitmap sync test fixture should satisfy this invariant"); + } + app.update(); + assert!(app.world().resource::>().is_empty()); + } +} From a3eb2646ad11a78c88ba7f456baeefac9a459dca Mon Sep 17 00:00:00 2001 From: wipesides Date: Thu, 16 Jul 2026 16:36:14 +0300 Subject: [PATCH 04/10] fix(terminal): advertise bitmap-compatible client capabilities --- src/inline.rs | 122 +++++++++++++++++++++++++++++++++++++++--------- src/kitty.rs | 109 ++++++++++++++++++++++++++++++++++++++++++ src/runtime.rs | 69 ++++++++++++++++++++++++++- src/terminal.rs | 27 +++++++++++ src/vt.rs | 12 +++++ 5 files changed, 314 insertions(+), 25 deletions(-) diff --git a/src/inline.rs b/src/inline.rs index a1440dd..3933a1c 100644 --- a/src/inline.rs +++ b/src/inline.rs @@ -139,7 +139,7 @@ impl TerminalInlineObjects { ) -> Vec> { self.consume_pty_output_with_limit( chunk, - parser, + runtime, camera_updates, terminal_output, MAX_BITMAP_APC_BYTES, @@ -147,27 +147,27 @@ impl TerminalInlineObjects { } #[cfg(test)] - fn consume_pty_output_with_bitmap_limit( + fn consume_pty_output_with_bitmap_limit( &mut self, chunk: &[u8], - parser: &mut vt100::Parser, + runtime: &mut TerminalRuntime, bitmap_apc_limit: usize, ) -> Vec> { let mut camera_updates = Vec::new(); let mut terminal_output = false; self.consume_pty_output_with_limit( chunk, - parser, + runtime, &mut camera_updates, &mut terminal_output, bitmap_apc_limit, ) } - fn consume_pty_output_with_limit( + fn consume_pty_output_with_limit( &mut self, mut chunk: &[u8], - parser: &mut vt100::Parser, + runtime: &mut TerminalRuntime, camera_updates: &mut Vec, terminal_output: &mut bool, bitmap_apc_limit: usize, @@ -201,7 +201,7 @@ impl TerminalInlineObjects { let take = chunk.len().min(INGEST_BLOCK_BYTES).min(block_limit); self.pending_bytes.extend_from_slice(&chunk[..take]); chunk = &chunk[take..]; - replies.extend(self.process_pending_bytes(parser, camera_updates, terminal_output)); + replies.extend(self.process_pending_bytes(runtime, camera_updates, terminal_output)); if self .pending_bytes @@ -214,9 +214,9 @@ impl TerminalInlineObjects { replies } - fn process_pending_bytes( + fn process_pending_bytes( &mut self, - parser: &mut vt100::Parser, + runtime: &mut TerminalRuntime, camera_updates: &mut Vec, terminal_output: &mut bool, ) -> Vec> { @@ -406,6 +406,21 @@ impl TerminalInlineObjects { match operation { KittyOperation::Pending | KittyOperation::Ignored => (true, None), + KittyOperation::Query { + image_id, + result, + quiet, + } => { + let reply = match result { + Ok(()) if quiet == 1 => None, + Err(_) if quiet == 2 => None, + Ok(()) => Some(format!("\x1b_Gi={image_id};OK\x1b\\").into_bytes()), + Err(error) => { + Some(format!("\x1b_Gi={image_id};EINVAL:{error}\x1b\\").into_bytes()) + } + }; + (true, reply) + } KittyOperation::TransmitOnly { object_id, image } => { self.objects .insert(object_id, InlineObject::KittyImage(image.rasterize())); @@ -857,6 +872,7 @@ mod tests { use super::*; const BITMAP_SUPPORT_REPLY: &[u8] = b"\x1b_ratty;i;s;v=1;fmt=png;frame=rgba8;payload=1;chunk=1;placement=1;crop=1;fit=contain|cover|fill;filter=nearest|linear;opacity=1\x1b\\"; + const RATATUI_IMAGE_KITTY_QUERY: &[u8] = b"\x1b_Gi=31,s=1,v=1,a=q,t=d,f=24;AAAA\x1b\\"; const PNG_2X2: &[u8] = &[ 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x02, 0x08, 0x06, 0x00, 0x00, 0x00, 0x72, @@ -865,21 +881,28 @@ mod tests { 0xab, 0xb6, 0x0d, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82, ]; - fn parser() -> vt100::Parser { - vt100::Parser::new(24, 80, 0) + fn parser() -> TerminalRuntime { + TerminalRuntime::for_test(24, 80) + } + + fn contents(runtime: &TerminalRuntime) -> String { + vt::visible_row_texts(&runtime.term) + .join("\n") + .trim_end() + .to_owned() } fn consume( objects: &mut TerminalInlineObjects, chunk: &[u8], - parser: &mut vt100::Parser, + parser: &mut TerminalRuntime, ) -> Vec> { let mut camera_updates = Vec::new(); let mut terminal_output = false; objects.consume_pty_output(chunk, parser, &mut camera_updates, &mut terminal_output) } - fn register_bitmap(objects: &mut TerminalInlineObjects, parser: &mut vt100::Parser) { + fn register_bitmap(objects: &mut TerminalInlineObjects, parser: &mut TerminalRuntime) { let payload = base64::engine::general_purpose::STANDARD.encode(PNG_2X2); let command = format!("\x1b_ratty;i;r;id=7;fmt=png;source=payload;more=0;{payload}\x1b\\"); assert!(consume(objects, command.as_bytes(), parser).is_empty()); @@ -893,7 +916,60 @@ mod tests { let replies = consume(&mut objects, b"left\x1b_ratty;i;s\x1b\\right", &mut parser); assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); - assert_eq!(parser.screen().contents(), "leftright"); + assert_eq!(contents(&parser), "leftright"); + } + + #[test] + fn kitty_query_reports_support_without_storing_an_image() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + + let replies = consume(&mut objects, RATATUI_IMAGE_KITTY_QUERY, &mut parser); + + assert_eq!(replies, [b"\x1b_Gi=31;OK\x1b\\".to_vec()]); + assert!(objects.objects.is_empty()); + assert!(objects.anchors.is_empty()); + } + + #[test] + fn invalid_kitty_query_reports_error_without_mutating_state() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + + let replies = consume( + &mut objects, + b"\x1b_Gi=9,s=2,v=2,a=q,t=d,f=24;AAAA\x1b\\", + &mut parser, + ); + + assert_eq!( + replies, + [b"\x1b_Gi=9;EINVAL:invalid pixel data\x1b\\".to_vec()] + ); + assert!(objects.objects.is_empty()); + assert!(objects.anchors.is_empty()); + } + + #[test] + fn kitty_query_quiet_levels_suppress_the_requested_reply_class() { + let mut objects = TerminalInlineObjects::default(); + let mut parser = parser(); + + let ok_replies = consume( + &mut objects, + b"\x1b_Gi=31,s=1,v=1,a=q,t=d,f=24,q=1;AAAA\x1b\\", + &mut parser, + ); + let error_replies = consume( + &mut objects, + b"\x1b_Gi=9,s=2,v=2,a=q,t=d,f=24,q=2;AAAA\x1b\\", + &mut parser, + ); + + assert!(ok_replies.is_empty()); + assert!(error_replies.is_empty()); + assert!(objects.objects.is_empty()); + assert!(objects.anchors.is_empty()); } #[test] @@ -902,11 +978,11 @@ mod tests { let mut parser = parser(); assert!(consume(&mut objects, b"before\x1b_ratty;i;", &mut parser).is_empty()); - assert_eq!(parser.screen().contents(), "before"); + assert_eq!(contents(&parser), "before"); let replies = consume(&mut objects, b"s\x1b\\after", &mut parser); assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); - assert_eq!(parser.screen().contents(), "beforeafter"); + assert_eq!(contents(&parser), "beforeafter"); } #[test] @@ -932,7 +1008,7 @@ mod tests { ); assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); - assert_eq!(parser.screen().contents(), "beforeafter"); + assert_eq!(contents(&parser), "beforeafter"); assert!(!objects.discarding_oversized_bitmap_apc); assert!(objects.pending_bytes.len() <= limit); } @@ -955,7 +1031,7 @@ mod tests { ); assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); - assert_eq!(parser.screen().contents(), "tail"); + assert_eq!(contents(&parser), "tail"); assert!(!objects.discarding_oversized_bitmap_apc); } @@ -969,7 +1045,7 @@ mod tests { let replies = objects.consume_pty_output_with_bitmap_limit(b"s\x1b\\", &mut parser, limit); assert_eq!(replies, vec![crate::rgp::support_reply()]); - assert!(parser.screen().contents().is_empty()); + assert!(contents(&parser).is_empty()); } #[test] @@ -980,7 +1056,7 @@ mod tests { let replies = consume(&mut objects, b"\x1b_ratty;i;s\x9c", &mut parser); assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); - assert!(parser.screen().contents().is_empty()); + assert!(contents(&parser).is_empty()); } #[test] @@ -998,7 +1074,7 @@ mod tests { replies, vec![BITMAP_SUPPORT_REPLY.to_vec(), crate::rgp::support_reply()] ); - assert!(parser.screen().contents().is_empty()); + assert!(contents(&parser).is_empty()); } #[test] @@ -1015,7 +1091,7 @@ mod tests { assert_eq!(replies, vec![BITMAP_SUPPORT_REPLY.to_vec()]); assert!(objects.bitmap.bitmap(7).is_some()); - assert!(parser.screen().contents().is_empty()); + assert!(contents(&parser).is_empty()); } #[test] @@ -1030,7 +1106,7 @@ mod tests { ); assert!(replies.is_empty()); - assert_eq!(parser.screen().contents(), "beforeafter"); + assert_eq!(contents(&parser), "beforeafter"); } #[test] diff --git a/src/kitty.rs b/src/kitty.rs index 6cd7324..cebf55f 100644 --- a/src/kitty.rs +++ b/src/kitty.rs @@ -53,6 +53,40 @@ impl KittyParserState { let action = params.get("a").copied().unwrap_or("T"); match action { + "q" => { + let image_id = params + .get("i") + .and_then(|value| value.parse().ok()) + .unwrap_or(0); + let quiet = params + .get("q") + .and_then(|value| value.parse().ok()) + .unwrap_or(0); + let format = params + .get("f") + .and_then(|value| value.parse().ok()) + .unwrap_or(100); + let width = params + .get("s") + .and_then(|value| value.parse().ok()) + .unwrap_or(0); + let height = params + .get("v") + .and_then(|value| value.parse().ok()) + .unwrap_or(0); + let medium = params.get("t").copied().unwrap_or("d"); + let result = base64::engine::general_purpose::STANDARD + .decode(payload) + .map_err(|_| "invalid pixel data") + .and_then(|payload| { + validate_direct_query(format, width, height, medium, &payload) + }); + Some(KittyOperation::Query { + image_id, + result, + quiet, + }) + } "T" | "t" => { let starts_new_transfer = self.transfer.is_none() || params.contains_key("a") @@ -187,6 +221,15 @@ pub enum KittyOperation { Pending, /// Indicates the sequence was ignored. Ignored, + /// Capability query result, returned without storing image state. + Query { + /// Image identifier echoed in the protocol reply. + image_id: u32, + /// Validation result for the probed payload. + result: Result<(), &'static str>, + /// Kitty `q` response-suppression level. + quiet: u8, + }, /// Image registration without placement. TransmitOnly { /// Object identifier. @@ -217,6 +260,33 @@ pub enum KittyOperation { }, } +fn validate_direct_query( + format: u32, + width: u32, + height: u32, + medium: &str, + payload: &[u8], +) -> Result<(), &'static str> { + if medium != "d" { + return Err("unsupported transmission medium"); + } + let pixels = u64::from(width).saturating_mul(u64::from(height)); + let expected = match format { + 24 => pixels.saturating_mul(3), + 32 => pixels.saturating_mul(4), + 100 => { + return image::load_from_memory_with_format(payload, image::ImageFormat::Png) + .map(|_| ()) + .map_err(|_| "invalid PNG data"); + } + _ => return Err("unsupported pixel format"), + }; + if payload.len() as u64 != expected { + return Err("invalid pixel data"); + } + Ok(()) +} + struct KittyTransfer { action: String, object_id: u32, @@ -361,3 +431,42 @@ pub fn refresh_kitty_placeholder_anchors( changed } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn kitty_query_parses_valid_direct_transfer_probe() { + let mut state = KittyParserState::default(); + + let operation = + state.consume_sequence(b"\x1b_Gi=31,s=1,v=1,a=q,t=d,f=24;AAAA\x1b\\", (0, 0)); + + assert!(matches!( + operation, + Some(KittyOperation::Query { + image_id: 31, + result: Ok(()), + quiet: 0, + }) + )); + } + + #[test] + fn kitty_query_preserves_quiet_level() { + let mut state = KittyParserState::default(); + + let operation = + state.consume_sequence(b"\x1b_Gi=9,s=2,v=2,a=q,t=d,f=24,q=2;AAAA\x1b\\", (0, 0)); + + assert!(matches!( + operation, + Some(KittyOperation::Query { + image_id: 9, + result: Err("invalid pixel data"), + quiet: 2, + }) + )); + } +} diff --git a/src/runtime.rs b/src/runtime.rs index c745cc0..66fbe57 100644 --- a/src/runtime.rs +++ b/src/runtime.rs @@ -28,6 +28,12 @@ pub struct RuntimeOptions { pub working_dir: Option, } +fn apply_terminal_identity(command: &mut CommandBuilder) { + command.env("RATTY_SESSION", "1"); + command.env("TERM_PROGRAM", "ratty"); + command.env("TERM_PROGRAM_VERSION", env!("CARGO_PKG_VERSION")); +} + /// Running PTY and parser state. /// /// The `!Sync` PTY handles (the output channel receiver and the master) live @@ -125,6 +131,33 @@ fn find_git_bash() -> Option { } impl TerminalRuntime { + #[cfg(test)] + pub(crate) fn for_test(rows: u16, cols: u16) -> Self { + let (_tx, rx) = mpsc::channel::>(); + let sink = TerminalEventSink::default(); + let term = Crosswords::new( + CrosswordsSize::new(usize::from(cols.max(1)), usize::from(rows.max(1))), + CursorShape::Block, + sink.clone(), + WindowId::from(0), + 0, + 1000, + ); + + Self { + rx: SyncCell::new(rx), + writer: Arc::new(Mutex::new(None)), + master: SyncCell::new(None), + child: None, + reader_thread: None, + term, + processor: Processor::default(), + sink, + pty_disconnected: false, + shutdown_started: false, + } + } + /// Spawns the shell PTY runtime. /// /// # Errors @@ -173,6 +206,7 @@ impl TerminalRuntime { for (key, value) in &config.env { cmd.env(key, value); } + apply_terminal_identity(&mut cmd); let child = pair .slave @@ -278,8 +312,18 @@ impl TerminalRuntime { // rio-vt reflows content and resets the scrolling region natively, so // the grid resize is the whole operation — no snapshot and replay. - self.term - .resize(CrosswordsSize::new(usize::from(cols), usize::from(rows))); + let pixel_width = u32::from(pw); + let pixel_height = u32::from(ph); + let cell_width = pixel_width.div_ceil(u32::from(cols)); + let cell_height = pixel_height.div_ceil(u32::from(rows)); + self.term.resize(CrosswordsSize::new_with_dimensions( + usize::from(cols), + usize::from(rows), + pixel_width, + pixel_height, + cell_width, + cell_height, + )); } /// Returns the active kitty keyboard enhancement flags. @@ -326,3 +370,24 @@ impl Drop for TerminalRuntime { self.shutdown(); } } + +#[cfg(test)] +mod tests { + use std::ffi::OsStr; + + use super::*; + + #[test] + fn ratty_child_environment_identifies_the_terminal() { + let mut command = CommandBuilder::new("rchat-tui"); + + apply_terminal_identity(&mut command); + + assert_eq!(command.get_env("RATTY_SESSION"), Some(OsStr::new("1"))); + assert_eq!(command.get_env("TERM_PROGRAM"), Some(OsStr::new("ratty"))); + assert_eq!( + command.get_env("TERM_PROGRAM_VERSION"), + Some(OsStr::new(env!("CARGO_PKG_VERSION"))) + ); + } +} diff --git a/src/terminal.rs b/src/terminal.rs index d355cc4..a2479a9 100644 --- a/src/terminal.rs +++ b/src/terminal.rs @@ -226,6 +226,17 @@ impl TerminalSurface { UVec2::new(width, height) } + /// Returns the physical pixel dimensions of one terminal cell. + pub fn cell_pixel_dimensions(&self) -> (u16, u16) { + let pixels = self.pixmap_dimensions(); + let width = pixels.x.div_ceil(u32::from(self.cols.max(1))); + let height = pixels.y.div_ceil(u32::from(self.rows.max(1))); + ( + width.clamp(1, u32::from(u16::MAX)) as u16, + height.clamp(1, u32::from(u16::MAX)) as u16, + ) + } + /// Returns the current terminal layout. fn layout(&self) -> TerminalLayout { TerminalLayout::new( @@ -763,6 +774,22 @@ mod tests { assert_eq!(rendered[0], "你 好 e\u{0301}z"); } + #[test] + fn cell_pixel_dimensions_divide_the_physical_texture_by_the_grid() { + let mut terminal = TerminalSurface::new(&AppConfig::default()) + .expect("default terminal surface should initialize"); + terminal.resize(37, 11); + let pixels = terminal.pixmap_dimensions(); + + assert_eq!( + terminal.cell_pixel_dimensions(), + ( + pixels.x.div_ceil(37).clamp(1, u32::from(u16::MAX)) as u16, + pixels.y.div_ceil(11).clamp(1, u32::from(u16::MAX)) as u16, + ) + ); + } + /// Regression test for vertical-only zoom steps (#97): with fractional /// cell quantization, every font-size step must grow both axes. #[test] diff --git a/src/vt.rs b/src/vt.rs index 91676ce..0f0d3c4 100644 --- a/src/vt.rs +++ b/src/vt.rs @@ -689,6 +689,18 @@ mod tests { assert!(harness.sink.take_replies().is_empty(), "replies must drain"); } + #[test] + fn cell_size_query_reports_current_pixel_dimensions() { + let mut harness = Harness::new(24, 80); + harness.term.resize(CrosswordsSize::new_with_dimensions( + 80, 24, 800, 480, 10, 20, + )); + + harness.feed(b"\x1b[16t"); + + assert_eq!(harness.sink.take_replies(), [b"\x1b[6;20;10t".to_vec()]); + } + /// rio-vt reports the engine's capabilities, not ratty's. Sixel and OSC 52 /// must not survive to the PTY, or applications will emit payloads that /// silently go nowhere. From de40c0455bb15631956a261ba0ecdcb4c7f1d8dd Mon Sep 17 00:00:00 2001 From: wipesides Date: Thu, 16 Jul 2026 16:36:46 +0300 Subject: [PATCH 05/10] examples(bitmap): demonstrate placement and live frames --- examples/bitmap_frames.rs | 674 ++++++++++++++++++++++++++++++++++++ examples/bitmap_pan_zoom.rs | 648 ++++++++++++++++++++++++++++++++++ 2 files changed, 1322 insertions(+) create mode 100644 examples/bitmap_frames.rs create mode 100644 examples/bitmap_pan_zoom.rs diff --git a/examples/bitmap_frames.rs b/examples/bitmap_frames.rs new file mode 100644 index 0000000..bb99f23 --- /dev/null +++ b/examples/bitmap_frames.rs @@ -0,0 +1,674 @@ +use std::{ + io::{self, Stdout, Write}, + thread, + time::{Duration, Instant}, +}; + +use anyhow::{Context, Result, ensure}; +use base64::Engine as _; +use clap::Parser; +use image::ImageEncoder as _; +use ratatui::crossterm::{ + cursor::{Hide, MoveTo, Show}, + execute, queue, + style::Print, + terminal::{ + self, Clear, ClearType, EnterAlternateScreen, LeaveAlternateScreen, disable_raw_mode, + enable_raw_mode, + }, +}; + +const BITMAP_ID: u32 = 42; +const PLACEMENT_ID: u32 = 7; +const MAX_BASE64_CHUNK: usize = 4096; +const APC_PREFIX: &str = "\u{1b}_ratty;i;"; +const APC_END: &str = "\u{1b}\\"; + +#[derive(Parser)] +#[command(about = "Stream generated RGBA8 frames through Ratty's bitmap surface protocol")] +struct Args { + /// Frames per second. + #[arg(long, default_value_t = 15)] + fps: u32, + + /// Run duration in seconds. + #[arg(long, default_value_t = 10.0)] + duration: f64, + + /// Bitmap width in pixels. + #[arg(long, default_value_t = 320)] + width: u32, + + /// Bitmap height in pixels. + #[arg(long, default_value_t = 180)] + height: u32, +} + +fn encode_registration(bitmap_id: u32, encoded_png: &str) -> Vec> { + debug_assert_eq!(MAX_BASE64_CHUNK % 4, 0); + let chunks: Vec<_> = encoded_png.as_bytes().chunks(MAX_BASE64_CHUNK).collect(); + let last = chunks.len().saturating_sub(1); + + chunks + .into_iter() + .enumerate() + .map(|(index, payload)| { + let payload = std::str::from_utf8(payload).expect("base64 is ASCII"); + let more = u8::from(index != last); + if index == 0 { + encode_command(format!( + "r;id={bitmap_id};fmt=png;source=payload;more={more};{payload}" + )) + } else { + encode_command(format!("r;id={bitmap_id};more={more};{payload}")) + } + }) + .collect() +} + +fn encode_placement( + bitmap_id: u32, + placement_id: u32, + row: u16, + col: u16, + columns: u32, + rows: u32, +) -> Vec { + encode_command(format!( + "p;id={bitmap_id};pid={placement_id};row={row};col={col};w={columns};h={rows};fit=contain;filter=linear;opacity=1" + )) +} + +fn encode_frame( + bitmap_id: u32, + sequence: u32, + width: u32, + height: u32, + rgba: &[u8], +) -> Result>> { + ensure!(width > 0 && height > 0, "frame dimensions must be nonzero"); + let expected_len = width + .checked_mul(height) + .and_then(|pixels| pixels.checked_mul(4)) + .and_then(|bytes| usize::try_from(bytes).ok()) + .context("frame dimensions overflow")?; + ensure!( + rgba.len() == expected_len, + "RGBA8 frame length does not match its dimensions" + ); + + let encoded = base64::engine::general_purpose::STANDARD.encode(rgba); + debug_assert_eq!(MAX_BASE64_CHUNK % 4, 0); + let chunks: Vec<_> = encoded.as_bytes().chunks(MAX_BASE64_CHUNK).collect(); + let last = chunks.len().saturating_sub(1); + + Ok(chunks + .into_iter() + .enumerate() + .map(|(index, payload)| { + let payload = std::str::from_utf8(payload).expect("base64 is ASCII"); + let more = u8::from(index != last); + if index == 0 { + encode_command(format!( + "f;id={bitmap_id};seq={sequence};fmt=rgba8;w={width};h={height};more={more};{payload}" + )) + } else { + encode_command(format!( + "f;id={bitmap_id};seq={sequence};more={more};{payload}" + )) + } + }) + .collect()) +} + +fn encode_deletion(bitmap_id: u32, placement_id: u32) -> [Vec; 2] { + [ + encode_command(format!("d;pid={placement_id}")), + encode_command(format!("d;id={bitmap_id}")), + ] +} + +fn encode_command(body: String) -> Vec { + format!("{APC_PREFIX}{body}{APC_END}").into_bytes() +} + +struct LatestFrameScheduler { + interval: Duration, + next_sequence: Option, +} + +impl LatestFrameScheduler { + const fn new(interval: Duration) -> Self { + Self { + interval, + next_sequence: Some(1), + } + } + + fn due_sequence(&mut self, elapsed: Duration) -> Option { + let next_sequence = self.next_sequence?; + let latest_tick = elapsed.as_nanos() / self.interval.as_nanos(); + let latest_tick = u32::try_from(latest_tick).unwrap_or(u32::MAX); + if latest_tick < next_sequence { + return None; + } + + self.next_sequence = latest_tick.checked_add(1); + Some(latest_tick) + } + + fn next_deadline(&self) -> Duration { + self.next_sequence.map_or(Duration::MAX, |sequence| { + self.interval.saturating_mul(sequence) + }) + } +} + +#[derive(Debug)] +struct TimingConfig { + interval: Duration, + run_duration: Duration, +} + +fn validate_timing(fps: u32, duration_seconds: f64) -> Result { + ensure!(fps > 0, "--fps must be greater than zero"); + ensure!( + duration_seconds.is_finite() && duration_seconds > 0.0, + "--duration must be a finite positive number" + ); + + let interval = Duration::try_from_secs_f64(1.0 / f64::from(fps)) + .context("--fps cannot be represented as a frame interval")?; + ensure!(!interval.is_zero(), "--fps is too large"); + let run_duration = Duration::try_from_secs_f64(duration_seconds) + .context("--duration is too large to represent")?; + + // The loop emits only while elapsed < run_duration. At nanosecond + // resolution, this is the largest tick that can become due. + let maximum_due_tick = run_duration.as_nanos().saturating_sub(1) / interval.as_nanos(); + ensure!( + maximum_due_tick <= u128::from(u32::MAX), + "--fps and --duration can exceed the u32 frame sequence capacity" + ); + + Ok(TimingConfig { + interval, + run_duration, + }) +} + +fn generate_frame(width: u32, height: u32, sequence: u32) -> Result> { + let len = width + .checked_mul(height) + .and_then(|pixels| pixels.checked_mul(4)) + .and_then(|bytes| usize::try_from(bytes).ok()) + .context("frame dimensions overflow")?; + let mut rgba = vec![0; len]; + let square_size = (width.min(height) / 5).max(1); + let travel = width.saturating_sub(square_size).saturating_add(1); + let square_x = sequence.wrapping_mul(6) % travel; + let square_y = height.saturating_sub(square_size) / 2; + + for y in 0..height { + for x in 0..width { + let offset = usize::try_from((y * width + x) * 4).expect("validated frame fits usize"); + rgba[offset] = ((u64::from(x) * 255) / u64::from(width)) as u8; + rgba[offset + 1] = ((u64::from(y) * 255) / u64::from(height)) as u8; + rgba[offset + 2] = sequence.wrapping_mul(3) as u8; + rgba[offset + 3] = 255; + + if x >= square_x + && x < square_x + square_size + && y >= square_y + && y < square_y + square_size + { + rgba[offset..offset + 4].copy_from_slice(&[255, 240, 32, 255]); + } + } + } + + Ok(rgba) +} + +fn encode_png(width: u32, height: u32, rgba: &[u8]) -> Result> { + let mut png = Vec::new(); + image::codecs::png::PngEncoder::new(&mut png) + .write_image(rgba, width, height, image::ExtendedColorType::Rgba8) + .context("failed to encode initial PNG")?; + Ok(png) +} + +trait TerminalBackend { + fn enable_raw(&mut self) -> io::Result<()>; + fn enter_alternate(&mut self) -> io::Result<()>; + fn hide_cursor(&mut self) -> io::Result<()>; + fn prepare_screen(&mut self) -> io::Result<()>; + fn show_cursor(&mut self) -> io::Result<()>; + fn leave_alternate(&mut self) -> io::Result<()>; + fn disable_raw(&mut self) -> io::Result<()>; +} + +#[derive(Default)] +struct TerminalSetupState { + raw_enabled: bool, + alternate_entered: bool, + cursor_hidden: bool, +} + +fn setup_terminal(backend: &mut impl TerminalBackend) -> io::Result { + let mut state = TerminalSetupState { + raw_enabled: true, + ..TerminalSetupState::default() + }; + if let Err(error) = backend.enable_raw() { + restore_terminal(backend, &mut state); + return Err(error); + } + + state.alternate_entered = true; + if let Err(error) = backend.enter_alternate() { + restore_terminal(backend, &mut state); + return Err(error); + } + + state.cursor_hidden = true; + if let Err(error) = backend.hide_cursor() { + restore_terminal(backend, &mut state); + return Err(error); + } + + if let Err(error) = backend.prepare_screen() { + restore_terminal(backend, &mut state); + return Err(error); + } + + Ok(state) +} + +fn restore_terminal(backend: &mut impl TerminalBackend, state: &mut TerminalSetupState) { + if std::mem::take(&mut state.cursor_hidden) { + let _ = backend.show_cursor(); + } + if std::mem::take(&mut state.alternate_entered) { + let _ = backend.leave_alternate(); + } + if std::mem::take(&mut state.raw_enabled) { + let _ = backend.disable_raw(); + } +} + +struct CrosstermBackend { + stdout: Stdout, +} + +impl TerminalBackend for CrosstermBackend { + fn enable_raw(&mut self) -> io::Result<()> { + enable_raw_mode() + } + + fn enter_alternate(&mut self) -> io::Result<()> { + execute!(self.stdout, EnterAlternateScreen) + } + + fn hide_cursor(&mut self) -> io::Result<()> { + execute!(self.stdout, Hide) + } + + fn prepare_screen(&mut self) -> io::Result<()> { + execute!(self.stdout, Clear(ClearType::All), MoveTo(0, 0)) + } + + fn show_cursor(&mut self) -> io::Result<()> { + execute!(self.stdout, Show) + } + + fn leave_alternate(&mut self) -> io::Result<()> { + execute!(self.stdout, LeaveAlternateScreen) + } + + fn disable_raw(&mut self) -> io::Result<()> { + disable_raw_mode() + } +} + +struct TerminalSession { + backend: CrosstermBackend, + setup: TerminalSetupState, + bitmap_id: u32, + placement_id: u32, +} + +impl TerminalSession { + fn enter(bitmap_id: u32, placement_id: u32) -> io::Result { + let mut backend = CrosstermBackend { + stdout: io::stdout(), + }; + let setup = setup_terminal(&mut backend)?; + Ok(Self { + backend, + setup, + bitmap_id, + placement_id, + }) + } + + fn write_commands(&mut self, commands: I) -> io::Result<()> + where + I: IntoIterator>, + { + for command in commands { + self.backend.stdout.write_all(&command)?; + } + self.backend.stdout.flush() + } + + fn draw_status(&mut self, sequence: u32, fps: u32) -> io::Result<()> { + queue!( + self.backend.stdout, + MoveTo(0, 0), + Clear(ClearType::CurrentLine), + Print(format!( + "Ratty RGBA8 bitmap stream | target {fps} FPS | sequence {sequence}" + )) + )?; + self.backend.stdout.flush() + } +} + +impl Drop for TerminalSession { + fn drop(&mut self) { + for command in encode_deletion(self.bitmap_id, self.placement_id) { + let _ = self.backend.stdout.write_all(&command); + } + let _ = self.backend.stdout.flush(); + restore_terminal(&mut self.backend, &mut self.setup); + } +} + +fn main() -> Result<()> { + let args = Args::parse(); + ensure!( + args.width > 0 && args.height > 0, + "--width and --height must be greater than zero" + ); + + let timing = validate_timing(args.fps, args.duration)?; + let initial_rgba = generate_frame(args.width, args.height, 0)?; + let png = encode_png(args.width, args.height, &initial_rgba)?; + let encoded_png = base64::engine::general_purpose::STANDARD.encode(png); + + let mut terminal = + TerminalSession::enter(BITMAP_ID, PLACEMENT_ID).context("failed to enter terminal mode")?; + terminal.write_commands(encode_registration(BITMAP_ID, &encoded_png))?; + let (columns, rows) = terminal::size().context("failed to read terminal size")?; + terminal.write_commands([encode_placement( + BITMAP_ID, + PLACEMENT_ID, + 1, + 0, + u32::from(columns.max(1)), + u32::from(rows.saturating_sub(1).max(1)), + )])?; + terminal.draw_status(0, args.fps)?; + + let started = Instant::now(); + let mut scheduler = LatestFrameScheduler::new(timing.interval); + loop { + let elapsed = started.elapsed(); + if elapsed >= timing.run_duration { + break; + } + + if let Some(sequence) = scheduler.due_sequence(elapsed) { + let rgba = generate_frame(args.width, args.height, sequence)?; + terminal.write_commands(encode_frame( + BITMAP_ID, + sequence, + args.width, + args.height, + &rgba, + )?)?; + terminal.draw_status(sequence, args.fps)?; + continue; + } + + let wait = scheduler + .next_deadline() + .saturating_sub(elapsed) + .min(timing.run_duration.saturating_sub(elapsed)); + thread::sleep(wait); + } + + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + const APC_END: &str = "\u{1b}\\"; + + fn command_text(commands: &[Vec]) -> Vec<&str> { + commands + .iter() + .map(|command| { + std::str::from_utf8(command).expect("valid example test input should succeed") + }) + .collect() + } + + fn payload(command: &str) -> &str { + command + .strip_suffix(APC_END) + .expect("valid example test input should succeed") + .rsplit_once(';') + .expect("valid example test input should succeed") + .1 + } + + #[derive(Clone, Copy, Debug, PartialEq, Eq)] + enum TerminalAction { + EnableRaw, + EnterAlternate, + HideCursor, + PrepareScreen, + ShowCursor, + LeaveAlternate, + DisableRaw, + } + + struct MockTerminalBackend { + fail_at: TerminalAction, + actions: Vec, + } + + impl MockTerminalBackend { + fn new(fail_at: TerminalAction) -> Self { + Self { + fail_at, + actions: Vec::new(), + } + } + + fn record(&mut self, action: TerminalAction) -> io::Result<()> { + self.actions.push(action); + if action == self.fail_at { + Err(io::Error::other("injected terminal failure")) + } else { + Ok(()) + } + } + } + + impl TerminalBackend for MockTerminalBackend { + fn enable_raw(&mut self) -> io::Result<()> { + self.record(TerminalAction::EnableRaw) + } + + fn enter_alternate(&mut self) -> io::Result<()> { + self.record(TerminalAction::EnterAlternate) + } + + fn hide_cursor(&mut self) -> io::Result<()> { + self.record(TerminalAction::HideCursor) + } + + fn prepare_screen(&mut self) -> io::Result<()> { + self.record(TerminalAction::PrepareScreen) + } + + fn show_cursor(&mut self) -> io::Result<()> { + self.record(TerminalAction::ShowCursor) + } + + fn leave_alternate(&mut self) -> io::Result<()> { + self.record(TerminalAction::LeaveAlternate) + } + + fn disable_raw(&mut self) -> io::Result<()> { + self.record(TerminalAction::DisableRaw) + } + } + + #[test] + fn partial_terminal_setup_attempts_reverse_cleanup() { + let mut backend = MockTerminalBackend::new(TerminalAction::HideCursor); + + assert!(setup_terminal(&mut backend).is_err()); + + assert_eq!( + backend.actions, + vec![ + TerminalAction::EnableRaw, + TerminalAction::EnterAlternate, + TerminalAction::HideCursor, + TerminalAction::ShowCursor, + TerminalAction::LeaveAlternate, + TerminalAction::DisableRaw, + ] + ); + } + + #[test] + fn lifecycle_registers_and_places_once_then_sends_monotonic_rgba_frames_and_deletes() { + let mut commands = encode_registration(BITMAP_ID, &"A".repeat(4100)); + commands.push(encode_placement(BITMAP_ID, PLACEMENT_ID, 2, 1, 80, 24)); + commands.extend( + encode_frame(BITMAP_ID, 1, 2, 2, &[0; 16]) + .expect("valid example test input should succeed"), + ); + commands.extend( + encode_frame(BITMAP_ID, 3, 2, 2, &[1; 16]) + .expect("valid example test input should succeed"), + ); + commands.extend(encode_deletion(BITMAP_ID, PLACEMENT_ID)); + + let commands = command_text(&commands); + assert_eq!( + commands + .iter() + .filter(|command| command.contains(";r;id=") && command.contains("fmt=png")) + .count(), + 1 + ); + assert_eq!( + commands + .iter() + .filter(|command| command.contains(";p;")) + .count(), + 1 + ); + assert_eq!( + commands + .iter() + .filter(|command| command.contains(";f;") && command.contains("fmt=rgba8")) + .map(|command| *command) + .collect::>(), + vec![ + "\u{1b}_ratty;i;f;id=42;seq=1;fmt=rgba8;w=2;h=2;more=0;AAAAAAAAAAAAAAAAAAAAAA==\u{1b}\\", + "\u{1b}_ratty;i;f;id=42;seq=3;fmt=rgba8;w=2;h=2;more=0;AQEBAQEBAQEBAQEBAQEBAQ==\u{1b}\\", + ] + ); + assert!(commands[commands.len() - 2].contains(";d;pid=7")); + assert!(commands[commands.len() - 1].contains(";d;id=42")); + } + + #[test] + fn frame_chunks_base64_on_aligned_4096_character_boundaries() { + let rgba = vec![7; 3_075]; + + let chunks = encode_frame(BITMAP_ID, 9, 1, 3_075 / 4, &rgba) + .expect_err("invalid example test input should be rejected"); + assert!(chunks.to_string().contains("length")); + + let rgba = vec![7; 4_096 * 3 / 4 + 4]; + let chunks = encode_frame(BITMAP_ID, 9, 1, rgba.len() as u32 / 4, &rgba) + .expect("valid example test input should succeed"); + assert_eq!(chunks.len(), 2); + for (index, command) in command_text(&chunks).iter().enumerate() { + assert!(payload(command).len() <= 4096); + assert_eq!(payload(command).len() % 4, 0); + assert_eq!(command.contains("fmt=rgba8;w=1;h=769"), index == 0); + assert_eq!(command.contains("more=0"), index == 1); + if index == 1 { + assert!(command.starts_with("\u{1b}_ratty;i;f;id=42;seq=9;more=0;")); + } + } + } + + #[test] + fn scheduler_skips_obsolete_ticks_instead_of_bursting() { + let mut scheduler = LatestFrameScheduler::new(std::time::Duration::from_millis(100)); + + assert_eq!( + scheduler.due_sequence(std::time::Duration::from_millis(99)), + None + ); + assert_eq!( + scheduler.due_sequence(std::time::Duration::from_millis(100)), + Some(1) + ); + assert_eq!( + scheduler.due_sequence(std::time::Duration::from_millis(450)), + Some(4) + ); + assert_eq!( + scheduler.due_sequence(std::time::Duration::from_millis(451)), + None + ); + assert_eq!( + scheduler.next_deadline(), + std::time::Duration::from_millis(500) + ); + } + + #[test] + fn timing_validation_rejects_duration_conversion_overflow() { + let error = + validate_timing(15, 1e300).expect_err("invalid example test input should be rejected"); + + assert!(error.to_string().contains("--duration")); + } + + #[test] + fn timing_validation_rejects_sequence_capacity_overflow_and_accepts_boundary() { + let overflow = f64::from(u32::MAX) + 2.0; + let boundary = f64::from(u32::MAX) + 1.0; + + assert!(validate_timing(1, overflow).is_err()); + assert!(validate_timing(1, boundary).is_ok()); + } + + #[test] + fn scheduler_emits_u32_max_at_most_once() { + let mut scheduler = LatestFrameScheduler::new(std::time::Duration::from_nanos(1)); + let saturated = std::time::Duration::from_nanos(u64::from(u32::MAX)); + + assert_eq!(scheduler.due_sequence(saturated), Some(u32::MAX)); + assert_eq!( + scheduler.due_sequence(saturated.saturating_add(std::time::Duration::from_secs(1))), + None + ); + assert_eq!(scheduler.next_deadline(), std::time::Duration::MAX); + } +} diff --git a/examples/bitmap_pan_zoom.rs b/examples/bitmap_pan_zoom.rs new file mode 100644 index 0000000..c751739 --- /dev/null +++ b/examples/bitmap_pan_zoom.rs @@ -0,0 +1,648 @@ +use std::{ + env, fs, + io::{self, Stdout, Write}, +}; + +use anyhow::{Context, Result}; +use base64::Engine as _; +use image::GenericImageView as _; +use ratatui::crossterm::{ + cursor::{Hide, MoveTo, Show}, + event::{self, Event, KeyCode}, + execute, queue, + style::Print, + terminal::{ + self, Clear, ClearType, EnterAlternateScreen, LeaveAlternateScreen, disable_raw_mode, + enable_raw_mode, + }, +}; + +const BITMAP_ID: u32 = 42; +const PLACEMENT_ID: u32 = 7; +const MAX_BASE64_CHUNK: usize = 4096; +const APC_PREFIX: &str = "\u{1b}_ratty;i;"; +const APC_END: &str = "\u{1b}\\"; + +#[derive(Clone, Copy)] +struct Destination { + row: u16, + col: u16, + columns: u32, + rows: u32, +} + +impl Destination { + const fn new(row: u16, col: u16, columns: u32, rows: u32) -> Self { + Self { + row, + col, + columns, + rows, + } + } +} + +#[derive(Clone, Copy, Debug)] +enum Fit { + Contain, + Cover, + Fill, +} + +impl Fit { + const fn protocol_value(self) -> &'static str { + match self { + Self::Contain => "contain", + Self::Cover => "cover", + Self::Fill => "fill", + } + } +} + +#[derive(Clone, Copy, Debug)] +enum Filter { + Nearest, + Linear, +} + +impl Filter { + const fn protocol_value(self) -> &'static str { + match self { + Self::Nearest => "nearest", + Self::Linear => "linear", + } + } +} + +#[derive(Clone, Copy)] +enum Zoom { + In, + Out, +} + +#[derive(Clone, Copy)] +struct ViewState { + bitmap_width: u32, + bitmap_height: u32, + x: u32, + y: u32, + width: u32, + height: u32, + fit: Fit, + filter: Filter, + opacity: f32, +} + +impl ViewState { + fn new(bitmap_width: u32, bitmap_height: u32) -> Self { + Self { + bitmap_width, + bitmap_height, + x: 0, + y: 0, + width: bitmap_width, + height: bitmap_height, + fit: Fit::Contain, + filter: Filter::Linear, + opacity: 1.0, + } + } + + fn pan(&mut self, horizontal: i64, vertical: i64) { + self.x = offset_clamped( + self.x, + horizontal, + self.bitmap_width.saturating_sub(self.width), + ); + self.y = offset_clamped( + self.y, + vertical, + self.bitmap_height.saturating_sub(self.height), + ); + } + + fn zoom(&mut self, direction: Zoom) { + let (new_width, new_height) = match direction { + Zoom::In => ((self.width / 2).max(1), (self.height / 2).max(1)), + Zoom::Out => ( + self.width.saturating_mul(2).min(self.bitmap_width), + self.height.saturating_mul(2).min(self.bitmap_height), + ), + }; + let center_x = self.x.saturating_add(self.width / 2); + let center_y = self.y.saturating_add(self.height / 2); + self.width = new_width; + self.height = new_height; + self.x = center_x + .saturating_sub(new_width / 2) + .min(self.bitmap_width.saturating_sub(new_width)); + self.y = center_y + .saturating_sub(new_height / 2) + .min(self.bitmap_height.saturating_sub(new_height)); + } + + fn cycle_fit(&mut self) { + self.fit = match self.fit { + Fit::Contain => Fit::Cover, + Fit::Cover => Fit::Fill, + Fit::Fill => Fit::Contain, + }; + } +} + +fn offset_clamped(current: u32, delta: i64, maximum: u32) -> u32 { + let next = i64::from(current).saturating_add(delta); + next.clamp(0, i64::from(maximum)) as u32 +} + +fn encode_registration(bitmap_id: u32, encoded_png: &str) -> Vec> { + debug_assert_eq!(MAX_BASE64_CHUNK % 4, 0); + let chunks: Vec<_> = encoded_png.as_bytes().chunks(MAX_BASE64_CHUNK).collect(); + let last = chunks.len().saturating_sub(1); + + chunks + .into_iter() + .enumerate() + .map(|(index, payload)| { + let payload = std::str::from_utf8(payload).expect("base64 is ASCII"); + let more = u8::from(index != last); + if index == 0 { + encode_command(format!( + "r;id={bitmap_id};fmt=png;source=payload;more={more};{payload}" + )) + } else { + encode_command(format!("r;id={bitmap_id};more={more};{payload}")) + } + }) + .collect() +} + +fn encode_placement(bitmap_id: u32, placement_id: u32, destination: Destination) -> Vec { + encode_command(format!( + "p;id={bitmap_id};pid={placement_id};row={};col={};w={};h={};fit=contain;filter=linear;opacity=1", + destination.row, destination.col, destination.columns, destination.rows + )) +} + +fn encode_update(placement_id: u32, view: ViewState) -> Vec { + encode_command(format!( + "u;pid={placement_id};src_x={};src_y={};src_w={};src_h={};fit={};filter={};opacity={:.3}", + view.x, + view.y, + view.width, + view.height, + view.fit.protocol_value(), + view.filter.protocol_value(), + view.opacity + )) +} + +fn encode_deletion(bitmap_id: u32, placement_id: u32) -> [Vec; 2] { + [ + encode_command(format!("d;pid={placement_id}")), + encode_command(format!("d;id={bitmap_id}")), + ] +} + +fn encode_command(body: String) -> Vec { + format!("{APC_PREFIX}{body}{APC_END}").into_bytes() +} + +trait TerminalBackend { + fn enable_raw(&mut self) -> io::Result<()>; + fn enter_alternate(&mut self) -> io::Result<()>; + fn hide_cursor(&mut self) -> io::Result<()>; + fn prepare_screen(&mut self) -> io::Result<()>; + fn show_cursor(&mut self) -> io::Result<()>; + fn leave_alternate(&mut self) -> io::Result<()>; + fn disable_raw(&mut self) -> io::Result<()>; +} + +#[derive(Default)] +struct TerminalSetupState { + raw_enabled: bool, + alternate_entered: bool, + cursor_hidden: bool, +} + +fn setup_terminal(backend: &mut impl TerminalBackend) -> io::Result { + let mut state = TerminalSetupState { + raw_enabled: true, + ..TerminalSetupState::default() + }; + if let Err(error) = backend.enable_raw() { + restore_terminal(backend, &mut state); + return Err(error); + } + + state.alternate_entered = true; + if let Err(error) = backend.enter_alternate() { + restore_terminal(backend, &mut state); + return Err(error); + } + + state.cursor_hidden = true; + if let Err(error) = backend.hide_cursor() { + restore_terminal(backend, &mut state); + return Err(error); + } + + if let Err(error) = backend.prepare_screen() { + restore_terminal(backend, &mut state); + return Err(error); + } + + Ok(state) +} + +fn restore_terminal(backend: &mut impl TerminalBackend, state: &mut TerminalSetupState) { + if std::mem::take(&mut state.cursor_hidden) { + let _ = backend.show_cursor(); + } + if std::mem::take(&mut state.alternate_entered) { + let _ = backend.leave_alternate(); + } + if std::mem::take(&mut state.raw_enabled) { + let _ = backend.disable_raw(); + } +} + +struct CrosstermBackend { + stdout: Stdout, +} + +impl TerminalBackend for CrosstermBackend { + fn enable_raw(&mut self) -> io::Result<()> { + enable_raw_mode() + } + + fn enter_alternate(&mut self) -> io::Result<()> { + execute!(self.stdout, EnterAlternateScreen) + } + + fn hide_cursor(&mut self) -> io::Result<()> { + execute!(self.stdout, Hide) + } + + fn prepare_screen(&mut self) -> io::Result<()> { + execute!(self.stdout, Clear(ClearType::All), MoveTo(0, 0)) + } + + fn show_cursor(&mut self) -> io::Result<()> { + execute!(self.stdout, Show) + } + + fn leave_alternate(&mut self) -> io::Result<()> { + execute!(self.stdout, LeaveAlternateScreen) + } + + fn disable_raw(&mut self) -> io::Result<()> { + disable_raw_mode() + } +} + +struct TerminalSession { + backend: CrosstermBackend, + setup: TerminalSetupState, + bitmap_id: u32, + placement_id: u32, +} + +impl TerminalSession { + fn enter(bitmap_id: u32, placement_id: u32) -> io::Result { + let mut backend = CrosstermBackend { + stdout: io::stdout(), + }; + let setup = setup_terminal(&mut backend)?; + Ok(Self { + backend, + setup, + bitmap_id, + placement_id, + }) + } + + fn write_commands(&mut self, commands: I) -> io::Result<()> + where + I: IntoIterator>, + { + for command in commands { + self.backend.stdout.write_all(&command)?; + } + self.backend.stdout.flush() + } + + fn draw_help(&mut self, view: ViewState) -> io::Result<()> { + queue!( + self.backend.stdout, + MoveTo(0, 0), + Clear(ClearType::CurrentLine), + Print("arrows pan | +/- zoom | f fit | n/l filter | [/] opacity | q quit"), + MoveTo(0, 1), + Clear(ClearType::CurrentLine), + Print(format!( + "crop {}x{}+{},{} | fit {} | filter {} | opacity {:.1}", + view.width, + view.height, + view.x, + view.y, + view.fit.protocol_value(), + view.filter.protocol_value(), + view.opacity + )) + )?; + self.backend.stdout.flush() + } +} + +impl Drop for TerminalSession { + fn drop(&mut self) { + for command in encode_deletion(self.bitmap_id, self.placement_id) { + let _ = self.backend.stdout.write_all(&command); + } + let _ = self.backend.stdout.flush(); + restore_terminal(&mut self.backend, &mut self.setup); + } +} + +fn main() -> Result<()> { + let path = env::args_os() + .nth(1) + .context("usage: cargo run --example bitmap_pan_zoom -- ")?; + let png = fs::read(&path) + .with_context(|| format!("failed to read PNG from {}", path.to_string_lossy()))?; + let image = image::load_from_memory_with_format(&png, image::ImageFormat::Png) + .context("input is not a valid PNG")?; + let (bitmap_width, bitmap_height) = image.dimensions(); + drop(image); + let encoded_png = base64::engine::general_purpose::STANDARD.encode(&png); + + let mut terminal = TerminalSession::enter(BITMAP_ID, PLACEMENT_ID) + .context("failed to enter interactive terminal mode")?; + let mut view = ViewState::new(bitmap_width, bitmap_height); + terminal.draw_help(view)?; + terminal.write_commands(encode_registration(BITMAP_ID, &encoded_png))?; + let (columns, rows) = terminal::size().context("failed to read terminal size")?; + terminal.write_commands([encode_placement( + BITMAP_ID, + PLACEMENT_ID, + Destination::new( + 2, + 1, + u32::from(columns.saturating_sub(2).max(1)), + u32::from(rows.saturating_sub(3).max(1)), + ), + )])?; + + loop { + let Event::Key(key) = event::read().context("failed to read terminal input")? else { + continue; + }; + if !key.is_press() { + continue; + } + + let pan_x = i64::from((view.width / 20).max(1)); + let pan_y = i64::from((view.height / 20).max(1)); + let changed = match key.code { + KeyCode::Char('q') => break, + KeyCode::Left => { + view.pan(-pan_x, 0); + true + } + KeyCode::Right => { + view.pan(pan_x, 0); + true + } + KeyCode::Up => { + view.pan(0, -pan_y); + true + } + KeyCode::Down => { + view.pan(0, pan_y); + true + } + KeyCode::Char('+') | KeyCode::Char('=') => { + view.zoom(Zoom::In); + true + } + KeyCode::Char('-') => { + view.zoom(Zoom::Out); + true + } + KeyCode::Char('f') => { + view.cycle_fit(); + true + } + KeyCode::Char('n') => { + view.filter = Filter::Nearest; + true + } + KeyCode::Char('l') => { + view.filter = Filter::Linear; + true + } + KeyCode::Char('[') => { + view.opacity = (view.opacity - 0.1).max(0.0); + true + } + KeyCode::Char(']') => { + view.opacity = (view.opacity + 0.1).min(1.0); + true + } + _ => false, + }; + + if changed { + terminal.write_commands([encode_update(PLACEMENT_ID, view)])?; + terminal.draw_help(view)?; + } + } + + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + const APC_END: &str = "\u{1b}\\"; + + #[derive(Clone, Copy, Debug, PartialEq, Eq)] + enum TerminalAction { + EnableRaw, + EnterAlternate, + HideCursor, + PrepareScreen, + ShowCursor, + LeaveAlternate, + DisableRaw, + } + + struct MockTerminalBackend { + fail_at: TerminalAction, + actions: Vec, + } + + impl MockTerminalBackend { + fn new(fail_at: TerminalAction) -> Self { + Self { + fail_at, + actions: Vec::new(), + } + } + + fn record(&mut self, action: TerminalAction) -> io::Result<()> { + self.actions.push(action); + if action == self.fail_at { + Err(io::Error::other("injected terminal failure")) + } else { + Ok(()) + } + } + } + + impl TerminalBackend for MockTerminalBackend { + fn enable_raw(&mut self) -> io::Result<()> { + self.record(TerminalAction::EnableRaw) + } + + fn enter_alternate(&mut self) -> io::Result<()> { + self.record(TerminalAction::EnterAlternate) + } + + fn hide_cursor(&mut self) -> io::Result<()> { + self.record(TerminalAction::HideCursor) + } + + fn prepare_screen(&mut self) -> io::Result<()> { + self.record(TerminalAction::PrepareScreen) + } + + fn show_cursor(&mut self) -> io::Result<()> { + self.record(TerminalAction::ShowCursor) + } + + fn leave_alternate(&mut self) -> io::Result<()> { + self.record(TerminalAction::LeaveAlternate) + } + + fn disable_raw(&mut self) -> io::Result<()> { + self.record(TerminalAction::DisableRaw) + } + } + + #[test] + fn enter_alternate_mutation_followed_by_error_still_attempts_reverse_cleanup() { + let mut backend = MockTerminalBackend::new(TerminalAction::EnterAlternate); + + assert!(setup_terminal(&mut backend).is_err()); + + assert_eq!( + backend.actions, + vec![ + TerminalAction::EnableRaw, + TerminalAction::EnterAlternate, + TerminalAction::LeaveAlternate, + TerminalAction::DisableRaw, + ] + ); + } + + #[test] + fn hide_cursor_mutation_followed_by_error_still_attempts_reverse_cleanup() { + let mut backend = MockTerminalBackend::new(TerminalAction::HideCursor); + + assert!(setup_terminal(&mut backend).is_err()); + + assert_eq!( + backend.actions, + vec![ + TerminalAction::EnableRaw, + TerminalAction::EnterAlternate, + TerminalAction::HideCursor, + TerminalAction::ShowCursor, + TerminalAction::LeaveAlternate, + TerminalAction::DisableRaw, + ] + ); + } + + #[test] + fn registration_chunks_base64_on_aligned_4096_character_boundaries() { + let encoded = "A".repeat(4096 * 2 + 8); + + let chunks = encode_registration(BITMAP_ID, &encoded); + + assert_eq!(chunks.len(), 3); + for (index, chunk) in chunks.iter().enumerate() { + let command = + std::str::from_utf8(chunk).expect("valid example test input should succeed"); + let payload = command + .strip_suffix(APC_END) + .expect("valid example test input should succeed") + .rsplit_once(';') + .expect("valid example test input should succeed") + .1; + assert!(payload.len() <= 4096); + assert_eq!(payload.len() % 4, 0); + assert_eq!(command.contains("fmt=png;source=payload"), index == 0); + assert_eq!(command.contains("more=0"), index == 2); + } + } + + #[test] + fn lifecycle_registers_and_places_once_then_only_updates_before_deletion() { + let mut commands = encode_registration(BITMAP_ID, "QUJDRA=="); + commands.push(encode_placement( + BITMAP_ID, + PLACEMENT_ID, + Destination::new(2, 1, 80, 24), + )); + let mut view = ViewState::new(640, 480); + view.zoom(Zoom::In); + commands.push(encode_update(PLACEMENT_ID, view)); + view.pan(16, 8); + commands.push(encode_update(PLACEMENT_ID, view)); + view.cycle_fit(); + commands.push(encode_update(PLACEMENT_ID, view)); + view.filter = Filter::Nearest; + commands.push(encode_update(PLACEMENT_ID, view)); + view.opacity = 0.5; + commands.push(encode_update(PLACEMENT_ID, view)); + commands.extend(encode_deletion(BITMAP_ID, PLACEMENT_ID)); + + let commands: Vec<_> = commands + .iter() + .map(|command| { + std::str::from_utf8(command).expect("valid example test input should succeed") + }) + .collect(); + assert_eq!( + commands + .iter() + .filter(|command| command.contains(";r;id=") && command.contains("fmt=png")) + .count(), + 1 + ); + assert_eq!( + commands + .iter() + .filter(|command| command.contains(";p;")) + .count(), + 1 + ); + assert!( + commands[2..commands.len() - 2] + .iter() + .all(|command| command.contains(";u;pid=")) + ); + assert!(commands[2].contains("src_w=320;src_h=240")); + assert!(commands[3].contains("src_x=176;src_y=128")); + assert!(commands[4].contains("fit=cover")); + assert!(commands[5].contains("filter=nearest")); + assert!(commands[6].contains("opacity=0.500")); + assert!(commands[7].contains(";d;pid=7")); + assert!(commands[8].contains(";d;id=42")); + } +} From d01f3a2e9a15342f5e696e565078c4d94eb50317 Mon Sep 17 00:00:00 2001 From: wipesides Date: Thu, 16 Jul 2026 16:37:06 +0300 Subject: [PATCH 06/10] docs(bitmap): document bitmap surface usage --- README.md | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/README.md b/README.md index 60c0e61..a0ea853 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,7 @@ Inspired by TempleOS | Built with Rust & Ratatui - Spinning rat cursor ([customizable](#changing-the-cursor)) - Traditional 2D and [new 3D mode](#3d-mode)! - [Inline 3D objects](#inline-3d-objects) +- [Inline 2D bitmap surfaces](#inline-2d-bitmap-surfaces) - [GPU-backed text rendering](#rendering-pipeline) - Image support (via Kitty Graphics Protocol >:\() @@ -233,6 +234,58 @@ A blazingly fast serial monitor with plotter TUI and 3D telemetry