Skip to content

Enhance tool evidence display and improve light mode contrast#35

Open
Szowesgad wants to merge 422 commits into
mainfrom
develop
Open

Enhance tool evidence display and improve light mode contrast#35
Szowesgad wants to merge 422 commits into
mainfrom
develop

Conversation

@Szowesgad

Copy link
Copy Markdown
Contributor

This pull request introduces several improvements to tool execution handling, image attachment processing, and hotkey gating logic in the agent/controller pipeline. The changes focus on making tool names more user-friendly, improving the handling and error reporting of image attachments, and refining the logic for hotkey events during agent turns. Comprehensive unit tests are added for all new logic.

Tool Execution and Labeling Improvements:

  • Added friendly_tool_name and tool_running_status functions to map raw tool identifiers to concise, human-readable labels and provide user-friendly status messages during tool execution. Updated tool execution/result handling to use these labels and avoid showing raw wire names in the conversation timeline. [1] [2]
  • Updated logging to include both raw and friendly tool names for debugging, and improved the grouping of tool activity in the UI.
  • Added comprehensive unit tests to ensure tool name mapping and status messages are correct and regression-proof.

Image Attachment Handling:

  • Implemented build_image_attachments_from_text to parse, load, and cap image attachments referenced in outgoing messages, ensuring images are properly forwarded as vision input and missing/unreadable images are reported to the user. [1] [2]
  • Added tests for image attachment parsing, loading, overflow capping, and error reporting.

Agent/Hotkey Gating Logic:

  • Refined hotkey gating logic to allow assistive "Talk Anytime" starts while an agent turn is in progress, but continue to block non-assistive dictation starts. Updated should_block_hotkey_during_agent_send and related controller logic for clarity and testability. [1] [2] [3]

Other Improvements:

  • Skipped empty image content blocks when building input items for the OpenAI provider, preventing request rejection due to empty image data.
  • Minor logging and import updates to support new features. [1] [2]

These changes collectively improve user experience, reliability, and maintainability of the agent/controller interaction pipeline.## Summary

What changed, and why?

User Impact

What does this improve or prevent for a real CodeScribe user?

Runtime Impact

  • Recording, hotkeys, overlays, settings, or release packaging changed
  • Public docs, onboarding, install path, or metadata changed
  • No runtime behavior changed

Verification

  • cargo fmt --all
  • cargo clippy -- -D warnings
  • cargo test
  • make semgrep

Add targeted commands, screenshots, recordings, or release-artifact checks here:

Release Notes

Should this appear in CHANGELOG.md?

  • Yes, added
  • No, internal-only

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several major enhancements to CodeScribe, including an onboarding lane chooser (Basic vs. Agentic), an Agentic-lane readiness probe, grouped tool activity in the voice chat timeline, Light Mode semantic contrast fixes, and a post-LLM lexicon pass to prevent protected term corruption. The review comments correctly identify opportunities to improve Rust stable compatibility by avoiding unstable let-chains, optimize string truncation performance to prevent O(N) traversal on large inputs, and fix a bug where missing image files are incorrectly reported as valid vision inputs.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread app/ui/voice_chat/api/messages.rs Outdated
Comment on lines +209 to +217
if let Some(idx) = state.active_tool_activity_index
&& state
.messages
.get(idx)
.map(|msg| msg.role == ChatRole::ToolActivity)
.unwrap_or(false)
{
return idx;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using let-chains (&& with let) is currently an unstable nightly feature in Rust. To ensure compatibility with stable Rust compilers, it is safer to use nested if let statements.

Suggested change
if let Some(idx) = state.active_tool_activity_index
&& state
.messages
.get(idx)
.map(|msg| msg.role == ChatRole::ToolActivity)
.unwrap_or(false)
{
return idx;
}
if let Some(idx) = state.active_tool_activity_index {
if state
.messages
.get(idx)
.map(|msg| msg.role == ChatRole::ToolActivity)
.unwrap_or(false)
{
return idx;
}
}

Comment thread app/ui/voice_chat/api/messages.rs Outdated
Comment on lines +733 to +739
let capped = if text.chars().count() > MAX_BUBBLE_DISPLAY_CHARS {
let mut t: String = text.chars().take(MAX_BUBBLE_DISPLAY_CHARS).collect();
t.push_str("\n… (truncated for display; full text preserved in the thread)");
t
} else {
text.to_string()
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Calling text.chars().count() traverses the entire string to count characters, which is highly inefficient ($O(N)$) for pathological or extremely large inputs (e.g., pasted base64 blobs or giant log files). Since we only care if the string exceeds MAX_BUBBLE_DISPLAY_CHARS, we can use char_indices().nth(MAX_BUBBLE_DISPLAY_CHARS) to find the truncation boundary in $O(\text{MAX})$ time, completely avoiding traversing the rest of the string.

Suggested change
let capped = if text.chars().count() > MAX_BUBBLE_DISPLAY_CHARS {
let mut t: String = text.chars().take(MAX_BUBBLE_DISPLAY_CHARS).collect();
t.push_str("\n… (truncated for display; full text preserved in the thread)");
t
} else {
text.to_string()
};
let capped = if let Some((idx, _)) = text.char_indices().nth(MAX_BUBBLE_DISPLAY_CHARS) {
let mut t = text[..idx].to_string();
t.push_str("\n… (truncated for display; full text preserved in the thread)");
t
} else {
text.to_string()
};

Comment thread app/ui/voice_chat/api/send.rs Outdated
Comment on lines 1128 to 1142
if codescribe_core::attachment::image_media_type(path).is_some() {
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
if size > codescribe_core::attachment::MAX_VISION_IMAGE_BYTES {
out.push_str(&format!(
"(image too large for vision input: {} bytes > {} max; not sent)\n",
size,
codescribe_core::attachment::MAX_VISION_IMAGE_BYTES
));
} else {
out.push_str("(image detected; will be sent as vision input)\n");
image_paths.push(display.to_string());
}
} else {
out.push_str("(skipped: not UTF-8 text)\n");
out.push_str("(skipped: unsupported image format or not UTF-8 text)\n");
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If the file does not exist or is unreadable, std::fs::metadata(path) will fail, and unwrap_or(0) will return 0. This causes the code to incorrectly report (image detected; will be sent as vision input) for a non-existent file. Handling the Result of metadata properly allows us to correctly report missing or unreadable files as skipped.

            if codescribe_core::attachment::image_media_type(path).is_some() {
                match std::fs::metadata(path) {
                    Ok(meta) => {
                        let size = meta.len();
                        if size > codescribe_core::attachment::MAX_VISION_IMAGE_BYTES {
                            out.push_str(&format!(
                                "(image too large for vision input: {} bytes > {} max; not sent)\n",
                                size,
                                codescribe_core::attachment::MAX_VISION_IMAGE_BYTES
                            ));
                        } else {
                            out.push_str("(image detected; will be sent as vision input)\n");
                            image_paths.push(display.to_string());
                        }
                    }
                    Err(_) => {
                        out.push_str("(skipped: image file not found or unreadable)\n");
                    }
                }
            } else {
                out.push_str("(skipped: unsupported image format or not UTF-8 text)\n");
            }

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the assistive voice-chat UX by (1) grouping and “de-wiring” tool evidence in the conversation timeline, (2) making image attachments reliably flow as real vision input with clearer user-visible error handling, and (3) improving Light Mode readability via deterministic contrast tokens; it also refines hotkey gating and adds an onboarding “Basic vs Agentic” lane with readiness probing.

Changes:

  • Added grouped Tool Activity blocks (per assistant turn) and friendly tool labels/status messaging; tool failures now carry an explicit is_error bit to the UI.
  • Centralized image-attachment marker parsing + vision loading in core::attachment, wired through both agent and legacy send paths, and added protection against empty image blocks.
  • Introduced deterministic Light/Dark contrast palettes (WCAG-checked) and added onboarding lane persistence + agentic readiness probe/step flow.

Reviewed changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
pico.save Removed an editor scratch file from the repo.
docs/architecture/attachment-vision-pipeline.md Added architecture doc for the attachment→vision pipeline and its shared marker contract.
core/tests/env_validation.rs Tightened env-loader test isolation for llm_endpoint.
core/quality/qube_report.rs Added protected-term loss detection to quality reporting.
core/pipeline/stream_postprocess.rs Added operator vocabulary + protected terms lexicons; added lexicon re-apply and loss detection helpers; expanded tests.
core/llm/ai_formatting.rs Reused shared attachment parsing/loading; increased image cap; applied lexicon post-LLM.
core/config/settings.rs Added persisted onboarding_mode to settings (incl. env promotion + tests).
core/attachment.rs Introduced shared image marker parsing + vision image loading + tests.
core/agent/session.rs Tool-result summaries now surface first error message; propagates is_error to UI event.
core/agent/event.rs Added is_error to AgentUiEvent::ToolResult for UI rendering.
assets/protected_terms.jsonl Added curated protected-term lexicon entries (proper nouns/brands/tools).
assets/programming.jsonl Updated Loctree canonical term/casing and expanded mispronunciations.
assets/operator_vocabulary.jsonl Added Polish UI-command/operator normalization seed lexicon.
app/ui/voice_chat/tool_activity.rs New pure module for per-turn grouped tool activity + evidence summaries + tests.
app/ui/voice_chat/state.rs Added tool-activity group storage and “last sent attachments” state.
app/ui/voice_chat/mod.rs Exported tool-activity APIs and module.
app/ui/voice_chat/handlers/menus.rs Added “Re-attach previous” menu item; updated attachment menu logic.
app/ui/voice_chat/handlers/classes.rs Registered new onAttachReattach: handler.
app/ui/voice_chat/handlers/attachments.rs Implemented re-attach behavior for last-sent attachments.
app/ui/voice_chat/api/tests.rs Added state-level tests for grouped tool activity rendering and turn boundaries.
app/ui/voice_chat/api/send.rs Cleared attachments after send but preserved for re-attach; improved image honesty in attachment block.
app/ui/voice_chat/api/mod.rs Imported tool-activity types used by the API layer.
app/ui/voice_chat/api/messages.rs Added tool-activity record APIs; added bubble text capping/breaking to prevent CoreText hangs; added click toggling behavior.
app/ui/voice_chat/api/lifecycle.rs Ensured tool-activity turn state resets on overlay clear.
app/ui/voice_chat/api/export.rs Included Tool Activity role in markdown export labels.
app/ui/shared/helpers/mod.rs Added deterministic Light/Dark contrast palette tokens + WCAG tests + appearance resolution.
app/ui/settings/engine_tab.rs Added agentic readiness rows (gated by persisted onboarding mode); refactored tone→color mapping.
app/ui/onboarding/window.rs Added onboarding lane chooser UI and readiness table UI scaffolding.
app/ui/onboarding/widgets.rs Added mode radio syncing and a system orange color helper.
app/ui/onboarding/tests.rs Added tests for onboarding mode persistence and lane-dependent step navigation.
app/ui/onboarding/steps.rs Inserted Mode step; added AgenticReadiness step; kept stable index flow array.
app/ui/onboarding/state.rs Added OnboardingModeChoice and initial mode selection from settings.
app/ui/onboarding/render.rs Rendered Mode/AgenticReadiness steps; lane-aware sidebar markers; readiness table rendering.
app/ui/onboarding/handlers.rs Added onModeSelected: handler wiring.
app/ui/onboarding/actions.rs Added lane-aware step navigation; persisted onboarding mode; removed hardcoded full-disk step index.
app/controller/tests.rs Updated hotkey gating tests; added predicate-level tests for assistive “Talk Anytime” vs raw start blocking.
app/controller/mod.rs Allowed assistive start events during agent in-flight sends; kept Busy-state hotkey blocking.
app/controller/helpers.rs Added friendly tool-name/status mapping; implemented image-attachment loading for agent send path; updated tool UI event handling; added tests.
app/agent/tools/mcp.rs Added agentic readiness probe and tests; classified prerequisites including PRView detection heuristic.
app/agent/openai_provider.rs Skipped empty image blocks when building provider input to avoid request rejection.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread app/ui/voice_chat/api/send.rs Outdated
Comment on lines 1128 to 1140
if codescribe_core::attachment::image_media_type(path).is_some() {
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
if size > codescribe_core::attachment::MAX_VISION_IMAGE_BYTES {
out.push_str(&format!(
"(image too large for vision input: {} bytes > {} max; not sent)\n",
size,
codescribe_core::attachment::MAX_VISION_IMAGE_BYTES
));
} else {
out.push_str("(image detected; will be sent as vision input)\n");
image_paths.push(display.to_string());
}
} else {
Comment thread core/attachment.rs
Comment on lines +282 to +285
/// Load an image file as `(bytes, media_type)` for vision input.
///
/// Returns `None` (with a warning) when the extension is not a vision-supported
/// image, the file is unreadable, or it exceeds `max_bytes`.
/// Loaded rules-only via `load_seed_jsonl` (seed format gives whole-word +
/// case control), so these common words never enter `protected_canonicals` and
/// never trip the downstream loss-detection gate. Canonicals were confirmed
/// real and high-frequency via `loct occurrences` before being chosen.
Comment thread app/controller/helpers.rs
Comment on lines +441 to +445
let mut chars = word.chars();
if let Some(first) = chars.next() {
out.extend(first.to_uppercase());
out.push_str(chars.as_str());
}
Comment thread app/controller/helpers.rs
Comment on lines +1294 to +1297
assert_eq!(
friendly_tool_name("mcp__github__create_issue"),
"Create Issue · Github"
);
Comment thread app/controller/helpers.rs
Comment on lines +1358 to +1361
assert_eq!(
tool_running_status("Create Issue · Github"),
"Running Create Issue · Github…"
);
Comment thread app/ui/voice_chat/tool_activity.rs Outdated
Comment on lines +374 to +378
fn prettify_source(server: &str) -> String {
let cleaned: String = server
.chars()
.map(|c| if c == '-' || c == '_' { ' ' } else { c })
.collect();
Comment thread core/llm/ai_formatting.rs
Comment on lines 461 to +463
fn build_responses_user_content(user_message: &str) -> Vec<InputContent> {
const MAX_IMAGES: usize = 4;
// Kept in sync with `MAX_AGENT_VISION_IMAGES` in the agent send path.
const MAX_IMAGES: usize = 16;
m-szymanska and others added 25 commits July 1, 2026 22:35
The Settings scene rendered as a fixed content-sized window (only a minWidth/
minHeight floor, no room to grow).

- Apply `.windowResizability(.contentMinSize)` to the Settings scene so the
  content frame's floor becomes the window minimum and it can grow from there.
- Relax the SettingsView frame to allow growth (maxWidth/maxHeight .infinity)
  so the content scales instead of clipping.

SwiftUI restores the window frame across launches automatically.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add os_log Logger breadcrumbs (subsystem com.vetcoders.codescribe,
category "attachments") along the composer attachment staging path to let
a single operator repro pinpoint where a picked image is lost:

- Composer.pickAttachments: panel presentation + completion (OK/cancel,
  url count, lastPathComponent list — no full paths).
- AgentChatStore.addAttachments: incoming vs post-dedupe count + resulting
  pendingAttachments.count.
- AgentChatStore.send: attachmentPaths.count + text.isEmpty at request build.
- RealChatEngine.streamReply: which bridge call runs (streamReply vs
  streamReplyWithAttachments) and attachment count.

Diagnostic only; no staging/send logic changed. NSOpenPanel kept modeless
(begin) — Composer has no NSWindow handle, so beginSheetModal would need a
bridging refactor out of scope for this cut.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tray's "Recording" pill and "Stop Dictation" row are driven off the
overlay recording lifecycle: handleRecordingStarted fires
onRecordingStarted?() (tray -> recording), but no code path ever fired
onRecordingStopped?(). The tray therefore only cleared via the popover's
one-shot onAppear poll, so a hotkey-driven stop (toggle/hold) left it
stuck showing "Recording" for the rest of the process lifetime.

Fire onRecordingStopped?() from finalizeTranscript() -- the single
authoritative stop funnel that finishControllerRecording, runStop, and
applySessionFinalised all route through -- gated on the finalize
transition so the AppModel onRecordingStopped closure's re-entrant
markStopped() cannot recurse or churn @published tray state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A sent user turn now carries its staged attachments onto the You bubble
and renders them as chips (mono filename + photo glyph, or a small inline
thumbnail when the source image still loads), reusing the composer's chip
style. This closes the UX gap where a message sent with an image left no
trace in the transcript.

Chips are in-memory for the current session: the persisted thread stores
composer images as `type: "image"` with data_omitted and no filename/path
(see thread_store), so a restored turn comes back without them. Surfacing
chips after restore would require carrying attachment metadata through the
FFI/persistence layer — left as a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…esult

The system-wide AX read behind "Save selection" can't see a SwiftUI
`Text.textSelection` selection made in our own agent window: opening the
tray popover steals key focus and SwiftUI doesn't expose `AXSelectedText`,
so the action silently no-ops (or saves stale clipboard). It also gave no
visible feedback — the OS notification an accessory app posts is not
guaranteed to surface.

- Harvest the live selection from the agent window's responder chain
  (`copy:` with pasteboard snapshot/restore) before falling back to the
  existing AX/clipboard path, so a drag-selected bubble reaches the daily
  note without a manual Cmd+C.
- Route every save outcome (saved / nothing-to-save / error) to a
  transient, permission-free banner in the still-open popover; covers both
  Save selection and Save last transcript so neither can fail silently.
- Add `notes` os_log breadcrumbs across the save path for diagnosis.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Message bodies were showing raw markdown (literal **bold**, `- `/`1. `
list markers, `#` headings); only backtick code spans were styled. Replace
CodeSpanText with a MarkdownText renderer: a small line-based block parser
(paragraphs, `#`-`###` headings, bullet/ordered lists, fenced ``` code
blocks) whose inline text is handed to AttributedString(markdown:) for
**bold**, *italic*, `code` spans (mono + olive) and [links](url)
(terracotta, opened via NSWorkspace). Parse failure falls back to the raw
string so a bubble is never empty. Inputs are value types, so SwiftUI
re-parses only the turn whose text changed — the single streaming turn,
not the whole history.

Add a subtle inline "copy" button in each turn's meta row (mono 10, faint
until hovered) that copies the raw pre-render text via the same chatCopy
path as the existing context-menu Copy, flipping to "copied" for ~1.5s.
For a text-less image turn it copies the attachment filenames instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Long names (e.g. "Screenshot 2026-07-02 at 1.37.37 PM.png") were hard
clipped on the right with no ellipsis, hiding the extension. Switch both
the composer staging chip and the sent-bubble AttachmentChip to
.truncationMode(.middle) so the start and the extension stay visible
(e.g. "Screenshot 20…PM.png").

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add an .onDrop file-URL target over the whole bottom composer strip so an
image dragged from Finder is staged through the existing
store.addAttachments path — a chip appears and it sends exactly like a
📎-picked image. Multiple files stage multiple chips. The input box
border lights up terracotta while a file is dragged over it. Accepted
types mirror the NSOpenPanel picker (PNG/JPEG/GIF/WebP/BMP/TIFF); other
files are rejected with an "attachments" os_log breadcrumb. Previously
there was no drop target at all, so Finder drags did nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Thread titles were derived from the first user message on every save, and
since agent chats open with a pasted instruction preamble, every thread in
the rail read "INSTRUKCJA UŻYTKOWNIKA: JESTEŚ AGENTEM…", making search useless.
There was also no way to rename a thread.

- core ThreadStore: add `set_thread_title` (persists title + marks it custom;
  leaves `updated_at` so a rename does not reorder the rail) and a
  `title_is_custom` flag on `Thread` (`#[serde(default)]`, back-compatible).
- bridge: expose `CodescribeThreads::rename_thread`; skip auto-titling when
  `title_is_custom` so a hand-set title is never clobbered on the next turn.
- derive_thread_title now walks the raw (newline-preserving) first message and
  drops leading instruction/all-caps-header lines, falling back to the previous
  collapsed-text behavior when every line looks like boilerplate.
- mirror the no-clobber guard in the legacy controller persist path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a Rename affordance to ThreadRail: context-menu "Rename" plus double-click
on the title swaps it for an inline TextField. Enter (or click-away) commits,
Esc reverts. Commits route through the store's `rename`, which persists via the
threads provider for on-disk threads and updates in memory otherwise; the chat
header reads `currentThread.title` so it refreshes reactively.

- ChatThreadsProviding + RealThreadsEngine: `renameThread` over the FFI.
- AgentChatStore: `rename(_:to:)` — no-op on empty/unchanged title.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ck loop

The dictation overlay could appear in a random one of three broken states
between shows, actively oscillated between two sizes while recording, ignored
clicks, and eventually hung the app.

Root cause (confirmed by a hung-process sample): the panel's content was hosted
in an NSHostingController whose `sizingOptions = []` only stopped the CONTROLLER
from driving `preferredContentSize`. The underlying NSHostingView still installed
Auto Layout min/max/intrinsic constraints derived from the (flexible, constantly
animating) SwiftUI fitting size and pushed them onto the `.resizable` window every
display cycle. That closed a feedback loop — window resized to the fitting size,
the flexible content (`maxHeight: .infinity`) re-fit to the new frame, produced a
different fitting size, and the two chased each other. The sample showed the main
thread ground in `NSWindow.updateConstraintsIfNeeded -> NSHostingView.updateConstraints
-> NSHostingController.preferredContentSize`, never settling, which is both the
visible oscillation and the hang. A separate facet of the same desync left the
content anchored past the window's left edge with dead space on manual resize.

Fix:
- Host the content directly as `panel.contentView = NSHostingView` and set the
  VIEW's `sizingOptions = []` so it exports no size constraints at all; the window
  is sized only by us and by the user's edge-drag. This structurally removes the
  loop (and the hang).
- Make the hosting view track the window bounds via autoresizing (`.width, .height`)
  so every resize reflows the SwiftUI layout (header top, footer bottom, transcript
  region absorbing the middle) instead of rendering at a fixed creation-time frame.
- Enforce the 460x300 floor for programmatic sizing too (`contentMinSize` + a
  `clamp(_:to:)` applied in `show()`), which AppKit's `minSize` does not cover.
- Replace `setFrameAutosaveName` (which persisted the loop's runaway sizes and
  restored a stale oversized frame -> ghost-outline/clipped states) with manual
  UserDefaults persistence, clamped to the min and the current screen on restore.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Agent Chat header carried inert decoration: a Drawer/Agent segmented
toggle for a mode that does not exist, a "restore" glyph for an
unbuilt thread-memory-restore feature, a mic glyph (dictation entry
belongs in the composer, landing separately), and a hard-coded "Idle"
pill and "thread memory on" label that never reflected runtime.

Remove the toggle, restore, and mic. Wire the remaining controls to
truth:
- status pill reads live store state — Idle (olive) / Thinking (amber)
  / Streaming (terracotta) via new AgentChatStore.isThinking/isStreaming
- gear opens the Settings scene via the openSettings environment action
  (same path the tray uses)
- title bar drops the false "thread memory on" text, keeps the turn count

Composer: trim the affordance row to the two truthful hints (streaming,
attach file / image) and simplify the placeholder to "Type a message…"
(there is no Fn-to-speak in chat yet). The header ellipsis is retained
inert; it gains a thread menu in a follow-up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reintroduce the chat-to-Markdown export dropped with the legacy voice-chat
UI (37efe51), rebuilt against the persisted ThreadStore model.

- core/agent/thread_export: pure, unit-tested `thread_to_markdown(&Thread,
  assistant_only)` — a thread-title H1, export metadata, then one
  `## Role · timestamp` section per turn. Content flattening is type-aware
  (keyed on the canonical content-block shapes) so it yields clean prose
  and drops tool-use / image payloads, rather than leaking structural
  field values the way a blind key walk would.
- bridge CodescribeThreads::export_thread_markdown loads a thread, formats
  it, and writes the file under ~/.codescribe/transcriptions/YYYY-MM-DD/
  with collision-avoidance, mirroring the old save_chat_markdown_to_history
  and returning the absolute path. `assistant_only` selects the
  assistant-replies-only variant.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Give the header ellipsis a real job: a menu for the current thread with
Rename, Favorite/Unfavorite, Export to Markdown (all turns), Export
assistant replies only, and a destructive Delete under a separator.

- Rename opens an alert with a title field and commits through the
  existing store.rename path (same core rename as the rail inline edit).
- Favorite/Delete reuse store.toggleFavorite / store.delete.
- Export calls the new export_thread_markdown FFI via a
  ChatThreadsProviding.exportThreadMarkdown seam (RealThreadsEngine) and
  AgentChatStore.exportMarkdown; on success it reveals the written .md in
  Finder (activateFileViewerSelecting — no file-access prompt, the path is
  under the app's own ~/.codescribe data dir). Export entries are hidden
  for a not-yet-persisted local thread (no backend id).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restored agent-thread messages collapsed into a single paragraph: the
live stream hands raw markdown to the block renderer, but the FFI flatten
that rebuilds a message's display text on load ran
`split_whitespace().join(" ")`, folding every newline into a space so the
block parser saw one giant line. Preserve intra-block newlines, trim only
each block's outer edges, and join distinct content blocks with a blank
line. Also drop the `type` metadata key from the display walk (it was
surfacing a literal "text" token after every block); the search-index
twin in thread_index.rs keeps it, harmless once lowercased.

Adds round-trip tests over flatten_message_text covering newline
preservation, block separation, and binary/blank skipping.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extend the chat markdown renderer (MarkdownText / MDBlock) with three
GFM block types beyond paragraphs, headings, lists and code fences:

- Tables: a pipe row gated by a `|---|---|` separator parses into a Grid
  with a mono, high-contrast header over a faint fill, hairline row
  separators, and inline-markdown cells that share width and wrap, so a
  wide table never overflows the bubble.
- Blockquotes: `>` lines (single- or multi-line; deeper `>>` collapses to
  one level) render with a terracotta hairline bar and dimmed body text.
- Task lists: `- [x]` / `- [ ]` render an SF Symbol checkbox (filled olive
  when done, faint square when open) instead of literal brackets, with
  done items slightly dimmed.

Parsing stays line-based and value-typed, so a bubble still re-parses only
when its own text changes; malformed input falls back to prose and the
streaming caret keeps working on the last block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Harden the block parser against cases the torture-test surfaced:

- Thematic breaks: a standalone `---`/`***`/`___` (3+, spaces allowed,
  including `- - -`) now renders as a hairline rule instead of literal
  text. Table separator rows are consumed by the table branch first, so
  a lone `---` is unambiguously an HR.
- Wide tables: replace the SwiftUI Grid with a custom Layout that weights
  each column by its longest cell but holds a per-column minimum, so one
  very long column can no longer crush the rest to word-per-line slivers.
- Nested fences: a code block opened with N backticks now closes only on
  a line whose backtick run is >= N (CommonMark), so a ````md block can
  carry an inner ```ts fence verbatim instead of leaking and leaving an
  empty block behind.
- Blockquotes parse their body recursively, so nested lists, code fences
  and `>>` quotes render structured instead of half-raw.
- Inline HTML is fully escaped (outside code spans): raw tags always show
  as literal text, never dropped or half-rendered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Build on the renderer edge-case fixes with the affordances and polish
the torture-test called for:

- GitHub alerts: a blockquote opening with `[!NOTE]`/`[!TIP]`/
  `[!IMPORTANT]`/`[!WARNING]`/`[!CAUTION]` renders as a callout — SF
  Symbol icon, tinted label and left bar over a faint wash, body parsed
  normally. Reuses the recursive blockquote parse path.
- Code blocks gain a hover-revealed copy button in the corner that copies
  the raw body and flips to a green "copied" for ~1.5s, matching the
  message-level copy affordance.
- Polish: inline code sits on a raised chip so it detaches from prose
  (also inside checkboxes and table cells); checked task boxes read in a
  fuller olive; links carry a subtle underline; deep (level 3+) bullets
  use a lighter hollow marker.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A file dropped directly onto the composer's NSTextField was consumed by
the field editor and pasted as a path string, bypassing the container
.onDrop staging added in 0903fab (which only fired for drops outside the
field). Layer a transparent drop-catcher above the input box that is
hit-testable only mid-drag, so it beats the field editor to a field drop
and routes it through handleDrop (chip staging) while leaving clicks and
typing untouched at rest. Drag-over state is OR'd across an outer target
(whole strip, bootstraps the drag) and the catcher's own isTargeted, so
the highlight stays stable as the pointer crosses from padding onto the
field.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… content or squares the corners

After the feedback-loop hotfix, the resizable dictation panel filled its
window via an autoresizing mask ([.width, .height]) on the NSHostingView.
That mask resizes by DELTAS measured from the view's initial frame, so on a
borderless resizable panel the hosting view drifted off the window's content
bounds after a user edge-drag: content spilled past the window edge (action
row clipped at the bottom, DICTATION pill/waveform anchored past the left
edge) and, because the SwiftUI rounded glass background was then painted
beyond the window rectangle, the visible corners squared off into a sharp
box.

Replace the mask with an absolute per-step frame sync: host the SwiftUI view
inside an OverlayContentContainer whose setFrameSize/layout re-assert
`hosting.frame = bounds`. The window resizes the container (its contentView)
on every resize step, including each step of a live edge-drag, so the glass
panel now covers the window 1:1 at any size — content never clips and the
rounded corners survive every resize.

The container exports no layout constraints and the hosting view keeps
`sizingOptions = []`, so the content-window sizing feedback loop that once
hung the app stays structurally dead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…il patches never silently miss

Pin the trim/offset contract to a single owner (the Rust emitter): session.rs
already trims at source (final_text = accumulated_text.trim()) so
UtteranceFinal.text == tail-patch committed_text by construction - this commit
adds the debug_assert invariant + comment guarding that identity against future
emitters, fixes the stale tail_patcher doc (claimed 'not yet wired into the
streaming hot path' while session.rs wires it at :132/:407/:1042), and adds a
regression test proving a leading/trailing-whitespace utterance still yields
Patches (not a silently Skipped out-of-range patch).

Verified: cargo test -p codescribe-core green (599 passed, includes new
regression tests); clippy --workspace -D warnings clean.

Authored-By: claude <agents@vetcoders.io>
session_id: 509ae15c-feea-43c2-9b85-7446deb07d4a
date: 2026-07-01T22:41:09 PDT
runtime: claude

(cherry picked from commit f8a9491)
…ffset contract

Remove the mutating trim in the keyed-by-utteranceId final-segment path
(applyFinal/upsertFinalSegment): the Rust emitter now owns the trim (see the
paired pipeline commit), so Swift-side trimming was the only mechanism left
that could shift ReplaceRange offsets and make the out-of-range guard silently
drop a valid patch. A debug-only assertion replaces it as a sensor for the
emitter guarantee (no Swift test target exists in project.yml to host a test).

Verified: make app PROFILE=debug with regenerated UniFFI bindings ->
BUILD SUCCEEDED.

Authored-By: claude <agents@vetcoders.io>
session_id: 509ae15c-feea-43c2-9b85-7446deb07d4a
date: 2026-07-01T22:41:23 PDT
runtime: claude

(cherry picked from commit b7824ba)
…transcription toggle

Surface two runtime knobs in Settings (Engine section): an STT engine selector
(Auto/Apple/Whisper -> CODESCRIBE_STT_ENGINE) and a Layered transcription
(experimental) toggle (off/phase1 -> CODESCRIBE_LAYERED_TRANSCRIPTION), wired
through the existing generic update_config FFI (no new bridge methods).

Deliberately NOT promoted to PROMOTED_SETTINGS_KEYS: the loader skips promoted
keys during .env file injection unconditionally, which would silently kill
documented .env workflows like CODESCRIBE_STT_ENGINE=onnx (the GUI intentionally
does not expose onnx). Instead settings.json only seeds the env under a
std::env::var().is_err() guard, so a manual .env line keeps winning - this was
caught by the adversarial review stage and fixed before commit.

Defaults preserve today's behavior exactly: engine auto, layered off.

Verified: cargo test --workspace (only the 2 known pre-existing failures:
e2e_env_registry / ffi mode_binding); clippy -D warnings clean; make app
PROFILE=debug BUILD SUCCEEDED with regenerated bindings.

Authored-By: claude <agents@vetcoders.io>
session_id: 509ae15c-feea-43c2-9b85-7446deb07d4a
date: 2026-07-01T22:41:39 PDT
runtime: claude

(cherry picked from commit 9d25a90)
…ropic catalog, keep Sonnet 5

Interim catalog correction after the operator refuted the audit's
'sonnet-5 is fictional' claim: Claude Sonnet 5 is REAL (premiered after the
line's CORRECTION.md snapshot of 2026-06-04). What the catalog actually
lacked was claude-sonnet-4-6 - the chosen formatting model, which
capability_policy already handled but neither the Rust catalog nor the
SettingsEngine.swift fallback ever offered. Both now list the full current
family (Opus 4.8 / Sonnet 5 / Sonnet 4.6 / Opus 4.7 / Haiku 4.5) and the
catalog typo-guard test covers all five.

Direction note (operator directive): static catalogs are interim only - model
lists must come from per-key auto-discovery (GET /v1/models); a dedicated
brief follows.

Verified: cargo test -p codescribe-core --lib llm::provider -> 16 passed.

Authored-By: claude <agents@vetcoders.io>
session_id: 509ae15c-feea-43c2-9b85-7446deb07d4a
date: 2026-07-02T12:54:44 PDT
runtime: claude
Add one INFO line per agent tool call (native + MCP) at the session
dispatch point: tool name, duration, ok/error, and on failure the error's
first line. Never logs arguments or full payloads; MCP calls surface the
server segment of the public name. Makes bare "failed <tool>" activity rows
diagnosable from codescribe.log after the fact.

For the MCP stdio client, pipe stderr (was null) and, on a
spawn-survived-but-handshake/call-failed exchange, emit a WARN enriched
with the process stderr (collapsed, truncated to 200 chars) bounded by a
short drain timeout so a still-alive child cannot block the diagnostic.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
m-szymanska and others added 30 commits July 9, 2026 18:41
- Preserve legacy delta sink spacing across utterance boundaries without duplicating boundary corrections
- Require low-confidence evidence before rejecting exact Polish transcription terms
- Count STT quality-gate drops in session Stats instead of leaving semantic_gate_drops at zero
- Add focused regression tests for sink deltas, PL hallucination terms, and gate-drop counting
fix(ci): pin Rust Quality jobs to macOS self-hosted runner
A bare tokio::spawn drove the agent turn with no abort path: a cancelled
turn kept executing tools (typing/clipboard/fs) to completion. The turn
task is now tied to its run_stream future through an abort-on-drop RAII
guard and registered in a per-thread TurnRegistry behind a new
CodescribeAgent.cancel_turn(thread_id) FFI method.

The explicit FFI cancel is required end-to-end: the generated UniFFI 0.30
Swift bindings poll Rust futures to completion and never propagate Swift
Task cancellation, so the guard alone would be unreachable from the app.
AgentChatStore.delete() now calls engine.cancelReply(threadId:) so
deleting a thread mid-stream actually stops the turn instead of only
silencing its UI updates.

An aborted turn is not persisted (the thread keeps its last completed
turn), aborting a finished task is a no-op (no completion race), and a
cancel with no active turn is a safe no-op.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Remove duplicate UI error events from fatal AgentSession failure paths.\n- Keep Swift-facing visibility through the existing thrown bridge error catch path.\n- Add bridge coverage proving provider errors are reported once.\n- Preserve turn cancellation semantics by leaving abort/cancel wiring untouched.
- Observe one-shot Settings deep-link changes while the window is already open
- Share a single SettingsView consumption path for appear and focus-time updates
- Clear pending targets only after a real target is consumed to avoid replay loops
- Use the time-indexed SpeechTranscriber preset and emit bridge segments from audioTimeRange attributes.
- Map bridge segment fixtures into RawTranscript with validation before downstream pipeline use.
- Cover the Apple segment path with a Rust fixture that reaches the Silero tail-drop filter.
…ing state

- Route composer dictation listener engine errors back to the main adapter.

- Report active engine failures through AgentChatStore so the composer leaves .recording.

- Release a stale dictationBlocked flag on the terminal composer error path.

- Keep successful stop-and-insert dictation flow unchanged.
- Wrap remaining SwiftUI #Preview blocks in debug-only compilation guards

- Match the existing guarded preview pattern without changing runtime code

- Keep batch scope limited to macOS Swift preview guard lines
- Keep full raw Whisper text when Silero did not drop segments but segment text is a strict subset
- Reduce decoder control-token suppression from 16 generated tokens to the pre-first-token guard
- Add focused regression tests for Silero text preservation and short utterance termination
- Avoid pre-commit hook side effects by committing only the STT engine file
- Confine config-owned process env seeding behind a bootstrap gate

- Persist runtime settings updates to settings/json env stores without set_var

- Read bridge settings snapshots from persisted stores instead of live env side effects

- Update lifecycle tests for the single-writer env contract
- Run Commit requests before Live requests at Critical thermal level.

- Keep Live requests throttled at Critical while preserving coalescing behavior.

- Deregister scheduler senders on shutdown and drop to prevent registry growth.

- Add scheduler tests for Critical thermal priority and registry cleanup.
Save migrated Keychain secrets before writing the settings.json completion sentinel.\n\n- Return early on migrated secret persistence failure so startup retries migration\n- Add a test-only save_key failure injector without changing the env registry\n- Cover retry success and one-time completion behavior for migrated API keys
Convert account auth runtime creation into a bridge Config error instead of panicking.\n\n- Propagate runtime initialization failure through start_account_login\n- Preserve the existing CsError contract for Swift callers\n- Keep account login server failures on the existing account_auth_to_cs path
- Default STT bench fixture selection to repo assets so bare runs ignore private ~/.codescribe corpora.

- Keep historical corpora available only through explicit --fixtures historical.

- Add --list-fixtures for deterministic no-model fixture-selection smoke checks.

- Stamp fixture_source in both honest failure reports and full benchmark reports.
- Remove the unused async downloader API and its private download helper.

- Drop the stale Silero VAD URL export while keeping legacy path helpers used by tests.

- Keep core/vad/mod.rs changes limited to the install re-export surface.
…onstant

- Replace the stale 45s Busy warning with STOP_TIMEOUT.as_secs().
- Remove stale 45s references from nearby watchdog comments.
- Keep timeout behavior unchanged; only message/comment text changed.
- Remove dead public VAD trigger helpers from RecordingController.\n- Confirmed both identifiers had zero callsites with loct literal scans.\n- Keep Busy watchdog behavior unchanged after EXT-1 falsification.
- Replace the recorder streaming buffer storage with VecDeque for O(1) front eviction.

- Preserve snapshot and stop callers by collecting retained samples into contiguous Vecs on cold paths.

- Extend the cap test to verify retained tail samples and absolute offset tracking.
- Gate semantic_gate_drops on final utterance items so interim previews do not inflate stats.

- Keep utterance confidence metadata fallback behavior intact while final items still take precedence.

- Update the counter test to cover ignored interim preview drops.
- Replace the redundant is_hallucination wrapper branch with a direct quality-aware call.

- Preserve behavior by passing avg_logprob through as Option<f32>, including None.
- Keep VAD-stop callbacks registered when auto_silence is disabled instead of clearing them during registration.\n- Warn from start() when a registered callback is inert under disabled auto_silence.\n- Gate callback emission on auto_silence and update recorder tests for retained-but-inert behavior.
- Clamp partially overlapping Apple STT segment starts to the previous end instead of dropping words.

- Keep fully contained or invalid segments filtered out.

- Preserve monotonic finite segment timestamps for downstream timeline consumers.
- Preserve raw Whisper text when Silero keeps all segments and only case or punctuation differs.

- Keep strict-subset preservation for no-drop segment text while retaining filtered fallback for real drops.

- Add regression coverage for no-drop equivalence and dropped-segment fallback.
- Post pendingSectionDidChange only when a real pending Settings section is set.

- Keep consume() as the one-shot clear path without triggering duplicate navigation.

- Preserve the existing SettingsView receiver contract for non-nil deep links.
- Replace the empty debug CARGO_FLAGS array with explicit debug/release cargo build branches.

- Keep make app behavior unchanged while making bash nounset safe on macOS.

- Unblock the required Swift app quality gate for debug builds.
fix: backlog cleanup batch 1 — audio, pipeline, agent lifecycle
fix: backlog cleanup batch 3 — reproducibility, dead code, stop-timeout truth (stacked on #59)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants