Skip to content

Use configurable transcribe.cpp output budget - #1633

Open
vladkalinichencko wants to merge 1 commit into
cjpais:mainfrom
vladkalinichencko:draft/transcribe-cpp-max-new-tokens
Open

Use configurable transcribe.cpp output budget#1633
vladkalinichencko wants to merge 1 commit into
cjpais:mainfrom
vladkalinichencko:draft/transcribe-cpp-max-new-tokens

Conversation

@vladkalinichencko

@vladkalinichencko vladkalinichencko commented Jul 8, 2026

Copy link
Copy Markdown

Depends on handy-computer/transcribe.cpp#74. Until that is merged and released, the Cargo patch points to the PR branch so this integration can be reviewed and tested against the exact implementation.

This keeps the Handy-side change small:

  • request a 4096-token output budget for transcribe.cpp runs
  • keep non-empty partial text when transcribe.cpp reports OUTPUT_TRUNCATED
  • show the backend error message when history re-transcription fails
  • update the integration to the current transcribe.cpp 0.2.0 API

Validated with:

  • cargo fmt --check
  • cargo check
  • 151 Rust tests
  • frontend production build
  • full Rust build
  • headless transcription through Handy with an installed Whisper Medium model on Metal

After #74 is released, the temporary Cargo patch entries can be replaced by the released crate version.

@vladkalinichencko
vladkalinichencko marked this pull request as ready for review July 14, 2026 08:33
@vladkalinichencko
vladkalinichencko force-pushed the draft/transcribe-cpp-max-new-tokens branch from bcff4da to 284b4f6 Compare July 30, 2026 12:43
@cjpais

cjpais commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Can you add a screenshot for this?

@vladkalinichencko

Copy link
Copy Markdown
Author

Yep, here you go:

clipboard

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants