-(demo sounds better in `.wav` but GitHub dosent suport that so its kinda bad)
+*(demo sounds better in `.wav` but GitHub doesn't support that so it's kinda bad)*
https://github.com/user-attachments/assets/c75e7141-5d73-40f4-b182-d4f5bc49ad1e
-## New in 3.2.0
+## New in Beta 4.0.0
-- **Cross-Platform Audio Playback:** Preview and JIT playback now go through `sounddevice`/`soundfile` instead of the Windows-only `winsound` module, removing a hard Windows dependency from `kokoro_engine.py`/`gui.py`.
+The rebuild. 3.2.0 was a CustomTkinter form over one `kokoro_engine.py`; 4.0.0 is a PySide6
+shell over a document with clips, tracks and characters, a pluggable engine layer with a second
+real backend, and projects that live in one file.
+
+- **Qt frontend, the only frontend.** `python main.py`/`run.bat` launches a PySide6 shell of
+ dockable panels (`kokoro_gui/qt/`) in a 2x2 grid: Transcript | Settings / Audio FX / Lexicon /
+ Voices tabs on top, Timeline | Transport underneath, under File / Edit / Options / Workspace
+ menus. Every panel is a dock you can drag; **Workspace > Advanced / Simple / Reset layout** are
+ saved layouts (Simple hides the timeline and gives the transcript the full height). The
+ CustomTkinter app (`gui.py`) is gone; PySide6 is a regular dependency in `requirements.txt`.
+- **A document, not a text box.** The old "generate this text" input is a project: a `Document`
+ of canonical text with `Clip`/`Track`/`Character` metadata layered on top (`kokoro_gui/daw/`).
+ The transcript is the source of truth and generated audio is a render of it, tracked per clip
+ with hash-based dirty detection. Generate regenerates every out-of-date clip in one pass with
+ bounded concurrency instead of the whole document every time; a document with no clips yet
+ still uses the whole-document pipeline. Auto-split turns a `[Speaker:FX]`-tagged document
+ (optionally per paragraph) into clips and generates them in one action.
+- **Transcript panel with character highlighting and a live gutter.** Each run is tinted by its
+ character, so speaker boundaries are visible without reading the inline `[Speaker:FX]:`
+ syntax, which converts into a real assignment the moment you finish a tagged line. Above the
+ editor sit two combos, Character and FX, that reflect the caret's clip and reassign the
+ selection (or the whole clip). The gutter labels once per character/FX change (`Narrator` /
+ `FX: Echo`) and shows a play button beside each out-of-date clip; click it to regenerate just
+ that clip. Out-of-date text is dash-underlined, thin rules show where clips end and where
+ Auto-split would cut. Copy/paste carries the character assignment along (with a setting for
+ whether a paste splits off its own run or inherits the destination's).
+- **A multi-track timeline on a real seconds axis.** One lane per character; clips sit end to end
+ in text order at their real duration once generated and an estimated one (dashed outline, no
+ waveform) before, learned from `generation_stats.json`. A ruler with a playhead, a fixed
+ track-header column, Ctrl+wheel zoom. Dragging a clip pins it to a time or moves it to another
+ character's track; dragging it before an earlier clip also moves its text there. Shift+drag
+ inside a clip carves out a sub-range and replaces it with fresh TTS under any character. Each
+ clip has its own FX button for overrides to its character's preset.
+- **Playback.** Play / pause / stop, click the ruler to seek, a playhead across all lanes, and
+ the transcript highlights and scrolls to the clip being played. Space toggles playback
+ anywhere but the text editor; Ctrl+Space toggles everywhere. Built on one
+ `sounddevice.OutputStream` that mixes the arrangement in the callback
+ (`kokoro_gui/audio/transport.py`), so the position is sample accurate. Loop toggle included.
+- **Export.** File > Export mixes every clip down to one file (wav/mp3/flac/ogg) at its timeline
+ position, optionally with a `.srt` and per-clip files (`The .tbaw project bundle
+Text based audio workflow. A .tbaw is a zip with a manifest, the document, the project's settings, its generated audio, and the voice and FX assets it names. This page is what you need to read or write one from a script, and what the app guarantees about it.
01 · LayoutOne zip, five kinds of entry
+Three JSON files at the root, then directories. Audio is stored uncompressed (ZIP_STORED) so a 400 MB audiobook doesn't pay for deflate on data that won't shrink; everything else is deflated. The archive is written with Zip64 enabled, so a bundle over 4 GB is fine.
Entries the app doesn't own are preserved. A directory a newer version or a fourth engine adds is copied through byte for byte on the next Save, so a bundle survives a round trip through an older build.
+manifest.json +document.json +project.json +audio/generated/ + <segment_key>_<index>.wav +audio/imported/ reserved, empty today +fx/ + <preset name>.json +engines/kokoro/voices/ + <mix name>.pt +engines/audio8/refs/ + <reference name>.wav + <reference name>.txt+
02 · manifest.jsonIdentity, versions, hashes
+Read first on Open. The welcome dialog's details pane reads only this file, so stats exists to make that cheap.
| Key | Meaning |
|---|---|
| format | Always "tbaw". Anything else is refused. |
| version | Integer, currently 1. A reader refuses a bundle whose version is higher than the one it implements. |
| requires | List of feature names the bundle depends on. Empty today. A reader refuses, by name, any feature it doesn't implement, which is how a future version can add something an old build can't fake. |
| project_id | Random id, stable across Save As. It names the working copy under cache/projects/ and is what crash recovery keys on, not the path. |
| created_by | e.g. "KokoroGUI 4.0.0". |
| created / modified | ISO 8601 local timestamps, seconds precision. created is carried from the previous manifest. |
| includes | {"generated_audio": bool, "imported_audio": bool}. Whether the audio directories were bundled. A bundle saved without generated audio regenerates its clips on open. |
| audio.format | "wav" or "flac": the format new segments are written in. |
| stats | {"clips", "characters", "duration_s"}. Counts and the sum of segment durations. Cosmetic; nothing reads audio to compute it. |
| engines | {"<engine id>": {"version": str, "meta": {}}} for each engine the document's characters use. version is what the backend reported (Kokoro: the package version; Audio8: its model id). meta is per-backend and empty for both shipped engines. |
| assets | Bundle path to SHA-256, for every entry under fx/ and engines/. Save skips re-hashing an asset whose hash is already in the working copy's index. |
Unknown manifest keys are kept on Open and written back on Save.
+03 · document.jsonThe document model
+The same shape the app autosaves to its working copy, with audio paths made bundle-relative (audio/generated/…). Five top-level keys.
| Key | Contents |
|---|---|
| runs | The text, in order, as a list of {text, clip_id, kind}. The document's full text is the join of every run's text. clip_id is null for narration nobody has assigned; kind mirrors the clip's source. |
| clips | {id, character_id, track_id, overrides, fx_override, timeline_timestamp, source, original_audio_path, segments}. A clip has no offsets of its own: its extent is wherever runs tags its id. overrides holds per-clip settings (voice, speed, take, fx_preset, apply_fx…); timeline_timestamp is null while the clip flows after its predecessor. source is "generated" or "imported". |
| clips[].segments | {id, order_index, text, cache_key, audio_path, duration, raw, engine_version}. One per engine-level chunk. cache_key is the segment key, and audio_path is audio/generated/<cache_key>_<order_index>.<ext>. raw is true for every segment 4.0.0 generates (FX are applied at read time); a segment without the field, from a 4.0 preview build, is treated as baked and regenerated once. engine_version is the version that produced it. |
| tracks | {id, name, character_id, order_index}. A lane. Kept separate from characters so a clip can sit on another character's track. |
| characters | {id, name, preset_data, highlight_color, backend_id}. preset_data is the presets/*.json shape: voice, speed, split pattern, fx_preset, apply_fx, and so on. backend_id is the engine the voice belongs to. |
| settings | The document's own defaults, the "nothing selected" scope of the Settings tab. |
Unknown fields ride along. Every one of these objects has an extra dict on the app side. A field this version doesn't know is parked there on load and written back on save, so a tool can add its own keys to a clip or character and they survive a round trip through the app.
04 · project.jsonPer-project settings
+The project_settings block. Small, and the only place the two bundle options live.
| Key | Contents |
|---|---|
| export | The Export dialog's remembered values: output folder, base filename, format, whether to write an .srt and per-clip files. |
| bundle | {"include_generated_audio": true, "include_imported_audio": true, "audio_format": "wav"} by default. What Save puts in the zip and how new segments are encoded. Editable from the Export dialog. |
| workspace | Optional. A workspace name that overrides the app-level one when this project opens. |
05 · Generated audioOne file per segment, named by what produced it
+Every generated segment is audio/generated/<segment_key>_<order_index>.<wav|flac>, raw model output with no FX, volume, pitch, normalize or trim applied. The file is written once and never overwritten: a regenerate of a clean clip bumps the clip's take, which changes the key, which is a new file. The old file stays for any other clip that plays it until the project closes, when files no segment references are deleted.
Two clips with the same text, voice, speed, language, engine version and take share a key, so they share a file. A bundle with includes.generated_audio false has no audio/generated/ entries; its segments keep their cache_key, and the app regenerates them on the first Generate (a cache hit if the machine's own cache has the key).
06 · Segment keysThe one hash
+A segment key is a SHA-256 over a pipe-joined k=v list. The same function names the file, stamps Segment.cache_key, and drives the out-of-date check in the transcript gutter, so the three can't disagree.
The voice enters as a name plus a content fingerprint, never a path, which is why a key is the same on every machine. Nothing after generation is in it: split pattern, FX, output format, normalize and trim all apply to the same raw segment, so they don't change the key and don't dirty a clip.
+Kokoro adds nothing to extra. Audio8 adds its reference transcript and sampling settings, since two clones of the same WAV with different transcripts are different voices.
schema_version=3 +engine_id=kokoro +engine_version=0.7.11 # kokoro package version +text=The old house stood… +voice=narrator_mix # basename, never a path +voice_fingerprint=9c1e… # file hash; the name if built-in +speed=0.943874 # speed ÷ 2^(pitch/12) +lang_code=a +extra_take=2 # only when non-zero +extra_ref_transcript=… # Audio8 only, + sampling + +sha256("|".join(f"{k}={v}"))+
07 · Engine assetsWhat each engine puts under engines/
+Save walks the document, collects the voice names its characters and clips use, and asks each backend for the files those names need. Only named assets ship; a built-in Kokoro voice is a name with no file.
+| Path | What it is |
|---|---|
| fx/<name>.json | An FX preset a character's fx_preset or a clip's override names. The Audio FX tab's "Save FX Preset" shape. |
| engines/kokoro/voices/<name>.pt | A custom mix made in the Mixing tab: a saved voice tensor. Loaded with torch.load(weights_only=True), see safety. |
| engines/audio8/refs/<name>.wav + .txt | A cloning reference and its word-for-word transcript. The encoded reference codes are derived data and stay in the machine's own cache; the app warms them on the first generate after open. |
On Open, the bundle's own assets take priority. The Voices tab lists a project's mixes and references before the global ones, and the same name in both resolves to the bundle's copy. Nothing writes back into engines/ except Open's extraction: saving a new mix or reference goes to the global store, and the next Save copies it in.
08 · CompatibilityVersions, engines, and the .json past
+Engine versions. A segment is compared under the engine_version it stores while its file is present. So a project made with one Kokoro release opens clean on a machine with another, and the status line says which version the clips came from. Only a clip you regenerate uses the installed version.
Preview-era projects. A .json project from the 4.0 previews (the document.json shape plus a top-level project_settings) still opens. The app builds a working copy, adopts any segment whose stored key matches the schema-2 formula and whose file exists (copying it in under its schema-3 key), and saves a .tbaw next to the original. The .json is left alone.
Forward compatibility. version and requires are the two gates. A reader refuses a higher version outright, and refuses a bundle that requires a feature it doesn't know, by name. Everything else, unknown entries, unknown manifest keys, unknown object fields, is carried through.
09 · The working copyWhere the app actually edits
+The app never edits a .tbaw in place. Open extracts it to cache/projects/<project_id>/, the working copy, and holds an OS lock on <dir>/lock for as long as the project is open. Autosave writes document.json and project.json there; clip generation writes into its audio/generated/. Save rebuilds the zip from the directory, into <path>.tmp, then swaps it over the old file, so a crash mid-save can't leave a truncated project.
A session.json next to the lock records which file the directory belongs to, the zip's size and mtime at extraction, a digest of the last saved document, and whether the directory is ahead of the file. That digest, not an mtime, is what the window title's * means. Opening a project whose directory is dirty offers to recover the unsaved session; Resume from the welcome dialog reuses a matching directory without re-extracting, which is why it's fast.
A clean close deletes audio no segment references and evicts other clean, unlocked working copies, keeping the last project's. Neither session.json nor lock is ever written into the bundle.
10 · Safety on openA bundle is untrusted input
+Before extracting, every entry name is checked: an absolute path on either OS, a drive-relative path, a .. component or a symlink is refused and nothing is written. Free space is checked against the uncompressed size from the central directory first.
A bundle carrying a .pt voice mix is refused unless the machine's torch is 2.6 or newer, where torch.load defaults to weights_only=True and can't execute a pickle payload. Upgrade torch to open it; there's no override.