Audio pipeline, vocabulary and STT cleanup - #417
Merged
Merged
Conversation
…nswer that has not started
…names and triggers at runtime
…s, and a benchmark script
… word, protect more common words
…ar citizen list on by default
…r migration is saved
…r.cpp, OpenAI and Groq transcription removed
…gman's name and punctuation are not part of the phrase
…d no clip is lost on a split, a release or a long capture
…gman's name, and hotword list edits survive a settings save
…er it said them; a vocabulary change reaches an open settings page
…man's own words, so stop and talking over it work through speakers
…e hears are transcribed every second, so stop and talking over it act before the wingman finishes
…speaks; that cost a transcription per second and still came late
…ayback by loudness against the wingman's own output; a setting turns that off
… stop in the first words is not measured as echo; a stop word anywhere over the wingman is a stop
…loudness curves, and the expected echo rings on after the output stops
…nd a bar that rises after a hit that was the echo after all
…ff by default and meant for headsets
… on the way out, so a duplex device keeps the microphone alive while a wingman speaks
…ms do not reconfigure a duplex device under each other
…ot burst out before the first chunk; the after-playback tail is half a second
…e everyday German words are protected; "dir" no longer becomes Adir
… a hold nobody released ends after two minutes, and a refused press says who holds the microphone
… closeness to small talk; the remember tool is for what the user says should last; moods are not facts
Shackless
added a commit
that referenced
this pull request
Sep 17, 2026
…#418) Reworks persistent memory around checkpoints and episodes, and moves the model defaults after a benchmark against the real subscription models. Sits on top of #417. ## Memory Two kinds of memory, kept apart because they age differently: - **Facts** are timeless and rewritten as a whole. A newer statement replaces an older one that contradicts it, so "sold the Aurora, bought a Freelancer" leaves one fact, not two. - **Episodes** are one entry per session: what happened, what was memorable, what is still open. The three most recent within 30 days go into the system prompt with their age, so "where were we?" names the current mission and can bring up an older moment. An episode with nothing memorable and nothing open is dropped, so a session of pure commands leaves no log. Writes happen at **checkpoints**: every 25 user turns or 20 minutes, at session end (unload, reset, 30 minutes idle), and through `memory_remember`. One support call gets the stored facts, the episode so far and the new messages, and returns the whole list plus the episode. This replaces both the extraction prompt and the separate consolidation pass. Reads no longer search. Every fact (capped at 40, newest first) and the recent episodes go into the system prompt as one block that only changes at a checkpoint, so the provider's prompt cache covers it. The per-turn similarity recall, the two tuned thresholds and the "N memories recalled" line are gone. Embeddings stay for deduplication, `memory_forget` and the memory tab search. The client shows "Memory: N facts and M sessions loaded" once at session start. Old `session_summary` rows are read as episodes, no data migration. ## Model defaults Benchmarked all subscription models on the real pipeline (bench and findings under `evals/`, gitignored): - **Local support model → Qwen3.5-4B.** On 19 conversations the bundled 2B returned nothing usable in 17 (score 0.35); the 4B scores 0.93, level with the cloud models, at the same speed. The 3.2.2 migration moves the old 2B default over; a model the user picked stays. The 2B stays selectable. - **Chat model unpinned on upgrade.** The 3.2.2 migration empties a pinned `conversation_deployment` so everyone follows the new plan default (gpt-4.1-mini, set on the backend). A specific model can be picked again in Settings. The cloud support default (deepseek-v4.1-flash) and the chat default (gpt-4.1-mini) are backend lane changes, in a separate backend branch. ## Tests Full suite green. New: `tests/test_memory_checkpoint.py` (checkpoints, episodes, the block, the fact cap); `tests/test_migration_321_to_322.py` gains the 4B move and the chat unpin. `test_memory_recall_gate.py` and `test_memory_placement.py` are removed with the code they covered.
Shackless
added a commit
that referenced
this pull request
Sep 20, 2026
## The crash
A tester's 3.2.2 install ended every start with:
```
Invalid config 'C:\Users\...\3_2_2\configs\settings.yaml':
1 validation error for SettingsConfig
stt Field required
...
pydantic_core._pydantic_core.ValidationError: 9 validation errors for SettingsConfig
stt Field required [type=missing, input_value={}, input_type=dict]
```
Two errors, one call. `load_settings_config` reported the file as
invalid, then
fell back to `SettingsConfig()` — which is not constructible, because no
field
in `SettingsConfig` has a default. The exception came out of
`ConfigManager`'s
constructor, so Core died in `main.py:97`. The migration that would have
repaired the file runs in `main.py:559` and never got there; every
following
start hit the same wall.
## Why his settings.yaml was out of date
`b3eb4b4f` bumped `LOCAL_VERSION` to 3.2.2 on 2026-09-15. `948ba0ca`
(#417)
added the required `stt` block to `settings.yaml` a day later. An
unstable dev
build from between those two commits writes
`3_2_2/configs/settings.yaml`
without `stt` and a completed `.migration` marker beside it — so the
3.2.1 → 3.2.2 migration, which does write `stt` correctly, never runs
for that
install again.
Checked the other two: `defaults.yaml` and `mcp.template.yaml` from that
same
commit still validate against the current models
(`condense_keep_recent_tokens`
and `disabled_tools` both have defaults). `settings.yaml` is the only
one that
breaks.
## What changed
- `load_settings_config` validates the file as before. If that fails, it
fills
the missing keys from `templates/configs/settings.yaml` (deep merge,
user
values always win), writes the repaired file back and says so in the
log.
- A file that fails on a **value** instead of a missing key is left on
disk
untouched — nobody can guess what the user meant — and Core comes up on
the
shipped defaults so Settings is there to correct it in.
- A `settings.yaml` that already validates is not rewritten.
- `create_settings_config` writes the template instead of an empty file.
An
empty file has no value for any required field, i.e. the same crash.
- In the migration: a settings/defaults step that fails validation now
persists
what it migrated. It used to leave the previous version's file in the
new
version's folder while `perform_migration` wrote the `.migration`
marker,
freezing that state for good.
- `_deep_merge_over` in the migration service now calls the shared
`deep_merge_configs`.
## Tests
`tests/test_settings_repair.py` (local only, `tests/` is gitignored): a
3.2.1
settings.yaml boots and keeps the user's values, an empty file boots on
the
template, a bad value boots on defaults and leaves the file alone, a
current
file is not rewritten. The 98 tests in the config/migration area pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reworks how Wingman AI listens, transcribes and corrects what it heard.
Listening
services/audio/) replaceAudioRecorder. Push-to-talk and voice activation share the stream and the transcription worker; utterances are transcribed in order and none is dropped.Vocabulary
services/audio/vocabulary.py): fuzzy match sized by the shorter word, no letter may change on a split or merge, common words are protected, exact-only entries in quotes.heard=correctentries, wingman tools to teach spellings, names and triggers at runtime. Markdown headings and bullets in prompts are not names; spoken triggers are.Speech-to-text
Commands
Notes
tests/(gitignored), 259 pass.