feat(settings): the settings foundation — one declaration, manifest v2, and docs that can't drift - #66
Merged
Merged
Conversation
…manifest
The config write surface lived in four hand-synced places: config_update's
per-section key tuples, settings_manifest's _CONFIG_WRITE_ROUTES mirror
(carrying a "keep in lockstep" comment), and prose in OpenApi.yaml and
endpoints.md. They had drifted, invisibly and in both directions:
- `research` and `web_research` were writable but reported read-only,
so the manifest under-reported 9 keys.
- `ambassador` claimed `speech_model`/`voice`/`transcription_model` were
writable while the handler's tuple omitted them and dropped every save
— the same shape as the Images regression this repo already ate once.
- `search.source_policy` advertised per-leaf writability, so a partial
patch of `trusted` silently wiped `blocked` and `goggle`.
settings_registry.py is now the one declaration. It records deltas only —
type and default still come from DEFAULT_CONFIG and the memory kit's
pydantic Settings — so a section declares just what a generic walker
cannot infer: which keys are writable, which accept an explicit null,
which dict leaves are written whole, and the presentation axes. A key that
isn't declared is read-only; it can't be silently writable.
Writes are two-phase. plan_config_update validates the whole payload and
returns the ops it would apply; apply_ops mutates. A payload rejected
halfway (models.roles has always 400'd on a malformed value) no longer
leaves the process-global ConfigManager half-written until the next
reload. Sections whose accept-set is computed per request (providers) or
that carry a verb no other section has (context_limits) declare a planner
instead of a key list, so the write surface stays in one file even where
it can't be a static list.
The retired trajectory-compression bridge is removed. Those keys were
written through to data/config.json from /api/memory/settings with no
validation (pydantic's extra="ignore" waved them past the schema) and
against a second copy of the defaults that could drift from
DEFAULT_CONFIG. The client stopped writing them releases ago; the
canonical route is /api/config/update. Sending one now gets a 400 naming
where it moved, rather than a silent drop.
Manifest v2 adds constraints, tier, ui_section, and authored help, each
emitted only where declared so v1 consumers see no new noise. Help itself
is authored once in settings_help.yaml (summary / what / how / why /
manage) — the settings UI and the generated reference will both read it,
so they cannot disagree. Recall is authored in full as the golden section;
the rest ship registry-only for now.
Constraints are opt-in per key: only keys that declare a range validate on
write, so every existing stored value and every undeclared key keeps
exactly today's pass-through behaviour.
Also unified the three divergent copies of secret classification (the
config redactor, the manifest, and the subtree list) onto one definition,
and gave the display-only consolidation keys a single source so the
manifest and the POST handler agree they aren't writable.
Tests: the old lockstep guard scraped `_SEARCH_KEYS` out of views.py as
source text and compared it to the manifest's mirror — a guard for a
lockstep that no longer exists. Replaced with structural coverage that
asserts the property for every section at once: every declared writable
key round-trips through the endpoint, manifest writability equals registry
derivation, a rejected payload leaves config untouched, whole-write leaves
stay atomic, declared ranges reject and undeclared keys accept anything,
and the golden section's help is complete with no orphaned entries.
1239 backend tests pass, including under the sterile-config runner.
Assisted-by: Opus 5
…w, boundaries The settings surface knew nothing about the settings it renders. Defaults were re-typed as `?? 50` literals at 131 call sites, so a control could disagree with the server about its own default and nothing would notice. Nothing showed what you'd changed. Help was a `hint` string with nowhere to put the rest. The client now reads GET /api/settings/manifest once per open and hands each control its own entry. Three things follow from that. **Field chrome.** A control bound to its manifest entry gains a dot when it differs from the shipped default, a reset that puts it back, its declared bounds, and a help popover carrying the authored prose — what it is, how it works, when to change it, what to watch. All of it is opt-in: pass no binding and the field renders exactly as before, so the sections that haven't been migrated are untouched. Reset rides the normal autosave path (it's just an update to the default value), so there's no second write path to keep honest. **Accessibility.** Five of the eight primitives rendered a <Label> with no htmlFor beside a control with no id, so the label announced nothing. NumberField was worse: its accessible name resolved to `title` — the hint sentence rather than the setting's name, which is what a screen reader would read out for "Max Results". FieldShell now owns the id and hands it to the control, and hints are wired through aria-describedby so both get announced. SliderField deliberately keeps its aria-label instead: Radix puts role="slider" on the thumb, so a `for` pointing at the root would just move the orphaned label somewhere new. **Overview.** Settings opened on Model Providers — an API-key admin page as the answer to "where do I start". It now opens on Overview: what you've changed from the defaults, grouped by the section that owns it, and tiles for everything else. The digest is computed from the manifest rather than curated, so it can't fall behind. Secrets are excluded — their values arrive redacted, so "changed" is unanswerable for them and printing the comparison would be worse than saying nothing. Empty strings read as "empty", never "default": several model keys ship with the literal default `inherit`, and "inherit → default" would mean the opposite of what it says. Settings with no owning section say so rather than pretending to be clickable. Rows navigate to the section. Landing on the exact control — scroll, focus, flash — needs anchors this doesn't invent yet; that's the next wave. **Boundaries.** Each section renders inside its own error boundary, keyed by section id. A section that threw used to escape to the page-level boundary and blank the app — which is precisely what Web Search did when it hit a Select it couldn't render. Now the failure stays in the pane, the rest of settings keeps working, and switching sections clears it. Pending edits survive the unmount because useSettingsAutosave already flushes there. Also fixed the dead `'servers'` default in useSettingsNavigation, which named a section that stopped existing several reorgs ago and only ever worked because every call site passed something else. Verified against a live server: the manifest reports 331 entries, and Overview correctly finds all 24 settings this install has moved off their defaults, grouped across 12 sections plus the set-elsewhere group. 448 client tests pass. Assisted-by: Opus 5
Recall is the most bespoke surface in Settings, so it goes first and sets the pattern the other sections follow. The panel was organised by *when a setting was added* rather than by what it belongs to. HyDE's model and temperature sat under "HyDE Settings" while its token budget sat three sections away in "Advanced"; Self-Query was split the same way, with its temperature and token budget separated from its model. Eight sections, and tuning one technique meant hunting through several of them. Now each technique's toggle owns every knob it governs, indented beneath it and shown only when it's on. Eight sections become three: Retrieval Techniques, Two-Stage Rerank, and an Advanced disclosure holding the genuinely cross-cutting knobs (min confidence, RRF k, graph depth, the first-person guard) collapsed by default. Nothing was removed — every setting is still present and still writable, and the collapsed ones are one click away. All 24 keys now bind to the settings manifest, which kills the 25 `?? 50` literals that re-stated server defaults at the call site. Each control gets its declared bounds, a dot when it differs from the shipped default, a reset that writes that default back through the normal autosave path, and a help popover carrying the authored prose — what it is, how it works, when to change it, what it interacts with. The failure state gained the retry button it never had. Also fixed a real defect the manifest surfaced: `is_secret_path` matched its markers as substrings, so "token" inside `max_tokens` classified 24 settings as credentials — every token budget on the platform. Their defaults came back `***`, which the settings UI and the generated reference would both have shown where a number belongs. Markers now match whole words, with an `endswith` arm so unseparated names like `apikey` stay covered; over-redacting a non-secret is fine, leaking one is not. Verified live: 22 help triggers on the panel, 6 technique groups, Advanced collapsed on load (aria-expanded=false, height 0), and help rendering the full five-field treatment. Settings now opens on Overview listing this install's 24 changed settings. 454 client tests pass. Assisted-by: Opus 5
Per-setting documentation didn't exist. `configuration.md` showed one illustrative `config.json` blob covering four of the roughly 180 keys, with no types, no ranges, and no descriptions — everything added since (search, context, alloy, ambassador, images, reasoning, compression) was simply absent. What explanation there was lived as `hint` props in JSX, which is to say it lived nowhere a reader could find it. scripts/gen_settings_reference.py renders the whole catalogue from the declarations the app itself reads: settings_help.yaml for prose, settings_registry.py for bounds and tiers, DEFAULT_CONFIG and the memory kit's Settings for types and defaults. Nothing on the page is authored by the page, so the documentation and the settings screen say the same thing by construction rather than by discipline. The generator is deliberately pure — it imports declarations and never calls get_config_manager() or get_settings(). Reading live config would bake one machine's values into a committed file and make the output depend on whose laptop ran it. settings_manifest grows build_reference_entries() for exactly this: the same entries as the manifest, minus `value`. Wired into `task docs:check` (and so into check:fast, release:check, and CI) with the severity convention the other generated artifact uses: a page that has drifted warns and tells you to regenerate; a page that is missing errors. A stale generated file should degrade, never lie. Settings that carry authored help get the long form — what it is, how it works, when to change it, what to watch. Recall is written up in full as the golden section; everything else is tabulated with its type, default, and write route, and gets written up on the section cadence. Also gave ui_section ids human labels (`Memory → Recall`, not `memory-recall`), since the headings are read by people. Assisted-by: Opus 5
Records the decision the last four commits implement, and brings the hand-written contract docs in line with it. ADR-17 states the rule: one declaration drives the config write path, the manifest, and the generated reference; writes validate-all-then-apply-all; constraints are opt-in per key (the zero-friction invariant — no stored value becomes un-writable); help is authored once. It also records what was rejected and why: `Field()` on the 150 pydantic fields (would add validation nobody asked for and put UI governance in runtime kit code), and keeping the trajectory bridge (a second blessed write path contradicts the whole point). OpenApi and endpoints.md stop enumerating the allowlists by hand. Those paragraphs were two of the four hand-synced copies the registry exists to collapse, and they had drifted like the rest — they now point at the manifest and the generated reference, and document the behaviour that actually matters (undeclared keys ignored, atomic rejection, `""` as a meaningful "follow the role" value). The memory-settings docs record the retired bridge and the display-only keys. Also: `aria-current` on the active settings-nav item, so the section you're on is announced as current rather than merely highlighted. CLAUDE.md gains the "declare it, don't hand-write a handler" rule. It was 17 bytes under its ceiling, so several entries got tightened to make room — the ratchet is meant to go down, not up. Verified: 1240 backend tests, 454 client tests, tsc clean, ruff clean, pyright at baseline 0, `task docs:check` green with no warnings. Live: settings open on Overview listing 25 changed-from-default settings grouped by section; Recall renders each technique's knobs beneath it with the full help popover; manifest v2 serves constraints, tiers and help. Assisted-by: Opus 5
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 1 of the settings campaign. Settings quality had crept: 21 sections, ~200 knobs, every section individually well-written but the aggregate hard to enter — "even for LLM experts, you look at it and wonder where to start."
This wave builds the foundation the rest of the campaign hangs off, and pays down the defects that foundation exposes.
The problem it fixes
The config write surface lived in four hand-synced places:
config_update's key tuples, the manifest's_CONFIG_WRITE_ROUTESmirror (carrying a literal "keep in lockstep" comment), and prose inOpenApi.yamlandendpoints.md. They had drifted, invisibly and in both directions — and the class of bug it invites had already shipped once (the Images section silently dropped every save while toasting success).Found live during this work, now fixed by construction:
research/web_researchwere writable but reported read-only — 9 keys under-reported.ambassador'sspeech_model/voice/transcription_modelwere advertised as writable while the handler dropped every save.search.source_policyadvertised per-leaf writes; patching one leaf wiped its siblings.models.rolescould 400 mid-walk, leaving the process-global config half-written until the next reload./api/memory/settingswas an unvalidated write path intoconfig.jsonvia the legacytrajectory_compression_*bridge, against a second copy of the defaults.What landed
settings_registry.py— one declaration. Records deltas only; type and default still come fromDEFAULT_CONFIGand the memory kit's pydanticSettings. Undeclared ⇒ read-only, never silently writable. Bespoke sections (providers' per-request accept-set,context_limits' wildcard subtree) register planners, so even the irregular cases live in one file. Writes are two-phase — validate the whole payload, then apply.Manifest v2 adds
constraints,tier,ui_section,nullable,empty_means,write_mode, and authoredhelp, each emitted only where declared. Writability now derives from the registry rather than mirroring it.The client reads it. Field kit v2 shows each control's shipped default, marks what you've changed, and offers to put it back; help is a first-class popover, not a
title=attribute. Fixed the a11y defects while the fields were open — five primitives had unassociated labels, andNumberField's accessible name resolved to its hint sentence. Sections now render inside error boundaries, so a section that throws no longer takes the app down.Settings open on Overview — what you've changed from the defaults, grouped by section — instead of an API-key admin page.
Golden Recall refit. Each technique owns its knobs (HyDE and Self-Query were rendering in two places), tiers replace the three ad-hoc "Advanced"/"Experimental" dialects, and all 24 keys carry the full write-up: what it is, how it works, when to change it, what to watch. This is the register the other sections refit toward on a cadence.
The docs generate themselves.
scripts/gen_settings_reference.pyrenders the whole catalogue from the same declarations, gated bytask docs:check. Settings and documentation can no longer drift — the guarantee is structural, not a discipline.Behaviour changes worth knowing
trajectory_compression_*on/api/memory/settingsnow 400s, naming/api/config/update. The client stopped writing these releases ago; the route was an unvalidated hole.Nothing was removed from the settings surface: requirement was "no setting lost — only abstracted, gated, or managed", and coverage is now a test, not a promise.
Verification
tscclean ·ruffclean · pyright at baseline 0task docs:checkgreen with zero warningsDeliberately out of scope (Wave 2)
Per-setting search (today's search still matches 21 section labels and hand-typed keyword arrays, not the ~200 settings), jump-to-control, deep links, and palette section commands. This wave leaves only the anchors they need. Tracked in
todo/backlog/genome-advisor.md.Decision recorded as ADR-17.
Assisted-by: Opus 5