Feat/followups - #21
Merged
Merged
Conversation
Finalizes detection details per implementation plan: contradiction verdict JSON + fail-open semantics, follow-up 14-day/no-completion rules, revisit bookend behavior, v1.2 ranking priority (person > contradiction > follow_up ~ amount > similar > revisit), and the new config toggles/threshold in config.example.yaml.
Detects recurring-idea patterns from the semantic-similar matches already gathered for Type 1: 3+ matches spanning >6 months emit one 'you've returned to this idea N times since [month year]' connection, anchored to the oldest note with the newest as excerpt. - Connection gains a json-hidden CreatedAt (populated by findSimilar) so detectors can reason about target-note age without re-querying - Priority map extended to the v1.2 order: person 5 > contradiction 4 > follow_up 3 ~ amount 3 > similar 2 > revisit 1 - Config: types.revisit toggle + contradiction_threshold field (defaulted to 0.80; consumed by step 3) - Table tests: span boundary, match-count floor, missing timestamps
Surfaces old intent notes ('need to follow up with bob...') whose
person never appears again afterwards:
- queue.FindFollowupCandidates: FTS5 intent-keyword phrases joined
against the person entity index, notes older than the 14-day floor
- queue.PersonMentionedSince: STRICTLY-after existence check with
multi-path exclusion — the intent note itself and the triggering
capture must not count as their own completion (>= with a single
exclude suppressed everything; caught by tests)
- Detector emits at most one follow_up per person, fail-open per
person, label per SPEC
Tests: surfaces old intent / fresh intent below floor / completion
suppresses / no keywords no candidates.
Two-step detection per SPEC: semantic candidates above
contradiction_threshold (default 0.80) get one LLM verdict call each -
strict JSON {"contradicts": bool, "because": string} parsed defensively,
fail-open per candidate on garbage or errors. Label carries the target
note date plus the verdict reason when present.
- connections.ContradictionChecker: narrow interface over the LLM;
nil checker skips the type entirely so tests stay LLM-free. Worker
adapts w.llm via optional-interface assertion (same pattern as the
consolidation client).
- Find signature gains the checker; candidates filtered to those at or
above the threshold before any LLM spend, capped at 5.
- CheckContradiction system prompt added to defaults with explicit
non-contradiction guidance to keep the false-positive rate down.
- Existing tests updated for the new Find parameter.
Full suite + lint green.
Live testing against the real pipeline surfaced and fixed three issues unit tests alone never could: - Verdict prompts omitted the new note's own body: the model saw one lone excerpt and answered always-false. Find now fetches self content (queue.GetNoteContent, falling back to chunk text when job payloads are unavailable) and both sides ship to the model. - One high-priority type flooded max_per_capture and silenced every other detector. rankAndLimit now dedupes per (note_path, type) and reserves one slot per present type before filling by rank; same-note dual-type rows kept as complementary info. - processJob wiped connections/memory jobs' stored note_path on completion (empty local var persisted); guard added. Verdict system prompt restructured around claim-restatement steps: qwen2.5:3b went from inconsistent to 3/3 correct on the borderline pair. End-to-end verified live: person + contradiction(0.87, reasoned) + similar all present in one stored result.
Multiple detector types matching the same target note each emitted a wikilink, producing duplicate entries in the connections block.
User verification found the connections data never reached the UI and flare chips only appeared on manual refresh: - vault.Reader now parses the connections: frontmatter block (written by SetConnections) and folds it into Related, which the note API already exposed but never populated - NoteHandler resolves Obsidian-basename wikilinks to real vault paths via queue lookup, dropping unresolvable entries - NoteView renders related notes as tappable 'linked notes' chips that switch the sheet to that note in place - Queue view: WebSocket done-events for ingest jobs trigger a flare rehydrating queue fetch (expansion preserved) so gold chips appear the moment a capture finishes, without manual refresh Verified live: /v1/notes resolves the contradiction test note's related links to full paths; 94 vitest green; assets rebuilt.
The first cut rendered raw note slugs with hash suffixes as link text
— unacceptable. Related links now resolve server-side to typed
{note_path, title} objects (titles via BatchGetNoteTitles, basename
fallback), and NoteView renders them as a proper linked-notes panel:
gold-accented full-width rows with link icons, two-line ellipsis for
long titles, hover state — consistent with the flare chip language.
94 vitest green, assets rebuilt, live-verified against the hates-note
(both links resolve to real human titles).
The chips kept :hover/:focus after switching notes because React reused the same DOM nodes. The linked-notes block is now keyed by the current note path (remounts on switch) and chips prevent focus on mousedown, so no state lingers after tapping through.
…ypography Scope: ideas 1-6 + 9 from the UX review. - GET /v1/media: streams media-dir files with extension-derived Content-Type, path pinned to the media dir, private cache headers; PWA fetches via token-header blob (media URLs never carry tokens) and previews image captures at the top of the note sheet - Entity chips: people (gold) / amounts / dates from frontmatter are tappable -> search tab pre-filled (new initialQuery flow mirroring the capture-query pattern); note API now exposes entities - Linked notes moved above content; each row shows WHY via type icons + labels (contradiction/revisit/follow_up/person/similar) hydrated from the note's connections result payload - Copy-as-markdown header button; source URL shows hostname only; FullNoteView renders raw as markdown consistently (the query-time plain-text degradation is gone — highlighting stays in excerpt view) - Typography pass: 580px sheet on desktop, 1.7 line-height, styled prose (headings/code/lists/pre), type-label footer on link rows Tests: media handler 5 cases; 94 vitest green; assets rebuilt.
source_file stores vault-relative paths (khayal/media/x) while the first handler cut assumed media-relative — normalize to inbox-relative before joining, then require containment inside the media dir. Both conventions return 200 image/jpeg live; traversal 400; no-token 401. Unit test config now sets Media.DefaultDir explicitly (prod-like).
Live verification of the connections flow exposed that model-extracted
name variants ('Sara') never joined with stored variants ('Sarah'),
silently killing follow_up detection and person connections.
- queue.GetPersonVariants: case-insensitive, shared-prefix (>=3 chars),
and edit-distance-<=1 (min length 4) matching over distinct person
entity values; stored data is never rewritten — matching happens at
read time
- FindFollowupCandidates + PersonMentionedSince expand to variants so
intent detection and completion checks are variant-proof
- findByEntity expands per person and dedupes across variant matches
Live-verified end-to-end: seeded 'Wren' intent (July) + fresh capture
'meeting wren' -> follow_up + person connections both fired through the
variant join; the suppressed cases (intermediate contact) still
suppress correctly. Unit tests cover matcher edge cases and the
regression scenario.
YAML frontmatter stores amounts as numbers; the chips passed them raw to search which called .trim() on a number (TypeError). All entity values are coerced to strings at render time.
Radix Dialog requires a DialogTitle for assistive tech; NoteView's
sheet only had a styled h2. Added sr-only SheetTitle/SheetDescription
and aria-describedby={undefined} to silence the warning properly.
Sheet test mock extended to cover the new exports.
README: features list now covers proactive connections, AI answers, capture intelligence, vault care, encrypted backups, and the realtime PWA; command tables gained kl delete / kl search --answer / khayal vault.* / backup+restore; config sample includes connections and memory blocks; roadmap marked with shipped status; data table covers memory.md and trash. TECH_STACK: Go 1.25, slog (not Zerolog), gorilla/websocket and filippo.io/age rows, consolidation-model row, accurate PWA stack (no router/Zustand) + vitest/playwright. API_CLIENT: DeleteNote + MediaBlob examples, WebSocket first-message auth contract, typed RelatedLink/Entities response docs.
A jargon-free walkthrough of khayal via the magic-notebook analogy: capture, the local robot helper (Ollama), forgotten-thought resurfacing (revisit/contradiction/follow-up connections), search and AI answers, local-only storage, trash-not-void deletes, and the backup treasure chest. Linked from the README header.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.