Skip to content

feat: semantic compression — rolling digests and recall deprioritization (#9)#127

Merged
rahilp merged 3 commits into
mainfrom
9-feature-semantic-compression-rolling-summaries-and-project-digests
Jun 7, 2026
Merged

feat: semantic compression — rolling digests and recall deprioritization (#9)#127
rahilp merged 3 commits into
mainfrom
9-feature-semantic-compression-rolling-summaries-and-project-digests

Conversation

@rahilp

@rahilp rahilp commented Jun 5, 2026

Copy link
Copy Markdown
Owner

Summary

  • Nightly cron compresses tags with >10 compressible entries into a single synthesized digest entry
  • endpoint for on-demand compression triggered from the Settings UI
  • source entries penalized 0.4× in recall scoring so digests surface above originals
  • Digest header is clean; each source entry gets appended in D1 only (no re-embedding), enabling future un-digest
  • Stats query mirrors filters exactly (importance < 4, ) so UI counts match reality
  • Settings sheet renders eligible tags with Digest buttons; menu scrolls safely at
  • cards render at 0.65 opacity; cards get accent-colored left border
  • tightened: threshold raised to 10, 48h cooldown via D1 check to prevent spam
  • Originals are never deleted — digests are additive

Test plan

  • Open Settings, confirm tags with ≥10 compressible entries appear in the digest section
  • Click Digest on a tag — confirm button shows spinner, then success with entry count
  • Run for a query related to the compressed tag — synthesized digest should rank above rolled-up originals
  • Confirm rolled-up cards appear dimmed (0.65 opacity) and digest cards have accent border in both Recall and Recent views
  • Confirm originals are still searchable via with high or tag filter
  • confirm nightly handler logs per-tag compression
  • Run test suite — 249 tests pass

…ion (#9)

- Nightly cron compresses tags with >10 entries into a synthesized digest
- GET /digest?tag=X endpoint for on-demand compression
- rolled-up entries penalized 0.4× in rerankWithTimeDecay so digests surface first
- Digest header is clean (no source_ids); each source entry gets [Digest: {id}]
  appended in D1 only — no re-embedding, enabling future un-digest
- Stats digest_candidates query mirrors compressTag filters exactly (importance < 4,
  HAVING count > 10) so UI counts match what actually compresses
- Settings sheet shows eligible tags with Digest button, scroll-safe at 80vh
- rolled-up cards render at 0.65 opacity; synthesized cards get accent border
- derivePattern tightened: threshold 10, 48h cooldown via D1 check
- 11 new compressTag unit tests; D1Mock extended with tag-filter and cooldown handlers
@rahilp rahilp linked an issue Jun 5, 2026 that may be closed by this pull request
@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 70.03% 437 / 624
🔵 Statements 69.91% 509 / 728
🔵 Functions 69.69% 69 / 99
🔵 Branches 64.16% 299 / 466
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
src/index.ts 67.65% 60.43% 67.03% 67.97% 94-141, 161, 432, 439, 480, 674, 785, 791, 800, 808-828, 878, 918, 931, 946-1295, 1304-1308, 1321-1346, 1364, 1405, 1442, 1574-1584, 1606, 1616
Generated in workflow #123 for commit c2797e9 by the Vitest Coverage Report Action

@rahilp rahilp added this to the v1.8 milestone Jun 5, 2026
@rahilp rahilp added the enhancement New feature or request label Jun 5, 2026
@rahilp rahilp merged commit 826c774 into main Jun 7, 2026
1 check passed
@rahilp rahilp deleted the 9-feature-semantic-compression-rolling-summaries-and-project-digests branch June 7, 2026 04:09
@github-project-automation github-project-automation Bot moved this from Todo to Done in Second Brain Jun 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Feature] Semantic compression — rolling summaries and project digests

1 participant