From abf0ab1ff1030a5831f2a2bf8b135a608850f754 Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 17:52:38 -0400 Subject: [PATCH 01/10] docs: roadmap + personal digest design spec --- ROADMAP.md | 63 +++++++++ .../2026-08-18-personal-digest-design.md | 131 ++++++++++++++++++ 2 files changed, 194 insertions(+) create mode 100644 ROADMAP.md create mode 100644 docs/superpowers/specs/2026-08-18-personal-digest-design.md diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..eae23eb --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,63 @@ +# Roadmap + +[`BACKLOG.md`](BACKLOG.md) is explicit that it's *"not a feature wishlist — it's the +production-hardening layer."* This file is the other half: new capability, not hardening — ideas +for Roger doing more without the owner personally driving every step, and for growing a small, +personal-interest Discord server that doesn't have a lot going on day to day. + +Community/culture planning (what to post about, tone, events — not code) lives separately in +[`community-discord`](https://github.com/R055LE/community-discord), since Roger and the server it +runs in are different things. This file is the "how Roger could help" side. + +Same effort key as BACKLOG: **S** ≈ an afternoon, **M** ≈ a day or two, **L** ≈ multi-day. + +--- + +## 1. Personal digest — **S/M** — *spec written* + +A feed roundup DM'd to the owner only, separate list and schedule from the public digest. Directly +answers "I'm out of the loop" — reuses Digest's existing RSS/feeds/curation plumbing rather than +building anything new, per [ADR-style discussion in the +spec](docs/superpowers/specs/2026-08-18-personal-digest-design.md). + +*Why it's first:* smallest of the four, reuses infrastructure that already exists and is already +tested, and is the most direct fix for the stated pain. + +## 2. Proactive public content ("Spark") — **L** — *idea only* + +The flagship. A new scheduled capability that posts unprompted to a public channel to spark +discussion — reacting to real fetched items (AI/tech headlines, story recs, art highlights, open +questions), not free-generated commentary that could confidently state something wrong. Same risk +shape as Digest (§9 of `ARCHITECTURE.md`): scheduled, no user in the path, posts a message, nothing +destructive. + +Needs its own design pass — content grounding, cadence, tone, and probably its own ADR given it's a +new class of unprompted public output. Not spec'd yet. + +*Why it's not first:* biggest unknown, and the personal digest's grounding-vs-hallucination question +(item 1) is a smaller version of the same problem worth solving first. + +## 3. Model allocation pass — **S** — *idea only* + +Give content-generation brains (Spark once it exists, Giga Brain now) their own tuned/creative model +chain distinct from admin's tool-calling chain. There's real headroom to do this — Giga Brain's spend +has been low. Mostly config + evaluation, not new code. + +*Why it's not standalone:* rides along with items 1 and 2 rather than being picked up on its own — +worth doing once there's actual creative-generation output to tune against. + +## 4. Parked — self-serve commands for other members + +Not a stated pain point today (the server's small, and the ask was about the owner's own bottleneck, +not delegating to others). Noted so it doesn't get lost if the server grows enough that it becomes +one. + +--- + +## Deliberately not doing (for now) + +- **Scraping non-RSS sources** for items 1/2. Feedparser-only, matching Digest's existing + constraint. A site with no feed is a future idea, not a blocker. +- **A distinct model/budget for the personal digest.** It shares Digest's `MODEL_DIGEST` chain and + `DAILY_TOKENS_DIGEST`/`DAILY_USD_DIGEST` cap — a second job, not a second brain. Revisit if the + editorial angle (item 2 territory) ever gets folded into it. diff --git a/docs/superpowers/specs/2026-08-18-personal-digest-design.md b/docs/superpowers/specs/2026-08-18-personal-digest-design.md new file mode 100644 index 0000000..e13a671 --- /dev/null +++ b/docs/superpowers/specs/2026-08-18-personal-digest-design.md @@ -0,0 +1,131 @@ +# Personal digest — design + +Tracks `ROADMAP.md` item 1. A feed roundup DM'd to the owner only, separate feed list and schedule +from the public digest (§9), so staying current doesn't depend on the owner going and looking for +news themselves. + +## Goal + +Reuse Digest's existing mechanism as a second job rather than building anything new: same +feedparser/dedup logic, same curation-tool shape, same delivery pattern Giga Brain already uses for +private, owner-only output. The only genuinely new things are a second feed list and a second +schedule. + +## Storage + +New `personal_feeds` table, identical shape to the existing `feeds` table: + +```sql +CREATE TABLE IF NOT EXISTS personal_feeds ( + url TEXT PRIMARY KEY, + title TEXT, + added_ts REAL NOT NULL +); +``` + +Purely additive (`CREATE TABLE IF NOT EXISTS`) — no migration of existing tables, no risk to live +data. `seen` stays exactly as-is and is shared between both lists: dedup is keyed on +`(feed_url, entry_id)`, which is already globally unique regardless of which list curated the URL. + +Five new `Store` methods, each a direct mirror of the existing feed method of the same shape: +`list_personal_feeds`, `add_personal_feed`, `remove_personal_feed`, `seed_personal_feeds`, +`count_personal_feeds`. + +This mirrors the codebase's own precedent for "same concept, second brain" — `ambient_log` / +`admin_log` / `gigabrain_log` are three separate tables with three separate method pairs, not one +table with a `brain` column. A `scope` column on `feeds` would work too, but duplication is the +established pattern here, so this follows it. + +## Config + +Three new settings in `roger/config.py`, next to the existing `digest_*` block: + +``` +personal_digest_feeds: str = "" # seed, same shape as digest_feeds +personal_digest_channel_id: int | None = None # unset = DM the owner +personal_digest_hour: int = 7 # own schedule, independent of digest_hour +``` + +`personal_digest_channel_id` unset means DM — same fallback shape `gigabrain_channel_id` already +uses, not a new pattern. A `personal_feeds` derived property (mirrors the existing `feeds` property) +splits `personal_digest_feeds` on commas. + +No new model or budget settings. The job shares the `digest` brain's model chain +(`MODEL_DIGEST`) and daily cap (`DAILY_TOKENS_DIGEST` / `DAILY_USD_DIGEST`) — it's a second job, not +a second brain, per `ROADMAP.md`'s "deliberately not doing" note. Revisit if the editorial angle +(`ROADMAP.md` item 2 territory) ever lands here. + +## `roger/brains/digest.py` + +Two new functions, each a close mirror of an existing one: + +- **`seed_personal_feeds_if_empty(store, settings)`** — copy of `seed_feeds_if_empty`, seeding from + `settings.personal_feeds` into `personal_feeds` instead of `feeds`. + +- **`run_personal_digest_job(*, client, settings, llm, store)`** — copy of `run_digest_job`'s + fetch/dedup/summarize path (`_collect_new`, `_summarize` are reused as-is — both already take a + feed list and a store, neither is Digest-table-specific), but: + - Sources `store.list_personal_feeds()` instead of `store.list_feeds()`. + - Delivery copies `run_gigabrain_suggestion`'s DM-or-channel pattern exactly: if + `personal_digest_channel_id` is set, post there; otherwise `client.fetch_user(settings.owner_id)` + → `create_dm()`. This replaces `run_digest_job`'s channel-required early return — "not + configured" now means "no feeds," not "no channel," since a DM destination is always + reachable in principle. + - Same `BudgetExceeded` / `LLMConfigError` handling as `run_digest_job`, same "mark seen only + after a successful post" ordering. + - Embed title `"Roger's personal digest — {date}"` in place of `"Roger's digest — {date}"`. + - Still calls `llm.complete("digest", messages)` — same brain identity, so it shares the budget + per the Config section above. + +## Curation tools + +Four new owner-only tools under `roger/tools/`, mirroring `list_feeds` / `suggest_feeds` / +`add_feed` / `remove_feed` exactly — same `ToolSpec` shape, schema in `schemas.py`, executor in +`executors.py` (the existing feed tools don't touch `guard.py`, so neither do these), same "no +confirm-gating" (the originals aren't gated; nil blast radius applies the same way to a second +list): + +- `list_personal_feeds` +- `suggest_personal_feeds` (validates a candidate feed against the live web before proposing it — + same as `suggest_feeds`) +- `add_personal_feed` +- `remove_personal_feed` + +Registered the same way the existing four are — available to the admin brain, owner-gated by the +existing `user.id == OWNER_ID` check before any tool runs (§2.3), nothing new in the trust boundary. + +## Scheduling + +`bot.py` gains `_personal_digest_loop`, a `tasks.loop` matching the existing `_digest_loop` / +`_gigabrain_loop` shape: default `time=datetime.time(hour=7)`, `change_interval`'d to +`settings.personal_digest_hour` at startup (mirrors how `digest_hour` and `gigabrain_hour` are +wired), `before_loop` waits for the client to be ready. Calls `run_personal_digest_job` the same way +`_digest_loop` calls `run_digest_job`. + +`seed_personal_feeds_if_empty` gets called at boot alongside the existing `seed_feeds_if_empty` +call. + +## Docs + +- `ARCHITECTURE.md` §9 gets a short addition noting the personal digest as a sibling scheduled job + sharing Digest's mechanism and budget, plus a `personal_feeds` row in the §10 table. +- `ROADMAP.md` item 1 flips to *shipped* once this lands. +- `README.md`'s Status section: one line, matching how Digest is already described there. + +## Testing + +Extends `tests/test_digest.py` with personal-digest equivalents of the existing +`run_digest_job` / `seed_feeds_if_empty` cases (not-configured, posts-and-marks-seen, +budget-exceeded, dead-feed-doesn't-kill-the-run). New tool tests mirroring the existing feed tool +tests. `tests/test_config.py` covers the three new settings and the `personal_feeds` property. +`tests/test_compose.py`'s "every setting is forwarded" gate (the same one the dollar-budget-gate +work tripped) applies here too — the three new env vars need forwarding in `compose.yaml`. + +## Out of scope + +- Non-RSS sources (scraping). Feedparser-only, matching Digest's existing constraint. +- The "editorial" commentary angle — reacting/opining rather than summarizing. Noted for + `ROADMAP.md` item 2, not this round. +- A distinct model or budget for this job (see Config). +- `/status` enrichment — the public digest doesn't surface feed count or last-run there today + either, so this doesn't add scope beyond what already exists. From 773ae653f585c1b6efeadce3b2b9f01681b8be20 Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 18:12:11 -0400 Subject: [PATCH 02/10] docs: personal digest implementation plan --- .../plans/2026-08-18-personal-digest.md | 1073 +++++++++++++++++ 1 file changed, 1073 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-18-personal-digest.md diff --git a/docs/superpowers/plans/2026-08-18-personal-digest.md b/docs/superpowers/plans/2026-08-18-personal-digest.md new file mode 100644 index 0000000..0f0132c --- /dev/null +++ b/docs/superpowers/plans/2026-08-18-personal-digest.md @@ -0,0 +1,1073 @@ +# Personal Digest Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** a feed roundup DM'd to the owner only, on its own feed list and schedule, separate from +the existing public digest. + +**Architecture:** every piece mirrors an existing Digest or Giga Brain mechanism exactly — a second +`personal_feeds` table (mirrors `feeds`), a second scheduled job sharing Digest's model/budget +(mirrors `run_digest_job`, with delivery copied from `run_gigabrain_suggestion`'s DM-or-channel +pattern), and four mirrored curation tools. No new abstractions. + +**Tech Stack:** `aiosqlite`, `feedparser`, `discord.py` `tasks.loop`, `pydantic-settings` — all +already in use, nothing new. + +## Global Constraints + +- Spec: `docs/superpowers/specs/2026-08-18-personal-digest-design.md` — this plan implements it + exactly; where anything here seems to conflict, the spec governs and the discrepancy should be + flagged, not silently resolved. +- No new model or budget config — the job shares `MODEL_DIGEST` / `DAILY_TOKENS_DIGEST` / + `DAILY_USD_DIGEST` via `llm.complete("digest", messages)`. +- `personal_digest_channel_id` unset means DM the owner; set means post there instead — same + fallback shape as `gigabrain_channel_id`. +- `PERSONAL_DIGEST_FEEDS` seeds `personal_feeds` once; after that the store is authoritative (same + "seed once" rule as `DIGEST_FEEDS`). +- Feedparser/RSS only — no scraping, no new source types. +- Every new `Settings` field must be forwarded in `compose.yaml`'s `environment:` block in the same + task that adds it — `tests/test_compose.py::test_every_setting_is_forwarded_by_compose` fails + otherwise, and per ADR precedent (the compose-vars-fix during the dollar-budget-gate work) this is + not optional cleanup, it's part of the task. +- Run `pytest -q` and `ruff check .` before every commit. Both must be clean. + +--- + +### Task 1: Storage — the `personal_feeds` table + +**Files:** +- Modify: `roger/store.py` +- Test: `tests/test_store.py` + +**Interfaces:** +- Produces: `Store.list_personal_feeds() -> list[dict]` (rows with `url`, `title`, `added_ts`, + ordered by `added_ts, url`), `Store.add_personal_feed(url: str, title: str | None) -> bool`, + `Store.remove_personal_feed(url: str) -> bool`, `Store.seed_personal_feeds(urls: list[str]) -> + int`, `Store.count_personal_feeds() -> int`. Every later task that touches storage uses these + exact names. + +- [ ] **Step 1: Write the failing tests** + +Add to `tests/test_store.py`, right after `test_feed_crud_and_dedupe` (around line 73): + +```python +async def test_personal_feed_crud(tmp_path): + store = await Store(str(tmp_path / "roger.db")).open() + try: + assert await store.count_personal_feeds() == 0 + assert await store.add_personal_feed("http://a", "A") is True + assert await store.add_personal_feed("http://a", "A") is False # duplicate URL ignored + assert await store.add_personal_feed("http://b", None) is True + assert [f["url"] for f in await store.list_personal_feeds()] == ["http://a", "http://b"] + + assert await store.remove_personal_feed("http://a") is True + assert await store.remove_personal_feed("http://a") is False # already gone + assert [f["url"] for f in await store.list_personal_feeds()] == ["http://b"] + finally: + await store.close() + + +async def test_personal_feeds_are_isolated_from_the_public_list(tmp_path): + store = await Store(str(tmp_path / "roger.db")).open() + try: + await store.add_feed("http://shared", None) + await store.add_personal_feed("http://shared", None) + assert await store.count_feeds() == 1 + assert await store.count_personal_feeds() == 1 + await store.remove_feed("http://shared") + assert await store.count_feeds() == 0 + assert await store.count_personal_feeds() == 1 # independent lists + finally: + await store.close() +``` + +Add to `tests/test_store.py`, right after `test_seed_feeds_ignores_existing` (end of file): + +```python + + +async def test_seed_personal_feeds_ignores_existing(tmp_path): + store = await Store(str(tmp_path / "roger.db")).open() + try: + await store.add_personal_feed("http://a", None) + await store.seed_personal_feeds(["http://a", "http://b"]) # "http://a" already present + assert {f["url"] for f in await store.list_personal_feeds()} == {"http://a", "http://b"} + finally: + await store.close() +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `pytest tests/test_store.py -k personal_feed -v` +Expected: FAIL — `AttributeError: 'Store' object has no attribute 'add_personal_feed'` (and +similarly for the other three new methods). + +- [ ] **Step 3: Add the table and the five methods** + +In `roger/store.py`, add to `_SCHEMA` right after the `feeds` table definition (after line 86): + +```python +CREATE TABLE IF NOT EXISTS personal_feeds ( + url TEXT PRIMARY KEY, + title TEXT, + added_ts REAL NOT NULL +); +``` + +Add the five methods right after `seed_feeds` (after line 352, before the `# --- digest dedupe (§9) +---` comment): + +```python + # --- personal digest feed list (owner-only; seeded once from PERSONAL_DIGEST_FEEDS) --- + + async def list_personal_feeds(self) -> list[dict[str, Any]]: + cursor = await self._conn.execute( + "SELECT url, title, added_ts FROM personal_feeds ORDER BY added_ts, url" + ) + return [dict(row) for row in await cursor.fetchall()] + + async def count_personal_feeds(self) -> int: + cursor = await self._conn.execute("SELECT COUNT(*) FROM personal_feeds") + row = await cursor.fetchone() + return int(row[0]) if row else 0 + + async def add_personal_feed(self, url: str, title: str | None) -> bool: + """Insert a personal feed. Returns True if newly added, False if the URL already existed.""" + cursor = await self._conn.execute( + "INSERT OR IGNORE INTO personal_feeds (url, title, added_ts) VALUES (?, ?, ?)", + (url, title, time.time()), + ) + await self._conn.commit() + return cursor.rowcount > 0 + + async def remove_personal_feed(self, url: str) -> bool: + """Delete a personal feed by exact URL. Returns True if a row was removed.""" + cursor = await self._conn.execute("DELETE FROM personal_feeds WHERE url = ?", (url,)) + await self._conn.commit() + return cursor.rowcount > 0 + + async def seed_personal_feeds(self, urls: list[str]) -> int: + now = time.time() + await self._conn.executemany( + "INSERT OR IGNORE INTO personal_feeds (url, title, added_ts) VALUES (?, ?, ?)", + [(url, None, now) for url in urls], + ) + await self._conn.commit() + return len(urls) +``` + +No migration is needed — `CREATE TABLE IF NOT EXISTS` handles a live DB that predates this table the +same way it already handles every other table. + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `pytest tests/test_store.py -v` +Expected: PASS (all tests in the file, not just the new ones — confirms nothing else broke). + +- [ ] **Step 5: Commit** + +```bash +git add roger/store.py tests/test_store.py +git commit -m "feat: add personal_feeds table and CRUD methods" +``` + +--- + +### Task 2: Config — settings, env forwarding, and the derived property + +**Files:** +- Modify: `roger/config.py` +- Modify: `roger.env.example` +- Modify: `compose.yaml` +- Test: `tests/test_config.py` + +**Interfaces:** +- Consumes: nothing from Task 1. +- Produces: `settings.personal_digest_feeds: str`, `settings.personal_digest_channel_id: int | + None`, `settings.personal_digest_hour: int`, `settings.personal_feeds: list[str]` (property). + Task 3 and Task 5 read these exact names. + +- [ ] **Step 1: Write the failing tests** + +Add to `tests/test_config.py`, right after `test_empty_digest_channel_id_becomes_none` (after line +62): + +```python + + +def test_personal_digest_defaults(monkeypatch): + _set_required(monkeypatch) + settings = Settings() + assert settings.personal_digest_feeds == "" + assert settings.personal_feeds == [] + assert settings.personal_digest_channel_id is None + assert settings.personal_digest_hour == 7 + + +def test_personal_feeds_is_parsed_to_list(monkeypatch): + _set_required(monkeypatch) + monkeypatch.setenv("PERSONAL_DIGEST_FEEDS", "http://a, http://b ,http://c") + assert Settings().personal_feeds == ["http://a", "http://b", "http://c"] + + +def test_empty_personal_digest_channel_id_becomes_none(monkeypatch): + _set_required(monkeypatch) + monkeypatch.setenv("PERSONAL_DIGEST_CHANNEL_ID", "") + assert Settings().personal_digest_channel_id is None +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `pytest tests/test_config.py -k personal -v` +Expected: FAIL — `pydantic_core._pydantic_core.ValidationError` or `AttributeError` (the field +doesn't exist yet). + +- [ ] **Step 3: Add the settings, property, and validator entry** + +In `roger/config.py`, add to the `# --- digest ---` block (after line 69, `digest_hour: int = 8`): + +```python + + # --- personal digest (owner-only, DM by default) --- + personal_digest_feeds: str = "" + # unset = DM the owner directly; set = post there instead (same shape as digest_channel_id). + personal_digest_channel_id: int | None = None + personal_digest_hour: int = 7 +``` + +Update the validator (line 84-86) to include the new field: + +```python + @field_validator( + "digest_channel_id", + "ops_channel_id", + "gigabrain_channel_id", + "personal_digest_channel_id", + mode="before", + ) +``` + +Add the derived property right after `feeds` (after line 112): + +```python + + @property + def personal_feeds(self) -> list[str]: + return _split_csv(self.personal_digest_feeds) +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `pytest tests/test_config.py -v` +Expected: PASS. + +- [ ] **Step 5: Forward the new settings in `roger.env.example` and `compose.yaml`** + +In `roger.env.example`, add right after `DIGEST_HOUR=8` (line 67), before `TZ=America/Detroit`: + +``` +# --- personal digest --- +# comma-separated RSS/Atom URLs, curated separately from the public digest above. Seeds once, same +# rule as DIGEST_FEEDS. +PERSONAL_DIGEST_FEEDS= +# unset = DM the owner directly; set = post there instead (same shape as DIGEST_CHANNEL_ID) +PERSONAL_DIGEST_CHANNEL_ID= +# local hour 0-23 +PERSONAL_DIGEST_HOUR=7 +``` + +In `compose.yaml`, add right after `DIGEST_HOUR: ${DIGEST_HOUR:-8}` (line 45), before +`OPS_CHANNEL_ID`: + +```yaml + PERSONAL_DIGEST_FEEDS: ${PERSONAL_DIGEST_FEEDS:-} + PERSONAL_DIGEST_CHANNEL_ID: ${PERSONAL_DIGEST_CHANNEL_ID:-} + PERSONAL_DIGEST_HOUR: ${PERSONAL_DIGEST_HOUR:-7} +``` + +- [ ] **Step 6: Run the compose-forwarding gate and the full suite** + +Run: `pytest tests/test_compose.py tests/test_config.py -v` +Expected: PASS — confirms the three new fields are forwarded and nothing dangles. + +Run: `pytest -q` +Expected: PASS (full suite). + +- [ ] **Step 7: Commit** + +```bash +git add roger/config.py roger.env.example compose.yaml tests/test_config.py +git commit -m "feat: add PERSONAL_DIGEST_* settings" +``` + +--- + +### Task 3: Digest brain — the second job + +**Files:** +- Modify: `roger/brains/digest.py` +- Test: `tests/test_digest.py` + +**Interfaces:** +- Consumes: `Store.list_personal_feeds`, `Store.add_personal_feed` (Task 1); + `settings.personal_feeds`, `settings.personal_digest_channel_id` (Task 2); the existing + module-private `_collect_new(feeds, store)` and `_summarize(entries, llm)` (unchanged, reused + as-is). +- Produces: `seed_personal_feeds_if_empty(store, settings) -> int`, + `run_personal_digest_job(*, client, settings, llm, store) -> dict[str, Any]`. Task 5's scheduling + loop calls both by these exact names. + +- [ ] **Step 1: Write the failing tests** + +Add to `tests/test_digest.py`, at the end of the file: + +```python + + +# --------------------------------------------------------------------------- personal digest + + +class FakeDMChannel: + def __init__(self): + self.sent = [] + + async def send(self, embed=None, content=None): + self.sent.append(embed if embed is not None else content) + + +class FakeUser: + def __init__(self, raise_on_create_dm=None): + self._raise_on_create_dm = raise_on_create_dm + self.dm_channel = FakeDMChannel() + + async def create_dm(self): + if self._raise_on_create_dm is not None: + raise self._raise_on_create_dm + return self.dm_channel + + +class FakePersonalClient: + def __init__(self, user=None, channel=None): + self._user = user + self._channel = channel + + def get_channel(self, channel_id): + return self._channel + + async def fetch_user(self, user_id): + return self._user + + +def _http_error(kind, status): + """Build a real discord HTTP error without a live aiohttp response.""" + response = SimpleNamespace(status=status, reason="test") + return kind(response, "boom") + + +def _personal_settings(channel_id=None, tz="America/Detroit", owner_id=1): + return SimpleNamespace(personal_digest_channel_id=channel_id, tz=tz, owner_id=owner_id) + + +async def _personal_store(tmp_path, feeds=("http://pf",)): + store = await Store(str(tmp_path / "pdig.db")).open() + for url in feeds: + await store.add_personal_feed(url, None) + return store + + +async def test_personal_seed_if_empty_is_one_shot(tmp_path): + store = await _personal_store(tmp_path, feeds=()) # start empty + try: + seeded = await digest.seed_personal_feeds_if_empty( + store, SimpleNamespace(personal_feeds=["http://s1", "http://s2"]) + ) + assert seeded == 2 + assert await store.count_personal_feeds() == 2 + # A later env change does NOT re-seed once the store is populated. + again = await digest.seed_personal_feeds_if_empty( + store, SimpleNamespace(personal_feeds=["http://s3"]) + ) + assert again == 0 + finally: + await store.close() + + +async def test_personal_not_configured_when_no_feeds(tmp_path): + store = await _personal_store(tmp_path, feeds=()) + try: + user = FakeUser() + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(), + llm=FakeLLM([]), + store=store, + ) + assert "not configured" in out["status"] + assert user.dm_channel.sent == [] + finally: + await store.close() + + +async def test_personal_no_new_items_skips(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([])) + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=FakeUser()), + settings=_personal_settings(), + llm=FakeLLM([]), + store=store, + ) + assert out["status"] == "no new items" + finally: + await store.close() + + +async def test_personal_posts_via_dm_when_no_channel_configured(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([_entry("n1")])) + user = FakeUser() + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(channel_id=None), + llm=FakeLLM([_resp("summary")]), + store=store, + ) + assert out["status"] == "posted" and out["count"] == 1 + assert len(user.dm_channel.sent) == 1 + assert isinstance(user.dm_channel.sent[0], discord.Embed) + + out2 = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(channel_id=None), + llm=FakeLLM([]), + store=store, + ) + assert out2["status"] == "no new items" # marked seen after the first post + finally: + await store.close() + + +async def test_personal_posts_to_channel_when_configured(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([_entry("n1")])) + channel = FakeChannel() + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=FakeUser(), channel=channel), + settings=_personal_settings(channel_id=99), + llm=FakeLLM([_resp("summary")]), + store=store, + ) + assert out["status"] == "posted" + assert len(channel.sent) == 1 + finally: + await store.close() + + +async def test_personal_dm_creation_failure_is_reported(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([_entry("n1")])) + user = FakeUser(raise_on_create_dm=_http_error(discord.Forbidden, 403)) + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(channel_id=None), + llm=FakeLLM([_resp("summary")]), + store=store, + ) + assert "DM failed" in out["status"] + finally: + await store.close() + + +async def test_personal_budget_skips_post_and_stays_retryable(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([_entry("n1")])) + user = FakeUser() + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(channel_id=None), + llm=FakeLLM([BudgetExceeded("digest", 100, 50)]), + store=store, + ) + assert "budget" in out["status"] + assert user.dm_channel.sent == [] + assert len(await _collect_new(["http://pf"], store)) == 1 # not marked seen + finally: + await store.close() +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `pytest tests/test_digest.py -k personal -v` +Expected: FAIL — `AttributeError: module 'roger.brains.digest' has no attribute +'seed_personal_feeds_if_empty'` (and similarly once that's added, for `run_personal_digest_job`). + +- [ ] **Step 3: Implement the two functions** + +In `roger/brains/digest.py`, add right after `seed_feeds_if_empty` (after line 85): + +```python + + +async def seed_personal_feeds_if_empty(store: Store, settings: Any) -> int: + """One-time bootstrap: import PERSONAL_DIGEST_FEEDS the first time the table is empty. + + Same one-shot rule as ``seed_feeds_if_empty`` — after the initial seed the store is + authoritative. + """ + if await store.count_personal_feeds() > 0: + return 0 + return await store.seed_personal_feeds(settings.personal_feeds) + + +async def run_personal_digest_job( + *, client: Any, settings: Any, llm: LLM, store: Store +) -> dict[str, Any]: + """Like ``run_digest_job``, but sourced from the personal feed list and delivered privately. + + Delivery copies ``run_gigabrain_suggestion``'s DM-or-channel pattern: the configured channel + if set, else a DM to the owner. Unlike the public digest, no channel is required to be + "configured" — "not configured" here means "no feeds," since a DM destination is always + reachable in principle. + """ + feeds = [row["url"] for row in await store.list_personal_feeds()] + if not feeds: + return {"status": "personal digest not configured (no feeds)"} + + entries = await _collect_new(feeds, store) + if not entries: + return {"status": "no new items"} + + try: + summary = await _summarize(entries, llm) + except BudgetExceeded: + log.warning("personal digest skipped: daily token budget hit") + return {"status": "budget exceeded; skipped"} + except LLMConfigError as exc: + return {"status": f"digest brain not configured ({exc})"} + + channel_id = settings.personal_digest_channel_id + if channel_id is not None: + destination = client.get_channel(channel_id) + if destination is None: + return {"status": f"personal digest channel {channel_id} not found"} + else: + try: + owner = await client.fetch_user(settings.owner_id) + destination = await owner.create_dm() + except discord.DiscordException: + log.exception("failed to open a DM with the owner for the personal digest") + return {"status": "DM failed; digest not delivered"} + + today = datetime.datetime.now(ZoneInfo(settings.tz)).strftime("%Y-%m-%d") + embed = discord.Embed(title=f"Roger's personal digest — {today}", description=summary[:4096]) + try: + await destination.send(embed=embed) + except discord.DiscordException: + log.exception("failed to deliver the personal digest") + return {"status": "delivery failed; digest not sent"} + + # Mark seen only after a successful send, so a failed delivery retries the same items. + await store.mark_seen([(entry["feed_url"], entry["id"]) for entry in entries]) + return {"status": "posted", "count": len(entries)} +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `pytest tests/test_digest.py -v` +Expected: PASS (full file). + +- [ ] **Step 5: Commit** + +```bash +git add roger/brains/digest.py tests/test_digest.py +git commit -m "feat: run_personal_digest_job with DM-or-channel delivery" +``` + +--- + +### Task 4: Curation tools + +**Files:** +- Modify: `roger/tools/schemas.py` +- Modify: `roger/tools/executors.py` +- Test: `tests/test_executors.py` + +**Interfaces:** +- Consumes: `Store.list_personal_feeds`, `Store.add_personal_feed`, `Store.remove_personal_feed` + (Task 1); the existing module-private `validate_feed(url)` and `_need_store(ctx)` in + `executors.py` (unchanged, reused as-is). +- Produces: four new registry entries — `list_personal_feeds`, `suggest_personal_feeds`, + `add_personal_feed`, `remove_personal_feed` — available to the admin brain automatically (it + builds its tool list from `list(schemas.REGISTRY)`, so no `roger/brains/admin.py` change is + needed). + +- [ ] **Step 1: Write the failing tests** + +Add to `tests/test_executors.py`, right after `test_feed_tool_without_store_raises_guard_error` +(end of the feed-tools section, before the `# ---... read-only server info` comment): + +```python + + +async def test_add_personal_feed_validates_and_persists(feeds): + feeds.responses["http://good"] = _good_feed(title="Good Blog", n=5) + out = await executors.add_personal_feed( + None, AddPersonalFeedArgs(url="http://good"), feeds.ctx + ) + assert out["added"] is True + assert out["title"] == "Good Blog" + assert [f["url"] for f in await feeds.store.list_personal_feeds()] == ["http://good"] + + +async def test_add_personal_feed_rejects_non_feed(feeds): + out = await executors.add_personal_feed( + None, AddPersonalFeedArgs(url="http://nope"), feeds.ctx + ) + assert out["added"] is False + assert await feeds.store.count_personal_feeds() == 0 + + +async def test_add_personal_feed_is_idempotent(feeds): + feeds.responses["http://good"] = _good_feed() + await executors.add_personal_feed(None, AddPersonalFeedArgs(url="http://good"), feeds.ctx) + out = await executors.add_personal_feed( + None, AddPersonalFeedArgs(url="http://good"), feeds.ctx + ) + assert out["added"] is False + assert out["note"] == "already in the personal feed list" + + +async def test_remove_personal_feed_hit_and_miss(feeds): + feeds.responses["http://good"] = _good_feed() + await executors.add_personal_feed(None, AddPersonalFeedArgs(url="http://good"), feeds.ctx) + hit = await executors.remove_personal_feed( + None, RemovePersonalFeedArgs(url="http://good"), feeds.ctx + ) + assert hit["removed"] is True + miss = await executors.remove_personal_feed( + None, RemovePersonalFeedArgs(url="http://good"), feeds.ctx + ) + assert miss["removed"] is False + + +async def test_list_personal_feeds_returns_current(feeds): + feeds.responses["http://a"] = _good_feed(title="A") + await executors.add_personal_feed(None, AddPersonalFeedArgs(url="http://a"), feeds.ctx) + out = await executors.list_personal_feeds(None, ListPersonalFeedsArgs(), feeds.ctx) + assert out["count"] == 1 + assert out["feeds"][0] == {"url": "http://a", "title": "A"} + + +async def test_suggest_personal_feeds_validates_without_persisting(feeds): + feeds.responses["http://ok"] = _good_feed(title="OK", n=2) + out = await executors.suggest_personal_feeds( + None, SuggestPersonalFeedsArgs(urls=["http://ok"]), feeds.ctx + ) + by_url = {c["url"]: c for c in out["candidates"]} + assert by_url["http://ok"]["ok"] is True + assert await feeds.store.count_personal_feeds() == 0 # suggest never writes + + +async def test_personal_feed_tool_without_store_raises_guard_error(): + with pytest.raises(GuardError): + await executors.list_personal_feeds(None, ListPersonalFeedsArgs(), None) +``` + +Add the four new names to the existing `from roger.tools.schemas import (...)` block near the top of +`tests/test_executors.py` (wherever `AddFeedArgs`, `ListFeedsArgs`, `RemoveFeedArgs`, +`SuggestFeedsArgs` are currently imported) — `AddPersonalFeedArgs`, `ListPersonalFeedsArgs`, +`RemovePersonalFeedArgs`, `SuggestPersonalFeedsArgs`, kept alphabetical alongside the others. + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `pytest tests/test_executors.py -k personal_feed -v` +Expected: FAIL — `ImportError` (the new arg classes don't exist yet), or once that's stubbed, +`AttributeError: module 'roger.tools.executors' has no attribute 'add_personal_feed'`. + +- [ ] **Step 3: Add the four arg models and registry entries in `schemas.py`** + +In `roger/tools/schemas.py`, add right after `RemoveFeedArgs` (after line 177): + +```python + + +class ListPersonalFeedsArgs(ToolArgs): + """No arguments — returns the owner's personal digest feed list.""" + + +class SuggestPersonalFeedsArgs(ToolArgs): + urls: list[str] = Field(min_length=1, max_length=8) # candidate feed URLs to vet + + +class AddPersonalFeedArgs(ToolArgs): + url: str # RSS/Atom feed URL; validated live before it is stored + + +class RemovePersonalFeedArgs(ToolArgs): + url: str # exact stored URL (from list_personal_feeds) +``` + +Add to `REGISTRY` right after the `"remove_feed"` entry (after line 441, before `"set_presence"`): + +```python + "list_personal_feeds": ToolSpec( + name="list_personal_feeds", + description="List the RSS/Atom feeds in the owner's personal digest (DM'd privately, " + "separate from the public digest). Read-only.", + args_model=ListPersonalFeedsArgs, + ), + "suggest_personal_feeds": ToolSpec( + name="suggest_personal_feeds", + description=( + "Validate candidate RSS/Atom feed URLs WITHOUT adding them to the personal digest. " + "Returns, per URL, whether it's a live feed, its title, and how many items it has. " + "Use this to vet feeds you propose before calling add_personal_feed." + ), + args_model=SuggestPersonalFeedsArgs, + ), + "add_personal_feed": ToolSpec( + name="add_personal_feed", + description=( + "Validate and add one RSS/Atom feed to the owner's personal digest. Fails if the URL " + "isn't a live feed. Idempotent — adding an existing feed is a no-op." + ), + args_model=AddPersonalFeedArgs, + ), + "remove_personal_feed": ToolSpec( + name="remove_personal_feed", + description=( + "Remove a feed from the owner's personal digest by its exact URL. Call " + "list_personal_feeds first to get the exact URL." + ), + args_model=RemovePersonalFeedArgs, + ), +``` + +- [ ] **Step 4: Add the four executor functions in `executors.py`** + +Add the four new names to the existing `from roger.tools.schemas import (...)` block (lines 28-58), +kept alphabetical: `AddPersonalFeedArgs` (between `AddMemberRoleArgs` and `AddReactionArgs`), +`ListPersonalFeedsArgs` (between `ListInvitesArgs` and `ListRoleMembersArgs`), +`RemovePersonalFeedArgs` (between `RemoveMemberRoleArgs` and `RemoveReactionArgs`), +`SuggestPersonalFeedsArgs` (after `SuggestFeedsArgs`). + +Add the four functions right after `list_feeds` (after line 698, before the +`# ---... toys (self / read)` comment): + +```python + +# --------------------------------------------------------------------------- personal digest feeds + + +async def suggest_personal_feeds( + guild: discord.Guild, args: SuggestPersonalFeedsArgs, ctx: ToolContext | None = None +) -> dict[str, Any]: + candidates = await asyncio.gather(*(validate_feed(url) for url in args.urls)) + return {"candidates": list(candidates)} + + +async def add_personal_feed( + guild: discord.Guild, args: AddPersonalFeedArgs, ctx: ToolContext | None = None +) -> dict[str, Any]: + store = _need_store(ctx) + checked = await validate_feed(args.url) + if not checked["ok"]: + return {"added": False, "url": args.url, "error": checked["error"]} + added = await store.add_personal_feed(args.url, checked.get("title")) + return { + "added": added, + "url": args.url, + "title": checked.get("title"), + "entries": checked.get("entries"), + "note": None if added else "already in the personal feed list", + } + + +async def remove_personal_feed( + guild: discord.Guild, args: RemovePersonalFeedArgs, ctx: ToolContext | None = None +) -> dict[str, Any]: + store = _need_store(ctx) + removed = await store.remove_personal_feed(args.url) + return { + "removed": removed, + "url": args.url, + "note": None + if removed + else "no feed with that exact URL (call list_personal_feeds first)", + } + + +async def list_personal_feeds( + guild: discord.Guild, args: ListPersonalFeedsArgs, ctx: ToolContext | None = None +) -> dict[str, Any]: + store = _need_store(ctx) + rows = await store.list_personal_feeds() + return { + "feeds": [{"url": r["url"], "title": r["title"]} for r in rows], + "count": len(rows), + } +``` + +Add the four functions to the `EXECUTORS` dict right after `"list_feeds": list_feeds,`: + +```python + "suggest_personal_feeds": suggest_personal_feeds, + "add_personal_feed": add_personal_feed, + "remove_personal_feed": remove_personal_feed, + "list_personal_feeds": list_personal_feeds, +``` + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `pytest tests/test_executors.py -v` +Expected: PASS (full file). + +Run: `pytest -q` +Expected: PASS (full suite — confirms the wider `REGISTRY`/`EXECUTORS` dicts didn't break an +existing test that enumerates them, e.g. an admin-tools-are-all-registered check). + +- [ ] **Step 6: Commit** + +```bash +git add roger/tools/schemas.py roger/tools/executors.py tests/test_executors.py +git commit -m "feat: add personal digest curation tools" +``` + +--- + +### Task 5: Scheduling and the ops alert + +**Files:** +- Modify: `roger/bot.py` +- Test: `tests/test_ops.py` + +**Interfaces:** +- Consumes: `run_personal_digest_job`, `seed_personal_feeds_if_empty` (Task 3); + `settings.personal_feeds`, `settings.personal_digest_hour` (Task 2); the existing `_DAY_S` + constant and `self._ops.alert(key, message, cooldown_s=...)` method (unchanged). +- Produces: `_personal_digest_problem(status: str) -> str | None` (module-level, pure — mirrors + `_digest_problem`). No later task depends on anything new here. + +- [ ] **Step 1: Write the failing tests** + +Add to `tests/test_ops.py`, right after `test_digest_problem_flags_failures` (after line 71, before +the gigabrain-problem tests): + +```python + + +def test_personal_digest_problem_none_for_success_statuses(): + assert _personal_digest_problem("posted") is None + assert _personal_digest_problem("no new items") is None + + +def test_personal_digest_problem_flags_failures(): + assert _personal_digest_problem("personal digest not configured (no feeds)") is not None + assert _personal_digest_problem("DM failed; digest not delivered") is not None + assert _personal_digest_problem("budget exceeded; skipped") is not None +``` + +Update the import at the top of `tests/test_ops.py` (line 3) to add `_personal_digest_problem`: + +```python +from roger.bot import ( + OpsNotifier, + _budget_alert, + _digest_problem, + _gigabrain_problem, + _personal_digest_problem, +) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `pytest tests/test_ops.py -k personal_digest_problem -v` +Expected: FAIL — `ImportError: cannot import name '_personal_digest_problem'`. + +- [ ] **Step 3: Add the problem helper** + +In `roger/bot.py`, add right after `_digest_problem` (after line 418, before the +`# Gigabrain statuses...` comment): + +```python + +# Personal digest statuses that mean "ran fine, nothing to flag"; anything else is worth an ops +# ping — same OK-prefix shape as the public digest. +_PERSONAL_DIGEST_OK_PREFIXES = ("posted", "no new items") + + +def _personal_digest_problem(status: str) -> str | None: + """The personal digest status if it signals a problem worth alerting on, else None (pure).""" + if any(status.startswith(prefix) for prefix in _PERSONAL_DIGEST_OK_PREFIXES): + return None + return status +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `pytest tests/test_ops.py -v` +Expected: PASS. + +- [ ] **Step 5: Wire the scheduled loop** + +Update the import at the top of `roger/bot.py` (line 32): + +```python +from roger.brains.digest import ( + run_digest_job, + run_personal_digest_job, + seed_feeds_if_empty, + seed_personal_feeds_if_empty, +) +``` + +In `setup_hook` (around line 466-468), add right after the existing feed-seeding block: + +```python + seeded = await seed_feeds_if_empty(self.store, self.settings) + if seeded: + log.info("seeded %d feed(s) from DIGEST_FEEDS into the store", seeded) + personal_seeded = await seed_personal_feeds_if_empty(self.store, self.settings) + if personal_seeded: + log.info( + "seeded %d feed(s) from PERSONAL_DIGEST_FEEDS into the store", personal_seeded + ) +``` + +In `setup_hook`, add right after the existing digest-loop start block (after line 485, before the +gigabrain block): + +```python + # Turned on by configuring at least one seed feed — DM delivery needs no channel to be set, + # unlike the public digest's channel-required gate. + if self.settings.personal_digest_feeds: + self._personal_digest_loop.change_interval( + time=datetime.time( + hour=self.settings.personal_digest_hour, tzinfo=ZoneInfo(self.settings.tz) + ) + ) + self._personal_digest_loop.start() + log.info( + "personal digest scheduled daily at %02d:00 %s", + self.settings.personal_digest_hour, + self.settings.tz, + ) +``` + +Add the loop method right after `_before_digest` (after line 664, before the `_gigabrain_loop` +definition): + +```python + @tasks.loop(time=datetime.time(hour=7)) + async def _personal_digest_loop(self) -> None: + result = await run_personal_digest_job( + client=self, settings=self.settings, llm=self.llm, store=self.store + ) + status = str(result.get("status", "")) + log.info("scheduled personal digest: %s", status) + problem = _personal_digest_problem(status) + if problem: + await self._ops.alert( + f"personal_digest:{time.strftime('%Y-%m-%d')}", + f"⚠️ **personal digest problem** — {problem}", + cooldown_s=_DAY_S, + ) + + @_personal_digest_loop.before_loop + async def _before_personal_digest(self) -> None: + await self.wait_until_ready() +``` + +- [ ] **Step 6: Run the full suite** + +Run: `pytest -q` +Expected: PASS (full suite). + +Run: `ruff check .` +Expected: clean. + +- [ ] **Step 7: Commit** + +```bash +git add roger/bot.py tests/test_ops.py +git commit -m "feat: schedule the personal digest and alert on failure" +``` + +--- + +### Task 6: Docs + +**Files:** +- Modify: `ARCHITECTURE.md` +- Modify: `ROADMAP.md` +- Modify: `README.md` + +No test — this is a prose-only task. Verification is a full-suite run plus a read-through. + +- [ ] **Step 1: Update `ARCHITECTURE.md` §9** + +Add a new bullet to the `## §9 Digest brain` section, right after the "Exactly-once posting" bullet +(after line 259): + +```markdown +- **A personal, DM'd sibling.** `run_personal_digest_job` is the same mechanism — fetch, dedupe, + summarize — pointed at a second, separately-curated `personal_feeds` list, and delivered to the + owner only (`PERSONAL_DIGEST_CHANNEL_ID` if set, else a DM — same fallback shape Giga Brain's + periodic check-in uses, §12). It shares the `digest` brain's model and daily budget; it's a + second job, not a second brain. Curated the same way — `suggest_personal_feeds` / `add_personal_feed` + / `remove_personal_feed` / `list_personal_feeds` mirror the public digest's four curation tools + exactly. +``` + +- [ ] **Step 2: Update `ARCHITECTURE.md` §10** + +Add a row to the persistence table (after the `feeds` row, around line 274): + +```markdown +| `personal_feeds` | The owner's personal digest feed list, curated separately from `feeds` (§9) | +``` + +- [ ] **Step 3: Flip `ROADMAP.md` item 1 to shipped** + +In `ROADMAP.md`, change the item 1 heading from: + +```markdown +## 1. Personal digest — **S/M** — *spec written* +``` + +to: + +```markdown +## 1. Personal digest — **S/M** — *shipped* +``` + +- [ ] **Step 4: Update `README.md`'s Status section** + +In the `## Status` section's Digest bullet, add one sentence noting the personal digest. Find the +bullet starting `- **Digest** — a scheduled daily RSS/Atom summary...` and append after its existing +sentence: + +```markdown + A second, privately-curated feed list can also be DM'd to the owner only + (`PERSONAL_DIGEST_FEEDS`), on its own schedule. +``` + +- [ ] **Step 5: Run the full suite one more time** + +Run: `pytest -q` +Expected: PASS. + +Run: `ruff check .` +Expected: clean. + +- [ ] **Step 6: Commit** + +```bash +git add ARCHITECTURE.md ROADMAP.md README.md +git commit -m "docs: personal digest in ARCHITECTURE, ROADMAP, README" +``` From ae17070f20580d485da0013392ca2204bf0a26c7 Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 21:15:04 -0400 Subject: [PATCH 03/10] feat: add personal_feeds table and CRUD methods --- roger/store.py | 43 +++++++++++++++++++++++++++++++++++++++++++ tests/test_store.py | 40 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+) diff --git a/roger/store.py b/roger/store.py index 023464e..f60f843 100644 --- a/roger/store.py +++ b/roger/store.py @@ -85,6 +85,12 @@ class AuditStatus(StrEnum): added_ts REAL NOT NULL ); +CREATE TABLE IF NOT EXISTS personal_feeds ( + url TEXT PRIMARY KEY, + title TEXT, + added_ts REAL NOT NULL +); + CREATE TABLE IF NOT EXISTS meta ( key TEXT PRIMARY KEY, value TEXT NOT NULL @@ -351,6 +357,43 @@ async def seed_feeds(self, urls: list[str]) -> int: await self._conn.commit() return len(urls) + # --- personal digest feed list (owner-only; seeded once from PERSONAL_DIGEST_FEEDS) --- + + async def list_personal_feeds(self) -> list[dict[str, Any]]: + cursor = await self._conn.execute( + "SELECT url, title, added_ts FROM personal_feeds ORDER BY added_ts, url" + ) + return [dict(row) for row in await cursor.fetchall()] + + async def count_personal_feeds(self) -> int: + cursor = await self._conn.execute("SELECT COUNT(*) FROM personal_feeds") + row = await cursor.fetchone() + return int(row[0]) if row else 0 + + async def add_personal_feed(self, url: str, title: str | None) -> bool: + """Insert a personal feed. Returns True if newly added, False if the URL already existed.""" + cursor = await self._conn.execute( + "INSERT OR IGNORE INTO personal_feeds (url, title, added_ts) VALUES (?, ?, ?)", + (url, title, time.time()), + ) + await self._conn.commit() + return cursor.rowcount > 0 + + async def remove_personal_feed(self, url: str) -> bool: + """Delete a personal feed by exact URL. Returns True if a row was removed.""" + cursor = await self._conn.execute("DELETE FROM personal_feeds WHERE url = ?", (url,)) + await self._conn.commit() + return cursor.rowcount > 0 + + async def seed_personal_feeds(self, urls: list[str]) -> int: + now = time.time() + await self._conn.executemany( + "INSERT OR IGNORE INTO personal_feeds (url, title, added_ts) VALUES (?, ?, ?)", + [(url, None, now) for url in urls], + ) + await self._conn.commit() + return len(urls) + # --- digest dedupe (§9) --- async def filter_unseen(self, feed_url: str, entry_ids: list[str]) -> set[str]: diff --git a/tests/test_store.py b/tests/test_store.py index 4a99707..0dc3759 100644 --- a/tests/test_store.py +++ b/tests/test_store.py @@ -72,6 +72,36 @@ async def test_feed_crud_and_dedupe(tmp_path): await store.close() +async def test_personal_feed_crud(tmp_path): + store = await Store(str(tmp_path / "roger.db")).open() + try: + assert await store.count_personal_feeds() == 0 + assert await store.add_personal_feed("http://a", "A") is True + assert await store.add_personal_feed("http://a", "A") is False # duplicate URL ignored + assert await store.add_personal_feed("http://b", None) is True + assert [f["url"] for f in await store.list_personal_feeds()] == ["http://a", "http://b"] + + assert await store.remove_personal_feed("http://a") is True + assert await store.remove_personal_feed("http://a") is False # already gone + assert [f["url"] for f in await store.list_personal_feeds()] == ["http://b"] + finally: + await store.close() + + +async def test_personal_feeds_are_isolated_from_the_public_list(tmp_path): + store = await Store(str(tmp_path / "roger.db")).open() + try: + await store.add_feed("http://shared", None) + await store.add_personal_feed("http://shared", None) + assert await store.count_feeds() == 1 + assert await store.count_personal_feeds() == 1 + await store.remove_feed("http://shared") + assert await store.count_feeds() == 0 + assert await store.count_personal_feeds() == 1 # independent lists + finally: + await store.close() + + async def test_usage_accumulates_tokens_and_cost(tmp_path): store = await Store(str(tmp_path / "roger.db")).open() try: @@ -157,3 +187,13 @@ async def test_seed_feeds_ignores_existing(tmp_path): assert {f["url"] for f in await store.list_feeds()} == {"http://a", "http://b"} finally: await store.close() + + +async def test_seed_personal_feeds_ignores_existing(tmp_path): + store = await Store(str(tmp_path / "roger.db")).open() + try: + await store.add_personal_feed("http://a", None) + await store.seed_personal_feeds(["http://a", "http://b"]) # "http://a" already present + assert {f["url"] for f in await store.list_personal_feeds()} == {"http://a", "http://b"} + finally: + await store.close() From 4f52911171103bdc8ff8ae7093b18edb0957537b Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 21:19:58 -0400 Subject: [PATCH 04/10] feat: add PERSONAL_DIGEST_* settings --- compose.yaml | 3 +++ roger.env.example | 9 +++++++++ roger/config.py | 16 +++++++++++++++- tests/test_config.py | 21 +++++++++++++++++++++ 4 files changed, 48 insertions(+), 1 deletion(-) diff --git a/compose.yaml b/compose.yaml index 13d2236..7bdb64b 100644 --- a/compose.yaml +++ b/compose.yaml @@ -43,6 +43,9 @@ services: DIGEST_FEEDS: ${DIGEST_FEEDS:-} DIGEST_CHANNEL_ID: ${DIGEST_CHANNEL_ID:-} DIGEST_HOUR: ${DIGEST_HOUR:-8} + PERSONAL_DIGEST_FEEDS: ${PERSONAL_DIGEST_FEEDS:-} + PERSONAL_DIGEST_CHANNEL_ID: ${PERSONAL_DIGEST_CHANNEL_ID:-} + PERSONAL_DIGEST_HOUR: ${PERSONAL_DIGEST_HOUR:-7} OPS_CHANNEL_ID: ${OPS_CHANNEL_ID:-} METRICS_PORT: ${METRICS_PORT:-9108} TZ: ${TZ:-America/Detroit} diff --git a/roger.env.example b/roger.env.example index 43afb97..741a564 100644 --- a/roger.env.example +++ b/roger.env.example @@ -65,6 +65,15 @@ DIGEST_FEEDS= DIGEST_CHANNEL_ID= # local hour 0-23 DIGEST_HOUR=8 + +# --- personal digest --- +# comma-separated RSS/Atom URLs, curated separately from the public digest above. Seeds once, same +# rule as DIGEST_FEEDS. +PERSONAL_DIGEST_FEEDS= +# unset = DM the owner directly; set = post there instead (same shape as DIGEST_CHANNEL_ID) +PERSONAL_DIGEST_CHANNEL_ID= +# local hour 0-23 +PERSONAL_DIGEST_HOUR=7 TZ=America/Detroit # --- ops --- diff --git a/roger/config.py b/roger/config.py index 56ebeec..ed0668e 100644 --- a/roger/config.py +++ b/roger/config.py @@ -68,6 +68,12 @@ class Settings(BaseSettings): digest_channel_id: int | None = None digest_hour: int = 8 + # --- personal digest (owner-only, DM by default) --- + personal_digest_feeds: str = "" + # unset = DM the owner directly; set = post there instead (same shape as digest_channel_id). + personal_digest_channel_id: int | None = None + personal_digest_hour: int = 7 + # --- ops --- # where Roger posts its boot self-report; None disables the report (logs still fire). ops_channel_id: int | None = None @@ -82,7 +88,11 @@ class Settings(BaseSettings): log_level: str = "INFO" @field_validator( - "digest_channel_id", "ops_channel_id", "gigabrain_channel_id", mode="before" + "digest_channel_id", + "ops_channel_id", + "gigabrain_channel_id", + "personal_digest_channel_id", + mode="before", ) @classmethod def _empty_to_none(cls, value: object) -> object: @@ -111,6 +121,10 @@ def gigabrain_models(self) -> list[str]: def feeds(self) -> list[str]: return _split_csv(self.digest_feeds) + @property + def personal_feeds(self) -> list[str]: + return _split_csv(self.personal_digest_feeds) + def load_settings() -> Settings: return Settings() # type: ignore[call-arg] # values come from the environment diff --git a/tests/test_config.py b/tests/test_config.py index 6ae6125..ef39663 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -62,6 +62,27 @@ def test_empty_digest_channel_id_becomes_none(monkeypatch): assert Settings().digest_channel_id is None +def test_personal_digest_defaults(monkeypatch): + _set_required(monkeypatch) + settings = Settings() + assert settings.personal_digest_feeds == "" + assert settings.personal_feeds == [] + assert settings.personal_digest_channel_id is None + assert settings.personal_digest_hour == 7 + + +def test_personal_feeds_is_parsed_to_list(monkeypatch): + _set_required(monkeypatch) + monkeypatch.setenv("PERSONAL_DIGEST_FEEDS", "http://a, http://b ,http://c") + assert Settings().personal_feeds == ["http://a", "http://b", "http://c"] + + +def test_empty_personal_digest_channel_id_becomes_none(monkeypatch): + _set_required(monkeypatch) + monkeypatch.setenv("PERSONAL_DIGEST_CHANNEL_ID", "") + assert Settings().personal_digest_channel_id is None + + def test_missing_required_field_raises(monkeypatch): for key in _REQUIRED: monkeypatch.delenv(key, raising=False) From 626d6a6dcbb7f23c8bdddbf19d1e8792e23652a0 Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 21:25:23 -0400 Subject: [PATCH 05/10] feat: run_personal_digest_job with DM-or-channel delivery --- roger/brains/digest.py | 63 +++++++++++++++ tests/test_digest.py | 176 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 239 insertions(+) diff --git a/roger/brains/digest.py b/roger/brains/digest.py index 1068a4e..4239b33 100644 --- a/roger/brains/digest.py +++ b/roger/brains/digest.py @@ -85,6 +85,17 @@ async def seed_feeds_if_empty(store: Store, settings: Any) -> int: return await store.seed_feeds(settings.feeds) +async def seed_personal_feeds_if_empty(store: Store, settings: Any) -> int: + """One-time bootstrap: import PERSONAL_DIGEST_FEEDS the first time the table is empty. + + Same one-shot rule as ``seed_feeds_if_empty`` — after the initial seed the store is + authoritative. + """ + if await store.count_personal_feeds() > 0: + return 0 + return await store.seed_personal_feeds(settings.personal_feeds) + + async def run_digest_job(*, client: Any, settings: Any, llm: LLM, store: Store) -> dict[str, Any]: feeds = [row["url"] for row in await store.list_feeds()] channel_id = settings.digest_channel_id @@ -114,3 +125,55 @@ async def run_digest_job(*, client: Any, settings: Any, llm: LLM, store: Store) # Mark seen only after a successful post, so a failed post retries the same items. await store.mark_seen([(entry["feed_url"], entry["id"]) for entry in entries]) return {"status": "posted", "count": len(entries)} + + +async def run_personal_digest_job( + *, client: Any, settings: Any, llm: LLM, store: Store +) -> dict[str, Any]: + """Like ``run_digest_job``, but sourced from the personal feed list and delivered privately. + + Delivery copies ``run_gigabrain_suggestion``'s DM-or-channel pattern: the configured channel + if set, else a DM to the owner. Unlike the public digest, no channel is required to be + "configured" — "not configured" here means "no feeds," since a DM destination is always + reachable in principle. + """ + feeds = [row["url"] for row in await store.list_personal_feeds()] + if not feeds: + return {"status": "personal digest not configured (no feeds)"} + + entries = await _collect_new(feeds, store) + if not entries: + return {"status": "no new items"} + + try: + summary = await _summarize(entries, llm) + except BudgetExceeded: + log.warning("personal digest skipped: daily token budget hit") + return {"status": "budget exceeded; skipped"} + except LLMConfigError as exc: + return {"status": f"digest brain not configured ({exc})"} + + channel_id = settings.personal_digest_channel_id + if channel_id is not None: + destination = client.get_channel(channel_id) + if destination is None: + return {"status": f"personal digest channel {channel_id} not found"} + else: + try: + owner = await client.fetch_user(settings.owner_id) + destination = await owner.create_dm() + except discord.DiscordException: + log.exception("failed to open a DM with the owner for the personal digest") + return {"status": "DM failed; digest not delivered"} + + today = datetime.datetime.now(ZoneInfo(settings.tz)).strftime("%Y-%m-%d") + embed = discord.Embed(title=f"Roger's personal digest — {today}", description=summary[:4096]) + try: + await destination.send(embed=embed) + except discord.DiscordException: + log.exception("failed to deliver the personal digest") + return {"status": "delivery failed; digest not sent"} + + # Mark seen only after a successful send, so a failed delivery retries the same items. + await store.mark_seen([(entry["feed_url"], entry["id"]) for entry in entries]) + return {"status": "posted", "count": len(entries)} diff --git a/tests/test_digest.py b/tests/test_digest.py index 644892f..5b752ae 100644 --- a/tests/test_digest.py +++ b/tests/test_digest.py @@ -180,3 +180,179 @@ async def test_budget_skips_post_and_stays_retryable(tmp_path, monkeypatch): assert len(await _collect_new(["http://f"], store)) == 1 # not marked seen finally: await store.close() + + +# --------------------------------------------------------------------------- personal digest + + +class FakeDMChannel: + def __init__(self): + self.sent = [] + + async def send(self, embed=None, content=None): + self.sent.append(embed if embed is not None else content) + + +class FakeUser: + def __init__(self, raise_on_create_dm=None): + self._raise_on_create_dm = raise_on_create_dm + self.dm_channel = FakeDMChannel() + + async def create_dm(self): + if self._raise_on_create_dm is not None: + raise self._raise_on_create_dm + return self.dm_channel + + +class FakePersonalClient: + def __init__(self, user=None, channel=None): + self._user = user + self._channel = channel + + def get_channel(self, channel_id): + return self._channel + + async def fetch_user(self, user_id): + return self._user + + +def _http_error(kind, status): + """Build a real discord HTTP error without a live aiohttp response.""" + response = SimpleNamespace(status=status, reason="test") + return kind(response, "boom") + + +def _personal_settings(channel_id=None, tz="America/Detroit", owner_id=1): + return SimpleNamespace(personal_digest_channel_id=channel_id, tz=tz, owner_id=owner_id) + + +async def _personal_store(tmp_path, feeds=("http://pf",)): + store = await Store(str(tmp_path / "pdig.db")).open() + for url in feeds: + await store.add_personal_feed(url, None) + return store + + +async def test_personal_seed_if_empty_is_one_shot(tmp_path): + store = await _personal_store(tmp_path, feeds=()) # start empty + try: + seeded = await digest.seed_personal_feeds_if_empty( + store, SimpleNamespace(personal_feeds=["http://s1", "http://s2"]) + ) + assert seeded == 2 + assert await store.count_personal_feeds() == 2 + # A later env change does NOT re-seed once the store is populated. + again = await digest.seed_personal_feeds_if_empty( + store, SimpleNamespace(personal_feeds=["http://s3"]) + ) + assert again == 0 + finally: + await store.close() + + +async def test_personal_not_configured_when_no_feeds(tmp_path): + store = await _personal_store(tmp_path, feeds=()) + try: + user = FakeUser() + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(), + llm=FakeLLM([]), + store=store, + ) + assert "not configured" in out["status"] + assert user.dm_channel.sent == [] + finally: + await store.close() + + +async def test_personal_no_new_items_skips(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([])) + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=FakeUser()), + settings=_personal_settings(), + llm=FakeLLM([]), + store=store, + ) + assert out["status"] == "no new items" + finally: + await store.close() + + +async def test_personal_posts_via_dm_when_no_channel_configured(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([_entry("n1")])) + user = FakeUser() + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(channel_id=None), + llm=FakeLLM([_resp("summary")]), + store=store, + ) + assert out["status"] == "posted" and out["count"] == 1 + assert len(user.dm_channel.sent) == 1 + assert isinstance(user.dm_channel.sent[0], discord.Embed) + + out2 = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(channel_id=None), + llm=FakeLLM([]), + store=store, + ) + assert out2["status"] == "no new items" # marked seen after the first post + finally: + await store.close() + + +async def test_personal_posts_to_channel_when_configured(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([_entry("n1")])) + channel = FakeChannel() + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=FakeUser(), channel=channel), + settings=_personal_settings(channel_id=99), + llm=FakeLLM([_resp("summary")]), + store=store, + ) + assert out["status"] == "posted" + assert len(channel.sent) == 1 + finally: + await store.close() + + +async def test_personal_dm_creation_failure_is_reported(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([_entry("n1")])) + user = FakeUser(raise_on_create_dm=_http_error(discord.Forbidden, 403)) + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(channel_id=None), + llm=FakeLLM([_resp("summary")]), + store=store, + ) + assert "DM failed" in out["status"] + finally: + await store.close() + + +async def test_personal_budget_skips_post_and_stays_retryable(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([_entry("n1")])) + user = FakeUser() + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(channel_id=None), + llm=FakeLLM([BudgetExceeded("digest", 100, 50)]), + store=store, + ) + assert "budget" in out["status"] + assert user.dm_channel.sent == [] + assert len(await _collect_new(["http://pf"], store)) == 1 # not marked seen + finally: + await store.close() From 030ae9bac2d9c017fc81ded196fe68010c23c8e9 Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 21:30:22 -0400 Subject: [PATCH 06/10] test: add coverage for send() delivery failure in personal digest --- tests/test_digest.py | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/tests/test_digest.py b/tests/test_digest.py index 5b752ae..2582430 100644 --- a/tests/test_digest.py +++ b/tests/test_digest.py @@ -186,17 +186,20 @@ async def test_budget_skips_post_and_stays_retryable(tmp_path, monkeypatch): class FakeDMChannel: - def __init__(self): + def __init__(self, raise_on_send=None): self.sent = [] + self._raise_on_send = raise_on_send async def send(self, embed=None, content=None): + if self._raise_on_send is not None: + raise self._raise_on_send self.sent.append(embed if embed is not None else content) class FakeUser: - def __init__(self, raise_on_create_dm=None): + def __init__(self, raise_on_create_dm=None, raise_on_send=None): self._raise_on_create_dm = raise_on_create_dm - self.dm_channel = FakeDMChannel() + self.dm_channel = FakeDMChannel(raise_on_send=raise_on_send) async def create_dm(self): if self._raise_on_create_dm is not None: @@ -356,3 +359,21 @@ async def test_personal_budget_skips_post_and_stays_retryable(tmp_path, monkeypa assert len(await _collect_new(["http://pf"], store)) == 1 # not marked seen finally: await store.close() + + +async def test_personal_send_failure_is_reported(tmp_path, monkeypatch): + store = await _personal_store(tmp_path) + try: + monkeypatch.setattr(digest.feedparser, "parse", lambda url: _feed([_entry("n1")])) + user = FakeUser(raise_on_send=_http_error(discord.HTTPException, 500)) + out = await digest.run_personal_digest_job( + client=FakePersonalClient(user=user), + settings=_personal_settings(channel_id=None), + llm=FakeLLM([_resp("summary")]), + store=store, + ) + assert out["status"] == "delivery failed; digest not sent" + # Item not marked seen after send failure, so it's retryable + assert len(await _collect_new(["http://pf"], store)) == 1 + finally: + await store.close() From 9cab1c6967a6627f3321cc3dc9b0303f2218c8a9 Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 21:35:27 -0400 Subject: [PATCH 07/10] feat: add personal digest curation tools --- roger/tools/executors.py | 60 +++++++++++++++++++++++++++++++++++ roger/tools/schemas.py | 47 +++++++++++++++++++++++++++ tests/test_executors.py | 68 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 175 insertions(+) diff --git a/roger/tools/executors.py b/roger/tools/executors.py index 501f958..801a2c8 100644 --- a/roger/tools/executors.py +++ b/roger/tools/executors.py @@ -28,6 +28,7 @@ from roger.tools.schemas import ( AddFeedArgs, AddMemberRoleArgs, + AddPersonalFeedArgs, AddReactionArgs, CreateChannelArgs, CreateForumPostArgs, @@ -39,6 +40,7 @@ ListFeedsArgs, ListForumPostsArgs, ListInvitesArgs, + ListPersonalFeedsArgs, ListRoleMembersArgs, ListScheduledEventsArgs, ListStructureArgs, @@ -47,6 +49,7 @@ PostMessageArgs, RemoveFeedArgs, RemoveMemberRoleArgs, + RemovePersonalFeedArgs, RemoveReactionArgs, ReplyToForumPostArgs, RunDigestArgs, @@ -55,6 +58,7 @@ SetPermissionsArgs, SetPresenceArgs, SuggestFeedsArgs, + SuggestPersonalFeedsArgs, ) # --------------------------------------------------------------------------- snapshot @@ -697,6 +701,58 @@ async def list_feeds( } +# --------------------------------------------------------------------------- personal digest feeds + + +async def suggest_personal_feeds( + guild: discord.Guild, args: SuggestPersonalFeedsArgs, ctx: ToolContext | None = None +) -> dict[str, Any]: + candidates = await asyncio.gather(*(validate_feed(url) for url in args.urls)) + return {"candidates": list(candidates)} + + +async def add_personal_feed( + guild: discord.Guild, args: AddPersonalFeedArgs, ctx: ToolContext | None = None +) -> dict[str, Any]: + store = _need_store(ctx) + checked = await validate_feed(args.url) + if not checked["ok"]: + return {"added": False, "url": args.url, "error": checked["error"]} + added = await store.add_personal_feed(args.url, checked.get("title")) + return { + "added": added, + "url": args.url, + "title": checked.get("title"), + "entries": checked.get("entries"), + "note": None if added else "already in the personal feed list", + } + + +async def remove_personal_feed( + guild: discord.Guild, args: RemovePersonalFeedArgs, ctx: ToolContext | None = None +) -> dict[str, Any]: + store = _need_store(ctx) + removed = await store.remove_personal_feed(args.url) + return { + "removed": removed, + "url": args.url, + "note": None + if removed + else "no feed with that exact URL (call list_personal_feeds first)", + } + + +async def list_personal_feeds( + guild: discord.Guild, args: ListPersonalFeedsArgs, ctx: ToolContext | None = None +) -> dict[str, Any]: + store = _need_store(ctx) + rows = await store.list_personal_feeds() + return { + "feeds": [{"url": r["url"], "title": r["title"]} for r in rows], + "count": len(rows), + } + + # --------------------------------------------------------------------------- toys (self / read) # Where the persisted presence "outfit" lives in the meta table. bot.py reads this key on boot to @@ -1125,6 +1181,10 @@ async def preview(name: str, guild: discord.Guild, args: Any) -> str: "add_feed": add_feed, "remove_feed": remove_feed, "list_feeds": list_feeds, + "suggest_personal_feeds": suggest_personal_feeds, + "add_personal_feed": add_personal_feed, + "remove_personal_feed": remove_personal_feed, + "list_personal_feeds": list_personal_feeds, "set_presence": set_presence, "set_nickname": set_nickname, "server_stats": server_stats, diff --git a/roger/tools/schemas.py b/roger/tools/schemas.py index 70bf8fe..b78c03f 100644 --- a/roger/tools/schemas.py +++ b/roger/tools/schemas.py @@ -177,6 +177,22 @@ class RemoveFeedArgs(ToolArgs): url: str # exact stored URL (from list_feeds) +class ListPersonalFeedsArgs(ToolArgs): + """No arguments — returns the owner's personal digest feed list.""" + + +class SuggestPersonalFeedsArgs(ToolArgs): + urls: list[str] = Field(min_length=1, max_length=8) # candidate feed URLs to vet + + +class AddPersonalFeedArgs(ToolArgs): + url: str # RSS/Atom feed URL; validated live before it is stored + + +class RemovePersonalFeedArgs(ToolArgs): + url: str # exact stored URL (from list_personal_feeds) + + # --------------------------------------------------------------------------- toys (self / read) StatusName = Literal["online", "idle", "dnd", "invisible"] @@ -439,6 +455,37 @@ def needs_confirm(self, args: Any) -> bool: ), args_model=RemoveFeedArgs, ), + "list_personal_feeds": ToolSpec( + name="list_personal_feeds", + description="List the RSS/Atom feeds in the owner's personal digest (DM'd privately, " + "separate from the public digest). Read-only.", + args_model=ListPersonalFeedsArgs, + ), + "suggest_personal_feeds": ToolSpec( + name="suggest_personal_feeds", + description=( + "Validate candidate RSS/Atom feed URLs WITHOUT adding them to the personal digest. " + "Returns, per URL, whether it's a live feed, its title, and how many items it has. " + "Use this to vet feeds you propose before calling add_personal_feed." + ), + args_model=SuggestPersonalFeedsArgs, + ), + "add_personal_feed": ToolSpec( + name="add_personal_feed", + description=( + "Validate and add one RSS/Atom feed to the owner's personal digest. Fails if the URL " + "isn't a live feed. Idempotent — adding an existing feed is a no-op." + ), + args_model=AddPersonalFeedArgs, + ), + "remove_personal_feed": ToolSpec( + name="remove_personal_feed", + description=( + "Remove a feed from the owner's personal digest by its exact URL. Call " + "list_personal_feeds first to get the exact URL." + ), + args_model=RemovePersonalFeedArgs, + ), "set_presence": ToolSpec( name="set_presence", description=( diff --git a/tests/test_executors.py b/tests/test_executors.py index 7991085..1106012 100644 --- a/tests/test_executors.py +++ b/tests/test_executors.py @@ -13,6 +13,7 @@ from roger.tools.schemas import ( AddFeedArgs, AddMemberRoleArgs, + AddPersonalFeedArgs, ChannelGrant, CreateChannelArgs, CreateForumPostArgs, @@ -24,6 +25,7 @@ ListFeedsArgs, ListForumPostsArgs, ListInvitesArgs, + ListPersonalFeedsArgs, ListRoleMembersArgs, ListScheduledEventsArgs, ListWebhooksArgs, @@ -32,9 +34,11 @@ PostMessageArgs, RemoveFeedArgs, RemoveMemberRoleArgs, + RemovePersonalFeedArgs, ReplyToForumPostArgs, SetPermissionsArgs, SuggestFeedsArgs, + SuggestPersonalFeedsArgs, ) @@ -1246,6 +1250,70 @@ async def test_feed_tool_without_store_raises_guard_error(): await executors.list_feeds(None, ListFeedsArgs(), None) +async def test_add_personal_feed_validates_and_persists(feeds): + feeds.responses["http://good"] = _good_feed(title="Good Blog", n=5) + out = await executors.add_personal_feed( + None, AddPersonalFeedArgs(url="http://good"), feeds.ctx + ) + assert out["added"] is True + assert out["title"] == "Good Blog" + assert [f["url"] for f in await feeds.store.list_personal_feeds()] == ["http://good"] + + +async def test_add_personal_feed_rejects_non_feed(feeds): + out = await executors.add_personal_feed( + None, AddPersonalFeedArgs(url="http://nope"), feeds.ctx + ) + assert out["added"] is False + assert await feeds.store.count_personal_feeds() == 0 + + +async def test_add_personal_feed_is_idempotent(feeds): + feeds.responses["http://good"] = _good_feed() + await executors.add_personal_feed(None, AddPersonalFeedArgs(url="http://good"), feeds.ctx) + out = await executors.add_personal_feed( + None, AddPersonalFeedArgs(url="http://good"), feeds.ctx + ) + assert out["added"] is False + assert out["note"] == "already in the personal feed list" + + +async def test_remove_personal_feed_hit_and_miss(feeds): + feeds.responses["http://good"] = _good_feed() + await executors.add_personal_feed(None, AddPersonalFeedArgs(url="http://good"), feeds.ctx) + hit = await executors.remove_personal_feed( + None, RemovePersonalFeedArgs(url="http://good"), feeds.ctx + ) + assert hit["removed"] is True + miss = await executors.remove_personal_feed( + None, RemovePersonalFeedArgs(url="http://good"), feeds.ctx + ) + assert miss["removed"] is False + + +async def test_list_personal_feeds_returns_current(feeds): + feeds.responses["http://a"] = _good_feed(title="A") + await executors.add_personal_feed(None, AddPersonalFeedArgs(url="http://a"), feeds.ctx) + out = await executors.list_personal_feeds(None, ListPersonalFeedsArgs(), feeds.ctx) + assert out["count"] == 1 + assert out["feeds"][0] == {"url": "http://a", "title": "A"} + + +async def test_suggest_personal_feeds_validates_without_persisting(feeds): + feeds.responses["http://ok"] = _good_feed(title="OK", n=2) + out = await executors.suggest_personal_feeds( + None, SuggestPersonalFeedsArgs(urls=["http://ok"]), feeds.ctx + ) + by_url = {c["url"]: c for c in out["candidates"]} + assert by_url["http://ok"]["ok"] is True + assert await feeds.store.count_personal_feeds() == 0 # suggest never writes + + +async def test_personal_feed_tool_without_store_raises_guard_error(): + with pytest.raises(GuardError): + await executors.list_personal_feeds(None, ListPersonalFeedsArgs(), None) + + # ------------------------------------------------------------------ read-only server info From df94be84067880a94f99d7c68a04825a30de2f98 Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 21:42:49 -0400 Subject: [PATCH 08/10] feat: schedule the personal digest and alert on failure --- roger/bot.py | 57 ++++++++++++++++++++++++++++++++++++++++++++++- tests/test_ops.py | 19 +++++++++++++++- 2 files changed, 74 insertions(+), 2 deletions(-) diff --git a/roger/bot.py b/roger/bot.py index 1c8095f..5b25139 100644 --- a/roger/bot.py +++ b/roger/bot.py @@ -29,7 +29,12 @@ from roger import metrics from roger.brains.admin import handle_admin_request from roger.brains.ambient import AmbientLimiter, handle_ambient -from roger.brains.digest import run_digest_job, seed_feeds_if_empty +from roger.brains.digest import ( + run_digest_job, + run_personal_digest_job, + seed_feeds_if_empty, + seed_personal_feeds_if_empty, +) from roger.brains.gigabrain import handle_gigabrain_request, run_gigabrain_suggestion from roger.config import Settings, load_settings from roger.health import HEARTBEAT_PATH @@ -418,6 +423,18 @@ def _digest_problem(status: str) -> str | None: return status +# Personal digest statuses that mean "ran fine, nothing to flag"; anything else is worth an ops +# ping — same OK-prefix shape as the public digest. +_PERSONAL_DIGEST_OK_PREFIXES = ("posted", "no new items") + + +def _personal_digest_problem(status: str) -> str | None: + """The personal digest status if it signals a problem worth alerting on, else None (pure).""" + if any(status.startswith(prefix) for prefix in _PERSONAL_DIGEST_OK_PREFIXES): + return None + return status + + # Gigabrain statuses that mean "nothing to flag" — including its own self-gating no-ops. _GIGABRAIN_OK_PREFIXES = ("delivered", "not due yet", "periodic suggestions not configured") @@ -466,6 +483,11 @@ async def setup_hook(self) -> None: seeded = await seed_feeds_if_empty(self.store, self.settings) if seeded: log.info("seeded %d feed(s) from DIGEST_FEEDS into the store", seeded) + personal_seeded = await seed_personal_feeds_if_empty(self.store, self.settings) + if personal_seeded: + log.info( + "seeded %d feed(s) from PERSONAL_DIGEST_FEEDS into the store", personal_seeded + ) await self._maybe_prune() # tidy expired rows on boot; the watchdog repeats it daily self._heartbeat.start() # liveness for the Dockerfile HEALTHCHECK (always on) if self.settings.metrics_port: @@ -483,6 +505,20 @@ async def setup_hook(self) -> None: log.info( "digest scheduled daily at %02d:00 %s", self.settings.digest_hour, self.settings.tz ) + # Turned on by configuring at least one seed feed — DM delivery needs no channel to be set, + # unlike the public digest's channel-required gate. + if self.settings.personal_digest_feeds: + self._personal_digest_loop.change_interval( + time=datetime.time( + hour=self.settings.personal_digest_hour, tzinfo=ZoneInfo(self.settings.tz) + ) + ) + self._personal_digest_loop.start() + log.info( + "personal digest scheduled daily at %02d:00 %s", + self.settings.personal_digest_hour, + self.settings.tz, + ) # Same pattern as digest: a daily tick that self-gates on the configured interval (§12). if self.settings.gigabrain_interval_days > 0: self._gigabrain_loop.change_interval( @@ -663,6 +699,25 @@ async def _digest_loop(self) -> None: async def _before_digest(self) -> None: await self.wait_until_ready() + @tasks.loop(time=datetime.time(hour=7)) + async def _personal_digest_loop(self) -> None: + result = await run_personal_digest_job( + client=self, settings=self.settings, llm=self.llm, store=self.store + ) + status = str(result.get("status", "")) + log.info("scheduled personal digest: %s", status) + problem = _personal_digest_problem(status) + if problem: + await self._ops.alert( + f"personal_digest:{time.strftime('%Y-%m-%d')}", + f"⚠️ **personal digest problem** — {problem}", + cooldown_s=_DAY_S, + ) + + @_personal_digest_loop.before_loop + async def _before_personal_digest(self) -> None: + await self.wait_until_ready() + @tasks.loop(time=datetime.time(hour=9)) async def _gigabrain_loop(self) -> None: result = await run_gigabrain_suggestion( diff --git a/tests/test_ops.py b/tests/test_ops.py index 9c24579..59c876b 100644 --- a/tests/test_ops.py +++ b/tests/test_ops.py @@ -1,6 +1,12 @@ """Ops-channel alerting — the dedupe notifier and the pure alert-decision helpers (backlog 1.2).""" -from roger.bot import OpsNotifier, _budget_alert, _digest_problem, _gigabrain_problem +from roger.bot import ( + OpsNotifier, + _budget_alert, + _digest_problem, + _gigabrain_problem, + _personal_digest_problem, +) class _FakeClock: @@ -71,6 +77,17 @@ def test_digest_problem_flags_failures(): assert _digest_problem("digest brain not configured (no models)") is not None +def test_personal_digest_problem_none_for_success_statuses(): + assert _personal_digest_problem("posted") is None + assert _personal_digest_problem("no new items") is None + + +def test_personal_digest_problem_flags_failures(): + assert _personal_digest_problem("personal digest not configured (no feeds)") is not None + assert _personal_digest_problem("DM failed; digest not delivered") is not None + assert _personal_digest_problem("budget exceeded; skipped") is not None + + def test_gigabrain_problem_none_for_success_and_self_gated_statuses(): assert _gigabrain_problem("delivered") is None assert _gigabrain_problem("not due yet") is None From 9fd623eb5b5a3a296e3433148f759ed66154a9e7 Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 21:47:19 -0400 Subject: [PATCH 09/10] docs: personal digest in ARCHITECTURE, ROADMAP, README --- ARCHITECTURE.md | 8 ++++++++ README.md | 2 ++ ROADMAP.md | 2 +- 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ec45a3b..88dcd48 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -257,6 +257,13 @@ triggerable via the `run_digest` tool. There is **no user input anywhere in this are truncated to 500 chars before the model sees them. - **Exactly-once posting.** Items are marked **seen** (`seen` table) only *after* a successful post, so a failed post retries the same items next time rather than dropping them. +- **A personal, DM'd sibling.** `run_personal_digest_job` is the same mechanism — fetch, dedupe, + summarize — pointed at a second, separately-curated `personal_feeds` list, and delivered to the + owner only (`PERSONAL_DIGEST_CHANNEL_ID` if set, else a DM — same fallback shape Giga Brain's + periodic check-in uses, §12). It shares the `digest` brain's model and daily budget; it's a + second job, not a second brain. Curated the same way — `suggest_personal_feeds` / `add_personal_feed` + / `remove_personal_feed` / `list_personal_feeds` mirror the public digest's four curation tools + exactly. ## §10 Persistence @@ -272,6 +279,7 @@ behaviour adds rows, not migrations. | `admin_log` | Owner admin conversation memory, per channel (§6) | | `gigabrain_log` | Owner gigabrain conversation memory, per channel (§12) | | `feeds` | The curated digest feed list (§9) | +| `personal_feeds` | The owner's personal digest feed list, curated separately from `feeds` (§9) | | `meta` | Small key/value bot state (persisted presence outfit, gigabrain's last-run date) — never pruned | ## §11 LLM layer & budgets diff --git a/README.md b/README.md index 8117e2e..2a747b7 100644 --- a/README.md +++ b/README.md @@ -114,6 +114,8 @@ Feature-complete across the planned phases: - **Digest** — a scheduled daily RSS/Atom summary (also triggerable via `/roger run the digest now`), deduped so nothing posts twice. Roger curates its own feed list: `DIGEST_FEEDS` seeds it once, then Roger validates candidates against the live web and adds or drops them on request. + A second, privately-curated feed list can also be DM'd to the owner only + (`PERSONAL_DIGEST_FEEDS`), on its own schedule. Runs as a non-root, read-only-rootfs container. ~110 tests cover the guard rules, the tool loop (including channel creation with access presets and the confirm-gated edit, post, and reorder diff --git a/ROADMAP.md b/ROADMAP.md index eae23eb..a3ac7d8 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -13,7 +13,7 @@ Same effort key as BACKLOG: **S** ≈ an afternoon, **M** ≈ a day or two, **L* --- -## 1. Personal digest — **S/M** — *spec written* +## 1. Personal digest — **S/M** — *shipped* A feed roundup DM'd to the owner only, separate list and schedule from the public digest. Directly answers "I'm out of the loop" — reuses Digest's existing RSS/feeds/curation plumbing rather than From dc75a2d7bda58ef7784a3abde1f5f8c19ed02bd0 Mon Sep 17 00:00:00 2001 From: Ross Tomsic Date: Tue, 18 Aug 2026 22:00:18 -0400 Subject: [PATCH 10/10] fix: address final review findings (loop self-gating, channel check, seen-collision docs, registry table) --- ARCHITECTURE.md | 24 ++++++++++++++++++------ README.md | 2 +- roger.env.example | 4 +++- roger/bot.py | 31 +++++++++++++++++-------------- tests/test_ops.py | 2 +- 5 files changed, 40 insertions(+), 23 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 88dcd48..b5d0005 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -188,6 +188,10 @@ Registry: | `suggest_feeds` | no (validates only) | — | | `add_feed` | yes | no | | `remove_feed` | yes | no | +| `list_personal_feeds` | no | — | +| `suggest_personal_feeds` | no (validates only) | — | +| `add_personal_feed` | yes | no | +| `remove_personal_feed` | yes | no | | `set_presence` | self only (own status/activity, persisted) | no | | `set_nickname` | self only (own guild nickname) | no | | `server_stats` | no | — | @@ -258,12 +262,20 @@ triggerable via the `run_digest` tool. There is **no user input anywhere in this - **Exactly-once posting.** Items are marked **seen** (`seen` table) only *after* a successful post, so a failed post retries the same items next time rather than dropping them. - **A personal, DM'd sibling.** `run_personal_digest_job` is the same mechanism — fetch, dedupe, - summarize — pointed at a second, separately-curated `personal_feeds` list, and delivered to the - owner only (`PERSONAL_DIGEST_CHANNEL_ID` if set, else a DM — same fallback shape Giga Brain's - periodic check-in uses, §12). It shares the `digest` brain's model and daily budget; it's a - second job, not a second brain. Curated the same way — `suggest_personal_feeds` / `add_personal_feed` - / `remove_personal_feed` / `list_personal_feeds` mirror the public digest's four curation tools - exactly. + summarize — pointed at a second, separately-curated `personal_feeds` list, delivered to + `PERSONAL_DIGEST_CHANNEL_ID` if set, else DMed directly to the owner — same fallback shape and + the same "deploy owner's choice of destination and privacy, not Roger's" reasoning as Giga + Brain's periodic check-in (§12). It shares the `digest` brain's model and daily budget; it's a + second job, not a second brain. Curated the same way — `suggest_personal_feeds` / + `add_personal_feed` / `remove_personal_feed` / `list_personal_feeds` mirror the public digest's + four curation tools exactly. Scheduled unconditionally, same as Giga Brain's interval check — + the job itself decides "not configured" (no feeds) rather than the caller gating on whether a + seed env var happens to still be set, so feeds curated live via chat are actually delivered. +- **One caveat: `seen` is shared.** Dedup is keyed on `(feed_url, entry_id)` globally, not per + list — a URL curated into *both* `feeds` and `personal_feeds` is only ever delivered by + whichever job runs first that day (personal digest defaults to `PERSONAL_DIGEST_HOUR=7`, before + the public digest's `DIGEST_HOUR=8`). Don't add the same feed to both lists if you want it in + both digests. ## §10 Persistence diff --git a/README.md b/README.md index 2a747b7..c6c7eef 100644 --- a/README.md +++ b/README.md @@ -117,7 +117,7 @@ Feature-complete across the planned phases: A second, privately-curated feed list can also be DM'd to the owner only (`PERSONAL_DIGEST_FEEDS`), on its own schedule. -Runs as a non-root, read-only-rootfs container. ~110 tests cover the guard rules, the tool loop +Runs as a non-root, read-only-rootfs container. ~275 tests cover the guard rules, the tool loop (including channel creation with access presets and the confirm-gated edit, post, and reorder tools), the rate limiter, and the digest and feed-curation paths. diff --git a/roger.env.example b/roger.env.example index 741a564..4ba1d69 100644 --- a/roger.env.example +++ b/roger.env.example @@ -68,7 +68,9 @@ DIGEST_HOUR=8 # --- personal digest --- # comma-separated RSS/Atom URLs, curated separately from the public digest above. Seeds once, same -# rule as DIGEST_FEEDS. +# rule as DIGEST_FEEDS. Dedup is shared with the public digest's feed list (keyed on URL+entry, not +# per-list) -- avoid adding the same feed to both lists, or only whichever runs first each day +# actually delivers it. PERSONAL_DIGEST_FEEDS= # unset = DM the owner directly; set = post there instead (same shape as DIGEST_CHANNEL_ID) PERSONAL_DIGEST_CHANNEL_ID= diff --git a/roger/bot.py b/roger/bot.py index 5b25139..a7c2cb2 100644 --- a/roger/bot.py +++ b/roger/bot.py @@ -232,6 +232,7 @@ def _missing_permissions(perms: discord.Permissions) -> list[str]: ("digest_channel_id", "digest"), ("ops_channel_id", "ops"), ("gigabrain_channel_id", "gigabrain check-in"), + ("personal_digest_channel_id", "personal digest"), ) @@ -425,7 +426,7 @@ def _digest_problem(status: str) -> str | None: # Personal digest statuses that mean "ran fine, nothing to flag"; anything else is worth an ops # ping — same OK-prefix shape as the public digest. -_PERSONAL_DIGEST_OK_PREFIXES = ("posted", "no new items") +_PERSONAL_DIGEST_OK_PREFIXES = ("posted", "no new items", "personal digest not configured") def _personal_digest_problem(status: str) -> str | None: @@ -505,20 +506,22 @@ async def setup_hook(self) -> None: log.info( "digest scheduled daily at %02d:00 %s", self.settings.digest_hour, self.settings.tz ) - # Turned on by configuring at least one seed feed — DM delivery needs no channel to be set, - # unlike the public digest's channel-required gate. - if self.settings.personal_digest_feeds: - self._personal_digest_loop.change_interval( - time=datetime.time( - hour=self.settings.personal_digest_hour, tzinfo=ZoneInfo(self.settings.tz) - ) - ) - self._personal_digest_loop.start() - log.info( - "personal digest scheduled daily at %02d:00 %s", - self.settings.personal_digest_hour, - self.settings.tz, + # Always scheduled, unconditionally — DM delivery needs no channel or feeds pre-configured. + # run_personal_digest_job self-gates on "no feeds" the same way run_gigabrain_suggestion + # self-gates on its interval (§12): the job decides, not the caller. This also means feeds + # curated live via the admin tools (with PERSONAL_DIGEST_FEEDS left unset) actually get + # scheduled, not just seeded into a table nothing reads from. + self._personal_digest_loop.change_interval( + time=datetime.time( + hour=self.settings.personal_digest_hour, tzinfo=ZoneInfo(self.settings.tz) ) + ) + self._personal_digest_loop.start() + log.info( + "personal digest scheduled daily at %02d:00 %s", + self.settings.personal_digest_hour, + self.settings.tz, + ) # Same pattern as digest: a daily tick that self-gates on the configured interval (§12). if self.settings.gigabrain_interval_days > 0: self._gigabrain_loop.change_interval( diff --git a/tests/test_ops.py b/tests/test_ops.py index 59c876b..85f5e29 100644 --- a/tests/test_ops.py +++ b/tests/test_ops.py @@ -80,10 +80,10 @@ def test_digest_problem_flags_failures(): def test_personal_digest_problem_none_for_success_statuses(): assert _personal_digest_problem("posted") is None assert _personal_digest_problem("no new items") is None + assert _personal_digest_problem("personal digest not configured (no feeds)") is None def test_personal_digest_problem_flags_failures(): - assert _personal_digest_problem("personal digest not configured (no feeds)") is not None assert _personal_digest_problem("DM failed; digest not delivered") is not None assert _personal_digest_problem("budget exceeded; skipped") is not None